SmartCat is all about data science and data engineering. Storing and processing large amounts of data is in the heart of systems we architect, implement, tune or maintain. Apache Kafka, as a tool for implementing elastic and performant data pipelines, and Event Sourcing, as an architectural pattern, are often on our plate.
The GDPR regulative, inevitable as death and tax, is here to stay. And it’s actually a good thing. But it brings some new challenges, especially in the Big Data field and related technologies and implementation approaches. One of the challenges, the GDRP’s “right to be forgotten”, hits especially hard implementations based on the Event Sourcing pattern, where every piece of data is supposed to be preserved “forever”. Apache Kafka is usually chosen as the storage for such systems.
In this talk, we’ll cover some of the challenges generated by GDPR and one possible approach for solving them in systems that combines Event Sourcing and Apache Kafka, using data encryption.
Nikola Ivancevic
Co-founder & Senior consultant, SmartCat
All-rounder and problem solver with over 20 years of experience in engineering, ranging from hardware design, over embedded and distributed systems, networking, devops, to backend and frontend development. Recently, the main focus covers JVM-based technologies, distributed systems, DevOps and Cloud environments with a special attention to the Big Data technologies and working in data intensive environments using technologies such as Apache Kafka and Apache Cassandra.