The talk will be about Data engineering. The Stream Processing and Stream Processing Frameworks will be introduced in general, for building scalable pipelines. In Balabit we investigated several solutions and I present our conclusions, with these Distributed Stream Processing Frameworks. I will share my experiences in implementing a single node version of stream processing. Our solution is implemented in python, with an API matching to Apache Storm. This makes our code easy to run in an Apache Storm cluster or in a single machine, without changing the code or bringing the overhead of the Java-Python communication in the single machine version.
Zsigmond Ádám Olivér
Software Engineer, Balabit-Europe Kft.
I have a BSC in Software Technology. I started to work at Balabit 2 years ago. We are working in a User Behavior Analytic product since then. I have experiences in python pandas and machine learning algorithms written by our DataScience Team. The knowledge I am about to share is the experience i got from the recent architecture change in our product to make it easy to scale on multiple machines.