Structured Streaming, the next generation real-time data analytics component of Spark was released last year. Structured Streaming allows you to think about your incoming data flow as an unbounded, ever-growing table. You use it as if it was a simple static database table and your query results get updated real-time. Spark figures out scaling and the heavy lifting behind the scenes.
Datapao teamed up with POIZO, an award-winning Industry 4.0 startup from the Smart-Heating scene. In this demo I will show you a hands-on presentation of how you can analyse real-world and real-time sensor data using Spark Structured Streaming.
I design and implement Big Data and Spark Architectures at Datapao, mostly for online and Industry 4.0 clients. Besides working on Data Infrastructures, I’m work as a senior consultant and instructor at Databricks, the company created by the founders of Apache Spark. Earlier I worked on the Spark integration project in Rapidminer and led the Data Engineering and the Business Modeling teams of Prezi.