Kafka and Kubernetes are both complex distributed systems that have been widely adopted by enterprises. This talk dives deep into the challenges of running Kafka on Kubernetes, how they complement each other and describes the best practices that we’ve developed during our customer engagements. We introduce the open source Kafka operator for Kubernetes by Banzai Cloud, which is a radically different, yet much more productive approach to deploying Kafka on Kubernetes than existing alternatives. At the end of the talk we will show you a demo of the Banzai Cloud Kafka Spotguide, which has been built using the Kafka operator to provide a production-grade Kafka service for users of the Pipeline platform. Using the platform, Kafka clusters can be provisioned in on-prem datacenters and in six different clouds in single-, multi-, or hybrid-cloud configurations in minutes. Resiliency, monitoring, centralized log collection, security, health checks, and autoscaling are built-in tier-zero features that all applications and services running on the platform benefit from.
Molnár Bálint
Software Engineer, Banzai Cloud
Balint is a software engineer working for Banzai Cloud. His primary role is changing and adopting data frameworks as Spark, Zeppelin, Kafka and Tensorflow to run “natively” on Kubernetes and automate/oversimplify the user experience of using them in hybrid environments. Prior to joining Banzai Cloud he was working at Hortonworks, working on Apache Kafka.