Hard-won lessons after processing 6.7T events through PostgreSQL queues

Description

Many organizations reach for specialized streaming systems like Apache Kafka for high-throughput event processing. But is it always the best choice? What if you choose PostgreSQL instead.

This talk chronicles six years of battle-tested lessons learned while scaling PostgreSQL from a simple queue to a system processing 100,000 events per second, and delivering total 6.7T events.

Learn about the specific configuration values, query patterns, and architectural decisions that enabled PostgreSQL to compete with and often outperform dedicated messaging systems, while providing the operational simplicity and transactional guarantees that only PostgreSQL can offer.

Speaker

Aris Tzoumas

Staff Engineer, RudderStack

Aris Tzoumas is a Staff Software Engineer at RudderStack with over 20 years of experience building scalable, high-performance systems across telecommunications, banking, European institutions, and data infrastructure. A core contributor to the open-source rudder-server, he specializes in Go, PostgreSQL, Kubernetes, and cloud-native architectures. Aris is passionate about designing elegant solutions, driving engineering best practices, and sharing insights on scalable data pipelines, enterprise DevOps, and software architecture.

Scroll to top