Session Type
Breakout Session
Name
From Zero to Hero: petabyte-scale Tiered Storage lessons
Date
Tuesday, May 20, 2025
Time
5:30 PM - 6:15 PM
Location Name
Breakout Room 5
Description
Whether you’re running mission critical applications or just shipping logs in real time, Tiered Storage can make your Kafka Cluster cheaper, easier to manage and faster. To understand the benefits, tradeoffs and the development history join this talk where we’ll uncover KIP-405 and showcase how the community backed up this important feature for Apache Kafka. We’ll rollback the KIP history, starting from 2018, to understand the major milestones and share details on how major industry leaders like Apple, Datadog and Slack helped out and tested both the Tiered Storage functionality and the first AWS S3 open source plugin. Furthermore, we’ll share details, gotchas, and tradeoffs of users successfully adopting Tiered Storage in production at scale, surpassing 150 GB/s of throughput. If you want to optimize your Apache Kafka cluster for performance, cost, and overall health, this session is for you.
Francesco Tisiot Filip Yonov
Level
Intermediate
Target Audience
Architect, Developer, Executive (Technical), Executive (Non-technical)
Tags
Apache Kafka, Architecture, Storage