Session Type
Breakout Session
Name
All you need to know about Kafka v4.0 as a Kafka or Flink user
Date
Wednesday, March 19, 2025
Time
2:00 PM - 2:45 PM
Location Name
Scarlet 3
Description
Apache Kafka v4.0 introduces significant changes that will affect how you deploy, configure, and operate your Kafka clusters. In addition to the famous "zookeeper removal", what else does Kafka v4.0 bring to us? As a Kafka user, what changes will impact my existing kafka cluster? As a Flink user, am I safe from the kafka v4.0 upgrade when adopting flink kafka connector? In this session, I'll go through all the important changes in Apache Kafka v4.0, like the log4j2 upgrade(KIP-653), the next-generation rebalance protocol(KIP-848), the async consumer, the Eligible Leader Replicas(KIP-996), some default configuration changes(KIP-1030), and many component/API deprecation and removal...etc. Not just introducing these changes, most importantly, I'll let you know what they impact your existing cluster, for both Kafka and Flink. Like in the change of KIP-996, will it change the existing unclean leader election mechanism? like in the change of KIP-848, will the Flink kafka connector need to adopt the new async consumer for the new rebalance protocol..., like in the removal of old client protocol API versions(KIP-896), will it be incompatible to my existing kafka clients or Flink Kafka connector...etc. After this session, you will have better understanding of what changes the Apache Kafka v4.0 will have. And you can also know what "actions" you have to take for your existing clusters, no matter it's Kafka or Flink. Finally, you know you can upgrade to Kafka v4.0 without any "surprise".
Luke Chen
Level
Introductory
Target Audience
Operator/Administrator, Developer, Executive (Non-technical)
Industry
IT, Telecommunications, Banking/Finance, Technology
Tags
Apache Kafka, Apache Flink, Stream Processing