Real-time data streaming is no longer a luxury; it’s a necessity. Change Data Capture (CDC) pipelines, through their generated feed of data modification events, power everything from data replication to event-driven architectures. However, building a truly scalable and efficient solution is no small feat, especially when dealing with high-throughput workloads.

YugabyteDB, a distributed PostgreSQL-compatible database, is designed to handle massive data volumes while ensuring strong consistency and high availability. Its architecture makes it an excellent choice for CDC pipelines that need to scale without compromising performance.

In this talk, we’ll introduce the YugabyteDB CDC connector for Apache Flink, a solution already in use at New Relic to support demanding real-world scenarios. We’ll explore how it integrates with Flink’s powerful stream processing capabilities, the unique advantages it offers, and the key challenges we tackled during development. If you’re looking for a reliable and scalable way to process data changes in real time, maybe this session can offer you some valuable insights.