Session Type
Breakout Session
Name
Flink SQL Revolutions: Breaking Out of the Matrix with PTFs
Date
Wednesday, May 21, 2025
Time
9:00 AM - 9:45 AM
Location Name
Breakout Room 2
Description
SQL is more than just "a language"; it is the product of tough negotiations among industry leaders, generations of engineers, and data experts. It has evolved into an ecosystem that has proven to survive a constantly changing IT landscape. Yet, many believe that SQL alone is not enough for the instant world we live in today. This world is driven by stream processing and event-driven applications, which demand complex state machines, special types of joins, rule-based conditional logic, and time-based decision-making. Apache Flink was designed exactly for these use cases. However, it seems that Flink SQL users have been somewhat left behind. This has not only hindered the adoption of Flink SQL but also prevented interested users from leveraging a powerful CDC engine. The upcoming Flink version will change this significantly. Process Table Functions (PTFs) defined in FLIP-440 are the solution. These functions accept entire tables as arguments and are equipped to handle streaming primitives such as time, timers, and state. In this talk, I will showcase the full potential of PTFs and how they could replace microservices consuming data from Kafka. This will start a Flink SQL revolution, all while adhering to recent additions to the SQL ISO standard.
Timo Walther
Level
Intermediate
Target Audience
Data Engineer/Scientist, Developer, Architect
Industry
Banking/Finance, Retail/E-Commerce, Technology, IT
Tags
Apache Flink, Event-Driven Systems, Architecture