Short Abstract

"It works, most of the time…" It’s the one warning they don’t provide in the FlinkSQL documentation. While writing standard SQL for streaming apps sounds like a dream, the reality of production often feels like trying to survive the first day of middle school without a map. Between data type inference mapping that misses your target and circular application restarts due to malformed records, you need more than just a SELECT statement to get by. In this session, we’re declassifying the secrets of FlinkSQL. I’ll move past the "Flink 101" tutorials to show you how to utilize a Hybrid API approach to get the flexibility of streaming SQL and the battle -fought reliability of DataStream API and ProcessFunction.

________________________________________________________________________ 

Full Description

The official FlinkSQL interface promises a smooth ride, but as any survivor knows, production is where things get messy. At Riskified, we’ve hit the limits of "pure" SQL and had to write our own rules for survival. This session is your Declassified Survival Guide to the parts of FlinkSQL that aren’t in the handbook.

We’ll examine the architecture of a fault-tolerant FlinkSQL job and demonstrate how to bypass inherent limitations using a Hybrid API approach. We’ll show you exactly how to bridge the gap between SQL velocity and DataStream API control.

Results May Vary: Your Guide to Survival

Tip #1: The Hybrid Escape Hatch. Why you shouldn't settle for a pure TableAPI environment if you actually want to survive a production workload.
Tip #2: Don’t Let Data Types Bully You. How to bypass FlinkSQL rigid type inference and handle schema evolution without the 3:00 AM alerts, like Enum to String schema conversions.
Tip #3: The Safety Net (DLQ). A step-by-step look at building a Dead Letter Queue for SQL jobs to ensure one bad record doesn't take down your whole pipeline.
________________________________________________________________________ 

Key Takeaways

The Hybrid Blueprint: Practical code for bridging the Table and DataStream APIs together to achieve results.
Operational Resilience: How to implement DLQs and error-handling in a SQL-first environment.
Production-ready Streaming SQL: Don’t undercompensate your production pipeline in favor of FlinkSQL’s ease-of-use.