Waiting for real datasets, or building complex mocks, slows down streaming development. Apache Flink's Datagen connector lets you rapidly prototype pipelines without any external source, generating realistic, controllable data at scale. In this session, we'll show how to use Datagen to shape schema, control distributions, and push throughput so you can test operators, windows, joins, and sinks under production‑like conditions.

What You'll Learn

We'll cover the full spectrum of Datagen capabilities through hands-on demonstrations:

• Schema Design and Data Types: Learn to define complex schemas with nested structures, arrays, and maps. We'll show how to generate realistic data patterns including timestamps, UUIDs, sequences, and random values across Flink SQL data types.
• Distribution Control: Master techniques for controlling data distributions to simulate real-world scenarios. 
• Performance Testing: Discover how to configure throughput rates, control parallelism, and generate high-volume streams to stress-test your pipeline. We'll demonstrate techniques for simulating backpressure, testing watermark strategies, and validating checkpoint performance under load.
• Advanced Patterns: Explore sophisticated use cases including generating time-series data with realistic gaps, creating multi-stream scenarios for join testing, and simulating late-arriving events to validate your windowing logic.

Stop waiting for data and start building. Datagen transforms streaming development from a slow, dependency-heavy process into a fast, iterative workflow.