Real-time analytics systems face the dual challenge of serving high-concurrency queries at millisecond latency while keeping up with ever-growing, dynamic datasets. Apache Pinot excels as a low-latency serving layer, but low-latency serving directly from a data lake is a completely different challenge with its own unique complexities.

In this talk, we present native Iceberg support in Apache Pinot, enabling the industry’s first low-latency serving directly on data lakes. We cover the evolution of Pinot’s architecture—from tightly coupled storage and compute to tiered cloud storage, and now from Pinot segment format to natively supporting Parquet format. We will walk through key innovations such as the Parquet forward indexes, the ability to use all of Pinot’s indexes directly on top of Parquet files, index pinning, Parquet page-level reads, and chunk prefetching.

These optimizations go far beyond metadata-based pruning typically offered by Parquet, enabling highly precise, high-concurrency query execution. Attendees will gain insight into how these design choices unlock interactive, real-time analytics on large-scale lakehouse datasets, bridging the gap between serving and modern open data formats.