Apache Iceberg with a versioned catalog, on your own storage

Data lake and data warehouse in one architecture

A data lake holds data in shared storage for different processing tasks. A data warehouse provides structured data for recurring analysis. The lavalake lakehouse architecture combines object storage, managed Iceberg tables and SQL queries through Trino. Teams can prepare data for metrics while storage remains in their infrastructure.

Apache Parquet and Iceberg: file versus table

Apache Parquet is a column-oriented file format. Iceberg manages the table, its metadata and its snapshots above the files. This distinction matters: an individual Parquet file does not describe the complete state of an Iceberg table. Consistent access also needs to account for the associated table metadata.

Big data architecture: storage, catalog and compute

In lavalake, S3-compatible storage holds data, Nessie provides the versioned catalog and Trino handles SQL processing. This separation makes each component’s role visible. Sizing depends on data volume, query shape and concurrent users; validate the appropriate capacity against your own workloads.

Process changes selectively

Streams track a table’s read position and expose net changes for downstream processing. Time travel and working copies help inspect earlier states and prepare changes.

Why the format is the decision that matters

In a classic warehouse your tables sit in a format only that warehouse reads. Moving to another vendor is then not a configuration step but an export project. Iceberg inverts that: a table is Parquet files plus an openly specified metadata layer. Spark, Flink and Trino read the same table without anyone copying anything.

For a product whose promise is data sovereignty, that is not a side detail. Sovereignty is not only about the disk sitting in your own server room — it also means you can keep using the data without the vendor's cooperation.

An open location and an open format are two different things. Only both together add up to independence.

The catalog is versioned

Above the Iceberg tables sits Nessie — a catalog that behaves like Git. There is a main branch, there are points in time, and there is the option to query an earlier state without touching the current one. In practice: a reporting date stays reproducible even though the table has been written to since.

-- Current state
SELECT count(*) FROM lakehouse.raw.orders;

-- The same view, as of August 1
SELECT count(*) FROM lakehouse.raw.orders
FOR TIMESTAMP AS OF TIMESTAMP '2026-08-01 00:00:00';

The difference from a database backup is substantial. Point-in-time recovery rolls an entire cluster back to a moment. Time travel queries a single table as of a moment while everything else stays current — in production, from an ordinary query.

Storage and compute are separate

The data sits in object storage and the engine reaches it from outside. That is the same architectural principle cloud warehouses use — except both halves are yours. Storage grows without compute having to grow with it, and the other way round.

For evaluation lavalake bundles SeaweedFS, so the whole stack runs on one machine. In production you point it at the object storage you already operate instead.

Scheduled Iceberg maintenance

Schedules can compact files, expire old snapshots and remove orphan files. Retention is configurable from 7 to 365 days, with a default of 7. These tasks only run after a maintenance schedule has been configured; time travel then depends on the snapshots that remain.

Workload classes on one machine

The Compose stack assigns queries to S, M and L classes by user role. These limit concurrent queries and set soft memory limits. They manage existing compute capacity; additional workers expand the shared cluster. Dedicated warehouse clusters are available on Kubernetes; real-cluster KEDA acceptance remains outstanding.

What Trino enforces along the way

Every statement runs under the exchanged token of the logged-in user. Trino's own grants and the row-access and masking policies therefore decide what lands in a result set — not the interface that submitted the query. That chain is why search results and dashboard tiles can differ per viewer.

• ACID transactions at table level
• Schema evolution without rewriting the table
• Time travel and branching through the Nessie catalog
• Partitioning that may change without touching old data
• Dedicated warehouse clusters on Kubernetes; KEDA acceptance pending