The actual question

Insights

These are not three grades of the same product. They differ in where schema is enforced, who guarantees that a reader sees a consistent set of files, and which maintenance work becomes yours rather than a vendor's. That third difference is what you will live with, so it deserves more weight than any feature table. Here is the mechanism behind each design, the failure modes it hands you, and how to choose without buying a platform nobody is staffed to operate.

The question people actually bring is narrower than the title suggests. It is usually: our warehouse bill is climbing, or nobody trusts the lake we built, and someone has proposed a lakehouse, so do we move? A feature comparison will not answer that. All three keep columnar data on commodity storage and answer SQL. What separates them is where schema is enforced, who guarantees that a reader sees a consistent set of files, and which maintenance work becomes yours instead of a vendor's. Weigh that last one heavily, because it is the part you live with.

A warehouse enforces schema on write. You declare a table, the system validates data on the way in, and it owns the storage layout, the statistics, the transaction log and the access model. In exchange you get atomic commits, an optimiser with real statistics, predictable concurrency for dashboards, and governance down to the column. You give up control of the format, and historically you coupled your compute to one vendor.

A data lake is object storage plus files, with schema applied on read. A table is a directory prefix, and whatever Parquet happens to sit under that prefix is the table. It is cheap, engine-agnostic and accepts everything, including data you have not modelled: logs, images, audio, PDFs, model weights. The guarantee list, though, is essentially durability. There is no atomic multi-file write, no concept of a table version, and no way for a reader to distinguish a half-finished job from a finished one except by convention. That is the mechanism behind every data-swamp story you have heard. It is a missing primitive, not sloppy engineering.

A lakehouse keeps the same files and adds a metadata layer stating precisely which files constitute version N of a table. Iceberg, Delta Lake and Hudi all do this, with different mechanics. A commit becomes an atomic swap of a pointer to new metadata, usually guarded by optimistic concurrency. That one change buys snapshot isolation, time travel, schema evolution without rewriting history, and per-file statistics an engine can use to skip data it does not need. Nearly everything good and everything irritating about lakehouses follows from it.

Small files first. Every commit writes new files, so frequent commits, streaming ingest or a wide writer fan-out produce very large numbers of small Parquet files. Planning cost scales with file count, per-file overhead starts to dominate, and compression gets worse. The remedy is compaction, and compaction is a scheduled job somebody must size, pay for and de-conflict with writers. It rarely appears in the migration plan.

Then snapshots. Time travel is retention by another name. If old snapshots never expire, the files they reference are never deleted, so storage and metadata both grow and listing slows. Iceberg expires snapshots, Delta vacuums; either way the retention window is a decision with a cost attached, not a housekeeping detail.

Concurrent writers are next. Optimistic concurrency means conflicting commits are rejected and retried, so two jobs touching overlapping partitions will fight. The fix is organisational: one writer per table, or partition ownership drawn so writers cannot collide. Skip it and you get load-dependent failures that only reproduce under production concurrency.

Row-level deletes and updates force an explicit choice. Copy-on-write rewrites the affected files, giving slower writes and clean reads. Merge-on-read writes a delete marker, giving fast writes, but now every read reconciles and you have acquired another compaction job. Decide per table, on the write pattern, rather than inheriting whichever default you got.

And the one that catches people legally: a delete commit does not remove data. Older snapshots still reference the files containing that row, so it stays readable and restorable until retention expires and those files are physically removed. If you have promised somebody erasure within a window, your snapshot retention policy is part of that promise, and the two numbers should agree.

Parquet is open. The Iceberg and Delta specifications are open. That is real, and it is the strongest argument for taking a lakehouse seriously. But a table is more than its files: something has to resolve a table name to its current metadata pointer, and that something is the catalog. The catalog also tends to accumulate identity, grants, masking, lineage and audit. Lock-in did not disappear. It moved from the storage format to the catalog and the governance layer, where it is far less visible.

So when a vendor says your data stays in your own bucket in an open format, believe it, then ask the second question: what happens to your grants, your row filters, your lineage and your table-name resolution when a different engine points at the same bucket? If the answer amounts to "you rebuild them", that is the size of your exit cost.

Related and underrated: two engines reading one table can disagree. Timestamp and timezone handling, type coercion, NULL ordering, decimal precision, collation, regex dialect. "One copy, many engines" is exact at the byte level and looser at the semantic level. If a figure will appear in a board pack and in an analyst's notebook, decide now which engine is authoritative for it.

Which format to choose matters less than it did, and interoperability is moving quickly enough that I would not trust any article on its current state, this one included. Check it at decision time. The durable question is who operates the catalog and whether you can leave.

I am not going to quote prices. They change, they are regional, and a number in an article is a liability. The shapes are stable enough to reason about. Warehouses mostly meter compute in a vendor unit, whether credits, DBUs or slot-hours, or they charge per byte scanned. Either way it is one line item, which makes it easy to attribute and therefore easy to govern.

A lakehouse splits into at least four: object storage, query compute, a catalog or platform charge, and maintenance compute for compaction and expiry. The fourth is the one missing from business cases. It also scales with ingest frequency rather than query volume, so it drifts away from whatever finance is watching.

Two traps worth naming. Per-byte-scanned pricing plus poor partitioning means exploratory analysts repeatedly full-scan your largest tables, and the fix is data layout, not a discount. And if compute sits in a different provider or network from storage, you pay for the crossing on every query, forever. What any of this costs for you, I cannot say, and nor can anybody who has not read your query logs and your ingest rate.

Open table formats are what made running this yourself realistic. The guarantee that used to require a database engine now lives in files, so object storage on your own kit, plus Iceberg or Delta, plus an engine of your choosing, is a working stack with no cloud account in it. Be honest about what comes with it: drive failures, erasure coding, capacity planning, restore drills (a snapshot in the same bucket is not a backup), version upgrades, and no autoscaler to conceal a badly sized compaction job. That is a staffing question, and it has a legitimate answer in both directions.

The mistake in the other direction is more common. Single-node compute has grown faster than most organisations' data. One server with a few hundred gigabytes of RAM and a vectorised engine will answer a large share of real analytical workloads faster than a small distributed cluster, with none of the coordination cost. Measure whether one machine already does it before designing for scale-out. Often it does, and the distributed design was bought against an assumption nobody tested.

If your data is tabular, your consumers are SQL and BI tools, and your volumes are ordinary, a warehouse is the boring correct answer, and boring is a feature here. In my experience organisations usually hold less queryable data than their architecture diagrams imply — measure yours rather than taking that on trust.

The commonest failure is not picking the wrong one. It is running two of them "temporarily": two copies, two sets of semantics, and a reconciliation job with no owner and no documentation which, a year later, is the most load-bearing pipeline in the company. If you migrate, the deliverable is the date the old system is switched off, not the date the new one starts accepting writes.

And bronze, silver and gold are naming conventions, not a design. They say nothing about grain, ownership, refresh SLAs or who is permitted to break what. Layer names are free; the contracts between layers are the work.

Get three numbers off the system you already run: how much data you genuinely query in a typical month rather than how much you store, how often you ingest, and how many concurrent interactive users you serve. Those decide this more than any vendor comparison will. Then take the least complicated option that satisfies them, name the person who owns compaction and retention before go-live rather than after, and fix a date for switching off whatever you are replacing. If you cannot name that person, you are not ready for a lakehouse yet, and that is a perfectly respectable place to be.

Talk to us about this