The question underneath the question

Insights

All three will hold your data and run SQL over it fast enough. The real choice is about which organisation you already are, which billing model you can live inside, and which platform's characteristic failure mode you would rather debug at eleven at night.

Someone has asked you to pick a data platform and the shortlist arrived pre-written: Databricks, Snowflake, Microsoft Fabric. The comparison you will be handed is about engines and benchmark charts. That is not the decision. All three will store your data, run SQL over it quickly enough for the people asking, and connect to your BI tool. The decision is which one fits the organisation you actually have — its existing cloud, its existing skills, its licensing relationships, its tolerance for a bill that moves — and which failure mode you would rather own, because each of the three has a distinctive one and you will meet it.

Databricks grew out of Apache Spark. Its centre is compute: clusters, notebooks, jobs, and a table format it wrote and open-sourced. Governance arrived later, as Unity Catalog, and moving an existing estate off the old Hive metastore onto it is a project rather than a setting. The platform assumes you employ engineers who are comfortable with a distributed runtime, and it rewards them generously.

Snowflake started as a SQL warehouse and grew outwards. Its founding idea — storage separated from compute, compute as disposable virtual warehouses you switch on and off — is now so widely copied that it is easy to forget it was once contentious. The real achievement is that the warehouse is boring. You do not tune a storage engine, you do not size executors, and a mixed team of analysts and engineers can operate it without a dedicated platform specialist. Python and ML have been added competently, but SQL is still the first-class citizen and it shows.

Fabric is not an engine. It is a SaaS bundle over several engines Microsoft already had — Spark for data engineering, a T-SQL warehouse, a KQL engine for time-series and log data, and Power BI — unified by three things: one storage layer (OneLake), one table convention (Delta Parquet), and one meter (capacity). Its centre of gravity is Power BI, and that is the single most useful thing to know about it. If Power BI already holds your semantic model and your reporting, Fabric is a short walk. If it does not, much of what makes Fabric coherent simply does not apply to you.

Snowflake and Databricks both bill consumption by time. Credits and DBUs accrue while compute runs, so idle compute is pure waste, and the most consequential setting in a Snowflake account is auto-suspend. Consumption pricing rewards bursty work and punishes anything left switched on. It also hands every engineer a lever that resembles a fix: doubling a warehouse usually makes a bad query finish sooner, which looks like success and is not. Nobody regrets putting query-level cost attribution in place early. Plenty of teams discover a year in that they cannot say which department caused the bill.

Databricks adds a second bill. You pay Databricks for DBUs and the cloud provider for the machines underneath, and the DBU rate differs by the kind of compute you chose — which makes running production jobs on interactive all-purpose clusters both a hygiene problem and a pricing one. Enforced tagging from the first week is worth more than any optimisation exercise you run later.

Fabric inverts the model. You buy a capacity, everything running on it draws from the same pool, usage is smoothed over time, and you are throttled once you overrun. Finance often prefers this because the number is known in advance. The engineering consequence is that performance stops being a local property of your workload: another team's Spark job can slow your report. Your cost levers are pausing a capacity or buying a larger one, both blunt. Capacity and workspace design therefore becomes the most important design work on the project, and it happens before anyone writes a pipeline.

None of these are exotic. They are what goes wrong in the second year, once the migration is declared finished and the people who built it have moved on.

Delta and Iceberg have been converging for some time, and Databricks acquiring Tabular — the company founded by Iceberg's original authors — took most of the heat out of the argument. All three platforms now read the other side's tables to some useful degree, and Fabric standardised on Delta Parquet in OneLake from the outset. The sensible conclusion is that the file format is no longer where you are locked in.

The lock-in moved up a layer. What is expensive to move is the catalogue and everything hanging from it: permissions, row and column policies, lineage, shares, the semantic model, the orchestration graph, and the several thousand transformations somebody wrote in a specific dialect. Read interoperability also tends to be asymmetric — one engine writes the table properly, the others read it acceptably — so establish which direction your real write path runs before you treat open formats as an exit plan. An exit you have never rehearsed is not a plan, it is a hope.

Fabric, when Power BI is already the reporting and semantic layer, identity is already Entra, and the team is analysts fluent in T-SQL. Direct Lake is a real advantage in that world: no import refresh to manage, no DirectQuery penalty, as long as you monitor for fallback. Fabric is also the easiest of the three to buy inside an organisation that already buys everything from Microsoft, and that matters more to timelines than anyone admits in the technical review.

Snowflake, when you want a warehouse a mixed team can operate without a platform engineer, and when many concurrent BI users matter. The one feature I would weigh heavily is sharing: granting another account live read access without shipping copies. If your business hands data to partners or customers, that removes an entire category of file-transfer engineering, and it is the hardest capability on this page to reproduce yourself.

Databricks, when training and serving your own models is in scope, when heavy transformation logic wants to be Python or Scala rather than SQL, when streaming is real rather than aspirational, and when a meaningful share of the data is unstructured. It is the most capable of the three and the least forgiving. Choose it if you have, or can hire, engineers who want to operate a compute platform.

If two of them fit, you are close to a coin flip, and the right move is to let procurement leverage and local hiring decide it and then close the question. More value is lost to a six-month selection process than to choosing the second-best platform.

These comparisons quietly assume you have a scale problem. Check that. A single modern server takes hundreds of gigabytes of RAM and terabytes of fast local NVMe, and Postgres, DuckDB or Trino over Parquet in storage you control will absorb a surprising amount of genuine analytical work with no per-second meter running anywhere. If your largest fact table is in the hundreds of millions of rows and your concurrency is a couple of dozen analysts, you may be buying elasticity for a load that never arrives.

Where the crossover sits, I cannot tell you honestly in general terms. It depends on your data volumes, query concurrency, latency expectations, how much of the work is unstructured, and above all whether you have people who want to run infrastructure. Anyone who offers you a threshold figure without asking those questions is guessing at your expense.

And if the binding constraint is that the data cannot leave premises you control — regulated records, client confidentiality, a contractual restriction — then none of the three is on the list and this was never a platform-selection exercise. It is an architecture problem, and it should be run as one.

Take your ugliest existing transformation and your worst-performing report, and run both on the two finalists with your own data at your own concurrency. Ignore published benchmarks: Databricks and Snowflake have each published results favouring themselves and publicly disputed the other's methodology, which is a fair indication of what those figures are worth. Model cost against your own idle pattern rather than a vendor's calculator, because idle is exactly where consumption pricing and capacity pricing diverge. Write down what leaving would cost — catalogue, policies, semantic layer, transformations — and put that number in front of whoever signs. Then decide, in weeks rather than quarters, and spend the remaining year on the data itself, which is where the value was the whole time.

Talk to us about this