Cluster Ops TCO Calculator — on-prem MLX vs frontier API

Inputs

In millions. Slide to your typical monthly throughput across all calls.

Determines tok/sec per mini and which frontier API tier is the fair comparison.

Tighter latency = more headroom = more minis. Default 150 ms covers most chat + RAG.

If peak hour is 3× the average, you size for peak to keep latency.

36 months matches typical Mac Mini service life. Sets the TCO window below.

Cluster Ops sizing

Mac Mini count (peak)
4
Capex (one-time)
$8.0K
Monthly opex (power + colo + ops)
$110/mo
Cost per million tokens (Cluster Ops)
$3.32 / M tokens
TCO over horizon
$11.9K

Computed from your inputs and the assumptions listed below — nothing else. Open “Show the math” for every step.

vs frontier API — same horizon

  • GPT-4o

    $45.0K

    +$33.1K vs on-prem · $12.50/M

  • Claude Sonnet

    $32.4K

    +$20.5K vs on-prem · $9.00/M

  • Gemini Pro

    $18.0K

    +$6.1K vs on-prem · $5.00/M

Break-even vs cheapest comparable API
20.5 months (vs Gemini Pro)

Assumptions

Every figure above comes from your inputs and these values. Each default is the constant the original page kept in its source; change any of them and the results and the math trace follow.

Hardware and running cost

Legacy default $2,000: M4 Pro entry config, 64GB unified memory, 512GB SSD.

Legacy default 50 W, taken as a flat average over the month.

Legacy default $0.15, which it called an NYC commercial rate.

Legacy default $5.

Legacy default $17, which its source describes as about 5 minutes per mini per month at $200/hr.

Legacy default 36. Capex is spread over the shorter of this and the horizon for the cost-per-million figure only.

Throughput and API prices for the medium (30B–70B) class

Legacy defaults 120 / 35 / 12 for small / medium / large: MLX on an M4 Pro 64GB, Q4 quants.

Legacy default $12.50, a 2026-Q2 reference figure. Check the provider's current rate card.

Legacy default $9.00, a 2026-Q2 reference figure. Check the provider's current rate card.

Legacy default $5.00, a 2026-Q2 reference figure. Check the provider's current rate card.

Change the model class above to see and edit the other two tiers.

Formula constants

Legacy default 30: tokens/month ÷ (30 × 24 × 3600 s), and 720 hours of power.

Legacy default 200 ms. Above it no extra capacity is added.

Legacy default 50 ms per step of headroom.

Legacy default 20%: capacity × (1 + 20% for every 50 ms the target sits below 200 ms). A heuristic, not a measurement.

Show the math
inputs
  tokens/month         100M (100.0M)
  model class          medium
  latency target       150 ms p95
  peak/avg ratio       3.0×
  horizon              36 months

throughput
  avg tok/s needed     38.6  (100M × 1,000,000 / (30 d × 24 h × 3600 s))
  peak tok/s needed    115.7  (avg × 3.0)
  latency headroom     20%  (+20% per 50 ms of target below 200 ms)
  tok/s per mini       35  (medium (30B–70B))
  minis required       ceil((115.7 × 1.20) / 35) = 4  (never fewer than 1)

capex
  mini price           $2000
  capex total          4 × $2000 = $8.0K

opex (monthly)
  power                4 × 50W × 720h × $0.15/kWh = $22
  colo (amortized)     4 × $5 = $20
  ops                  4 × $17 = $68
  total opex           $110/mo

cost per million tokens
  capex amortized      $8.0K / 36mo = $222/mo  (min(service life 36, horizon 36))
  total / tokens       ($222 + $110) / 100M = $3.32/M

TCO over horizon
  capex + opex × mo    $8.0K + $110 × 36 = $11.9K  (full capex; no residual value)

vs frontier (horizon 36mo, 100M tok/mo)
  GPT-4o               $45.0K  ($12.50/M)
  Claude Sonnet        $32.4K  ($9.00/M)
  Gemini Pro           $18.0K  ($5.00/M)

break-even
  cheapest API         Gemini Pro @ $5.00/M = $500/mo
  monthly savings      $500 - $110 = $390/mo
  break-even months    20.5  (capex / monthly savings)

assumptions in force (edit them above; defaults are the legacy page's constants)
  hardware             Mac Mini at $2000 each  (legacy default $2,000: M4 Pro entry config, 64GB unified, 512GB SSD)
  power                50W average at $0.15/kWh  (legacy defaults 50W and $0.15, which it called an NYC commercial rate)
  colo                 $5/mini/mo  (legacy default $5)
  ops                  $17/mini/mo  (legacy default $17, "~5 min/mini/month at $200/hr")
  service life         36 months  (legacy default 36)
  tok/s per mini       35 for medium  (legacy defaults 120 / 35 / 12: MLX on M4 Pro 64GB, Q4 quants)
  frontier prices      per 1M tokens, blended 50/50 in+out  (legacy: 2026-Q2 reference; check current rate cards)

Want to walk through these numbers with your own volume and power rate? Contact Garnet →

Where the API wins. Low monthly volume (under ~30M tokens/month for medium-class), bursty traffic without sustained peak, frontier-quality requirements where on-prem is not yet a fair comparison (full-precision GPT-5/Opus class). The calculator says so when it’s honest.

Where on-prem wins. Sustained high volume (~100M+ tokens/mo medium-class), data-residency requirements (HIPAA / GDPR / public-sector), or workflows where the latency to a co-located mini beats round-trip to a hyperscaler region. Cluster Ops productizes this: hot-swap models, automated eviction, hardened deploys, monthly cost-per-token report.

Assumptions are visible (click "Show the math" above) and biased pessimistic for on-prem — we assume entry-level M4 Pro mini at $2,000, $0.15/kWh power, $5/mo colo amortized, and 1 ops-hour/mini/month at $200/hr. Plug in your own numbers; the page is pure client-side JS and the math is in plain view.

See the Cluster Ops subscription → Methodology vs frontier API / cloud GPU

Scope this with us →