LLM cost calculator: cloud API vs local GPU

For a given workload, estimate monthly cost on a cloud API offering and the cost of running an equivalent open-weight model on your own GPU hardware, with amortisation, energy and labour made explicit. Not a benchmark: local throughput is estimated from hardware specifications with efficiency factors you can override below. All amounts in USD. EUR display

Grok 4 Fast (Azure AI Foundry): no cache read price published, cacheHitRatio treated as 0.
Grok 4 Fast (Azure AI Foundry): no cache read price published, cacheHitRatio treated as 0.

Fairness controlsQuality match (local quantisation)

fp8 is approx. -0.5 benchmark points vs fp16 (approximate until Phase 2).

Tokenizer (for $ / 1M characters)Tokenizer corpus sliceUtilisation (30.0%)

Amortisation horizon24 mo36 mo48 mo

WorkloadPresetInput tokens / monthOutput tokens / month

Or compute from requests/day

Requests / dayAvg input tokens / requestAvg output tokens / request

Cache hit ratioCache write ratio (blank = 1 - hit)Batch sharePeak concurrencyInteractive (must meet a latency target)Latency target (tokens/s/user)

CloudModelHost / offeringCompare every other host of this model

$3.20 / month ($0.32 / Mtok effective)

$0.0901 / 1M chars (o200k, blended slice)

Breakdown
Input	$1.20
Output	$2.00
Cache read	$0.00
Cache write	$0.00
Batch saving	-$0.00
Per-request fees	$0.00

Other hosts of this model
Host	$/month	$/Mtok
Grok 4 Fast (Azure AI Foundry)	$3.20	$0.32

Local hardwareHardwareOpen-weight modelContext length (tokens, for KV sizing)

Custom hardware (used only when Hardware = Custom)

GPUsVRAM per GPU (GB)Memory bandwidth (GB/s, total)FP16 TFLOPS (total)GPU capex (USD)Host capex (USD)GPU TDP (W)GPU idle power (W)Host idle power (W)Host load power (W)InterconnectUnified memory

Cost assumptions

Electricity (USD/kWh)Electricity presetLabour hours / monthLabour rate (USD/h)Hosting (USD/month)Software (USD/month)Spares (fraction of capex)Residual value (fraction of GPU capex)

Advanced: measured throughput overrides

Measured decode tokens/s (overrides the modelled estimate)Measured prefill tokens/s

Does not fit: needs 57.6 GB, this hardware has 32 GB. Smallest fitting preset: rtx5090x2.

Cloud$3.20 / mo

Local, workload$– / Mtok

Local, capacity$– / Mtok

Cheapercloud

Breakeven and sensitivity

Breakeven volume: – tokens/month. Breakeven utilisation: –.

Monthly cost vs. tokens/month

Cloud cost scales linearly with volume; local cost steps up each time another box is needed. Log scale on tokens/month.

Assumptions

Key	Value	Source	Overridden
hoursPerMonth	730	CALCULATOR_SPEC.md §2	no
daysPerMonth	30.4	CALCULATOR_SPEC.md §3	no
weightsOverheadFactor	1.05	CALCULATOR_SPEC.md §6.1	no
fixedOverheadGb	2	CALCULATOR_SPEC.md §6.1	no
tpEffPcie	0.85	CALCULATOR_SPEC.md §6.1	no
tpEffNvlink	0.95	CALCULATOR_SPEC.md §6.1	no
tpEffUnified	1	CALCULATOR_SPEC.md §6.1	no
defaultContextTokens	8192	T08 default (not specified in CALCULATOR_SPEC.md)	no
decodeEff	0.7	CALCULATOR_SPEC.md §6.2	yes
prefillEff	0.5	CALCULATOR_SPEC.md §6.2	yes
batchEffK	0.04	CALCULATOR_SPEC.md §6.2	yes
utilisation	0.3	CALCULATOR_SPEC.md §6.2	no
maxUtilisation	0.95	CALCULATOR_SPEC.md §6.2	no
sparesPct	0.1	CALCULATOR_SPEC.md §6.3	no
residualPct	0.2	CALCULATOR_SPEC.md §6.3	no
loadFactor	0.8	CALCULATOR_SPEC.md §6.3	yes
horizonMonths	36	CALCULATOR_SPEC.md §6.3	no
labourHours	2	CALCULATOR_SPEC.md §6.3	no
labourRate	60	CALCULATOR_SPEC.md §6.3	no
hostingMonth	0	CALCULATOR_SPEC.md §6.3	no
softwareMonth	0	CALCULATOR_SPEC.md §6.3	no
electricityUsdKwh	0.1419	src/data/electricity_defaults.json (US); EIA Electric Power Monthly via commercialenergyadvisors.com	no
latencyTargetTps	20	CALCULATOR_SPEC.md §3	no
tokenizerSlice	blended	CALCULATOR_SPEC.md §7	yes

Permalink