Calculator method

How the cloud-vs-local calculator at /calc computes its numbers. Every default in a calculation's Assumptions table links to the section below that explains it, or to the external primary source for data such as electricity prices.

Open the calculator or return to the methodology overview.

1. Purpose

For a given workload: what does it cost per month on a cloud offering, and what would it cost to run an equivalent open-weight model on chosen hardware, with every hidden cost on both sides made explicit? This is not a benchmark tool. Throughput is estimated from first principles with efficiency factors you can override, and the same code runs on the server (permalinks, API) and in your browser.

2. Units

3. Workload

inputTokensMonth, outputTokensMonth   (or requestsDay * avgInput/avgOutput * 30.4)
cacheHitRatio [0,1]       share of input tokens read from cache (cloud only)
cacheWriteRatio [0,1]     share of input tokens written to cache (default 1 - cacheHitRatio)
batchShare [0,1]          share eligible for batch pricing
peakConcurrency           simultaneous streams at peak (local sizing)
interactive               local must meet latencyTargetTps
latencyTargetTps          default 20 tokens per second

Workload presets (coding agent, RAG chat, batch summarisation, classification, chat assistant, content generation) fill these fields with typical ratios; each preset states its rationale on the calculator page. Months are converted at 30.4 days.

4. Cloud cost

I, O = monthly input and output tokens; h = cache hit ratio; w = cache write ratio; b = batch share
uncachedIn = I(1-h); cachedIn = I*h; writtenIn = I*w (0 if no cache write price)
inBatch = uncachedIn*b; inStd = uncachedIn(1-b); outBatch = O*b; outStd = O(1-b)
costIn  = inStd*p.input  + inBatch*p.batchInput
costOut = outStd*p.output + outBatch*p.batchOutput
costCache = cachedIn*p.cacheRead + writtenIn*p.cacheWrite
long context: if the average input exceeds a tier's threshold, the tier prices apply to the whole request
cloudMonth = (costIn + costOut + costCache)/1e6 + requestsMonth*p.perRequest
effectivePerMtok = cloudMonth*1e6/(I+O)

If the offering has no published cache price, the cache hit ratio is forced to 0 and a warning is shown. If it has no batch price, the batch discount is used when published, otherwise the batch share is forced to 0. The same calculation is shown for every other host serving the same model.

5. Local hardware and models

Each hardware preset carries VRAM, memory bandwidth, FP16 throughput, power draw, a street price with its date and source, and a verified_at date. Presets: RTX 4090, RTX 5070 Ti, RTX 5090 (1, 2 and 4 cards), NVIDIA DGX Spark, Mac Studio with M5 Ultra (256 GB), A100 80 GB, H100 80 GB (1 and 8 cards), and a custom entry. Street prices move quickly; each preset states the date and source of its price, and where a figure could not be verified the preset says so.

The local model list is built from two sources. Open-weight models in the price database that have a known parameter count are listed automatically; their layer count and KV heads are estimated from the parameter count, and the calculator warns when it uses such an estimate. Models with a published configuration (layers, hidden size, KV heads, head dimension read from the model's own config) keep those exact numbers.

Quantisation is modelled by bytes per parameter only: 2.0 for fp16, 1.0 for fp8 and 0.56 for 4-bit with scales. The quality impact of quantisation is not modelled; the comparison is cost only.

6. Local cost

Local cost has three steps: does the model fit, how fast is it, and what does it cost per month.

6.1 Memory fit

weightsGb = paramsTotalB * bytesPerParam
kvPerTokenBytes = 2 * layers * kvHeads * headDim * kvBytes     (kvBytes 2 for fp16, 1 for fp8)
kvGb = kvPerTokenBytes * contextTokens * peakConcurrency / 1e9
neededGb = weightsGb*1.05 + kvGb + 2
fits = neededGb <= gpus*vramGb

Weights take 5% more than their raw size and 2 GB is reserved for the runtime. The context length used to size the KV cache defaults to 8192 tokens, a typical chat context, and can be edited. With several GPUs, tensor-parallel efficiency (tpEff) is 0.85 over PCIe, 0.95 over NVLink and 1.0 on a single unified-memory machine. If the model does not fit, the calculator reports the smallest preset that does and shows no cost.

6.2 Throughput

activeBytes = paramsActiveB*1e9*bytesPerParam
singleStreamTps = memBandwidthGbs*1e9*gpus*tpEff / activeBytes * decodeEff        (decodeEff 0.70)
batchEff(n) = 1/(1+0.04(n-1))
batchedTpsTotal = singleStreamTps*batchN*batchEff(batchN)     (batchN = min(peakConcurrency, memory limit))
prefillTps = fp16Tflops*1e12*gpus*tpEff*prefillEff / (2*paramsActiveB*1e9)       (prefillEff 0.50)

Decode speed is limited by memory bandwidth and reaches 70% of that bound; prefill is limited by compute and reaches 50% of peak. If the workload is interactive and the per-stream speed falls below the latency target, the batch size is reduced until the target is met. A measured tokens per second figure overrides both.

hoursActive = 730u           (u = utilisation, default 0.30)
capacityOut = batchedTpsTotal*3600*hoursActive; capacityIn = prefillTps*3600*hoursActive
hoursNeeded = O/(batchedTpsTotal*3600) + I/(prefillTps*3600)
boxesNeeded = ceil(hoursNeeded/(730*0.95))      (a box is never loaded above 95%)

6.3 Monthly cost

capex = (capexUsd + hostCapexUsd)*(1 + sparesPct)          sparesPct 0.10
residual = residualPct*capexUsd                              residualPct 0.20 of the GPU price
amortMonth = (capex - residual)/horizonMonths                horizon 36 (24 and 48 offered)
energyKwh(hActive) = (gpus*tdpW*loadFactor + hostLoadW)*hActive/1000
                   + (gpus*idleWGpu + hostIdleW)*(730-hActive)/1000     loadFactor 0.80
energyCost = energyKwh * electricityUsdKwh
labourMonth = labourHours * labourRate                        defaults 2 hours at $60
hostingMonth and softwareMonth default to 0
localMonthCapacity = amortMonth + energyCost(730u) + labour + hosting
localMonthWorkload = boxesNeeded*amortMonth + energyCost(hoursNeeded) + labour + hosting
capacityPerMtok = localMonthCapacity*1e6/(capacityIn + capacityOut)
workloadPerMtok = localMonthWorkload*1e6/(I + O)             the fair comparator against cloudMonth

Electricity defaults come from public statistics: the US figure is the commercial-sector average retail price from the EIA Electric Power Monthly, table 5.6.A; the EU figures are Eurostat nrg_pc_205 (non-household prices, band IB, excluding VAT); the UK figure is the DESNZ Quarterly Energy Prices non-domestic average. Each preset in the calculator shows the period it covers, and you can enter your own rate.

7. Tokenizer normalisation

Different tokenizers split the same text into different token counts, so prices per token are not directly comparable. The site measures characters per token for each tokenizer family on a fixed public-domain or self-authored corpus (English prose, code, German, Croatian, Chinese, JSON) and reports a blended average plus per-slice figures. The price per million characters is the price per million tokens divided by characters per token for the chosen slice; the default slice is the blended average. Families without tokenizer data are omitted rather than assumed.

8. Output

A result contains the cloud cost with breakdown and other hosts, the local fit, throughput and cost with breakdown, a comparison (cheaper side, ratio, break-even tokens per month and break-even utilisation, a plus and minus 50% sensitivity table, and a 24, 36 and 48 month horizon table), a quality note, the assumptions table with a source for every default and whether you overrode it, and any warnings. The break-even volume solves cloudMonth(T) = localMonthWorkload(T) with input and output scaled by T; break-even utilisation is the utilisation at which local capacity cost per million tokens equals the cloud effective price.