Total Cost of Ownership for AI Infrastructure

Overview#

Total Cost of Ownership is the discipline of summing every cost an infrastructure decision creates over its useful life, then comparing decisions on like-for-like totals rather than on advertised unit prices. The temptation — especially when comparing a cloud GPU-hour rate to an on-prem GPU server quote — is to compare only the headline numbers: $11.80/GPU/hr against $35,000 per server. That comparison is always wrong, because the on-prem number excludes most of the cost (power, cooling, networking, facility, people, idle utilisation, decommissioning) and the cloud number excludes none.

A defensible TCO model spans six buckets: capital expenditure (the kit), operating expenditure (power, cooling, connectivity, real estate, software), people (fully-loaded SRE, platform, network, security and procurement headcount), opportunity cost of capital (the discount rate against tied-up capex), opportunity cost of idle capacity (the hours you provisioned but did not consume), and decommissioning (decommission, secure data erasure, disposal). AI infrastructure pushes every bucket harder than general IT because power density is higher, refresh cadence is shorter, and idle GPUs are more expensive per hour than idle CPUs.

Yobitel uses TCO as a first-class ranking dimension across Omniscient Compute (the public-facing index of every relevant GPU provider) and the Yobitel NeoCloud commercial surface. Customers comparing a Yobitel NeoCloud reservation against a hyperscaler reservation see normalised three-year and five-year TCO side-by-side, not just headline rates — because at 70%+ sustained utilisation the headline rate is the least informative number on the page.

This entry helps you build a defensible TCO model for an AI-infrastructure decision (build, hyperscaler, neocloud, hybrid), pick the right horizon, account for the buckets most teams forget, and read Yobitel Omniscient Compute rankings and Yobitel NeoCloud reservation pricing in the same lens you apply to the rest of your portfolio.

How TCO works — the six-bucket model#

Every well-formed AI-infrastructure TCO decomposes into the six buckets below. The exact ratios shift with utilisation, horizon and architecture choice — but the buckets themselves rarely change.

Power and cooling — the bucket most models underweight#

AI infrastructure shifted power density an order of magnitude in two GPU generations. An H100 SXM module draws up to 700 W under load; an H200 module similar; a B200 module around 1,000 W; a B300 module around 1,200 W. An 8-GPU H100 HGX server with CPUs, NICs and storage draws ~10.2 kW; an 8-GPU B200 HGX equivalent can exceed 14 kW; a GB200 NVL72 rack pulls roughly 120 kW. Compare that to the 20 kW per rack most legacy enterprise datacentres were built for — five to seven times the design density.

Power-Usage Effectiveness (PUE) determines how much extra you pay for cooling, lighting and overhead on top of IT load. A modern direct-to-chip liquid-cooled AI facility runs at PUE 1.10 — 1.20; a legacy air-cooled facility can be PUE 1.50 — 1.80. Over a three-year horizon at sustained 70% utilisation, the PUE delta alone can shift TCO by 15-25%. UK industrial electricity prices around $0.18 — $0.24/kWh make the PUE delta worth six figures per rack per year at AI density.

Yobitel NeoCloud's facility footprint runs PUE 1.15 across the UK, EU and US-east footprint, with direct-to-chip liquid cooling on H100/H200/B200 racks and rear-door heat-exchangers on H100 PCIe nodes. The implication for the customer-facing TCO model is that the per-GPU-hour rate Omniscient Compute publishes for Yobitel NeoCloud already bundles the PUE delta — there is no separate cooling line to add.

Variants — when to apply each TCO horizon#

Three years is the most common modelling horizon for general AI infrastructure; it matches the aggressive depreciation schedule most CFOs use for accelerators. Five years is the standard for sovereign and regulated workloads where asset disposal is heavily constrained and the customer expects the kit to outlive a hardware generation. Seven years is unrealistic for current-generation accelerators; thermal stress alone makes the failure curve unfavourable past five.

• 3-year horizon — Standard for general-purpose AI infrastructure. Matches typical straight-line depreciation. The horizon hyperscaler Reserved Instances align to.
• 5-year horizon — Standard for sovereign builds (UK NCSC OFFICIAL workloads under multi-year compliance scope, EU Data Boundary residency contracts). Yobitel UK Sovereign reservations align to this horizon by default.
• Per-workload horizon — A foundation-model training run that lasts six weeks should be modelled at six weeks, not three years. Match the horizon to the decision being made.
• Match horizons across alternatives — Comparing a 3-year on-prem build to a 1-year cloud spend is the most common TCO error. Build the on-demand and reservation alternatives at the same horizon as the build before comparing.

Build vs hyperscaler vs neocloud vs hybrid — when each wins#

There is no universal answer to the build-versus-rent question. The answer depends on utilisation, horizon, access to capital, access to power and skilled operations talent, and whether sovereign-residency constraints rule alternatives out. The summary below maps where each option typically wins on a normalised TCO basis.

Trade-offs and known limitations#

TCO modelling has real limits, and serious practitioners are explicit about them. The model is only ever as defensible as the assumptions feeding it; the most common failures are listed below.

• Utilisation forecasts are aspirational. The realistic utilisation of a new AI cluster in year one is rarely above 50%; ramp to 70-80% only happens with mature scheduling, MIG slicing, and elastic workloads. Model the ramp explicitly, do not assume steady-state from day one.
• Hardware generation risk dominates 5-year models. A 3-year H100 build that is forced to refresh to B300 at year 2.5 because the workload mix shifted writes off significant remaining capex. Stress-test the model against an early-refresh scenario.
• Power availability has become a binding constraint in many UK and US markets. A model that assumes incremental capacity is buyable at $0.20/kWh is wrong in any market where the grid connection itself is the gating factor.
• Sovereign-residency carries TCO premium that is not always quantified. UK NCSC OFFICIAL or EU Data Boundary capacity is typically 15-25% more expensive than comparable hyperscaler regions; that premium is usually defensible against the alternative of failing a compliance audit ([[ncsc-cloud-security-principles]], [[gdpr-article-32]]).
• People costs scale non-linearly with cluster size. The first 256 GPUs need roughly the same SRE headcount as the next 1,000. Linear-scaling people assumptions overestimate the per-GPU cost at scale and underestimate it at small scale.
• Software opex is rising as commercial Kubernetes distributions, observability platforms and FinOps tools increasingly charge per node or per GPU. Model these as variable-with-fleet, not as flat overhead.

Practical implementation notes — how to actually build the model#

A defensible TCO spreadsheet (or Bazel/Python rollup, or warehouse view fed from FOCUS) follows the structure below. The same structure works for one-off business-case analysis and for the standing FinOps dashboard.

• Start with utilisation. Pick a sustained-utilisation forecast and a low/expected/high scenario range. Every other number flexes against this.
• Sum capex bucket by bucket. Use real integrator quotes for GPU servers, not list price — discounts of 15-30% off NVIDIA list are typical at quantity.
• Amortise capex straight-line over the horizon. Pair with an opportunity-cost-of-capital line at the firm's discount rate.
• Power and cooling — multiply IT load by PUE by USD/kWh by 8,766 hours per year. Use a sensitivity range on USD/kWh.
• People — fully load (salary + employer NI/payroll tax + benefits + tooling + overheads, typically 1.4x — 1.7x salary). Allocate fractional headcount honestly across multiple clusters.
• Idle-capacity penalty — express as (1 - utilisation) x amortised capex per period. This is the line most models omit and the line that flips build vs neocloud decisions.
• Compare against neocloud and hyperscaler effective rates pulled from FOCUS EffectiveCost ([[focus-spec]]) — not list rates. Yobitel Omniscient Compute publishes normalised TCO rankings already; cross-reference your spreadsheet against the index.
• Sensitivity analysis — tornado chart on utilisation, USD/kWh, and discount rate. Decisions that hold across all three sensitivities are robust; decisions that only hold in the optimistic corner are not.

Where TCO sits in the Yobitel stack#

Yobitel treats TCO as a published, normalised number across both Omniscient Compute (the public index) and the Yobitel NeoCloud commercial surface. Omniscient Compute ranks every indexed provider not only by current GPU-hour rate but by three-year normalised TCO with PUE, fabric, storage, sovereign-region premium and operational support folded in — so customers comparing Yobitel NeoCloud reservations against AWS P5 reservations, Azure ND H100 v5 reservations, or a build at a UK colocation see the same lens applied to each.

Yobitel NeoCloud's reservation instrument behaves like a FOCUS Savings Plan — see [[savings-plans]] — committing to a compute envelope rather than per-SKU. Customers commit at the dollar-per-hour level across H100, H200 and B200 capacity; the TCO model rolls up to a single effective rate that already includes power at UK industrial rates, PUE 1.15 cooling, NDR InfiniBand or 800GbE fabric, parallel filesystem storage, 24x7 SRE coverage and sub-processor transparency. There is no separate cooling, network or operations line for the customer to model.

Where customers prefer a build path, Yobitel Professional Services delivers TCO modelling and procurement support; where they want a hybrid, Managed Operations runs the 24x7 NOC over the customer-owned estate while Omniscient Compute keeps the FOCUS-conformant cost layer ([[focus-spec]]) unified across in-house, neocloud and hyperscaler footprints. Together this means the build-vs-buy decision can be revisited continuously against the actual numbers, not against the assumptions that were defensible eighteen months ago.