AI Compute Emissions — Training and Inference Methodology and Calculation Approach
The headline number for training a large model is real, large, and almost always misunderstood. The most-cited figure in the field — 284 tonnes of CO₂e for a neural architecture search — is not the cost of training a model; it is a one-time discovery cost that is amortised across every model that reuses the result.
AI carbon accounting fails when a one-off figure is read as a recurring one — and when training is counted while inference, which eventually dwarfs it, is ignored.
AI emissions split into two regimes: training, a discrete one-time event measured per run, and inference, a tiny per-query cost that compounds across billions of calls. For a deployed model, cumulative inference usually exceeds training.
The two regimes: training vs inference
An AI model’s lifetime carbon has two distinct shapes. Training is a discrete, front-loaded event: a single large burst of compute that produces the model, measured per run in tonnes of CO₂e. Inference is the opposite shape: a minuscule cost per query that recurs every time the deployed model answers a request, compounding across billions of calls. The two require different methodologies — training is a one-off energy figure; inference is a per-query chain multiplied by volume — and conflating them, or counting one and ignoring the other, is the central error in AI carbon accounting.
The lifetime crossover is the under-reported consequence: for a model serving meaningful traffic, cumulative inference passes the one-off training cost and keeps climbing. A methodology that reports only the dramatic training number understates the deployed model’s true footprint. Both regimes draw on the same physical infrastructure — the data-centre facilities treated in the PUE and operational methodology and the allocation logic in the cloud compute allocation methodology; this page is specifically about the AI workload running on top of it.
Provider directory
Need help keeping this up all year?
See who does this work. Every listing names the standards it works to, and paid placements are labelled. Including KERAMIDA and Small World Consulting.
Browse 6 carbon accounting & inventory providers →Do this work? A listing is US$390 a year. Get listed →
Training emissions: the per-run figure
Training emissions are computed as the energy consumed by the run multiplied by the grid carbon intensity of the data centre that ran it. The reference figures here are from Patterson et al. 2021, the foundational measured study of large neural-network training, carried in MasterBrain for six landmark models. Each value is the net tCO₂e for a single training run.
First, the data-centre overhead (PUE) is already folded into these energy-to-carbon figures — do not apply a separate PUE multiplier on top, or you double-count the cooling overhead. Second, the values are net of Google’s 24/7 carbon-free-energy accounting only where it was applied: GShard (gross 4.8 → net 4.3) and Switch Transformer (gross 72.2 → net 59.1) carry a CFE reduction; the other four do not. GPT-3, in particular, ran on Microsoft Azure with no Google CFE attribution, so its gross and net are identical at 552.1.
| Model | Net tCO₂e/run | Gross tCO₂e | Energy (MWh) | CFE% at run | Hardware |
|---|---|---|---|---|---|
| GPT-3 (175B params) | 552.1 | 552.1 | 1,287 | — (Azure) | V100 GPU |
| Meena (2.6B) | 96.4 | 96.4 | 232 | 30 | TPU v3 |
| Switch Transformer | 59.1 | 72.2 | 179 | 43 | TPU v3 |
| T5 (11B) | 46.7 | 46.7 | 85.7 | 19 | TPU v3 |
| GShard (600B) | 4.3 | 4.8 | 24.1 | 73 | TPU v3 |
| Evolved Transformer | 3.2 | 3.2 | 7.5 | 31 | TPU v2 (Google Iowa) |
Net tCO₂e values render live from MasterBrain; gross, energy, CFE%, and hardware are carried as authoritative row metadata sourced to Patterson 2021 Table 4. Parameter counts are descriptive, cited from the same source.
The spread is the story. A 600-billion-parameter sparse model (GShard) trained for a fraction of the carbon of a 175-billion-parameter dense one (GPT-3), because architecture, hardware generation, and grid all moved in GShard’s favour. Parameter count alone predicts almost nothing about training carbon — the four factors below do.
The four factors that move training carbon
Patterson’s framing identifies four largely independent levers, each capable of changing training emissions by a large multiple. They are why two models of similar size can differ by an order of magnitude in carbon.
| Factor | What it controls | Magnitude of effect |
|---|---|---|
| Model — architecture & algorithm | How much computation the training requires (sparse vs dense, efficient attention) | Up to ~5–10× between architectures of similar capability |
| Machine — accelerator generation | Work delivered per watt (TPU v2 → v3 → newer; GPU vs TPU) | ~2–5× across hardware generations |
| Mechanisation — datacentre efficiency | Facility overhead (PUE) — already folded into the figures above | ~1.1–1.7× depending on facility class |
| Map — grid location | Carbon intensity of the grid serving the run | Up to ~5–20× between clean and dirty grids |
The grid lever is the largest and the easiest to move. Patterson’s own anchors illustrate it: a run in Google’s Iowa facility at 0.08 kg CO₂e/kWh [GreenCalculus digital.ai.methodology.google_iowa_net_co2_intensity_2019 · PATTERSON LARGE 2021 · v2026.284] against the US grid average of 0.429 kg CO₂e/kWh [GreenCalculus digital.ai.methodology.us_grid_avg_co2_intensity_2021 · PATTERSON LARGE 2021] — a 5.4× difference in carbon for identical energy, purely from where the run executed. Choosing a clean-grid region for training is the single highest-leverage decision, exactly as it is for the operational facility footprint. The relationship between the grid lever and the dual location-based/market-based figure follows the market-based Scope 2 methodology; the CFE reductions in the training table are market-based instruments applied by the operator.
The neural architecture search misconception
The most-cited statistic in AI carbon — a 284-tonne CO₂e figure — comes from Strubell et al. 2019 and describes a neural architecture search: an automated procedure that trains and evaluates thousands of candidate architectures to discover a good one. It is routinely quoted as if it were the cost of training a single model. It is not, and Patterson 2021 reframes it explicitly.
The search is paid once; the architecture is reused across many downstream training runs. Per-use cost falls toward zero as reuse rises — Meena’s reuse alone returned roughly 15× the NAS cost within the first year.
The 284 tonnes is a one-time discovery cost amortised across every model that subsequently reuses the architecture. The Evolved Transformer training run carried in MasterBrain at 3.2 tCO₂e [GreenCalculus digital.ai.training.evolved_transformer · PATTERSON LARGE 2021] excludes the NAS — the two are deliberately separate quantities, because the search is paid once and the training run recurs. Quoting 284 tonnes as the carbon of training a model overstates a single run by roughly two orders of magnitude and ignores the amortisation that is the entire point of the result. The methodology treats a NAS cost as a capital-like one-off, amortised over its downstream reuses, not as a recurring per-model charge.
Report the 284 tCO₂e as a one-time search cost with its amortisation basis stated — number of downstream reuses and the resulting per-use cost — not as a standalone shock figure. The honest framing is “a NAS search cost 284 tonnes once and has been reused across many models since,” not “training this model emitted 284 tonnes.”
Inference emissions: per-query, built from primitives
Inference has no dedicated reference figure in the way training does — published per-run training studies exist, but per-query inference energy is model- and deployment-specific. The methodology therefore builds inference from the same energy-attribution primitives as any cloud workload: accelerator power from utilisation, scaled by the workload’s share of the node, by facility PUE, by grid intensity. The chain is identical to the one in the cloud compute allocation methodology; what differs is the functional unit — the query.
1 · Node power
Server draw
idle + util × (peak − idle)
2 · Share
Per query
node share × hold time
3 · Overhead
PUE
facility ÷ IT
4 · Carbon
Grid intensity
kg CO₂e/kWh
5 · Volume
Query count
→ service total
Where a published per-query energy figure exists for the specific model — measured in watt-hours per query or per thousand tokens — it can be cited as the energy term directly, replacing the node-power reconstruction. Either way the per-query figure is multiplied by the service’s query volume to reach the reported total. The volume multiplier is what makes inference dominate: a per-query cost of a few milligrams becomes tonnes per year at billion-query scale, and tens of tonnes for a heavily used service. Worked Example B builds this from primitives; Worked Example C shows the crossover against training.
There is no digital.ai.inference.* family. Inference is constructed from the energy-attribution primitives plus a cited per-query energy figure for the specific model; no turnkey inference coefficient exists.
Embodied accelerator carbon
The training and inference figures above are operational only — energy consumed in use. They exclude the embodied manufacturing carbon of the accelerators themselves: the GPUs and TPUs whose fabrication is among the most carbon-intensive manufacturing in the supply chain. For a full lifecycle figure this embodied carbon should be amortised over the accelerator’s useful life and added, exactly as server and facility embodied carbon is added at the data-centre level.
There is no GPU, TPU, or accelerator embodied-carbon row anywhere in the digital.* or materials.* keyspace, and no data-centre server embodied row either — only end-user-device embodied carbon exists. Accelerator embodied carbon must be sourced externally from manufacturer disclosures or an LCA database and clearly labelled as such; no value is fabricated here. The Patterson reference figures are explicitly operational, so a lifecycle AI footprint requires this term to be added separately.
Worked examples
Three audit-record snapshots. All figures are fixed — worked examples reconcile to their stated inputs regardless of future data updates. Net training values come from the live rows; gross/energy/CFE are the carried Patterson metadata; inference is built from primitives.
GShard: 24.1 MWh of energy, reported net of 73% carbon-free-energy matching at 4.3 tCO₂e (live). We show what the same energy would emit on a high-carbon grid with no CFE, to isolate the grid and CFE contributions.
The reported 4.3 tonnes reflects both a clean grid and applied CFE matching. The identical compute on a US-average grid with no matching would emit roughly 2.4× as much — the grid lever and the CFE instrument together account for the gap. Note we did not multiply by PUE: the energy figure already includes facility overhead.
A query that holds 4 of 64 vCPU on a modern node (idle 90 W, peak 450 W) at 50% utilisation for 0.8 seconds, on hyperscale infrastructure (PUE 1.2), US-average grid. Scaled to one billion queries a year.
Under 2 milligrams per query, yet 1.93 tonnes a year at a billion queries — and a billion is modest for a popular service. The per-query figure is trivial; the volume multiplier is everything.
Comparing GShard’s one-off training (4.3 tCO₂e, live) against the inference rate from Example B (1.9305 tCO₂e per billion queries) to find when cumulative inference overtakes training.
For a model serving a billion queries a year, cumulative inference passes the training cost in just over two years and keeps rising for the model’s deployed life. The dramatic training figure is the smaller number over a popular model’s lifetime. Any AI footprint that reports training alone is incomplete.
Scope placement and reporting
Where AI emissions land depends, as with all cloud, on who is reporting. The mapping mirrors the boundaries in the cloud allocation methodology.
| Activity | Model developer / operator | Customer using the model via API |
|---|---|---|
| Training compute | Scope 2 (own data centre) or Scope 3 Cat 1 (cloud-trained) | Scope 3 Cat 1 — embedded in the purchased service |
| Inference / serving compute | Scope 2 (own) or Scope 3 Cat 1 (cloud) | Scope 3 Cat 1 — purchased AI service |
| Accelerator embodied carbon | Scope 3 Cat 2 — capital goods | Scope 3 Cat 1 — embedded in the service |
A company training its own models in its own facilities reports the operational electricity as Scope 2 with the dual location-based and market-based figures; a company training on cloud, or consuming a third-party model through an API, reports it as Scope 3 Category 1 under the GHG Protocol Scope 3 Standard. The CFE reductions in the training figures are market-based instruments held by the operator, so a customer reconstructing a location-based estimate will arrive at a higher figure than the operator’s net number.
Edge cases and common errors
- Quoting the NAS 284 tonnes as a per-model training cost. It is a one-time architecture-search discovery cost amortised across all downstream reuses. Report it with its amortisation basis, not as the carbon of training one model.
- Applying a PUE multiplier to the Patterson figures. Datacentre overhead is already folded into those energy-to-carbon values. Multiplying again double-counts cooling.
- Treating all six training figures as CFE-reduced. Only GShard and Switch Transformer carry a gross-to-net CFE reduction. GPT-3 ran on Azure with gross = net = 552.1. State per model whether net differs from gross.
- Reporting training and ignoring inference. For a deployed model, cumulative inference overtakes training within a few years. A training-only figure understates lifetime carbon.
- Using a model’s net figure as its physical emission. Net figures reflect the operator’s market-based CFE instruments. A location-based estimate on the physical grid is higher, and both should be disclosed.
- Predicting carbon from parameter count. A 600B sparse model trained for far less carbon than a 175B dense one. Architecture, hardware, datacentre, and grid jointly determine the figure.
- Omitting accelerator embodied carbon silently. These figures are operational only; GPU/TPU embodied carbon is excluded and not in MasterBrain. Source it externally and disclose its absence rather than implying zero.
Governance and audit checklist
- Training run energy and grid documented. Each material training run records its energy (MWh), the data-centre grid intensity, and whether the reported figure is gross or net of CFE.
- PUE treatment stated once. The record states whether PUE is already embedded in the energy-to-carbon figure (Patterson-style) or applied separately — never both.
- NAS costs separated and amortised. Any architecture-search cost is recorded as a one-off, with the number of downstream reuses and resulting per-use amortisation, not folded into a single model’s training figure.
- Inference basis and volume documented. The per-query energy basis (reconstructed or cited) and the query volume used to scale it are both recorded; the lifetime training/inference split is shown.
- Location-based companion present. Net (market-based) AI figures are accompanied by an independently reconstructed location-based figure.
- Scope placement correct. Own-facility training is Scope 2; cloud or third-party-API AI is Scope 3 Category 1; accelerator embodied is Category 2 (operator) or Category 1 (customer).
- Embodied accelerator carbon disclosed or flagged. Operational-only figures are labelled as such, and accelerator embodied carbon is sourced externally or its absence disclosed.
- GWP basis stated. The figures are tagged AR6 100-year; any externally sourced term is reconciled to the same basis before summing.
Standards alignment
| Standard | Role for AI compute accounting |
|---|---|
| ML CO₂ Impact | The machine-learning emissions estimation method — the dedicated framework for estimating training and inference carbon from hardware, runtime, and grid intensity. |
| GHG Protocol Scope 3 Standard | Governs customer-side AI accounting — purchased AI services and cloud-trained models fall in Category 1; accelerator embodied in Category 2 for the operator. |
| GHG Protocol Scope 2 Guidance | Sets the dual location-based / market-based reporting that distinguishes a net (CFE-matched) training figure from its physical-grid equivalent. |
| Sustainable Web Design Model | Per-byte digital-service carbon estimation — a complementary proxy where AI is delivered through web serving and per-query telemetry is unavailable. |
| ISO/IEC 30134 | Defines PUE and the facility KPIs that govern the datacentre-overhead term in any AI energy-to-carbon calculation. |
Frequently asked questions
No — that figure is a neural architecture search, not a training run. The 284 tCO₂e from Strubell et al. 2019 is the cost of an automated search across thousands of candidate architectures to discover a good one. Patterson 2021 reframes it as a one-time discovery cost amortised across every model that later reuses the architecture. The Evolved Transformer it produced trains for about 3.2 tonnes per run, and that figure deliberately excludes the search. Quoting 284 tonnes as the cost of training a single model overstates a run by roughly two orders of magnitude.
Three reasons compound. GPT-3 is a 175-billion-parameter dense model that consumed 1,287 MWh — far more energy than the sparser models. It ran on V100 GPUs, an older and less efficient accelerator than the TPU v3s used for several of the others. And it ran on Microsoft Azure with no Google 24/7 carbon-free-energy attribution, so its gross and net figures are identical at 552.1 tonnes, whereas GShard and Switch Transformer carry CFE reductions that lower their reported numbers. Architecture, hardware, and grid all moved against GPT-3.
Over a deployed model’s lifetime, inference usually dominates. Training is a single front-loaded event; inference is a tiny per-query cost that recurs across every request. In the worked example, a model serving a billion queries a year passes its training cost in cumulative inference within about two years, then keeps climbing. A footprint that reports only the dramatic training number understates the deployed model’s true emissions. Both regimes must be accounted.
No. Datacentre overhead — the PUE multiplier — is already folded into the energy-to-carbon conversion in those figures. Applying PUE again double-counts cooling and power-conversion losses. This is different from building a footprint from raw IT energy, where PUE must be applied explicitly; the Patterson reference figures already include it.
Build it from the energy-attribution chain — accelerator power from utilisation, scaled by the query’s share of the node and its hold time, multiplied by PUE and grid intensity — then multiply the per-query figure by the service’s query volume. Where a published per-query energy figure exists for the specific model (watt-hours per query or per thousand tokens), cite it as the energy term directly. There is no turnkey inference factor; the per-query cost is model- and deployment-specific.
No — the training and inference figures are operational only, capturing energy consumed in use. The embodied manufacturing carbon of the accelerators is excluded and is not currently carried as a data value, so it must be sourced externally from manufacturer disclosures or an LCA database and added separately for a full lifecycle figure. Treat the operational number as a lower bound on lifecycle carbon, and disclose that the embodied term is excluded rather than implying it is zero.
It depends on who you are. If you train or serve models in your own data centres, the electricity is your Scope 2, reported with both location-based and market-based figures. If you train on cloud infrastructure or consume a third-party model through an API, it is Scope 3 Category 1 — a purchased service. Accelerator embodied carbon is Category 2 capital goods for the operator and Category 1 for the customer. A customer reconstructing a location-based estimate will arrive at a higher figure than an operator’s CFE-matched net number.
Related digital methodologies
This is one of GreenCalculus’s eight interlocking digital-emissions methodologies, all sharing the same grid-intensity and PUE factor lineage so their estimates reconcile. See also the Bitcoin emissions, the cloud compute allocation, the data-centre PUE, the end-user devices, the IT asset & e-waste lifecycle, the video streaming and the website digital carbon methodologies.