Training vs. Inference Data Centers: Two Different Buildings

AI workloads have split into two distinct development formats with different power profiles, site criteria, and underwriting logic. Building the wrong one for your tenant costs 18-36 months.

The data center industry has been talking about the AI boom as if it is one thing. It is not. AI compute breaks into two structurally different workloads -- training and inference -- and those workloads require facilities that share a name but diverge materially in site criteria, electrical design, cooling architecture, location logic, and tenant economics.

Developers who treat a training cluster and an inference facility as variations on the same product type will build the wrong building. The gap between the two widens every quarter as frontier AI models grow more compute-intensive and as inference demand scales to consumer-level volumes.

The Workload Difference

Training workloads build AI models. They connect tens of thousands of GPUs in tightly synchronized clusters where each GPU communicates with the others continuously. The computational graph cannot be split across facilities -- an interrupted training run from a power outage or network partition is a major operational event requiring the job to restart, often at significant cost. Training is latency-insensitive between regions: these facilities can be placed anywhere land, power, water, and fiber conditions are favorable. What they cannot tolerate is unreliable power delivery or network interruptions within the cluster.

Inference workloads serve trained models to users and applications. Unlike training, inference tasks are independent of each other -- each user query is processed separately. The workload is highly parallelizable and can be distributed across multiple facilities. What inference cannot tolerate is latency: a chatbot serving a real-time user must respond in milliseconds, not seconds. That latency budget directly constrains geography.

McKinsey's analysis of AI infrastructure development puts training racks at 100-200 kW per rack and frontier training systems approaching 1 MW per rack. Inference racks run 30-150 kW in mainstream configurations, though leading-edge inference accelerators -- including Nvidia's next-generation platforms -- are reporting rack power budgets of 370 kW for inference-optimized configurations. The inference density premium is driven by the same silicon: frontier inference uses the same GPU platforms as training, just in a different operational mode.

By 2030, inference compute is projected to reach over 90 GW globally (CAGR of 35%), outpacing training compute growth. The inference buildout is coming. It will not look like Northern Virginia.

Site Criteria: Where They Diverge

Training: Site selection follows power availability. Training facilities are latency-insensitive, so developers chase the grid. Texas, the Midwest, the Mountain West, and the Nordics all qualify -- anywhere utility reserve margins are adequate, interconnection timelines are manageable, and land is available at scale.

A single large training campus might consume 500-1,000 MW at full build. That power requirement means the site needs either existing high-voltage transmission infrastructure nearby or a credible path to a new substation. Water availability for cooling is a secondary constraint that is growing in significance as liquid and hybrid cooling systems increase water withdrawal volumes.

Power delivery reliability is critical. Training jobs that fail due to power interruptions require expensive restarts. A facility targeting hyperscale training tenants needs redundancy architecture at the utility entry point and an on-site backup generation strategy capable of carrying the full compute load through the restart cycle.

Inference: Site selection follows population and latency budgets. Inference facilities for consumer-facing applications need to be within a defined network distance of the end user. Three rough tiers govern placement in 2026:

• 50-200ms budget: regional cloud campuses, suitable for batch summarization and offline AI tasks

• 20-50ms budget: metropolitan proximity, required for live chat, enterprise copilots, and most agent workloads

• Sub-20ms budget: edge proximity, required for ad bidding, live trading, and emerging autonomous systems

The latency requirement makes inference a metro siting problem. Available sites are smaller, more expensive, more constrained by zoning, and more contested than greenfield training locations. Power availability is still a constraint, but the power requirement per facility is smaller -- 20-100 MW per campus rather than 500 MW+ -- which means sites that would not qualify for hyperscale training are viable for inference.

Community opposition, noise ordinances, and urban zoning restrictions that barely register as constraints for remote training campuses become material entitlement risks for metro-area inference facilities.

Cooling Architecture

Training: Advanced liquid cooling is now standard. Training clusters at 100+ kW per rack cannot be air-cooled sustainably. Direct-to-chip cold plate and rear-door heat exchangers are the predominant approaches in production today. Full immersion is in deployment at some hyperscale facilities. The cooling infrastructure is part of the facility design from day one -- it cannot be retrofitted from air after the fact.

Training facilities also require power-quality management infrastructure that inference facilities do not. GPU training clusters produce sharp, dynamic load fluctuations: power draw can surge or decline 30-60% within milliseconds as training jobs move between compute phases. This requires oversized power delivery networks, harmonic filtering, and fast-response UPS systems. The electrical infrastructure for a training facility is more complex and more expensive per MW than a comparable inference facility.

Inference: Mainstream inference deployments at 30-80 kW per rack can run on hybrid cooling -- direct-to-chip for the AI accelerator rows, air cooling for networking, storage, and general compute. Leading-edge inference at 80-200+ kW per rack requires the same liquid cooling infrastructure as training.

The practical implication: a facility designed for mainstream inference can be air-cooled or lightly liquid-cooled with lower structural requirements, lower HVAC capital cost, and lower water use. A facility designed for frontier inference must be fully liquid-cooled -- the same design requirement as training, but in a metro location where land, structural costs, and permitting complexity are all higher.

Underwriting Logic

Training: The underwriting case is built on power delivery certainty and tenant credit. Hyperscale training tenants -- the major cloud providers -- sign long-term leases with significant take-or-pay provisions. The developer underwrites a single credit: can this tenant pay, and for how long? The lease structure is complex (power floors, ramp provisions, PUE caps, expansion options), but the revenue stream is predictable once a tenant is signed.

Spec training development -- building without a signed lease -- has become unusual above 50 MW because the capital required to finance a facility at that scale without tenant pre-commitment creates refinancing risk that most development capital structures cannot absorb.

Inference: The underwriting case is less proven. Inference facilities are often smaller and serve a more diverse tenant mix: enterprise AI deployments, mid-tier cloud providers, and colo operators who want GPU-enabled space. The lease terms are shorter and the tenant credit is more variable than hyperscale.

The geographic premium for metro inference sites -- proximity to population centers, lower latency to users -- means land costs are higher. The combination of higher land cost, shorter leases, and more complex entitlement processes creates a different risk profile than suburban or exurban training campuses. Investors underwriting inference development should model lease-up assumptions conservatively. Inference demand is real, but the supply of inference-capable metro capacity is also growing.

Two Practical Implications for Developers

The most frequent mistake is designing a facility for both workloads simultaneously without segmenting the electrical infrastructure. A data hall designed for 30 kW average density with provision for future liquid cooling "if needed" will not serve training tenants who require 100+ kW today and full liquid cooling infrastructure from the first day of occupancy. The mixed-design compromise serves neither workload well.

The better approach is to segment the facility at the data hall level: air-cooled halls for inference and general compute tenants, liquid-cooled halls for training and frontier inference. That segmentation requires a clear view of the tenant mix at the time of design, not at the time of leasing.

The second practical implication is site selection discipline. Training site selection is a power problem. Inference site selection is a location problem. The teams that confuse the two -- chasing remote greenfield power for a facility that needs metro proximity, or fighting for expensive urban infill for a facility that could be located anywhere -- will waste 18-36 months before correcting the mistake.

Oracle, Google, and Nvidia described at Data Center World 2026 the result of getting this right: a bimodal development portfolio where large, high-density training campuses are placed in power-advantaged remote locations, and inference clusters are co-located within existing metro campuses and purpose-built metro facilities close to population centers. That portfolio logic is available to institutional developers as well as hyperscalers. But it requires building the two facility types as distinct products, not as variations on the same building.