You did not buy the cheaper one. You bought a different system.

The business question

Whoever decides on price is not wrong about the price. They are wrong about what the price is measuring.

Two proposals for the same workload. Same accelerator, same count, same capacity on paper. One is considerably cheaper, and the only word available to explain the difference to the board is “price”.

The answer, before the argument. With a memory-bandwidth-bound workload, the system specified at the floor delivers 69% of the throughput of the other. To do the same work you need 1.45 of it. So the cheaper proposal has to be 31% cheaper just to break even, and that is before electricity, drive replacement and the upgrade that becomes a repurchase. If capacity is reached by populating half the channels, the discount required rises to 62%.

That number rests on premises, and all of them are declared and replaceable by yours in the final section. What sits between here and there are ten engineering decisions nobody writes in the quote, and which decide for how many years the system stays the system that was bought.

There are two ways to arrive at a system proposal. The first starts from the catalogue: take what is available, what is compatible, what closes the price, and assemble it. The second starts from the application: measure the workload, declare the horizon, and choose every part to survive it. Both deliver something that powers on the first day. Only one of them answers for how long.

The first is cheaper, and it has to be. It is selling less system.

And what is at stake here is a single vertex. The ten decisions in this text live entirely inside compute: the node that does the arithmetic. An accelerated computing system also has the power that feeds it, the thermal circuit that keeps it inside its envelope, the network that joins the nodes to one another, the storage that supplies them, and the operating system and application layer that decides whether any of the rest becomes science. Each of those carries its own list of decisions nobody writes down. If one vertex hides ten, the question worth asking is how many the other five hide.

Frame · the ten decisions and what each one shortens

#	Decision	Measured in	What you give up
1	Assumed horizon	years on the schedule, work per watt in the field	A five-year schedule against an 18-to-36-month refresh cycle, and nearly 5× the electricity for the same work
2	Processor generation	memory channels × DDR5 rate	461 against 614 GB/s per socket, between two consecutive generations
3	Single-socket SKU	vendor suffix (P, U)	Expansion. The documented path becomes replacement
4	Modules per channel	DPC and the MT/s derate	31% of the bandwidth at 2DPC, or 62% if half the channels sit empty
5	Endurance class	DWPD and TBW	One replacement inside the schedule: 3.2 years against 5
6	Power supply class	certified efficiency by load point	Around US$ 22 thousand of electricity per rack over five years
7	Thermal envelope and chassis	°C at the inlet, airflow through the board	The warranty, when telemetry shows the envelope exceeded
8	Lane budget	lanes, GB/s, ns of latency	32 lanes for all networking and all storage
9	Link generation, at both ends	PCIe generation of the slot and of the device	Half the bandwidth at the same lane width
10	Architectural openness	conformance to a published definition	The market price of every part added after year one

Arraste para o lado para ver a tabela inteira.

What the frame hides. The ten are not independent. Decisions 2, 8 and 9 are the same purchase seen three times. Decision 10 multiplies whatever all the others cost to fix later. Adding the ten naively overstates the case; the final section shows how to add them.

The discipline behind this

Nothing here is invention and nothing is taste. The ten decisions belong to an engineering discipline with its own standards, method and arithmetic, practised wherever a system has to be available and not merely delivered.

It has a name, RAMS, for reliability, availability, maintainability and safety. It has a method for resolving a composite system into paths, the reliability block diagram, which lays the system out in elements in parallel and in series: redundant power supplies in parallel, the fabric in series, the storage layer in series. Series is the unforgiving arrangement, because a chain in series is only as available as its weakest link, and the weakest link is rarely the component that appears in the proposal.

And it has an equation that decides more than people expect:

Availability  =  MTBF / (MTBF + MTTR)

  MTBF   mean time between failures.   A property of the hardware.
  MTTR   mean time to repair.          A property of the contract.

The second term is the one that usually goes unspecified, and it is the one that dominates. With a system MTBF of 67 hours, which is an input measured on your own fleet and not a number published here:

  MTTR  4 h   on-site field service     A = 94.4%     494 h/year   ≈ 21 days
  MTTR  8 h   next business day         A = 89.3%     934 h/year   ≈ 39 days

Same hardware, same failure rate, same everything the quote describes. Eighteen days of production a year separate the two, and the difference was never an equipment specification.

It is the lesson running underneath the ten decisions that follow: a high MTBF does not rescue a poor MTTR, and availability is not a component you can buy. It is a property of the whole chain, calculated rather than assumed.

1 · Two clocks, and the quote names neither

Every proposal assumes a number of years. It is almost never written down, and it is almost always three to five. That single number is doing the work of two, and they are not the same number.

The first clock is the depreciation schedule. It is an accounting instrument. It spreads the purchase price across a number of years and reduces taxable income in each of them, which is to say it is a claim against tax. US tax practice classifies computers, servers and network equipment as five-year property. A federal agency’s internal property policy uses seven for systems above a value threshold. Both are institutional sources of equal weight and they disagree, because they describe different universes. Neither measures how long the hardware works, and that is not a defect in them: measuring hardware is not what a depreciation schedule is for.

The second clock is the service life. It is an engineering and operating fact: how long the system goes on producing useful work at a cost that still justifies running it. Physics governs part of it, through endurance and the thermal envelope, and economics governs the rest, principally the energy it takes to buy a unit of work.

Confusing the two is where the money goes. The schedule tells you how you recover the cost of the asset on paper and against tax. The service life tells you how you recover the investment in fact, in work produced. The two are not in competition and the point is not to choose between them. The point is to buy a system whose service life is at least as long as the schedule you are writing it off over, and then to run it that long. That is what a well-made investment in a computing estate looks like: the tax benefit taken in full, and the system still earning while you take it.

The failure mode is the mismatch. A system that goes economically obsolete years before its schedule closes leaves finance still deducting depreciation on equipment whose power bill already costs more than the work it returns. The asset is on the books; the value is not in the room.

And the horizon moves, in public, with a figure attached. A hyperscale operator states in its annual filing that it extended the useful life of its servers from five to six years in January 2024, reducing depreciation expense by US$ 3.2 billion and adding US$ 2.5 billion to net income for the year. Twelve months later, the same document records the reversal: a useful-life study completed in the fourth quarter of 2024 and the life of a subset of servers cut from six years back to five, at an expected cost of US$ 0.7 billion of operating income. In the same quarter it decided to retire servers early and recorded US$ 920 million of accelerated depreciation.

The document gives the reason in one sentence:

These two changes above are due to an increased pace of technology development, particularly in the area of artificial intelligence and machine learning.

Read it as the two clocks: the accounting clock was reset to chase the engineering one, and then reset again when the engineering one moved faster than expected. The operator with more visibility into hardware ageing than any buyer of a cluster extended the horizon, booked two and a half billion in profit, and within a year gave part of it back. The horizon is not a fact about the equipment. It is an estimate, it gets revised, and the revision has a price.

The service-life clock is the one that actually retires installations. A system does not stop working; it stops being worth running. The measure is work per unit of energy, and it moves fast enough to show up in a public ranking: the Green500 leader went from 15.1 GFLOPS per watt in June 2019 to 73.3 in June 2026.

The same work, on two efficiency frontiers

  system on the 2019 frontier   needs  ≈ 4.8 ×  the electricity
  system on the 2026 frontier   needs      1 ×

  The monthly power bill does not fall as the system ages.
  The work it buys does.

That is why working HPC installations get retired. The hardware did not fail. The same power bill started buying a fraction of what the same budget buys on a current system, and at some point the difference finances the replacement on its own. A depreciation schedule cannot see this, because it measures the asset and not the electricity.

In practice: ask every quote to declare the horizon and say where the number came from. A system sized against a five-year schedule and a workload on a three-year cycle are two different purchases wearing the same cover.

2 · The generation you inherit

Server generation is not a marketing label. It is a fixed count of memory channels, a ceiling on DDR5 rate and a lane budget, and all three are published per generation.

Within a current family, the previous generation offers twelve DDR5 channels at 4800 MT/s and the current one keeps twelve channels while raising the rate to 6400, with the same PCIe 5.0 lane count. Two generations back it was eight DDR4 channels. Channels and rate are the two numbers that fix aggregate bandwidth, and neither of them changes after the purchase.

In practice: a previous-generation socket is not a discount on the CPU. It is a permanent ceiling on memory bandwidth for every workload the system will ever run, and memory-bound inference feels it first.

3 · One socket, and the expansion that is not one

Both current families publish an official SKU designation that restricts the processor to single-socket configuration. One uses the P suffix, the other U, documented by the vendor as supported in one-socket configuration only.

The naming is not perfectly consistent: the vendor itself notes that some workload-optimised SKUs are also single-socket without carrying the suffix. The part number decides, not the letter.

The documented path from one socket to two is removal and replacement of the processor with a matched pair, not the addition of a second unit.

In practice: a single-socket SKU converts a future expansion into a future replacement. It is a legitimate choice and often the right one. It is not legitimate silence in a quote.

4 · Capacity is not bandwidth

Two systems can carry the same total memory and deliver different bandwidth, because bandwidth comes from channels and rate, not from gigabytes.

A memory channel is a path between processor and memory, and the board offers a fixed number of them. Each channel has one or two sockets. Filling both is called 2DPC, two DIMMs per channel; filling one is 1DPC. Reaching capacity at 2DPC forces the platform to derate the transfer rate, measured in MT/s, megatransfers per second.

On a twelve-channel board with twenty-four sockets, with aggregate bandwidth calculated as channels × rate × eight bytes:

Configuration                           Capacity     Aggregate bandwidth

  12 × 64 GB    1DPC   6400 MT/s          768 GB        614 GB/s
  24 × 32 GB    2DPC   4400 MT/s          768 GB        422 GB/s     69%
   6 × 128 GB   1DPC   4800 MT/s          768 GB        230 GB/s     38%
  (half the channels populated)

The third is the one that hurts, and it is the one a capacity-driven quote runs toward, because larger modules in smaller numbers assemble more cheaply. Under 40% of the bandwidth, for the life of the system, and the quote says 768 GB in all three.

In practice: compare quotes by populated channels and sustained rate, never by installed gigabytes. Two configurations claiming the same capacity can differ by a factor of two and a half.

5 · Endurance, or the drive that closes before the schedule does

NVMe is the interface through which storage talks to the processor, and it says nothing about how long the flash lasts. That is a separate and normative specification. DWPD, drive writes per day, is how many times the entire drive can be overwritten per day for the duration of the warranty. TBW, terabytes written, is the same fact expressed as a total. Both are defined under a JEDEC standard.

The reference workloads differ by class, and the difference is the point. The client class is measured at 40 °C, eight hours a day. The enterprise class, at 55 °C, twenty-four hours a day, with a mixed write distribution and a stricter error rate. A drive qualified against the first is not the same product as one qualified against the second, whatever the capacity label says.

A worked example, with the write volume declared as an input because no published number exists for it. A 3.84 TB drive and a checkpointing pipeline writing 6 TB per day:

Class      Guaranteed writes             Life at 6 TB/day     Against a 5-year schedule

  1 DWPD      7,008 TB  (1×3.84×365×5)     1,168 d  ≈ 3.2 y    one replacement inside it
  3 DWPD     21,024 TB  (3×3.84×365×5)     3,504 d  ≈ 9.6 y    clears it by 4.6 y

Same capacity, same form factor, same interface. One of the two has to be bought twice inside the horizon the quote assumes, and the quote does not say which.

In practice: endurance is the one number in the quote that can expire before the depreciation schedule does. Ask for DWPD and the warranty term, and run the division above against your measured volume.

6 · The power supply that charges by the hour

Power supply efficiency is certified by 80 PLUS, which measures how much of the alternating current drawn from the wall reaches the equipment as usable direct current, and publishes the result as thresholds at fixed load points. The rest is heat.

At 50% load, where the curve peaks in every class, Gold requires 90% and Titanium requires 94%. At 20% load, 87 and 92. At full load, 87 and 90.

Two things follow. The gap between classes is not constant: four points at the peak and five points at 20%. And efficiency falls at both ends of the curve, so a power supply oversized for the node operates permanently on the inefficient side.

This talks to redundancy. N+1 adds one shared spare unit; 2N duplicates the entire supply, each side sized to carry everything on its own. Under 2N each unit operates at half the load it would carry under N+1, which is to say the redundancy that protects availability pushes the operating point toward where the gap between classes is widest.

The arithmetic is direct, and it is the same structure for any rack:

A rack of 75 kW IT load, power supplies at 50% load, five years
US average commercial tariff: 14.19 ¢/kWh (EIA, June 2026)

  Gold        90%     75 / 0.90  =  83.3 kW drawn from the wall
  Titanium    94%     75 / 0.94  =  79.8 kW drawn from the wall
                                     3.5 kW of permanent difference

  3.5 kW × 8,760 h × 5 years  =  155 thousand kWh
  × US$ 0.1419                =  US$ 22 thousand per rack

In practice: power supply class is not a line item, it is a tariff. Substitute your own load and your own tariff; the structure of the arithmetic does not change, and its sign never changes.

7 · The thermal envelope, measured at the inlet, and the chassis that does not cool

The envelope is standardised. ASHRAE, the engineering body whose thermal guidelines govern data centre design, defines a recommended inlet range of 18 to 27 °C for all classes, with allowable ranges that widen by class up to 45 °C. The measurement point is the equipment inlet, not the room average, and that distinction settles arguments.

There is a second thermal decision, and it is about the box, not the room. Data centre accelerators are usually passive: the board carries a heatsink and no fan of its own, because it was designed for a chassis that pushes air through those fins at high static pressure. On these boards cooling is a property of the system, and the board has no way to compensate for a chassis that does not deliver it.

The requirement is published and it is not modest. A data centre design guide specifies a minimum of 157 cubic feet per minute of airflow per kilowatt of load, and the same vendor’s eight-accelerator system calls for 1,105 CFM front to back with the fans at eighty percent.

The board’s limits are published too: thermal qualification of 87 °C on the processor and 95 °C on the memory stacks, with hardware clock reduction triggered two degrees below the limit and shutdown at five. The vendor’s management documentation names both mechanisms and describes the hardware one as reducing clocks “by a factor of 2 or more”.

A chassis not designed to deliver that airflow moves air over the board rather than through it. The board throttles first, which looks like a performance problem and is the protection working, and degrades later, which looks like a defect.

In practice: two questions, not one. At what inlet temperature was the quoted density validated, and by whom. And whether the chassis was designed to cool passive boards at that power, with an airflow and static pressure figure, not with a verbal assurance.

8 · The lanes that run out

A two-socket platform on the current generation offers up to 160 PCIe 5.0 lanes. Eight accelerators at sixteen lanes each consume 128. That leaves 32 for everything else: all networking, all NVMe storage, all management.

A 400 Gb/s adapter takes sixteen. Each NVMe drive takes four. The budget closes before an enterprise networking and storage configuration does.

When it closes, the answer is a PCIe switch: silicon that multiplies downstream ports from a fixed number of host lanes, with pass-through latency below 115 nanoseconds on a current part. The cost is not uniform: the vendor’s own measurements show latency varying with packet size and with the mix of port widths, from roughly 138 to 314 nanoseconds. Negligible in large sequential transfer, material in small frequent transactions, which is exactly the pattern of collective communication.

In practice: a switch is not a defect, it is an engineering answer with a declared cost. The defect is the quote that does not say it is there.

9 · The generation of the slot is not the generation of the part

Two systems can both say PCIe 5.0 and neither be wrong, because the phrase describes the slot and says nothing about what is plugged into it. The rates are published by the standards body: 8 GT/s on generation 3, 16 on 4, 32 on 5, 64 on 6, a doubling at every step.

Platform below the accelerator. A previous-generation processor brings a previous-generation root complex with it, and the vendor documents the consequence without editorialising: a current accelerator’s product brief lists supported interface modes as Gen5 x16, Gen5 x8 or Gen4 x16, three modes presented as equivalent. That is the vendor stating, in a specification table, that a Gen4 slot with sixteen lanes carries what eight Gen5 lanes carry. Half.

Part below the platform. The inverse costs less and therefore happens more: a Gen5 platform populated with Gen4 or Gen3 NVMe. The bus is current, the drives are not, and the storage layer runs at the generation of the drives. Capacity and count are identical across the three generations, so the quote reveals nothing.

In practice: slot generation, device generation and lane width are three separate numbers. Treat a quote that gives one of the three as having given none.

10 · The parts you are permitted to buy

The first nine are taken once and become physics. The tenth is taken once and governs every subsequent purchase, including the ones that would fix the other nine.

There is a formal open server architecture initiative, the Open Compute Project, a neutral foundation that publishes hardware designs and the principles a design has to meet to carry the name. Its definition of openness is explicit: the measure is the ability of a third party to build, modify or customise the device from the contribution. It is a testable property: either the platform meets a published definition, or it does not.

In practice: the other nine are priced at signature. This one prices everything that comes afterwards, because it decides whether a replacement drive, an added accelerator or a memory expansion comes at market price or at catalogue price.

The arithmetic that closes

Now the arithmetic promised in the opening, opened up, with every premise declared and replaceable by yours.

One rack, eight nodes, a five-year horizon, a memory-bandwidth-bound workload, which is the declared premise everything below depends on. Two specifications for the same work. System A is current generation, twelve channels at 1DPC and 6400, Gen5 slots, 3 DWPD drives, Titanium supplies and thermal headroom for the next generation. System B is previous generation, twelve channels at 2DPC and 4400, Gen4 slots, 1 DWPD drives, Gold supplies and no headroom.

Memory throughput, channels × rate × 8 bytes

  A     12 channels × 6400        614 GB/s        100%
  B     12 channels × 4400        422 GB/s         69%

To deliver the same work as A, you need 1.45 of B.
So B has to be 31% cheaper JUST TO BREAK EVEN on throughput.

And after that it still loses:

  electricity       + US$ 22 thousand per rack over five years (Gold against Titanium)
  storage           + one drive replacement at year 3.2 (1 DWPD against 3)
  upgrade           the year-three upgrade becomes a repurchase, for lack of thermal headroom
  expansion         at catalogue price, if the architecture is closed

Thirty-one percent before anything else. And the number worsens quickly: if capacity is reached by populating half the channels with larger modules, throughput falls to 38% and the discount required to break even rises to 62%.

Two honest caveats. The premise that the workload is memory-bandwidth-bound is declared, not measured, and in a compute-bound scenario the effect is smaller. And the ten decisions do not add up simply: 2, 8 and 9 are the same purchase seen three times, so counting them separately triples a single difference, while 10 multiplies whatever the others cost to fix later.

What the arithmetic does is not to give you your number. It is to say which question to ask your two quotes: is the cheaper one cheap enough? Most of the time the answer is arithmetic, and it takes an afternoon.

Common specification mistakes

1 · Sizing the enclosure for today’s board. Power and thermal envelopes have grown with every accelerator generation. A chassis without headroom converts the next upgrade into the next purchase, and the saving at signature is the smaller of the two numbers.

2 · Comparing installed capacity instead of populated channels. Two systems with the same total memory can differ by a factor of two and a half. The gigabyte is the number that appears in the quote and the one that decides least.

3 · Reading a single-socket SKU as a discount. It is closed expansion. The documented path is processor replacement, and that cost appears in year three, not at signature.

4 · Sizing NVMe by capacity and ignoring endurance class. A read-intensive drive under checkpoint load exhausts its warranty before the accounting schedule closes. The conversion is arithmetic and takes one line.

5 · Assuming the parts you buy today are the parts you will buy in year three. Openness has a published definition and an auditable conformance programme. Either the platform meets it or it does not, and the answer decides whether every later repair is priced by the market or by a catalogue.

6 · Accepting an accelerator count without a lane budget or a generation. Eight accelerators leave 32 lanes for all networking and all storage, and a previous-generation slot halves what each lane carries.

7 · Putting a passive board in a chassis that was not built to cool it. The board has no fan. If the chassis does not force air through the fins at the required static pressure, the board throttles, then degrades, and the failure is thermal rather than a manufacturing defect.

What is not yet known

This entry declares its gaps rather than filling them, and there are five.

We did not establish a published figure for the daily write volume of a training workload. The closest is a production study of more than eighty-five thousand checkpoints, which offers a sizing formula rather than a number: write rate equals parameter count times fourteen bytes, divided by the checkpoint interval. It is a method you run, not a figure anyone publishes.

We did not establish a quantified reduction in service life from operating outside the thermal envelope. The throttling mechanism is documented, as cited above. What it costs the asset over years is not.

We did not establish institutional documentation of expansion-part restriction by licence or firmware, and we have marked what we have as observation rather than evidence.

We did not establish a total cost of ownership study comparing open and proprietary architecture with a declared methodology.

We did not establish the acquisition cost ratio between classes for any of the ten decisions, from any analyst, standards body or vendor. That is why the fifth column of the frame is in GB/s, years and kilowatt-hours rather than in money, and why the arithmetic that closes this text uses your price difference as an input, and not ours.

A close

The cheapest proposal is not the one that costs least. It is the one whose costs are all visible at signature.

Every decision in this text is legitimate. There are workloads for which a previous-generation socket, a single-socket SKU and Gold supplies are exactly right, and paying more would be waste. What separates a good quote from a cheap one is not which choices it makes. It is whether it says which choices it made, and for how many years it expects to be right.

A system assembled from the cheapest compatible parts can be delivered, and it will work on the first day. Whether it can be relied on for the horizon it was sold against is a different question, answered by a different discipline, and that answer is designed before it is assembled.