A 1996 Pentium Pro chess test: check software before GPUs

The thesis

Before you approve more compute, make someone show what better software would have delivered on the compute you own. The default assumption in most AI budgets runs the other way.

On 1 November Nicholas Carlini asked whether an off-the-shelf 1996 desktop with smarter software could have beaten Kasparov, without Deep Blue's custom hardware (independent, Carlini). He ran it on real hardware: a 200 MHz Pentium Pro with 64MB of RAM. Nobody played Kasparov. This is an estimate from engine matches.

The evidence

All Elo figures are Carlini's (independent). Now our maths, at 1 second a move:

• Same software, new hardware: 2614 − 1999 = +615.
• Same hardware, better software: 2720 − 1999 = +721.
• Same hardware, tuned Stockfish port: 2846 − 1999 = +847.

Carlini reports "+720 of the +1335 gained" for better programming. The first two lines add to within a point of that, so we read his split as the same comparison. That is our reading; he does not define the baseline. Software's share is 720 ÷ 1335 = 54%. The error bars are wide, so read it as "at least comparable", not "beats".

Carlini estimates about 3300 at classical time for the port, extrapolated from shorter games, against just under 2800 for Kasparov and 2700 for Deep Blue. In a 100-game match, the modern engine on the old machine scored 78 wins, 17 draws, and 5 losses against 1996 Crafty on modern hardware (independent).

One more line for engineering leaders. Carlini says he asked LLMs to write the code, naming Fable 5.1, Opus 5.5, and GPT-6. The cost of finding out what your software could do just fell. So the excuse for skipping the test is weaker than it was.

The strongest counterargument

Chess is the best possible case for this argument, and your workload is not chess.

• Chess has a perfect score. Run games, read the Elo, keep what wins. Most enterprise systems have no such number, so tuning can drift without anyone noticing.
• Stockfish is the product of decades of work by many people. Carlini ported that gain; he did not create it. Your codebase has no such store to draw on.
• Decades of software progress measured here are not a forecast. Carlini himself says the toy experiment "doesn't answer that question definitively".
• Sometimes the constraint is real. If a queue is full at peak, tuning next quarter does not help this quarter.

All of that is fair, and it narrows the claim. It does not remove it. We are not saying software beats hardware. We are saying the comparison is rarely run, and it should be.

What to do with it

1. Step 1. Require a baseline. No capacity request without a measured number on current hardware. Owner: head of engineering.
2. Step 2. Define the score. Latency, cost per task, or accuracy against a held-out test set. If you cannot define one, you cannot tell whether capacity worked either. Owner: product owner with engineering.
3. Step 3. Time-box the software pass. Give one engineer with coding agents a fixed window to try the obvious gains, then compare with the capacity quote. Owner: tech lead.

Hardware is the vendor's job to sell. Knowing what you already paid for is yours.

Missing

• Independent replication. This is one researcher's experiment, and we have not rerun it.
• A match against a human. The 3300 is an extrapolation across rating scales that Carlini says can diverge.
• The compute that went into tuning and evaluation. We could not establish how much modern hardware the work needed offline, and that bears directly on the thesis.
• Carlini's prose and tables differ slightly on upgraded Crafty (2718 against 2720). We use the table.
• Any enterprise measurement. Prestanda has no client figure to offer here, so this note carries none.