IBM open-weight claim for AI budgets: the call

The call

On IBM’s third-quarter call, Arvind Krishna said open-weight models deliver “anywhere from 25 percent to 10 percent of the token cost of the frontier models” and are “now 90 percent as effective”, and asked: “Would you like something that’s 10 percent of the cost if it’s 90 percent as good as the best, or do you want to pay that 10 times higher cost?” (vendor, via The Register, 22 Oct, 04:26 UTC). The line reached the Hacker News front page today at 206 points (independent, Zeli).

What it is, and what it is not

What it is. A CEO with an on-prem hardware business saying most enterprise work, such as summarising email, reviewing contracts, and answering customer calls, runs well enough on cheaper models, and that most businesses can build useful AI infrastructure in two or three racks (vendor, via The Register). It lands as closed prices fall too: OpenAI cut GPT-5.6 Luna by 80% in July, and Max Woolf argues open-weight models now cap what closed vendors can charge (independent, minimaxir.com).

What it is not. It is not a benchmark. Krishna named no models and no test. On his own call he put the late-2025 gap at 70 against 50, which is 71%, not 90% (our maths). It is not a price list either: on-prem cost is racks and people, not tokens. Krishna’s own words on the constraint: “Where are my kernel engineers?” Governance? Unaddressed.

The number nobody is quoting

“A tenth of the cost” is measured against the frontier. Against the cheapest closed tier, the popular open-weight API is dearer. And once you count wrong answers, the frontier model wins whenever a failed call costs more than about 14 cents to fix.

Index points are not failure rates, so read the break-even as a shape, not a quote. The shape holds: if a person has to fix a wrong answer, the token saving disappears at a few cents per fix. Cheap models win where a miss is cheap and caught automatically.

Missing

What IBM has not told you, as of 26 October:

• The benchmark, the models, and the date behind “90 percent as effective”.
• Whether 90% means task success on enterprise work or a ratio of scores.
• The all-in cost of two or three racks: hardware, power, and the kernel, networking, and GPU staff Krishna says are scarce.
• Licence terms per model; open weights do not mean one licence.
• Measured throughput for the on-prem boxes now being sold; the Lenovo figures are estimates, and Lenovo has published no measured number.
• What happens to the ratio when closed prices fall again; Luna fell 80% in one cut.

Who owns what

Token price is their job. The cost of a wrong answer is ours.

In a Salesforce org, put the model behind an Apex callout through a Named Credential, whether it points at a hosted API or your own inference endpoint, and write the model’s answer and the agent’s correction to the case. That record is what turns “90% as good” into a number you own. Start with work where a miss is caught: case classification checked against the final queue, or summaries a service agent edits anyway.

Commercial decision: on-prem buys control of the data path. It does not buy a lower bill until the racks and the staff are counted.

Monday morning

1. Step 1. Pick one task with 1,000 or more calls a month and a checkable answer. Owner: Salesforce service lead.
2. Step 2. Price a miss. Minutes a person spends fixing one wrong answer, at your loaded rate. Owner: operations lead.
3. Step 3. Run three models in shadow on the same cases: your current model, the cheapest closed tier, and one open-weight model. Owner: platform engineer.
4. Step 4. Report cost per correct answer, tokens plus fixes, not cost per token. Owner: AI budget owner.
5. Step 5. Price on-prem only after Step 4, with hardware, power, and staff in the quote. Owner: CIO.

We move this to WATCH if IBM names the benchmark and it turns out to be narrow, or if closed small-model prices fall below the open-weight hosts again this quarter.