Tested 2026-10-28
Writing spreadsheet formulas from plain English: Anthropic: Claude Opus 5.5 won, 4 of 5 models failed a must-have
Turn plain-English requests into a single working spreadsheet formula for the layout described — for a three-person bookkeeping firm.
What this project actually needed
- takes written text as input — must have
- produces written text — must have
- returns one formula that works on the described layout — must have: a correct formula on every task
- is good at coding — decides the order
- cost at your volume — decides the order
The winner
Anthropic: Claude Opus 5.5 passed every must-have and ranked first by how this job said to choose.
OpenRouter's automatic picker sent this job to deepseek/deepseek-v4.1-flash-1017 and was not ranked. It picks one message at a time and never sees this job's must-haves.
Who failed, in their own words
OpenAI: o4 Mini High
- gives an answer:
no answer came back after three tries (HTTP 404: b'{"error":{"message":"openai/o4-mini-high is no l)
Mistral: Mistral Nemo
- returns one formula that works on the described layout:
`=SUMIF(A2:A500, ">=10/1/2026", B2:B500)`
Auto Router
- returns one formula that works on the described layout:
=SUMPRODUCT((C2:C300="unpaid")/COUNTIFS(A2:A300,A2:A300,C2:C300,C2:C300))
Meta: Muse Wonder
- returns one formula that works on the described layout:
=IFERROR(COUNTA(UNIQUE(FILTER(A2:A300,(C2:C300="unpaid")*(A2:A300<>"")))),0)
Every model on every requirement
| Model | is good at coding | cost at your volume | produces written text | takes written text as input | returns one formula that works on the described layout |
|---|---|---|---|---|---|
| Auto Router | scored 0.83 of 1 across 3 tasks (graded) | $0.0062 a day (measured on your tasks) | not applicable — the picker chooses a model per request | not applicable — the picker chooses a model per request | passed 2 of 3 tasks |
| Meta: Muse Wonder | scored 0.77 of 1 across 3 tasks (graded) | $0.7438 a day (measured on your tasks) | yes (estimate — not observed) | yes (estimate — not observed) | passed 2 of 3 tasks |
| OpenAI: o4 Mini High | not judged — no graded answers | not measured | yes (estimate — not observed) | yes (estimate — not observed) | not judged — no graded answers |
| Mistral: Mistral Nemo | scored 0.30 of 1 across 3 tasks (graded) | $0.0001 a day (measured on your tasks) | yes (estimate — not observed) | yes (estimate — not observed) | passed 0 of 3 tasks |
| Anthropic: Claude Opus 5.5 | scored 1.00 of 1 across 3 tasks (graded) | $0.2725 a day (measured on your tasks) | yes (estimate — not observed) | yes (estimate — not observed) | passed 3 of 3 tasks |
5 models tested on spreadsheet formulas; cost of this test $0.1132.
Your app does something like this? XF Model Watch keeps it on the best model. XF Model Watch re-tests your own job as new models arrive and emails you only when one passes every check. Watch this job