Tested daily

Which model did each job actually need?

Every day we take a realistic job, work out what it really requires, and test the shortlist — plus brand-new models and OpenRouter's own automatic picker — on the same tasks. Failures are quoted word for word.

Short product descriptions for a small online shop: Meta: Llama 3.1 8B Instruct won, 2 of 8 models failed a must-have; OpenRouter's own picker came 4th of 6
2026-10-16
Sorting support tickets into categories as JSON: Mistral: Mistral Nemo won, 2 of 6 models failed a must-have; OpenRouter's own picker passed every must-have but cost 51× the winner (4th of 4)
2026-10-15
Replies to customer support emails: Qwen: Qwen3.7 Max won, 6 of 7 models failed a must-have
2026-10-15
Sorting support tickets into categories as JSON: Mistral: Mistral Nemo won, 2 of 6 models failed a must-have; OpenRouter's own picker passed every must-have but cost 72× the winner (4th of 4)
2026-10-15
Replying to customer reviews: Mistral: Mistral Nemo won, 3 of 7 models failed a must-have; OpenRouter's own picker passed every must-have but cost 44× the winner (4th of 4)
2026-10-15
Translating product listings into Spanish: Auto Router won, 0 of 7 models failed a must-have
2026-10-15
Scoring inbound leads from a contact form: inclusionAI: Ling 3.0 Flash won, 3 of 7 models failed a must-have; OpenRouter's own picker passed every must-have but cost 8× the winner (4th of 4)
2026-10-15
Pulling fields out of invoice text into JSON: inclusionAI: Ling 3.0 Flash won, 2 of 6 models failed a must-have; OpenRouter's own picker passed every must-have but cost 4× the winner (4th of 4)
2026-10-15
Short product descriptions for a small online shop: Meta: Llama 3.1 8B Instruct won, 3 of 7 models failed a must-have; OpenRouter's own picker came 4th of 4
2026-10-15

Find the model your project really needs — free