Tested 2026-10-15
Short product descriptions for a small online shop: Meta: Llama 3.1 8B Instruct won, 3 of 7 models failed a must-have; OpenRouter's own picker came 4th of 4
Write short product descriptions for a candle shop — for a one-person e-commerce shop.
What this project actually needed
- takes written text as input — must have
- produces written text — must have
- names the scent in every description — must have: scent named in every description
- cost at your volume — decides the order
The winner
Meta: Llama 3.1 8B Instruct passed every must-have and ranked first by how this job said to choose.
OpenRouter's automatic picker sent this job to deepseek/deepseek-v4-flash-0731, z-ai/glm-5.3-flash and came 4th of 4. It picks one message at a time and never sees this job's must-haves.
Who failed, in their own words
Mistral: Mistral Nemo
- facts match the task:
6 oz, 40-hour burn time
inclusionAI: Ling 3.0 Flash VL
- gives an answer:
no answer came back after three tries (HTTP 429: b'{"error":{"message":"Provider returned error","c)
Sao10K: Llama 3 8B Lunaris
- names the scent in every description:
smoky sweetness of vanilla
- facts match the task:
Burns for 60+ hours
Every model on every requirement
| Model | cost at your volume | produces written text | takes written text as input | names the scent in every description |
|---|---|---|---|---|
| Auto Router | $0.0045 a day (measured on your tasks) | not applicable — the picker chooses a model per request | not applicable — the picker chooses a model per request | passed 3 of 3 tasks |
| Mistral: Mistral Nemo | $0.0000 a day (measured on your tasks) | yes (estimate — not observed) | yes (estimate — not observed) | passed 3 of 3 tasks |
| Sao10K: Llama 3 8B Lunaris | $0.0000 a day (measured on your tasks) | yes (estimate — not observed) | yes (estimate — not observed) | passed 2 of 3 tasks |
| Meta: Llama 3.1 8B Instruct | $0.0000 a day (measured on your tasks) | yes (estimate — not observed) | yes (estimate — not observed) | passed 3 of 3 tasks |
| inclusionAI: Ling 3.0 Flash | $0.0006 a day (measured on your tasks) | yes (estimate — not observed) | yes (estimate — not observed) | passed 2 of 2 tasks |
| inclusionAI: Ling 3.1 Flash | $0.0030 a day (measured on your tasks) | yes (estimate — not observed) | yes (estimate — not observed) | passed 3 of 3 tasks |
| inclusionAI: Ling 3.0 Flash VL | not measured | yes (estimate — not observed) | yes (estimate — not observed) | not judged — no graded answers |
7 models tested on candle product descriptions; cost of this test $0.0356.
Your app does something like this? XF Model Watch keeps it on the best model. XF Model Watch re-tests your own job as new models arrive and emails you only when one passes every check. Watch this job