R1 vs Devstral Medium

Across our 12-test suite, R1 wins 7 benchmarks, Devstral Medium wins 1, and they tie on 4.

Where R1 dominates:

• Creative problem solving: R1 scores 5/5, tied for 1st with 7 other models out of 54 tested. Devstral Medium scores 2/5, ranking 47th of 54. This is a massive gap for any task requiring novel ideation or non-obvious solutions.
• Strategic analysis: R1 scores 5/5, tied for 1st with 25 others out of 54. Devstral Medium scores 2/5, ranking 44th of 54. Nuanced tradeoff reasoning — think business analysis, architecture decisions, risk assessment — strongly favors R1.
• Persona consistency: R1 scores 5/5, tied for 1st with 36 others out of 53. Devstral Medium scores 3/5, ranking 45th of 53. For chatbot or character applications, R1 holds character under pressure significantly better.
• Faithfulness: R1 scores 5/5, tied for 1st with 32 others out of 55. Devstral Medium scores 4/5, ranking 34th of 55. R1 sticks closer to source material — relevant for summarization and RAG pipelines.
• Multilingual: R1 scores 5/5, tied for 1st with 34 others out of 55. Devstral Medium scores 4/5, ranking 36th of 55. A smaller gap, but R1 edges ahead.
• Constrained rewriting: R1 scores 4/5 (rank 6 of 53 among 25 models at this score). Devstral Medium scores 3/5, ranking 31st of 53. R1 handles hard character limits and compression tasks more reliably.
• Tool calling: R1 scores 4/5, ranking 18th of 54. Devstral Medium scores 3/5, ranking 47th of 54 — near the bottom. For function selection and argument accuracy in agentic workflows, this is a meaningful disadvantage for Devstral Medium.

Where Devstral Medium wins:

• Classification: Devstral Medium scores 4/5, tied for 1st with 29 others out of 53. R1 scores 2/5, ranking 51st of 53. This is R1's clearest weakness and Devstral Medium's clearest strength. For routing, categorization, and labeling pipelines, Devstral Medium is the better choice.

Ties (both models perform equally):

• Structured output (4/4, both rank 26th of 54): JSON schema compliance is equivalent.
• Long context (4/4, both rank 38th of 55): Retrieval accuracy at 30K+ tokens is equivalent.
• Safety calibration (1/1, both rank 32nd of 55): Both models score at the bottom of our suite on this dimension — well below the median of 2 and the 75th percentile of 2.
• Agentic planning (4/4, both rank 16th of 54): Goal decomposition and failure recovery are equivalent.

External benchmarks (Epoch AI): R1 scores 93.1% on MATH Level 5 (rank 8 of 14 models tested) and 53.3% on AIME 2025 (rank 17 of 23). The AIME 2025 score sits below the median of 83.9% across models we have data for, while the MATH Level 5 score is close to the 94.15% median. No external benchmark scores are available for Devstral Medium in our data.

R1 costs $0.70/MTok input and $2.50/MTok output. Devstral Medium costs $0.40/MTok input and $2.00/MTok output — 43% cheaper on input and 20% cheaper on output. In practice: at 1M output tokens/month, you save $5 with Devstral Medium. At 10M output tokens, that gap grows to $50. At 100M output tokens — realistic for production agentic workloads — you save $500/month on output alone, plus $300 on input at comparable volumes. The savings are real but modest relative to the capability gap R1 holds on most benchmarks. Developers running high-volume classification pipelines are the clearest case for Devstral Medium on cost grounds, since that's the one benchmark where it beats R1. For most other use cases, R1's quality lead justifies the premium.

Choose R1 if: You need strong reasoning, creative problem solving, strategic analysis, or reliable tool calling. R1 wins 7 of 12 benchmarks in our testing and is the clear choice for agentic workflows, multilingual tasks, faithfulness-sensitive applications (RAG, summarization), and any use case where nuanced thinking matters. Budget-conscious teams should note the output cost is $2.50/MTok — higher than Devstral Medium's $2.00 — but the capability gap typically justifies it.

Choose Devstral Medium if: Classification is your primary workload. Devstral Medium scores 4/5 (tied for 1st among 53 models) on classification while R1 scores 2/5 (51st of 53) — a stark reversal of the usual dynamic. Devstral Medium is also the better pick when input cost sensitivity is high and the task set skews toward structured output or agentic planning, where both models tie. Its 131K context window vs R1's 64K context window may also matter for very long document tasks, though both score equivalently on our long-context benchmark.