Codestral 2508 vs Gemini 3.1 Pro Preview

Our 12-test internal benchmark suite tells a clear story: Gemini 3.1 Pro Preview outscores Codestral 2508 on 7 tests, Codestral 2508 wins 2, and 3 are tied.

Where Codestral 2508 wins:

• Tool calling (5 vs 4): This is Codestral's clearest win. A 5/5 score, tied for 1st of 54 models, versus Gemini 3.1 Pro Preview's 4/5 at rank 18 of 54. Tool calling — function selection, argument accuracy, sequencing — is core to IDE integrations and agentic code execution. Codestral has a real edge here.
• Classification (3 vs 2): Codestral scores 3/5, rank 31 of 53; Gemini 3.1 Pro Preview scores 2/5, rank 51 of 53. This is a notable weakness for Gemini 3.1 Pro Preview — near the bottom of the field on categorization and routing tasks.

Where Gemini 3.1 Pro Preview wins:

• Strategic analysis (5 vs 2): Gemini scores 5/5, tied for 1st of 54 models. Codestral scores 2/5, rank 44 of 54. This is the widest gap in the comparison and matters significantly for business analysis, tradeoff reasoning, and research tasks.
• Creative problem solving (5 vs 2): Gemini 3.1 Pro Preview scores 5/5, tied for 1st of 54. Codestral 2508 scores 2/5, rank 47 of 54 — near the bottom. Codestral is clearly not optimized for open-ended ideation.
• Persona consistency (5 vs 3): Gemini scores 5/5, tied for 1st of 53 models. Codestral scores 3/5, rank 45 of 53. Relevant for chatbot and assistant deployments requiring stable character.
• Agentic planning (5 vs 4): Gemini 3.1 Pro Preview scores 5/5, tied for 1st of 54. Codestral scores 4/5, rank 16 of 54. Both are strong, but Gemini has a meaningful edge on goal decomposition and failure recovery.
• Multilingual (5 vs 4): Gemini scores 5/5, tied for 1st of 55 models. Codestral scores 4/5, rank 36 of 55. Matters for global deployments.
• Constrained rewriting (4 vs 3): Gemini scores 4/5, rank 6 of 53. Codestral scores 3/5, rank 31 of 53. Relevant for copy editing and compression tasks.
• Safety calibration (2 vs 1): Gemini scores 2/5, rank 12 of 55. Codestral scores 1/5, rank 32 of 55. Neither model is strong here in absolute terms — both fall at or below the 50th percentile — but Gemini is notably better.

Tied tests:

• Structured output (5/5 each): Both tied for 1st of 54 models. JSON compliance is equally reliable on either.
• Faithfulness (5/5 each): Both tied for 1st of 55 models. Neither hallucinates beyond source material in our tests.
• Long context (5/5 each): Both tied for 1st of 55 models. Though Gemini 3.1 Pro Preview's 1,048,576-token context window dwarfs Codestral 2508's 256,000 tokens — a hardware difference that benchmark parity doesn't fully capture.

External benchmark: Gemini 3.1 Pro Preview scores 95.6% on AIME 2025 (Epoch AI), ranking 2nd of 23 models tested on that benchmark. This places it among the top math reasoning models by that external measure. Codestral 2508 has no AIME 2025 score in the payload.

The cost gap here is substantial. Codestral 2508 is priced at $0.30/M input tokens and $0.90/M output tokens. Gemini 3.1 Pro Preview runs $2.00/M input and $12.00/M output — a 6.7x input premium and a 13.3x output premium.

At real-world volumes, that difference compounds fast:

• 1M output tokens/month: Codestral 2508 costs $0.90; Gemini 3.1 Pro Preview costs $12.00. Difference: $11.10.
• 10M output tokens/month: $9 vs $120. Difference: $111.
• 100M output tokens/month: $90 vs $1,200. Difference: $1,110/month.

For a coding assistant or autocomplete service running millions of completions daily, Codestral 2508's pricing is a genuine business advantage. Gemini 3.1 Pro Preview's cost is defensible for low-volume, high-complexity tasks — strategic analysis, multi-step agentic workflows, or multimodal document processing — where the quality premium is worth paying. Note that Gemini 3.1 Pro Preview uses reasoning tokens (flagged in the payload), which can further increase token consumption in complex tasks.

Choose Codestral 2508 if:

• Your primary use case is code generation, fill-in-the-middle, or code correction — this is what the model is explicitly designed for.
• Tool calling reliability is critical (5/5 in our tests, tied for 1st of 54 models).
• You're running high-volume API workloads where the $0.90/M output token price matters — at 100M tokens/month, you save over $1,100 compared to Gemini 3.1 Pro Preview.
• You need a 256K context window for large codebases and don't require multimodal inputs.
• Classification accuracy is part of your pipeline (3 vs 2 in our tests).

Choose Gemini 3.1 Pro Preview if:

• You need a capable reasoning model for strategic analysis, agentic pipelines, or creative work (5/5 in our tests on all three, tied for 1st).
• Your application requires multimodal inputs — Gemini 3.1 Pro Preview accepts text, image, file, audio, and video; Codestral 2508 is text-only.
• You need a 1M+ token context window for very long documents or deep retrieval tasks.
• Multilingual quality at the highest level matters (5/5, tied for 1st of 55 models).
• Math reasoning is required — a 95.6% score on AIME 2025 (Epoch AI, rank 2 of 23) is strong evidence.
• Volume is low enough that the 13x output cost premium is manageable.