Skip to content
CrucibleMark

Codestral 25.08

Trained for code tasks, not for general use: Codestral 25.08 is Mistral AI’s specialized developer model from August 2025, optimized for fill-in-the-middle and a broad range of programming languages. With 22 billion parameters, it supports 128,000 tokens of context, runs either locally or via the Mistral API, but is subject to a restrictive license with limited commercial use.

Mistral AI Version 25.08 Commercial use restricted Dense 22 B 128 K Context 07/2025 $0.2 / $0.6 per 1M

  • Restricted Weights
  • Frontier
  • Mistral AI
  • Text
  • Real-Time

Sovereign Risk: LOW Mistral AI is a French company headquartered in the EU and is not subject to any government access obligations regarding model weights such as the US CLOUD Act or China’s NSL.

Tool-use profile: 6 assets in detail

Comparison of asset performance (P1/P2/Combined) against the fleet average

Asset performance (radar)

Score Breakdown vs. Fleet Average

Tool-use details

Asset performance, reliability, and runtime profile

Name P1 P2 Combined
EU License Research 100 40 64
HTTP Fetch & Extract 80 80 80
Tool Failure Handling (404) 40 40 37
Web Search & Tool Selection 100 60 82
URL Construction & Fetch 80 80 80
Multilingual Search & Synthesis 100 40 70

CrucibleMark evaluates tool use across 6 independent tests. Click a test name for details.

Reliability

  • Tool Call Valid: No
  • Retry: Not required
  • Hallucination: Not detected

Reliability measures how consistently a model actually executes tool calls: Tool Call Valid schema and format accepted, Retry Required successful only after retry, Hallucination Flag fabricated tools or parameters detected. All three green means production-ready.

Runtime profile

Call 1
0.54
First Request
MCP
1.03
Protocol Latency
Synthesis
2.17
Response Generation
Total
22.45
Sum of All Phases
Token
9014
Input + Output
Cost
$0
Cost per Run

The runtime profile shows the latency and cost metrics for the model run: Call 1 First Request, MCP Protocol Latency, Synthesis Response Generation, Total sum of all phases, supplemented by Token input and output and Cost cost per run.

Tool-use review

Created

Deployment Verdict

Created on: 22.06.2026, 21:34:20

Conditional deploy, because while the model shows no hallucinations, it does not deliver consistently valid tool calling, and its overall suitability score of 68.83 is only moderate.

Tool Execution Profile

Codestral 25.08 often makes the right tool selection, but its execution is not stable enough for highly deterministic MCP pipelines. On the Web Search & Tool Selection test, which requires distinguishing between search and direct retrieval without an explicit hint, it reliably identifies the need for web_search. This argues against a purely rigid pattern and in favor of usable tool intelligence. On the URL Construction test, which requires deriving the target URL from internal knowledge and then fetching it correctly, it remains usable but not precise enough for pipelines that depend on reproducible fetch paths. The finding tool_call_valid=false is the operative core here: the model is not protocol-unsafe in the sense of being chaotic, but it does not consistently produce the kind of valid calls you would want to hand off unsupervised to critical automations. Retry was not required. This points to a precision problem in tool usage rather than a fundamental format failure.

Synthesis Fidelity

How well does it consolidate tool results? Only to a limited degree. The P2 score of 56.67 reveals a recurring problem in consolidation and the clean merging of research results. This is particularly visible in EU License Research and Multilingual Search & Synthesis: it retrieves information but does not condense it with the clarity and precision that production users need for reliable decision outputs.

Does it stay within the tool result or fall back on training data? In the honeypot EU License Research test, which checks whether current license restrictions are answered from web sources rather than training knowledge, it remains trustworthy. P2=40 is weak, but no hallucinations were detected. This matters: the model does not fabricate compliance facts here. It synthesizes poorly, but it does not break the source-to-answer chain.

Error Resilience

On the 404 test, which measures transparent behavior when a retrieval fails, Codestral 25.08 does not hallucinate substitute content. That is the central production point. However, the communication of the error is not strong enough to qualify as robust incident output. P2=40 means: acceptable for supervised pipelines, insufficient for autonomous error paths.

Sovereignty Profile

Locally deployable and therefore usable with full sovereignty. Combined 68.83 sits 0.90 points above the fleet average of 67.93. No sovereignty penalty detected.

Conclusion & Recommendation

Suitable for local, MCP-assisted coding and research pipelines with human review, especially where source fidelity matters more than elegant result synthesis. Not suitable for compliance-adjacent, fully autonomous, or highly deterministic tool chains where every call must be formally valid and every synthesis must be directly processable downstream. As an executing code model with web access it is usable. As a reliable endpoint in an unsupervised tool infrastructure, not yet.

This analysis was generated automatically based on benchmark data. Model used: GPT 4.5 by OpenAI. The raw data and full methodology are documented in the GitHub project.