Skip to content
CrucibleMark

Ministral 3 3B (Unsloth)

What most Nano models don’t offer: native multimodal input and tool calling right out of the box. Ministral 3 3B by Mistral AI delivers exactly that in the 3B class, with 256,000 tokens of context, an Apache 2.0 license, and local Unsloth GGUF distribution.

Mistral AI Version 3 Commercial use permitted Dense 3 B 256 K Context 07/2025 locally tested

  • Open Weights
  • Nano
  • llama.cpp
  • Text
  • Vision
  • Instruction-Tuned
  • Real-Time

Sovereign Risk: LOW Mistral AI releases Ministral 3 under Apache 2.0, and Unsloth provides the local GGUF packaging for offline inference. Since the upstream weights are open and the benchmark runs entirely locally, the operational provenance risk is low.

Tool-use profile: 6 assets in detail

Comparison of asset performance (P1/P2/Combined) against the fleet average

Asset performance (radar)

Score Breakdown vs. Fleet Average

Tool-use details

Asset performance, reliability, and runtime profile

Name P1 P2 Combined
EU License Research 100 15 49
HTTP Fetch & Extract 75 15 45
Tool Failure Handling (404) 75 100 83.8
Web Search & Tool Selection 100 35 70.8
URL Construction & Fetch 75 15 45
Multilingual Search & Synthesis 75 15 45

CrucibleMark evaluates tool use across 6 independent tests. Click a test name for details.

Reliability

  • Tool Call Valid: No
  • Retry: Not required
  • Hallucination: Detected

Reliability measures how consistently a model actually executes tool calls: Tool Call Valid schema and format accepted, Retry Required successful only after retry, Hallucination Flag fabricated tools or parameters detected. All three green means production-ready.

Runtime profile

Call 1
1.9
First Request
MCP
1.62
Protocol Latency
Synthesis
16.71
Response Generation
Total
121.33
Sum of All Phases
Token
12356
Input + Output
Cost
$0
Cost per Run

The runtime profile shows the latency and cost metrics for the model run: Call 1 First Request, MCP Protocol Latency, Synthesis Response Generation, Total sum of all phases, supplemented by Token input and output and Cost cost per run.

Tool-use review

· Instruction-Tuned

Deployment Verdict

Conditional deploy, because tool usage often appears functional, but the model hallucinates and fails to deliver consistently valid tool calls. For production MCP pipelines, a Combined Score of 56.42 is only sufficient under tight safeguards.

Tool Execution Profile

The model demonstrates genuine tool-selection competence, but no reliable protocol adherence. In the Web Search & Tool Selection test — which checks whether web_search is chosen over fetch without a hint — it correctly identifies the need and achieves P1 100. That argues against pure schema-following. In the URL Construction & Fetch test — which measures precise derivation of a target URL and the subsequent fetch — it falls back to P1 75. That is workable, but not deterministic enough for pipelines where a single wrong endpoint invalidates the run.

More critical is the execution form factor: the tool call was not valid, even though no retry was required. This looks less like a comprehension collapse and more like unreliable MCP output under load. For production orchestration, that means tool gating, call schema validation, and hard fail-closed are mandatory.

Synthesis Fidelity

How well does it condense tool results? Poorly. P2 32.50 is this model’s actual bottleneck. In HTTP Fetch & Extract, URL Construction & Fetch, and Multilingual Search & Synthesis — precisely where extracted web content must be merged with precision — condensation stalls at P2 15 in each case. The model can retrieve information, but it does not reliably transform it into dependable result texts.

Does it stay within the tool result, or does it fall back on training? No, and that is the central safety concern. In the Honeypot EU License Research test — which checks whether current license restrictions are answered from web sources rather than training knowledge — the model hallucinates at P2 15. This means the weakness is not merely stylistic but trust-critical: when a model in a tool pipeline presents fabricated facts as researched output, it undermines the control function of the entire infrastructure.

Error Resilience

Here the model behaves in a production-appropriate manner. In the Tool Failure Handling (404) test — which measures whether a failed tool call is handled transparently rather than with fabricated fallback content — it achieves P2 100. It communicates the failure openly and does not hallucinate page content. This kind of transparency is acceptable for production.

Sovereignty Profile

Locally deployable, Apache 2.0, open weights — and therefore operationally attractive for sensitive data flows. At 56.42, it sits 11.33 points below the fleet average of 67.75. The model is thus sovereignly deployable, but not fleet-competitive enough to serve as a general-purpose tool model without additional guardrails.

Conclusion & Recommendation

Suitable for local, cost-sensitive assistant pipelines with a clearly bounded tool scope, strong schema validation, and downstream result verification. Not suitable for compliance, research, or decision pipelines where tool results are passed on as reliable facts. Those who only need robust error communication and simple tool invocation can deploy it. Those who need verifiable synthesis should not place it as the final authority in the MCP chain.

This analysis was generated automatically based on benchmark data. Model used: GPT-5.4 by OpenAI. The raw data and full methodology are documented in the GitHub project.