Skip to content
CrucibleMark

Qwen 3.5 4B (Unsloth)

Qwen 3.5 4B is a compact Open Weights model from Alibaba featuring a Thinking-Optional architecture and Unsloth Dynamic quantization at Q6. The Q6 tier offers higher fidelity than the Q4 variant at a moderate VRAM overhead, with a context window of 128,000 tokens. Deployable locally on resource-constrained hardware under the Apache 2.0 license.

Alibaba Version 3.5 Commercial use permitted Dense 4 B (4 B active) 128 K Context 06/2025 locally tested

  • Open Weights
  • Nano
  • llama.cpp
  • Text
  • Instruction-Tuned
  • Interactive

Sovereign Risk: LOW Fully local inference without cloud connection. The weights are publicly available (Apache 2.0, Unsloth quantization) and run entirely locally. NSL is not relevant, as no data is transmitted to Alibaba or Unsloth infrastructure.

Tool-use profile: 6 assets in detail

Comparison of asset performance (P1/P2/Combined) against the fleet average

Asset performance (radar)

Score Breakdown vs. Fleet Average

Tool-use details

Asset performance, reliability, and runtime profile

Name P1 P2 Combined
EU License Research 95 60 74
HTTP Fetch & Extract 80 80 80
Tool Failure Handling (404) 80 80 80
Web Search & Tool Selection 95 80 88.3
URL Construction & Fetch 80 80 80
Multilingual Search & Synthesis 95 40 67.5

CrucibleMark evaluates tool use across 6 independent tests. Click a test name for details.

Reliability

  • Tool Call Valid: Yes
  • Retry: Required
  • Hallucination: Not detected

Reliability measures how consistently a model actually executes tool calls: Tool Call Valid schema and format accepted, Retry Required successful only after retry, Hallucination Flag fabricated tools or parameters detected. All three green means production-ready.

Runtime profile

Call 1
6.48
First Request
MCP
0.74
Protocol Latency
Synthesis
9.48
Response Generation
Total
100.26
Sum of All Phases
Token
10285
Input + Output
Cost
$0
Cost per Run

The runtime profile shows the latency and cost metrics for the model run: Call 1 First Request, MCP Protocol Latency, Synthesis Response Generation, Total sum of all phases, supplemented by Token input and output and Cost cost per run.

Tool-use review

· Instruction-Tuned

Deployment Verdict

Conditional deploy, because it reliably produces valid tool calls and does not hallucinate, but synthesis fidelity for production-critical summarization is still too uneven.

Tool Execution Profile

Tool execution is clearly usable for a Nano model. The model generates valid MCP-compliant calls and scores at a strong level on Tool Execution. Particularly notable: on the Web Search & Tool Selection test — which checks without explicit hints whether web_search rather than fetch is required — it identifies the correct tool type with high confidence. This argues against mere schema-following and in favor of genuine tool selection in context.

Less robust is its performance on the URL Construction test, which checks whether the model derives the target URL from its own knowledge and then fetches it correctly. Execution there is usable, but not precise enough for fully deterministic flows. The pattern is clear: strong tool-type selection, weaker precision on concrete parameterization. The fact that a retry was needed reads more like a formatting or execution issue than a comprehension failure. That is operationally manageable, but relevant in tight tool loops.

Synthesis Fidelity

How well does it compress tool results? Solid, but not reliably sharp enough. P2 performance shows usable summarization on HTTP Fetch & Extract, Tool Failure Handling (404), and URL Construction & Fetch, but drops noticeably on EU License Research and especially on Multilingual Search & Synthesis. For pipelines where the model only needs to briefly summarize results, this is sufficient. For compliance, research exports, or multilingual result fusion, it is too inconsistent.

Does it stay within the tool result or fall back on training data? On the honeypot EU License Research test — which checks whether current license restrictions are actually retrieved from web sources — the trust signal holds: no hallucination, Content Verification State A. The model is therefore not the type that silently passes world knowledge off as a tool finding when certainty is lacking. For production trust, that matters more than the merely average summarization quality.

Error Resilience

Acceptable for production. On the 404 test — which measures whether a failing tool call is communicated transparently or replaced with fabricated content — the model stays clean. It does not hallucinate page content despite the error and handles the failure in a traceable manner. Exactly this behavior protects tool pipelines from silent factual errors.

Sovereignty Profile

Fully locally operable, Apache-2.0-licensed, and therefore well-suited for sovereign integration. On the performance side, it sits only 0.75 points below the fleet average of 66.55. For a locally running 4B dense generalist, that is a credible result.

Conclusion & Recommendation

Suitable for local MCP pipelines with well-defined tools, moderate synthesis requirements, and high priority on data sovereignty. Well applicable for research orchestration, web access, error transparency, and straightforward extraction jobs. Not the right choice for pipelines where the final response itself must be highly precise, multilingually consolidated, or formulated with compliance-level accuracy. As a tool operator: yes. As a final editorial authority: rather no.

This analysis was generated automatically based on benchmark data. Model used: GPT 4.5 by OpenAI. The raw data and full methodology are documented in the GitHub project.