Skip to content
CrucibleMark

Qwen 3.8 27B

Qwen3.8-27B is Alibaba’s dense 27.8-billion-parameter variant of the Qwen3.8 family (August 14, 2026), natively multimodal with a vision encoder for text, image, and video under the Apache 2.0 license. The 64-layer hybrid architecture of Gated DeltaNet plus Gated Attention blocks plus Multi-Token Prediction delivers 262,144 tokens of context (extensible to approximately one million via YaRN), configurable reasoning (xhigh/medium/low), and an optional no-thinking mode.

Alibaba Version 3.8 Commercial use permitted Dense 27.8 B (27.8 B active) 262 K Context 12/2025 locally tested

  • Open Weights
  • Workstation
  • vLLM
  • Text
  • Vision
  • Video
  • Instruction-Tuned
  • Long Context
  • Batch

Sovereign Risk: MEDIUM The model was developed by Alibaba, a company headquartered in China. This carries a theoretical risk due to Chinese legislation. However, since the weights have been released under the permissive Apache 2.0 license and are intended for local deployment, no data is transmitted to the manufacturer. The risk is rated ‘medium’: the origin lies in a high-risk jurisdiction, but the open license and local usage considerably minimize the practical risk.

Tool-use profile: 6 assets in detail

Comparison of asset performance (P1/P2/Combined) against the fleet average

Asset performance (radar)

Score Breakdown vs. Fleet Average

Tool-use details

Asset performance, reliability, and runtime profile

Name P1 P2 Combined
EU License Research 100 20 52
HTTP Fetch & Extract 80 80 80
Tool Failure Handling (404) 40 40 37
Web Search & Tool Selection 100 80 91
URL Construction & Fetch 80 80 80
Multilingual Search & Synthesis 100 60 80

CrucibleMark evaluates tool use across 6 independent tests. Click a test name for details.

Reliability

  • Tool Call Valid: No
  • Retry: Not required
  • Hallucination: Not detected

Reliability measures how consistently a model actually executes tool calls: Tool Call Valid schema and format accepted, Retry Required successful only after retry, Hallucination Flag fabricated tools or parameters detected. All three green means production-ready.

Runtime profile

Call 1
3.01
First Request
MCP
2.93
Protocol Latency
Synthesis
24.86
Response Generation
Total
184.85
Sum of All Phases
Token
10211
Input + Output
Cost
$0
Cost per Run

The runtime profile shows the latency and cost metrics for the model run: Call 1 First Request, MCP Protocol Latency, Synthesis Response Generation, Total sum of all phases, supplemented by Token input and output and Cost cost per run.

Tool-use review

Created · Instruction-Tuned · Long Context

Deployment Verdict

Conditional deploy: tool execution is strong, but overall synthesis remains too unreliable and tool calls throughout the run were not consistently valid. For production MCP pipelines, it works better as an executing model than as a trusted final synthesizer.

Tool Execution Profile

Qwen 3.8 27B demonstrates genuine tool intelligence. In the Web Search & Tool Selection test — which checks the choice between search and direct retrieval without explicit guidance — it correctly identifies the need for web_search. This argues against a rigid pattern and in favor of context-dependent tool selection. It also consistently reaches for external sources in Multilingual Search & Synthesis and EU License Research.

Precision in downstream execution is weaker. In the URL Construction test, it constructs the target URL adequately, but not stably enough for deterministic pipelines. The result is decent, not dependable. The fact that the run is marked overall as tool_call_valid=false is the most important operational caveat: the model plans correctly but does not consistently produce protocol-clean or fully valid calls. For MCP orchestration, this means: a validator before the tools, a verification step after them.

Synthesis Fidelity

How well does it compress tool results? Only moderately. The P2 score of 65.83 fits the task profile: HTTP Fetch & Extract and URL Construction & Fetch are adequate, but EU License Research drops sharply to P2=20. This is not a minor outlier — it is a warning signal for pipelines that need to build reliable conclusions or compliance statements from web findings. The model finds sources more reliably than it synthesizes them cleanly.

Does it stay within tool output or fall back on training? Not reliably enough. In the honeypot EU License Research test — which checks whether current license restrictions are actually drawn from web sources — it does not hallucinate overtly. Nevertheless, the confidence verdict is weak, because the synthesis nearly collapses. Put differently: it invents nothing, but it also fails to demonstrate that it accurately translates tool results into dependable statements.

Error Resilience

In the 404 test, which measures transparent handling of a failed tool call, Qwen 3.8 27B stays on the right side of the safety line. It does not hallucinate page content. This is a production-relevant positive. The quality of error communication, however, is only marginally useful at P2=40. The model is defensive rather than graceful: acceptable for production when the orchestrator handles error states clearly on its own.

Operational Profile

Call 1: 3.01s. MCP latency: 2.93s. Call 2: 24.86s. Total: 184.85s.
Slow for the performance delivered. Cost/run: local. Financially cheap, time-expensive.

Conclusion & Recommendation

Suitable for locally operated research, search-routing, and preprocessing pipelines where a second system validates, normalizes, or finally formulates the results. Not suitable as the sole final authority for compliance, license assessment, regulatory summaries, or other tool pipelines where the synthesis itself is the product. Anyone handing it a tool infrastructure should deploy it as a capable retriever and adequate executor — not as the last trusted voice.

This analysis was generated automatically based on benchmark data. Model used: GPT 4.5 by OpenAI. The raw data and full methodology are documented in the GitHub project.