Skip to content
CrucibleMark

Qwen 3.8 27B (Thinking)

Qwen3.8-27B is Alibaba’s dense 27.8-billion-parameter variant of the Qwen3.8 family (August 14, 2026), natively multimodal with a vision encoder for text, image, and video under the Apache 2.0 license. The 64-layer hybrid architecture of Gated DeltaNet plus Gated Attention blocks plus Multi-Token Prediction delivers 262,144 tokens of context (extensible to approximately one million via YaRN), configurable reasoning (xhigh/medium/low), and an optional no-thinking mode.

Alibaba Version 3.8 Commercial use permitted Dense 27.8 B (27.8 B active) 262 K Context 12/2025 locally tested

  • Open Weights
  • Workstation
  • vLLM
  • Text
  • Vision
  • Video
  • Instruction-Tuned
  • Long Context
  • Batch

Sovereign Risk: MEDIUM The model was developed by Alibaba, a company headquartered in China. This carries a theoretical risk due to Chinese legislation. However, since the weights have been released under the permissive Apache 2.0 license and are intended for local deployment, no data is transmitted to the manufacturer. The risk is rated ‘medium’: the origin lies in a high-risk jurisdiction, but the open license and local usage considerably minimize the practical risk.

Tool-use profile: 6 assets in detail

Comparison of asset performance (P1/P2/Combined) against the fleet average

Asset performance (radar)

Score Breakdown vs. Fleet Average

Tool-use details

Asset performance, reliability, and runtime profile

Name P1 P2 Combined
EU License Research 100 40 64
HTTP Fetch & Extract 80 80 80
Tool Failure Handling (404) 80 35 64.3
Web Search & Tool Selection 100 80 91
URL Construction & Fetch 80 80 80
Multilingual Search & Synthesis 100 40 70

CrucibleMark evaluates tool use across 6 independent tests. Click a test name for details.

Reliability

  • Tool Call Valid: No
  • Retry: Not required
  • Hallucination: Detected

Reliability measures how consistently a model actually executes tool calls: Tool Call Valid schema and format accepted, Retry Required successful only after retry, Hallucination Flag fabricated tools or parameters detected. All three green means production-ready.

Runtime profile

Call 1
5.75
First Request
MCP
1.56
Protocol Latency
Synthesis
47.12
Response Generation
Total
326.55
Sum of All Phases
Token
15252
Input + Output
Cost
$0
Cost per Run

The runtime profile shows the latency and cost metrics for the model run: Call 1 First Request, MCP Protocol Latency, Synthesis Response Generation, Total sum of all phases, supplemented by Token input and output and Cost cost per run.

Tool-use review

Created · Instruction-Tuned · Long Context

Deployment Verdict

Conditional deploy: Tool execution is strong, but the detected hallucination on an invalid tool call — and the invalid tool call itself — disqualify the model for unattended MCP pipelines.

Tool Execution Profile

Qwen 3.8 27B demonstrates genuine tool intelligence. In the Web Search & Tool Selection test, which checks the choice between search and direct retrieval without an explicit hint, it reliably recognizes that web_search is required before fetch. This argues against a rigid retrieval pattern. In the URL Construction & Fetch test, which measures the derivation of a target URL from internal knowledge, it performs adequately in principle, but not precisely enough for deterministic workflows. The profile is therefore strong in the planning phase and somewhat weaker in exact protocol execution. The P1 score supports this, but what matters most for production is: the tool call was invalid during the run. This is not a cosmetic formatting error — it is an integration risk at the MCP boundary. On the positive side, no retry was required. The model understands the task flow, but fails to handle formal interface requirements cleanly enough.

Synthesis Fidelity

How well does it consolidate tool results? Only with limited reliability. Synthesis quality is clearly the weak point overall. In HTTP Fetch & Extract and URL Construction & Fetch it consolidates cleanly, but in EU License Research and Multilingual Search & Synthesis precision drops sharply. The pattern is consistent: facts are found, but not reliably translated into dependable output text.

Does it stay within the tool result or fall back on training data? In the Honeypot EU License Research test it formally stays on the safe path. It does not hallucinate from training there, even though the synthesis remains weak. This matters. At the same time, the overall hallucination finding is a security risk, not merely a quality deficiency. Once a model can output fabricated content as a tool result, the entire tool infrastructure loses its trust anchor.

Error Resilience

This is where the production-critical defect lies. In the 404 test, which is designed to check for transparent handling of a failed tool call, Qwen 3.8 27B does not communicate the error cleanly — instead it hallucates page content despite the failure. This is categorically unacceptable for production. A pipeline can handle an openly reported fetch error. It cannot safely handle fabricated fallback content.

Operational Profile

Total 326.55s. Call 1 5.75s. Call 2 47.12s. MCP latency 1.56s. Slow for the overall quality achieved. Cost per run: local, therefore infrastructure-side inexpensive.

Conclusion & Recommendation

Suitable for assisted research, search, and orchestration pipelines with human review or hard downstream validators. Not suitable for compliance, documentation, incident, or retrieval workflows where tool failures must remain strictly transparent and synthesized outputs are consumed directly. Anyone deploying it should schema-validate tool outputs, hard-catch error paths, and verify every content summary against the raw sources.

This analysis was generated automatically based on benchmark data. Model used: GPT 4.5 by OpenAI. The raw data and full methodology are documented in the GitHub project.