Skip to content
CrucibleMark

Swift Qwen 3.8 27B

Swift Qwen 3.8 27B is UkisAI’s reasoning-efficiency fine-tune on Qwen 3.8 27B: up to 58 percent fewer thinking tokens at under one percent performance loss and roughly twice the throughput on reasoning tasks. NVFP4 quantization with 262,000 tokens of context, MTP head for speculative decoding, and documented tool use — license with a commercial ARR threshold.

UkisAI Version 3.8 Commercial use permitted Dense 28 B (28 B active) 262 K Context 12/2025 locally tested

  • Open Weights
  • Workstation
  • vLLM
  • Text
  • Vision
  • Gated-Weights
  • Batch

Sovereign Risk: MEDIUM TODO

Tool-use profile: 6 assets in detail

Comparison of asset performance (P1/P2/Combined) against the fleet average

Asset performance (radar)

Score Breakdown vs. Fleet Average

Tool-use details

Asset performance, reliability, and runtime profile

Name P1 P2 Combined
EU License Research 100 40 64
HTTP Fetch & Extract 80 80 80
Tool Failure Handling (404) 80 100 87
Web Search & Tool Selection 100 40 73
URL Construction & Fetch 80 80 80
Multilingual Search & Synthesis 80 40 60

CrucibleMark evaluates tool use across 6 independent tests. Click a test name for details.

Reliability

  • Tool Call Valid: No
  • Retry: Not required
  • Hallucination: Not detected

Reliability measures how consistently a model actually executes tool calls: Tool Call Valid schema and format accepted, Retry Required successful only after retry, Hallucination Flag fabricated tools or parameters detected. All three green means production-ready.

Runtime profile

Call 1
4.21
First Request
MCP
1.43
Protocol Latency
Synthesis
28.14
Response Generation
Total
202.69
Sum of All Phases
Token
9611
Input + Output
Cost
$0
Cost per Run

The runtime profile shows the latency and cost metrics for the model run: Call 1 First Request, MCP Protocol Latency, Synthesis Response Generation, Total sum of all phases, supplemented by Token input and output and Cost cost per run.

Tool-use review

Created · Gated-Weights

Deployment Verdict

Conditional deploy, as tool execution is overall viable, but tool calls were not consistently valid and synthesis quality is too uneven for trust-critical pipelines.

Tool Execution Profile

Swift Qwen 3.8 27B demonstrates genuine tool selection rather than pure default patterns. In the Web Search & Tool Selection test — which evaluates the choice between search and direct retrieval without an explicit hint — it correctly identifies the need for web_search and achieves full execution confidence. This speaks to workable planning logic in dynamic MCP pipelines.

Precision in the follow-through is weaker. In the URL Construction test, which checks whether the model derives the target URL itself and then uses fetch correctly, it remains usable but not deterministic enough for strictly automated flows. This aligns with the overall finding: P1 is strong, but tool_call_valid=false indicates that protocol adherence and call accuracy do not hold up cleanly in every run. For production tool chains, this means: good intent recognition, but hardening on the executor side remains mandatory.

Synthesis Fidelity

How well does it consolidate tool results? Only limitedly reliable. P2 performance falls visibly short of execution performance. Consolidation breaks down on EU License Research, Web Search & Tool Selection, and Multilingual Search & Synthesis. The model often retrieves information successfully but then formulates it too coarsely, drops relevant caveats, or fails to prioritize clearly enough for architect-level decisions.

Does it stay within tool results or fall back on training data? Predominantly yes, and that is the more important trust signal. In the honeypot EU License Research test — which checks whether current license restrictions are answered from web sources rather than training knowledge — it shows only weak synthesis utility, but no detected hallucination. This is not a quality win, but it is a trust signal: it does not openly fabricate current compliance facts.

Error Resilience

The model is production-ready here. In the 404 test, which measures transparent handling of a failing tool call rather than fabricated page content, it communicates the error cleanly and does not hallucinate substitute content. This is precisely the behavior a tool pipeline requires: visible failure rather than silent fiction.

Operational Profile

Total 202.69s per run. Call 1: 4.21s. Call 2: 28.14s. MCP latency: 1.43s. Operated locally, so infrastructurally favorable. Given the merely adequate overall performance, this is slow.

Conclusion & Recommendation

Suitable for local MCP pipelines where tool selection, research initiation, and robust error handling matter more than high-quality final consolidation. Well suited as a retrieval or preparatory agent stage with downstream validation. Not the right choice for compliance, policy, or executive summary pipelines where the response itself must be consolidated precisely, reliably, and without room for interpretation.

This analysis was generated automatically based on benchmark data. Model used: GPT-5.4 by OpenAI. The raw data and full methodology are documented in the GitHub project.