Skip to content
CrucibleMark

Gemma 4 26B-A4B Instruct (Thinking)

Google DeepMind deliberately sets Gemma 4 26B-A4B Instruct apart from earlier Gemma generations: it ships under a genuine Apache 2.0 license, with no restrictive Gemma terms of use. The Open Weights MoE activates only approximately 3.8 of 25.2 billion parameters per token and supports multi-token prediction for faster decoding. Multimodality for text and images, a 262,144-token context window, native function calling, and a configurable thinking mode round out the profile.

Google Version 4 Commercial use permitted MoE 25.2 B (3.8 B active) 262 K Context 02/2026 locally tested

  • Open Weights
  • Workstation
  • vLLM
  • Text
  • Vision
  • Instruction-Tuned
  • Batch

Sovereign Risk: MEDIUM Google DeepMind is a US company, so cloud/API usage (e.g., Google Cloud, OpenRouter) carries US CLOUD Act exposure. Unlike previous Gemma generations, Gemma 4 was released under a genuine Apache 2.0 license (no Gemma Terms of Use anymore), allowing fine-tuning and commercial use without restrictions. When running purely locally via llama.cpp/GGUF, CLOUD Act relevance is eliminated entirely, as no data is transmitted to Google. Weights are openly available on Hugging Face.

Tool-use profile: 6 assets in detail

Comparison of asset performance (P1/P2/Combined) against the fleet average

Asset performance (radar)

Score Breakdown vs. Fleet Average

Tool-use details

Asset performance, reliability, and runtime profile

Name P1 P2 Combined
EU License Research 100 40 64
HTTP Fetch & Extract 80 60 70
Tool Failure Handling (404) 80 60 73
Web Search & Tool Selection 100 80 91
URL Construction & Fetch 80 60 70
Multilingual Search & Synthesis 95 60 77.5

CrucibleMark evaluates tool use across 6 independent tests. Click a test name for details.

Reliability

  • Tool Call Valid: No
  • Retry: Not required
  • Hallucination: Not detected

Reliability measures how consistently a model actually executes tool calls: Tool Call Valid schema and format accepted, Retry Required successful only after retry, Hallucination Flag fabricated tools or parameters detected. All three green means production-ready.

Runtime profile

Call 1
5.78
First Request
MCP
1.17
Protocol Latency
Synthesis
41.85
Response Generation
Total
292.78
Sum of All Phases
Token
21995
Input + Output
Cost
$0
Cost per Run

The runtime profile shows the latency and cost metrics for the model run: Call 1 First Request, MCP Protocol Latency, Synthesis Response Generation, Total sum of all phases, supplemented by Token input and output and Cost cost per run.

Tool-use review

Created · Instruction-Tuned

Deployment Verdict

Conditional deploy, because tool execution is strong, but tool calls were not consistently formatted as valid, and synthesis fidelity at 60 clearly lags behind execution quality. The Combined Score of 74.25 is sufficient for productive use only where results are reviewed downstream.

Tool Execution Profile

Gemma 4 26B-A4B Instruct demonstrates genuine tool intelligence, not just rigid pattern matching. On the Web Search & Tool Selection test — which checks whether the model selects between search and direct retrieval without an explicit hint — the model correctly identifies the need for web_search and achieves P1 100. That is a strong signal for dynamic MCP pipelines.

Less clean is the precision on the URL Construction test, which checks whether the model derives the target URL from its own knowledge and then retrieves it correctly. P1 80 means: usable, but not deterministic enough for pipelines that require exact endpoints without a correction layer. It remains notable that the tool call was marked invalid overall despite no retry being necessary. This points to protocol or formatting weaknesses at isolated points rather than a fundamental comprehension problem.

Synthesis Fidelity

How well does it consolidate tool results? Only reliably enough in limited cases. The P2 score of 60 shows that the model usually merges retrieved content in a usable way, but is not stable enough in precision, weighting, and consolidation for high-quality result handoffs. This is also visible in EU License Research with P2 40, as well as in several tasks with solid execution but only average final presentation.

Does it stay within the tool result or fall back on training? In the honeypot EU License Research test — which checks whether current license restrictions are actually retrieved from web sources — no hallucination was detected. That is the more important trust signal. The model does not openly drift into fabricated recency, even if it subsequently fails to consolidate the retrieved content with sufficient rigor.

Error Resilience

On the 404 test, which checks for transparent behavior when a tool call fails, the model does not hallucinate substitute content. That is production-ready. P2 60 here means: communication about the failure is acceptable, but not particularly precise or actionable. For production operation this is manageable, because the safety boundary holds: no fabricated page content despite the error.

Sovereignty Profile

Locally deployable, Apache-2.0-licensed, and therefore attractive for sovereign deployments. With a Sovereignty Gap of -1.22 points below the fleet average of 66.87, it remains practically fleet-competitive. For organizations with a local deployment requirement, this is a tangible advantage.

Conclusion & Recommendation

Suitable for MCP pipelines with search, retrieval, and routing components where tool selection matters more than perfect result consolidation. A good fit for internal research agents, retrieval-augmented assistants, and sovereign on-premises setups with a review layer. Not the right choice for compliance, legal, or executive briefing pipelines where the final synthesis itself must be close to revision-ready. Anyone deploying this model should make tool call validation and a strict post-processing layer mandatory prerequisites.

This analysis was generated automatically based on benchmark data. Model used: GPT 4.5 by OpenAI. The raw data and full methodology are documented in the GitHub project.