Skip to content
CrucibleMark

Gemma 4 26B-A4B Instruct

Google DeepMind deliberately sets Gemma 4 26B-A4B Instruct apart from earlier Gemma generations: it ships under a genuine Apache 2.0 license, with no restrictive Gemma terms of use. The Open Weights MoE activates only approximately 3.8 of 25.2 billion parameters per token and supports multi-token prediction for faster decoding. Multimodality for text and images, a 262,144-token context window, native function calling, and a configurable thinking mode round out the profile.

Google Version 4 Commercial use permitted MoE 25.2 B (3.8 B active) 262 K Context 02/2026 locally tested

  • Open Weights
  • Workstation
  • vLLM
  • Text
  • Vision
  • Instruction-Tuned
  • Real-Time

Sovereign Risk: MEDIUM Google DeepMind is a US company, so cloud/API usage (e.g., Google Cloud, OpenRouter) carries US CLOUD Act exposure. Unlike previous Gemma generations, Gemma 4 was released under a genuine Apache 2.0 license (no Gemma Terms of Use anymore), allowing fine-tuning and commercial use without restrictions. When running purely locally via llama.cpp/GGUF, CLOUD Act relevance is eliminated entirely, as no data is transmitted to Google. Weights are openly available on Hugging Face.

Tool-use profile: 6 assets in detail

Comparison of asset performance (P1/P2/Combined) against the fleet average

Asset performance (radar)

Score Breakdown vs. Fleet Average

Tool-use details

Asset performance, reliability, and runtime profile

Name P1 P2 Combined
EU License Research 100 40 64
HTTP Fetch & Extract 75 80 77.5
Tool Failure Handling (404) 40 40 37
Web Search & Tool Selection 95 60 79.3
URL Construction & Fetch 75 60 67.5
Multilingual Search & Synthesis 95 40 67.5

CrucibleMark evaluates tool use across 6 independent tests. Click a test name for details.

Reliability

  • Tool Call Valid: No
  • Retry: Not required
  • Hallucination: Not detected

Reliability measures how consistently a model actually executes tool calls: Tool Call Valid schema and format accepted, Retry Required successful only after retry, Hallucination Flag fabricated tools or parameters detected. All three green means production-ready.

Runtime profile

Call 1
2.85
First Request
MCP
1.12
Protocol Latency
Synthesis
6.99
Response Generation
Total
65.75
Sum of All Phases
Token
11688
Input + Output
Cost
$0
Cost per Run

The runtime profile shows the latency and cost metrics for the model run: Call 1 First Request, MCP Protocol Latency, Synthesis Response Generation, Total sum of all phases, supplemented by Token input and output and Cost cost per run.

Tool-use review

Updated · Instruction-Tuned

Deployment Verdict

Conditional deploy, because tool execution is strong but tool calls are not consistently valid and synthesis at Combined 65.46 misses production-grade precision too often.

Tool Execution Profile

Gemma 4 26B-A4B Instruct demonstrates genuine tool selection rather than mere schema-following. On the Web Search & Tool Selection test — which evaluates the choice between search and direct retrieval without explicit hints — it scores P1 95 and identifies the need for web_search very reliably. This points to usable agent logic in dynamic MCP pipelines.

The second half of the chain is weaker. On the URL Construction test, which measures independent derivation of a target address and correct fetch behavior, it lands at P1 75. That is functional, but not deterministic enough for pipelines where URL formation and call format must be strict. The global finding “Tool call valid: false” is decisive here. The model appears tool-intelligent but not protocol-safe. Retry was not required. This argues against a pure formatting issue and more toward inconsistent execution at the final step.

Synthesis Fidelity

How well does it compress tool results? Only to a limited degree. P2 56.67 shows that Gemma often produces usable summaries but loses important precision. The pattern is consistent: HTTP Fetch & Extract is clean at P2 80, while EU License Research and Multilingual Search & Synthesis drop back to P2 40. For production tool pipelines this means: it often reads sources correctly but does not compress them stably enough for compliance, policy, or multilingual knowledge work.

Does it stay within tool output or fall back on training data? In the honeypot EU License Research test — which checks whether current license restrictions are actually retrieved from web sources — no hallucination was detected. That is the critical trust signal. The weak P2 score therefore reflects compression errors rather than fabricated facts. That is better than hallucination, but still requires oversight for sensitive outputs.

Error Resilience

On the 404 test, which measures how the model handles a failing tool call, it does not hallucinate page content. That is the minimum production requirement, and it is met. The transparency layer is problematic: P2 40 shows that the error condition is not communicated or contextualized cleanly enough. For robust systems this is acceptable when the orchestrator catches error states itself and does not pass responses to users unvalidated.

Sovereignty Profile

Fully operable on-premises under Apache-2.0, making it attractive for sovereign deployments. At Combined 65.46, it sits 1.73 points below the fleet average of 67.19.

Conclusion & Recommendation

Suitable for local research, routing, and preprocessing pipelines where tool selection matters more than perfect final synthesis and a downstream validator checks the output. Not suitable as the final authority for compliance, license assessment, incident communication, or multilingual decision documents. Those looking to run MCP orchestration locally and openly get a capable operator here — but not a reliable concluding reporter.

This analysis was generated automatically based on benchmark data. Model used: GPT 4.5 by OpenAI. The raw data and full methodology are documented in the GitHub project.