Skip to content
CrucibleMark

Gemma 4 31B Instruct (Thinking)

Gemma 4 31B Instruct is Google’s largest dense model, with 30.7 billion parameters across 60 layers and hybrid attention combining local sliding-window with global attention. Unlike the Gemma 4 MoE variant, this model activates all parameters per token. Multimodality for text and images, 256,000 tokens of context, native function calling, and a configurable thinking mode round out the profile. True Apache 2.0 license.

Google Version 4 Commercial use permitted Dense 30.7 B (30.7 B active) 256 K Context 01/2025 locally tested

  • Open Weights
  • Workstation
  • vLLM
  • Text
  • Vision
  • Instruction-Tuned
  • Unusable

Sovereign Risk: MEDIUM Google DeepMind is a US company, meaning Cloud/API usage (Google Cloud, Vertex AI, OpenRouter) carries US CLOUD Act exposure. Gemma 4 was released as the first Gemma generation under a true Apache 2.0 license (no more restrictive Gemma Terms of Use), enabling unrestricted fine-tuning and commercial use. For purely local deployment via llama.cpp/GGUF/NVFP4, CLOUD Act relevance is eliminated entirely, as no data is transmitted to Google. Official weights are available directly on Hugging Face.

Tool-use profile: 6 assets in detail

Comparison of asset performance (P1/P2/Combined) against the fleet average

Asset performance (radar)

Score Breakdown vs. Fleet Average

Tool-use details

Asset performance, reliability, and runtime profile

Name P1 P2 Combined
EU License Research 100 40 64
HTTP Fetch & Extract 80 60 70
Tool Failure Handling (404) 80 60 73
Web Search & Tool Selection 100 60 82
URL Construction & Fetch 80 60 70
Multilingual Search & Synthesis 100 60 80

CrucibleMark evaluates tool use across 6 independent tests. Click a test name for details.

Reliability

  • Tool Call Valid: No
  • Retry: Not required
  • Hallucination: Not detected

Reliability measures how consistently a model actually executes tool calls: Tool Call Valid schema and format accepted, Retry Required successful only after retry, Hallucination Flag fabricated tools or parameters detected. All three green means production-ready.

Runtime profile

Call 1
20.53
First Request
MCP
1.34
Protocol Latency
Synthesis
80.6
Response Generation
Total
614.83
Sum of All Phases
Token
13724
Input + Output
Cost
$0
Cost per Run

The runtime profile shows the latency and cost metrics for the model run: Call 1 First Request, MCP Protocol Latency, Synthesis Response Generation, Total sum of all phases, supplemented by Token input and output and Cost cost per run.

Tool-use review

Created · Instruction-Tuned

Deployment Verdict

Conditional deploy, because tool execution is solid overall, but synthesis fidelity — with a Combined score of 73.17 and notably weak compression — is insufficient to carry a tool infrastructure unsupervised.

Tool Execution Profile

Gemma 4 31B Instruct demonstrates genuine tool intelligence, not just rigid sequential execution. On the Web Search & Tool Selection test — which checks whether web_search is chosen over fetch without any hint — it scores P1 100 and cleanly identifies the appropriate access path. That is a strong signal for dynamic MCP pipelines. On the URL Construction test, which measures the autonomous derivation of a target URL followed by a fetch, it comes in lower at P1 80. This does not argue against tool usage, but it does argue against full determinism in URL formation. Critically, the overall tool call was flagged as invalid. This undermines confidence in protocol adherence more than the individual scores suggest. On the positive side, no retry was required. The issue therefore looks more like a localized call-form or argument-precision problem than a fundamental misunderstanding of the tooling model.

Synthesis Fidelity

How well does it compress tool results? Only moderately. P2 56.67 indicates a model that can retrieve sources but does not reliably convert the return into robust, concise working answers. This is also visible in the asset-level scores: HTTP Fetch & Extract, Tool Failure Handling (404), URL Construction & Fetch, and Multilingual Search & Synthesis each remain at P2 60, with EU License Research dropping to 40. For production chains, this means the retrieval stage is more reliable than the answer stage.

Does it stay within the tool result, or does it fall back on training? On the Honeypot EU License Research test — which checks whether current license restrictions are actually retrieved from web sources — no hallucination was detected. This is the central trust signal. Despite weak compression, the model does not visibly break the tool boundary or fabricate ostensibly researched compliance facts.

Error Resilience

On the 404 test, which measures transparent behavior when a tool call fails, the model does not hallucinate page content. That is acceptable for production. P2 60 indicates that error communication is not particularly precise or helpful, but it remains honest. For operational pipelines, that matters considerably more than elegant phrasing.

Sovereignty Profile

Locally deployable with Apache 2.0 weights and no cloud dependency. The model sits -1.22 points below the fleet average of 66.87, remaining sovereignty-competitive without a meaningful quality penalty for on-premises operation.

Conclusion & Recommendation

Well suited for local, sovereign MCP pipelines where tool selection, web research, and error transparency matter more than high-quality final compression. A good fit as a retrieval and orchestration component with a downstream validation or editorial stage. Not the first choice for compliance, executive, or customer-facing pipelines where the formulated synthesis itself must already be production-ready.

This analysis was generated automatically based on benchmark data. Model used: GPT 4.5 by OpenAI. The raw data and full methodology are documented in the GitHub project.