Hermes 4 70B

Hermes 4 70B is an open instruct and reasoning model by Nous Research from the Hermes 4 family with 70 billion parameters. The model combines optional thinking with advanced tool use and structured outputs, trained for high steerability and reduced Refusal rates. Available as an Open Weights model under a Modified MIT license for local or server-side deployment.

NousResearch Version 4 Commercial use permitted Dense 70 B 131 K Context 01/2025 $0.13 / $0.4 per 1M

Open Weights
Server
OpenRouter
Text
Instruction-Tuned
Real-Time

Sovereign Risk: MEDIUM Nous Research is a US-based company, which makes US jurisdiction and the CLOUD Act relevant. Since the models are Open Weights, they can be run locally, which minimizes the risk of uncontrolled data leakage via an API.

Tool-use details

Asset performance, reliability, and runtime profile

Name	P1	P2	Combined
EU License Research	100	60	76
HTTP Fetch & Extract	80	80	80
Tool Failure Handling (404)	80	80	80
Web Search & Tool Selection	100	60	82
URL Construction & Fetch	75	80	77.5
Multilingual Search & Synthesis	100	15	57.5

Name

Combined

EU License Research

100

HTTP Fetch & Extract

Tool Failure Handling (404)

Web Search & Tool Selection

100

URL Construction & Fetch

77.5

Multilingual Search & Synthesis

100

57.5

CrucibleMark evaluates tool use across 6 independent tests. Click a test name for details.

Reliability

Tool Call Valid: No

Retry: Not required

Hallucination: Detected

Reliability measures how consistently a model actually executes tool calls: Tool Call Valid schema and format accepted, Retry Required successful only after retry, Hallucination Flag fabricated tools or parameters detected. All three green means production-ready.

Runtime profile

Call 1: 0.98; First Request

MCP: 1.43; Protocol Latency

Synthesis: 5.77; Response Generation

Total: 49.14; Sum of All Phases

Token: 10314; Input + Output

Cost: $0; Cost per Run

The runtime profile shows the latency and cost metrics for the model run: Call 1 First Request, MCP Protocol Latency, Synthesis Response Generation, Total sum of all phases, supplemented by Token input and output and Cost cost per run.

Deployment Verdict

Conditional deploy, because tool execution is strong, but one invalid tool call and a confirmed hallucination limit confidence in production MCP pipelines. The overall impression is good, but not robust enough for unsupervised high-trust workloads.

Tool Execution Profile

Hermes 4 70B demonstrates clear tool intelligence in selecting the right tool. In the Web Search & Tool Selection test — which checks whether the model recognizes that a search is needed instead of fetch, without an explicit hint — the model reliably identifies the correct access path. This argues against rigid pattern behavior and in favor of genuine situational adaptation. It also operates confidently in EU License Research and Multilingual Search & Synthesis.

Protocol precision is weaker. The “Tool call valid: false” status and the result in the URL Construction & Fetch test show that deriving a target URL from internal knowledge is not deterministic enough. For MCP pipelines, this means: tool selection is usually correct, but concrete execution remains error-prone when the path is not already secured by the tooling layer. On the positive side, no retry was required. The issue therefore lies in first-attempt precision rather than repeated formatting failures.

Synthesis Fidelity

How well does it consolidate tool results? Only partially reliable. The P2 quality score of 41.67 is this model’s clear weak point. Multilingual Search & Synthesis is particularly notable: the research succeeds, but consolidation in German breaks down significantly. Hermes can retrieve information, but loses precision and prioritization during synthesis. For architectures where the model must not only find but also reliably summarize, this is a real production risk.

Does it stay within tool results or fall back on training data? In the EU License Research honeypot — which checks whether current license restrictions are actually retrieved from web sources — Hermes stays on the tool path and does not hallucinate there. This is an important trust signal. At the same time, the global hallucination finding is rated a security risk: once a model presents fabricated facts as tool output, it damages the trust chain of the entire tool infrastructure.

Error Resilience

Acceptable for production. In the Tool Failure Handling (404) test — which checks for transparent behavior when a fetch fails — Hermes communicates the error cleanly and does not fabricate substitute content. This behavior is exactly what production pipelines require, because orchestration can only respond reliably when given honest error states.

Sovereignty Profile

Locally deployable and fleet-competitive. The combined score is 75.50, placing the sovereignty gap n/a points below the fleet average of 67.19.

Conclusion & Recommendation

Suitable for locally operated MCP pipelines where the model primarily selects tools, opens search paths, and forwards results with downstream validation. Not suitable as the sole synthesis and trust authority for compliance, multilingual research consolidation, or deterministic fetch workloads with fragile URL construction. Hermes 4 70B is recommended as an orchestrating front model with strict tool schema validation, output checks, and a second review step for final responses.

This analysis was generated automatically based on benchmark data. Model used: GPT-5.4 by OpenAI. The raw data and full methodology are documented in the GitHub project.