Hermes 4 70B

Hermes 4 70B is an open instruct and reasoning model by Nous Research from the Hermes 4 family with 70 billion parameters. The model combines optional thinking with advanced tool use and structured outputs, trained for high steerability and reduced Refusal rates. Available as an Open Weights model under a Modified MIT license for local or server-side deployment.

NousResearch Version 4 Commercial use permitted Dense 70 B (70 B active) 131 K Context 01/2025 $0.13 / $0.4 per 1M

Open Weights
Server
OR
Text
Instruction-Tuned
Real-Time

Tool-use details

Asset performance, reliability, and runtime profile

Name	P1	P2	Combined
EU License Research	100	60	76
HTTP Fetch & Extract	80	35	57.5
Tool Failure Handling (404)	75	20	55.8
Web Search & Tool Selection	100	15	61.8
URL Construction & Fetch	75	60	67.5
Multilingual Search & Synthesis	100	60	80

Name

Combined

EU License Research

100

HTTP Fetch & Extract

57.5

Tool Failure Handling (404)

55.8

Web Search & Tool Selection

100

61.8

URL Construction & Fetch

67.5

Multilingual Search & Synthesis

100

CrucibleMark evaluates tool use across 6 independent tests. Click a test name for details.

Reliability

Tool Call Valid: No

Retry: Required

Hallucination: Detected

Reliability measures how consistently a model actually executes tool calls: Tool Call Valid schema and format accepted, Retry Required successful only after retry, Hallucination Flag fabricated tools or parameters detected. All three green means production-ready.

Runtime profile

Call 1: 0.84; First Request

MCP: 0.78; Protocol Latency

Synthesis: 3.05; Response Generation

Total: 28.05; Sum of All Phases

Token: 7599; Input + Output

Cost: $0.0014; Cost per Run

The runtime profile shows the latency and cost metrics for the model run: Call 1 First Request, MCP Protocol Latency, Synthesis Response Generation, Total sum of all phases, supplemented by Token input and output and Cost cost per run.

Deployment Verdict

Created on: 14.06.2026, 16:11:12

Conditional deploy, because tool execution is reliable, but synthesis quality — given the hallucination finding — limits confidence in unsupervised tool pipelines.

Tool Execution Profile

Hermes 4 70B performs strongly on the execution side. Tool calls are valid, MCP-compliant, and accurate in search-driven tasks. In the Web Search & Tool Selection test, which checks whether the model selects search over fetch without being prompted, the model reliably identifies the correct tool type. This argues against rigid pattern-following and in favor of genuine tool selection based on the task at hand.

It is weaker where it must derive the target address itself. In the URL Construction & Fetch test, it constructs the URL usably in most cases, but not consistently precisely enough for deterministic pipelines. This is not a protocol problem — it is a precision problem that occurs before the call. The fact that a retry was required fits this pattern: a correction in the flow or in target determination, not a fundamental misunderstanding of the tool interface.

Synthesis Fidelity

How well does it condense tool results? Only to a limited degree. P2 performance shows a clear gap between retrieval and processing. Hermes 4 70B retrieves information reliably, but frequently condenses it too coarsely, drops relevant details, or formulates the output without staying close enough to the tool finding. This is particularly visible in HTTP Fetch & Extract and Web Search & Tool Selection, where execution is strong but the final answer loses significant precision.

Does it stay within the tool result or fall back on training? In the Honeypot EU License Research, it stays within the verified web finding. That is the most important trust signal here. At the same time, the global hallucination finding remains a safety risk: once a model outputs fabricated facts as a result within a tool pipeline, it undermines the reliability of the entire infrastructure. For production use, this means: retrieval yes, synthesis only with downstream verification.

Error Resilience

In the 404 test, which checks for transparent handling of failed tool calls, Hermes 4 70B does not fabricate page content. This is the minimum requirement for production and it is met. The weak score stems from the fact that error communication is not condensed in a particularly useful way. Operationally this is acceptable: a brief, incomplete error message is preferable to fabricated fallback content.

Sovereignty Profile

Locally deployable with Open Weights, making it attractive for sovereign deployments. On the performance side, it sits 1.37 points below the fleet average of 67.84 — close to the fleet, but not leading.

Conclusion & Recommendation

Suitable for MCP pipelines in which the model selects tools, generates calls, and passes raw findings to a second verification or rendering step. Also viable for sovereign research workflows with a human in the loop. Not suitable as the sole final authority for compliance, high-density fact extraction, or automated user responses without verification. If you hand it the tool infrastructure, treat it as a retriever and orchestrator — not as an unsupervised final editor.

This analysis was generated automatically based on benchmark data. Model used: GPT 4.5 by OpenAI. The raw data and full methodology are documented in the GitHub project.