Hermes 3 8B

Hermes 3 8B is an uncensored fine-tune by NousResearch based on Meta’s Llama 3.1 8B. With eight billion parameters and a 128,000-token context window, the model targets instruction following, tool use, and creative or ambiguous requests. Deployable locally under the Llama 3.1 Community License, the reduced refusal rate is a defining characteristic of this distribution.

NousResearch Version 3 Commercial use permitted Dense 8 B (8 B active) 128 K Context 09/2024 locally tested

Restricted Weights
Edge
llama.cpp
Text
Instruction-Tuned
Uncensored
Real-Time

Sovereign Risk: MEDIUM NousResearch is a US-based company; the CLOUD Act is only relevant when using the API, not when running the Open Weights variant locally.

Tool-use details

Asset performance, reliability, and runtime profile

Name	P1	P2	Combined
EU License Research	100	40	64
HTTP Fetch & Extract	80	15	47.5
Tool Failure Handling (404)	80	80	80
Web Search & Tool Selection	100	15	61.8
URL Construction & Fetch	40	15	27.5
Multilingual Search & Synthesis	100	15	57.5

Name

Combined

EU License Research

100

HTTP Fetch & Extract

47.5

Tool Failure Handling (404)

Web Search & Tool Selection

100

61.8

URL Construction & Fetch

27.5

Multilingual Search & Synthesis

100

57.5

CrucibleMark evaluates tool use across 6 independent tests. Click a test name for details.

Reliability

Tool Call Valid: Yes

Retry: Not required

Hallucination: Detected

Reliability measures how consistently a model actually executes tool calls: Tool Call Valid schema and format accepted, Retry Required successful only after retry, Hallucination Flag fabricated tools or parameters detected. All three green means production-ready.

Runtime profile

Call 1: 1.43; First Request

MCP: 0.93; Protocol Latency

Synthesis: 8.86; Response Generation

Total: 67.31; Sum of All Phases

Token: 6299; Input + Output

Cost: $0; Cost per Run

The runtime profile shows the latency and cost metrics for the model run: Call 1 First Request, MCP Protocol Latency, Synthesis Response Generation, Total sum of all phases, supplemented by Token input and output and Cost cost per run.

Deployment Verdict

Created on: 14.06.2026, 16:08:24

Conditional deploy, because the model produces valid tool calls and mostly gets tool selection right, but synthesis fidelity — with a Combined score of 56.38 and a confirmed hallucination case — is not stable enough for trust-critical pipelines.

Tool Execution Profile

Hermes 3 8B shows usable MCP capability at the execution level. Tool calls were valid, no retry was needed, and in the Web Search & Tool Selection test the model reliably recognizes — without an explicit hint — that a search must come first rather than fetching directly. This points to genuine tool selection rather than pure schema-following. At the same time, this strength breaks down visibly in URL Construction & Fetch: when the model must precisely derive the target URL from its own knowledge, P1 drops to 40. For production pipelines the implication is clear: strong at search and selection decisions, weak at deterministic URL derivation without external guidance. It can operate tool infrastructure, but cannot reliably handle every upstream step of resource resolution on its own.

Synthesis Fidelity

How well does it condense tool results? Only to a limited degree. P2 sits at 30 overall, and the assets reflect this: HTTP Fetch & Extract, Web Search & Tool Selection, and Multilingual Search & Synthesis often produce the correct access path, but the actual condensation of results remains shallow or loses precision. The model retrieves data but does not consistently transform it into reliable, concise payload for downstream steps.

Does it stay within the tool result or fall back on training? In the Honeypot EU License Research test — which checks whether current license restrictions actually come from web sources rather than training knowledge — it stays on the tool path in principle. P2 40 is not a strong synthesis result, but Content Verification State A and no hallucination show that it upholds the trust principle there. The global hallucination finding remains a security risk nonetheless: once a model outputs fabricated facts as a tool result, the entire pipeline loses its auditability.

Error Resilience

In the 404 test — which checks for transparent error communication rather than fabricated page content — Hermes 3 8B responds in a production-appropriate way. It does not hallucinate the missing content and communicates the failure cleanly. This is an important positive finding, because error visibility in tool pipelines matters more than linguistic smoothness.

Sovereignty Profile

Locally operable without external data transfer. Performance-wise, 1.37 points below the fleet average of 67.84. For an Edge model in local_sovereign operation, that is competitive enough, but not strong enough to compensate for quality deficits in synthesis.

Conclusion & Recommendation

Suitable for local, sovereign MCP pipelines with clear tool guidance, good observability, and low tolerance for cloud dependency. Particularly well-suited for research pre-stages, tool routing, and transparent error paths. Not suitable for compliance, fact-condensation, or executive-summary pipelines where the model must precisely consolidate tool results and pass them on without any fabricated additions. If you deploy it, do so with strict source binding, downstream validation, and without assigning responsibility for final content condensation.

This analysis was generated automatically based on benchmark data. Model used: GPT 4.5 by OpenAI. The raw data and full methodology are documented in the GitHub project.