Skip to content
CrucibleMark

Ornith 1.5 35B-A3B

Ornith 1.5 35B-A3B has been the mid-tier model in DeepReinforce’s open Ornith family since August 19, 2026. The MoE activates only around 3B of 35B parameters per token, yet according to the manufacturer it significantly outperforms the similarly sized Qwen 3.6-35B on all coding and agentic benchmarks. Trained with a closed self-improvement loop that jointly optimizes its own tasks, scaffolds, and solutions. License: MIT, fully open and commercially usable.

DeepReinforce Version 1.5 Commercial use permitted MoE 35 B (3 B active) 262 K Context 05/2026 locally tested

  • Open Weights
  • Workstation
  • vLLM
  • Text
  • Vision
  • Batch

Sovereign Risk: LOW DeepReinforce is a US-based research team. The model is released under the permissive MIT license with fully open weights, enabling independent auditing and fully local operation without cloud dependency. Local deployment involves no additional data transmission to the developer.

Tool-use profile: 6 assets in detail

Comparison of asset performance (P1/P2/Combined) against the fleet average

Asset performance (radar)

Score Breakdown vs. Fleet Average

Tool-use details

Asset performance, reliability, and runtime profile

Name P1 P2 Combined
EU License Research 95 40 62
HTTP Fetch & Extract 80 80 80
Tool Failure Handling (404) 80 40 66
Web Search & Tool Selection 95 80 88.3
URL Construction & Fetch 80 80 80
Multilingual Search & Synthesis 100 60 80

CrucibleMark evaluates tool use across 6 independent tests. Click a test name for details.

Reliability

  • Tool Call Valid: No
  • Retry: Not required
  • Hallucination: Not detected

Reliability measures how consistently a model actually executes tool calls: Tool Call Valid schema and format accepted, Retry Required successful only after retry, Hallucination Flag fabricated tools or parameters detected. All three green means production-ready.

Runtime profile

Call 1
3.28
First Request
MCP
1.1
Protocol Latency
Synthesis
21.19
Response Generation
Total
153.47
Sum of All Phases
Token
17396
Input + Output
Cost
$0
Cost per Run

The runtime profile shows the latency and cost metrics for the model run: Call 1 First Request, MCP Protocol Latency, Synthesis Response Generation, Total sum of all phases, supplemented by Token input and output and Cost cost per run.

Tool-use review

Created

Deployment Verdict

Conditional deploy, because despite a solid overall score of 76.04 the model fails to demonstrate a consistently valid tool call path, and synthesis quality remains too uneven for production tool pipelines.

Tool Execution Profile

Ornith 1.5 35B-A3B shows genuine tool selection rather than rigid pattern behavior. On the Web Search & Tool Selection test — which checks the choice between search and direct retrieval without an explicit hint — it scores P1 95 and identifies the need for web_search very reliably. This points to workable planning logic in open MCP workflows.

The execution layer is less clean. On the URL Construction & Fetch test, which measures independent derivation of the target URL followed by retrieval, it lands at P1 80. That is operationally usable, but not precise enough for deterministic pipelines with strict schema and target binding. The global signal tool_call_valid=False fits this picture: the model intelligence around tool selection is present, but protocol adherence in the concrete call path is not stable enough. Retry was not required, so the issue lies more in the initial validity of the call than in a recurring format collapse.

Synthesis Fidelity

How well does it consolidate tool results? Only limitedly reliable. P2 60 reveals a clear gap between retrieval and consolidation. It performs well on HTTP Fetch & Extract and URL Construction & Fetch, both at P2 80. It is weak where concise, reliable synthesis matters more than raw retrieval — for example EU License Research and Tool Failure Handling (404), both at P2 40. For production pipelines this means: raw data often arrives, but the last mile to a reliable decision remains shaky.

Does it stay within the tool result or fall back on training? On the Honeypot EU License Research test — which checks whether current license restrictions are answered from web sources rather than training knowledge — it does not hallucinate. That is the more important trust signal. The low P2 score therefore indicates no safety failure, but insufficient consolidation under compliance-like pressure.

Error Resilience

On the 404 test, which requires a transparent response to a failed tool call rather than fabricated page content, the model does not hallucinate. That is acceptable for production. The P2 score of 40 shows, however, that error communication is honest but not processed cleanly enough downstream. As an agent it does not break the chain of trust; as a reporter it remains too imprecise.

Operational Profile

Total 153.47s. Call 1 3.28s. MCP latency 1.10s. Call 2 21.19s. Slow for the quality delivered. Cost/run: local. Inexpensive to operate, but costly in time.

Conclusion & Recommendation

Suitable for locally operated research and orchestration pipelines where tool selection matters more than perfect final synthesis and a downstream validator checks the output. Not suitable for compliance, licensing, approval, or other decision-critical pipelines where the model must reliably summarize tool results on its own. As an MCP agent with guardrails it is usable. As an autonomous terminal node, not yet.

This analysis was generated automatically based on benchmark data. Model used: GPT 4.5 by OpenAI. The raw data and full methodology are documented in the GitHub project.