Skip to content
CrucibleMark

Kimi K2.5

Kimi K2.5 is Moonshot AI’s flagship model featuring active chain-of-thought reasoning, multimodal input for text and images, and a focus on reasoning and agentic tasks. The MoE architecture activates 32 billion of the total one trillion parameters per token; the context window spans 128,000 tokens. Available as an Open Weights variant locally or via cloud, with Chinese jurisdiction as a cloud risk factor.

Moonshot AI Version k2.5 Commercial use permitted MoE 1000 B (32 B active) 128 K Context 09/2025 $0.44 / $2 per 1M

  • Open Weights
  • Frontier
  • OpenRouter
  • Text
  • Vision
  • Agentic Orchestrator
  • Batch

Tool-use profile: 6 assets in detail

Comparison of asset performance (P1/P2/Combined) against the fleet average

Asset performance (radar)

Score Breakdown vs. Fleet Average

Tool-use details

Asset performance, reliability, and runtime profile

Name P1 P2 Combined
EU License Research 100 35 61
HTTP Fetch & Extract 80 80 80
Tool Failure Handling (404) 80 80 80
Web Search & Tool Selection 100 80 91
URL Construction & Fetch 80 60 70
Multilingual Search & Synthesis 100 60 80

CrucibleMark evaluates tool use across 6 independent tests. Click a test name for details.

Reliability

  • Tool Call Valid: No
  • Retry: Not required
  • Hallucination: Detected

Reliability measures how consistently a model actually executes tool calls: Tool Call Valid schema and format accepted, Retry Required successful only after retry, Hallucination Flag fabricated tools or parameters detected. All three green means production-ready.

Runtime profile

Call 1
4.1
First Request
MCP
0.91
Protocol Latency
Synthesis
44.47
Response Generation
Total
296.87
Sum of All Phases
Token
18983
Input + Output
Cost
$0
Cost per Run

The runtime profile shows the latency and cost metrics for the model run: Call 1 First Request, MCP Protocol Latency, Synthesis Response Generation, Total sum of all phases, supplemented by Token input and output and Cost cost per run.

Tool-use review

Updated · Agentic Orchestrator

Deployment Verdict

Conditional deploy, because tool execution is strong, but the detected hallucination combined with an invalid tool call breaks trust for critical production pipelines.

Tool Execution Profile

Kimi K2.5 demonstrates clear orchestration strength. In the Web Search & Tool Selection test — which requires choosing correctly between search and fetch without an explicit hint — it selects the appropriate tool with confidence. This argues against rigid pattern behavior and in favor of genuine tool selection during execution. P1 also reaches 100 in Multilingual Search & Synthesis and EU License Research, meaning the correct research chain is triggered reliably.

The second half of execution is weaker: the tool call was not valid overall. This aligns with the result from URL Construction & Fetch, where the model constructs the target URL from its own knowledge in a usable but insufficiently precise way for deterministic pipelines. For MCP environments, this means: planning and tool selection are solid, but protocol and parameter precision are not consistent throughout. No retry was required. This therefore looks less like a format collapse and more like a localized last-mile execution issue.

Synthesis Fidelity

How well does it condense tool results? Only to a limited degree. A P2 score of 65.83 is too low for a Frontier model with an agentic focus to be considered a reliable synthesis layer. On the positive side, HTTP Fetch & Extract and Tool Failure Handling (404) show clean consolidation of extracted content. On the negative side, EU License Research drops off sharply, and synthesis in URL Construction & Fetch also remains too imprecise.

Does it stay within tool output or fall back on training data? No, not consistently. In the honeypot EU License Research — which tests whether current license restrictions are actually retrieved from web sources — the model hallucinates and achieves only P2=35. This is not merely a quality deficiency; it is a security risk. When a model outputs fabricated or training-reconstructed facts as the result of a tool pipeline, it undermines the verifiability of the entire infrastructure.

Error Resilience

In Tool Failure Handling (404), which measures how a failing call is handled, Kimi K2.5 responds in a production-appropriate manner. It communicates the error transparently and does not hallucinate substitute content. This is acceptable for real-world pipelines and suggests that failures do not automatically translate into incorrect responses.

Operational Profile

Call 1: 4.10s. Call 2: 44.47s. MCP latency: 0.91s. Total per run: 296.87s. Slow for the utility delivered. Cost: local.

Conclusion & Recommendation

Suitable for agentic research and routing pipelines where tool selection matters more than final fact synthesis and where downstream validators check every claim. Not suitable for compliance, licensing, policy, or other high-trust pipelines where tool outputs must remain strictly sourced. Anyone deploying Kimi K2.5 should use it as an orchestrator with hard output controls — not as the final arbiter of truth.

This analysis was generated automatically based on benchmark data. Model used: GPT 4.5 by OpenAI. The raw data and full methodology are documented in the GitHub project.