Skip to content
CrucibleMark

Kimi K2

Kimi K2 is Moonshot AI’s flagship model with an Open Weights license, focused on agentic tasks, coding, and multi-step reasoning. The MoE architecture activates 32 billion of the total one trillion parameters per token; the context window spans 128,000 tokens. Available as a text-only variant locally or via cloud providers — cloud usage under Chinese jurisdiction requires a separate evaluation.

Moonshot AI Version k2-0711 Commercial use permitted MoE 1000 B (32 B active) 128 K Context 06/2025 $0.6 / $2.5 per 1M

  • Open Weights
  • Frontier
  • OR
  • Text
  • Agentic Orchestrator
  • Real-Time

Tool-use profile: 6 assets in detail

Comparison of asset performance (P1/P2/Combined) against the fleet average

Asset performance (radar)

Score Breakdown vs. Fleet Average

Tool-use details

Asset performance, reliability, and runtime profile

Name P1 P2 Combined
EU License Research 100 60 76
HTTP Fetch & Extract 80 80 80
Tool Failure Handling (404) 80 80 80
Web Search & Tool Selection 100 35 70.8
URL Construction & Fetch 80 60 70
Multilingual Search & Synthesis 80 15 47.5

CrucibleMark evaluates tool use across 6 independent tests. Click a test name for details.

Reliability

  • Tool Call Valid: No
  • Retry: Not required
  • Hallucination: Detected

Reliability measures how consistently a model actually executes tool calls: Tool Call Valid schema and format accepted, Retry Required successful only after retry, Hallucination Flag fabricated tools or parameters detected. All three green means production-ready.

Runtime profile

Call 1
2.93
First Request
MCP
1.33
Protocol Latency
Synthesis
11.42
Response Generation
Total
94.08
Sum of All Phases
Token
6141
Input + Output
Cost
$0.0063
Cost per Run

The runtime profile shows the latency and cost metrics for the model run: Call 1 First Request, MCP Protocol Latency, Synthesis Response Generation, Total sum of all phases, supplemented by Token input and output and Cost cost per run.

Tool-use review

· Agentic Orchestrator

Deployment Verdict

Conditional deploy, because Kimi K2 produces valid tool calls and performs strongly in tool execution, but synthesis fidelity with a hallucination finding limits confidence in productive response layers.

Tool Execution Profile

Kimi K2 can in principle be handed off to an MCP-backed infrastructure. The tool calls are valid, protocol-compliant, and completed without retry. This argues against a pure formatting issue and in favor of a stable understanding of the call structure.

The decisive factor is tool selection. In the Web Search & Tool Selection test, which forces the correct choice between search and fetch without explicit guidance, the model selects the appropriate tool reliably. This does not look like rigid schema-following. It recognizes the information need and decides situationally. In the URL Construction & Fetch test, which requires deriving the target URL from internal knowledge and then fetching it correctly, it remains usable but less deterministic. The pattern is clear: strong tool intelligence at the selection stage, somewhat lower precision at execution when its own prior assumptions feed into URL construction.

Synthesis Fidelity

How well does it condense tool results? Only conditionally reliable. P2 performance is the clear weak point. Solid on HTTP Fetch & Extract and Tool Failure Handling (404), but noticeably weaker on Web Search & Tool Selection and especially on Multilingual Search & Synthesis, where condensation across language boundaries visibly breaks down. For pipelines in which the model merely references or passes on tool outputs briefly, this is tolerable. For compliance, research summaries, or customer-facing result presentation, it is too imprecise.

Does it stay within the tool result or fall back on training data? In the Honeypot EU License Research test, which probes exactly this trust failure, Kimi K2 stays with the web result and does not hallucinate. That is the more important finding. At the same time, a hallucination signal appears globally in the run. This is not merely a quality deficiency but a security risk: once a model outputs fabricated facts as apparent tool results, it undermines the reliability of the entire pipeline.

Error Resilience

Acceptable for production. In the 404 test, which checks for transparent handling of failing tool calls, Kimi K2 communicates the error rather than inventing page content. That is exactly the behavior required in operation. The failure is surfaced, not concealed.

Operational Profile

Call 1: 2.93s. MCP latency: 1.33s. Call 2: 11.42s. Total: 94.08s.
Slow overall run.
Cost per run: 0.006264. Inexpensive for the tool competence delivered.

Conclusion & Recommendation

Suitable for agentic pipelines where tool selection, flow control, and robust error handling matter more than high-quality final synthesis. Well suited for research orchestration, preprocessing, coder agents, and internal assistance layers with downstream verification. Not the first choice for multilingual research, decision-ready summaries, or any pipeline in which the model’s natural language output is delivered directly as a trusted final result.

This analysis was generated automatically based on benchmark data. Model used: GPT 4.5 by OpenAI. The raw data and full methodology are documented in the GitHub project.