LLMs head to head 7 use-case scenarios · Equal conditions · one results list

Basis of Calculation

Calculations are based on the provider’s public list prices at the time of testing. Local models carry no direct API costs and are labeled accordingly.

Informational Value

This figure shows what it costs to run a model through the full CrucibleMark benchmark once. It enables a direct price comparison between proprietary API models. For day-to-day usage it serves as a reference point, not a forecast — actual costs depend on your specific requests. Combined with the Total Score, it paints a price-to-performance picture: which model delivers how much quality per dollar.

License

The first badge identifies the license type and therefore the usage rights.

Commercial Proprietary model. Source code and weights are not public; the model is accessible exclusively via the provider’s official API.

Restricted Restricted Weights model. The weights are published, but the license restricts commercial or unrestricted use. Self-hosting is possible, but not unconditionally permitted.

Open Weight Open Weights model. The weights are freely available and may be downloaded, self-hosted, and used commercially without restrictions — locally or in the cloud.

Size Class

The middle badge assigns the model to a size class. It provides a reference for resource requirements and typical deployment context — not for quality.

Frontier The largest top-tier models (typically over 75B parameters). Generally only accessible via API or cloud; local operation is rarely practical.

Server Very large models (36–75B parameters) for dedicated server or cloud infrastructure.

Workstation Models (23–35B parameters) that can be run locally on high-performance workstation hardware.

Desktop Models (10–22B parameters) for standard desktop hardware, capable of running locally.

Edge Compact models (5–9B parameters) that run on typical consumer hardware (consumer PC, gaming rig).

Nano Very small models (≤4B parameters) with minimal resource requirements — run on the most modest hardware, such as laptops without a dedicated GPU.

Operation

The third badge shows how the model was operated during benchmarking.

API Accessible exclusively via the provider’s API (e.g. OpenAI, Anthropic, Google).

Cloud Open Weights model hosted in the cloud by a third-party provider (e.g. Together AI, Fireworks).

Local: Run locally on own hardware. Speed depends directly on the available hardware.

What the value refers to

The value applies to the respective deployment path (API, cloud, or local installation). For local models, it depends directly on the hardware used — two identical models can deliver very different speeds on different hardware.

Informative value

Speed is an average measurement across all modules. Short responses are processed proportionally faster than long ones; the mean smooths out these effects. For a direct comparison of two models, the value is solid as long as they run on the same deployment path. Comparing API models against local models is only partially meaningful, since the underlying hardware differs.

Value Meanings

Yes – the model supports Tool-Use and can execute structured function calls (Function Calling).

No – no native Tool-Use. The model responds to requests as plain text only.

– Unknown or not tested.

The Tool-Use Score

Beyond the yes/no indicator, there is a detailed Tool-Use Score determined across six independent tests. It measures not just whether a model calls tools, but how well it uses them — from tool selection and correct parameter passing to result synthesis. The full report is available on the model detail page under Tool-Use.

What the score measures

The score consolidates individual module results into a single number. A high ranking means the model performs well across all deployment scenarios, not just in one niche. A model that excels in a single discipline but underperforms elsewhere will rank lower accordingly.

What it does not tell you

The Total Score is a high-level indicator. For targeted decisions — such as “do I need the best reasoning or the best Tool-Use?” — it is worth examining the individual module scores. Use the module dropdown to sort the table by a specific metric.

Reasoning – Thinking Mode

Standard Classic processing without an explicit reasoning step. The model generates its response directly.

Thinking The model runs through a visible chain-of-thought process before responding. Slower, but more precise on complex tasks.

Adaptive With adaptive thinking, the model decides on its own whether a reasoning step is needed, depending on the complexity of the request.

When Each Mode Matters

Reasoning is not a quality indicator in itself — it’s a way of working. Thinking models shine on multi-step tasks but are slower and more expensive. Standard and Adaptive models respond faster and are well-suited for routine requests. Anyone evaluating latency or cost critically will find a first point of reference here.