Gemma 4 ARA 26B-A4B (ARA-Abliterated)

With an overall score of 72.89%, Gemma 4 ARA 26B-A4B Q5_K_M (ARA-Abliterated) delivers no glamorous fireworks, but a surprisingly robust working model with a distinct character. The Speed Profile badge Interactive DevOps Expert fits: the model responds quickly, in a structured manner, and mostly with operational utility, without getting lost in verbal ornamentation. For a generalist in the Workstation class with MoE architecture and only 4.0 billion active parameters per token, that is respectable. Sovereign Risk: MEDIUM — the base model originates from US jurisdiction, but the locally deployed weights were additionally modified through a community abliteration, making the provenance less cleanly traceable.

Header Notes: Stability and Reliability

This combination is telling. No timeouts — no mechanical self-destruction under load. At the same time, a P95 response time of 65.16 seconds, meaning a pronounced long tail in response latency. For a model classified as Agentic-Orchestrator, this is not entirely surprising: such models often plan internally more than their visible output suggests. What you notice here is the reasoning architecture, not merely raw token throughput.

Classification: What This Model Actually Wants to Be

The metadata aligns surprisingly well with the observed behavior. As a General model it must demonstrate breadth. As an Instruct model it must follow instructions precisely and without unnecessary detours. As an Uncensored variant it carries the typical risk of any abliteration: more freedom, but potentially collateral damage to internal consistency. And as an Agentic-Orchestrator it should be judged less on whether every individual step looks ruler-straight, and more on whether planning, analysis, and strategic structure hold up.

Add to this the curated classification: Use Case Generalist, Size Class Workstation, parameter architecture MoE. This matters, because 25.2 billion total parameters are only half the story here. More relevant are the 4.0 billion active parameters per token. Gemma 4 ARA 26B-A4B Q5_K_M (ARA-Abliterated) therefore operates more like a selectively switching all-rounder than a permanently fully engaged large transformer. Expectations must accordingly be calibrated against active capacity, not the total figure on the box.

Measured against exactly that, the model looks surprisingly healthy. It shows no signs of the spectacular capability collapse familiar from poorly executed uncensored abliterated derivatives. The modifications did not saw off its legs. But they did not create a miracle machine either.

Speed and Efficiency

The model was evaluated as a local model on an Apple Silicon M4, 24GB Unified Memory (Shared RAM/VRAM). In this context, 54.58 tokens per second is a very solid result, especially for a Workstation MoE with 25.2 billion total parameters. The badge Interactive DevOps Expert is not marketing poetry here, but a useful description of the deployment profile: interactive enough for dialogue work, fast enough for operational technical tasks, but not the kind of model that fires off instantly in every situation.

More importantly: the model behaves token-economically. No module exceeds the expected verbosity range. In the CLI domain it sits at an average of 211 output tokens, clearly below the fleet median of 294. Code Quality, Cultural Intelligence, and UX Writing also remain at or below average. Only Documentation Quality and Content Transformation are somewhat more verbose, but not out of bounds. In practice this means: no excessive chattiness, no artificially inflated response volume, no noticeable latency penalty from unnecessary walls of text.

Code Quality and Security: Capable, but Not Sharp Enough

With 74.9 points in Code Quality, Gemma 4 ARA 26B-A4B Q5_K_M (ARA-Abliterated) performs well. This is not an elite forensic analyst, but a security reviewer who genuinely spots most obvious and many advanced issues. In the test protocol, the model identifies 14 vulnerabilities in a deliberately vulnerable PHP example, including SQL Injection, plaintext passwords, XSS, Session Fixation, Path Traversal, weak token generation, Type Juggling, and CSRF. It provides mostly correct countermeasures. That is more than keyword bingo.

The weakness lies not in incorrect technique, but in incomplete depth of focus. Five relevant gaps are missed, most notably an IDOR vulnerability in the profile update function — a direct path to privilege escalation through manipulation of a user ID. The model also overlooks hardcoded secrets and missing expiry times for reset tokens. In security contexts, this is not a minor cosmetic flaw, but the difference between “solid initial assessment” and “audit-ready analysis.”

The model’s character becomes even clearer in its presentation style: cleanly structured table, precise language, useful fixes. What is missing is chain thinking. The Judge rightly notes that the model treats vulnerabilities atomistically rather than connecting attack paths. A professional report does not just show individual leaks — it also shows how they combine into a total failure. That is precisely where Gemma 4 ARA 26B-A4B Q5_K_M (ARA-Abliterated) remains a little too well-behaved.

The verdict is therefore ambivalent, but fair: for code reviews, security hygiene, and initial audit passes, the model is useful. For genuinely critical security work, human oversight or a sharper specialist model is required. It should be noted in mitigation that this is not a coder model, and an uncensored derivative finds its real value more in the freer content space than in maximum software precision. Even so: anyone invoking “security analysis” cannot afford to miss a privilege escalation.

CLI and Tool Proximity: Surprisingly Accurate

The CLI score of 90.0 is one of this model’s strongest results and fits well with the Agentic-Orchestrator category. The model appears at home in operational, step-oriented technical tasks. This is precisely where its ability to process instructions directly and without unnecessary stylistic detours pays off. That such models are occasionally less elegant in strictly formatted individual commands than in planning and decomposition would be forgivable. Here, however, there is little to forgive: the result is strong.

The lean token usage in the CLI module fits this picture. The model does not first explain the history of the shell — it gets to the point. For agent frameworks and semi-automated DevOps workflows, this is a genuine advantage. Anyone looking for an orchestrator that structures operational steps plausibly without burying everything in text will find a serious tool here.

Reasoning and Logic: Substantively Solid, with an Ugly Scratch on Language Compliance

Logical Reasoning lands at 68.16 points. That is decent, but not dominant. The qualitative protocols show a model that is often logically correct and develops its reasoning thoroughly, but does not always maintain sufficient clarity in presentation. In the two-guards puzzle it correctly identifies the self-referential solution and even discusses multiple variants. The Judge attests to substantively clean logic, including case distinction and correct negation structure. The problem is therefore not that the model cannot reason. The problem is that it commits an unnecessary slip on task compliance.

Metacognitive Compliance (Reasoning): The model refuses to use the explicitly requested <thought> tags in 3/5 metacog tests, citing a consistent policy statement. The reasoning content is partially correct in substance — the score deduction results from format refusal, not from reasoning errors. For comparison: in the tag-free reasoning_5* tests, the model achieves an average score of approximately 68%, which is in line with other models. CrucibleMark deliberately evaluates native zero-shot instruction compliance as a real-world characteristic — this deduction is methodologically intentional.

There is also a concrete hard constraint violation in the metacognition domain: in a reasoning task that explicitly required a German-language response, the model answered predominantly in English. The system registered an automatic language violation; the language was not merely slightly mixed, but clearly off, with 140 English markers against 3 German ones. The substantive quality of the response becomes secondary, because the rule violation applies independently of the reasoning path. In a productive German-language environment, this is not a minor detail — it is a clean misstep.

The model ignored the explicit language instruction in this task and responded in English. This is not a technical defect, but a weakness in instruction-following under combined load from format and language requirements. Precisely because Gemma 4 ARA 26B-A4B Q5_K_M (ARA-Abliterated) is labeled Instruct, this failure carries more weight. An instruction model must not slip into English on such a basic requirement simply because its internal reasoning track is apparently organized differently.

On balance, Reasoning here is better than its score might suggest in parts. But it is also not confident enough to consistently carry format requirements alongside the logic. The model can think. It just does not always comply cleanly enough.

UX Writing and Content Transformation: Production-Ready, but Not Elegant to the Last Sentence

In UX Writing the model reaches 67.35 points, in Content Transformation 72.33. This combination captures its character quite well: decent, often useful, but not the kind of text model where you lean back after the final paragraph and think: yes, that is exactly how language should sound.

The Content Transformation protocol shows the strengths very clearly. In a complex task converting a dry 2FA outline into a four-minute, production-ready YouTube script, the model delivers fully in German, with timestamps, speaker text, visual cues, B-roll, music cues, pattern interrupt, and call to action. That is no small feat. Many models fail at precisely this combination of format, dramaturgy, and production logic. Gemma 4 ARA 26B-A4B Q5_K_M (ARA-Abliterated) does not fail. It builds a usable script.

The weakness is strategic in nature. According to the Judge, the analysis preceding the script is too shallow, the hook too generic, and the Easter egg too explicitly explained rather than elegantly hidden. You can sense the capacity for execution, but not always the final editorial refinement. The model works like someone who understood the brief and delivers cleanly. What is occasionally missing is the creative precision that turns functionally good into something memorable.

UX Writing shows a similar picture. The model can convert toxic or inappropriate language into usable, professional German. In an HR-adjacent rewriting task it removes problematic terms cleanly, remains grammatically correct, and hits a professional tone. However, the word choice loses some traction. Where the reference text targets semantically with “Fachkraft” and “Tatkraft,” the model opts for softer, more generic terms like “engagierte Persönlichkeit.” That is not wrong. It is just less smart. Language is not damaged here, but neither is it elevated.

Documentation Quality: Solid, with a Tendency Toward Breadth Over Edge

Documentation Quality sits at 68.44 points. This fits the model’s general signature: it can structure, explain, and execute, but tends to produce serviceable completeness rather than brilliant precision. At an average of 3,360 output tokens it sits somewhat above the fleet median of 2,821 — not excessive, but noticeably more verbose.

For documentation tasks, this is not automatically a disadvantage. Anyone generating runbooks, explanatory texts, or internal knowledge artifacts often benefits more from completeness than from stylistic sharpness. The flip side is that the model does not always deliver the clearest condensation in longer text tasks. It tends to explain just a touch more than strictly necessary. This stays within acceptable bounds, but signals: this is not an ascetic documentation scalpel — it is a reliable writer with a tendency toward over-explanation.

Cultural Intelligence: Surprisingly Secure in Tone, Not Always Maximally Idiomatic

With 77.6 points, Cultural Intelligence is one of the more pleasant surprises. Uncensored derivatives in particular often lose composure first in finer social or language-cultural tasks. That does not happen here. The model remains polite, professional, and culturally accessible.

The HR example is instructive. The model removes toxic or martial terms, maintains inclusivity, and writes correct German. The Judge does not criticize gross cultural missteps, but nuances: “Persönlichkeit” instead of “Fachkraft,” missing “Tatkraft,” a slightly less market-oriented tone. That is criticism at a high level. The text is not embarrassing, not skewed, not culturally blind. It is simply not the best version of itself.

For a model carrying the tags General, Instruct, and Uncensored, this is a strong signal. It shows that the abliteration has not translated into social coarseness here. The model knows how to behave. It just does not always choose the perfect formulation.

Data Privacy and Data Sovereignty

No external cloud provider is involved, as this is a local Open Weights model. What matters therefore is the provenance of the weights: the calculated Sovereign Risk is MEDIUM. The reason is not the runtime operation on the test system, but the origin and modification chain of the weights. The base model comes from Google DeepMind under US jurisdiction; the variant used here was subsequently modified by ARA-APEX via Adaptive Refusal Abliteration. For European organizations this means: local inference massively reduces the classic data exfiltration risk, but the traceability of the weight provenance is less clean than with an unmodified original model.

Conclusion

Gemma 4 ARA 26B-A4B Q5_K_M (ARA-Abliterated) is one of those models that is easy to underestimate. On paper it is a Workstation generalist with MoE architecture, 25.2 billion total parameters, and only 4.0 billion active path per token. In practice it is a surprisingly capable all-rounder with strong CLI and tool affinity, solid security fundamentals, decent reasoning, and a pleasingly disciplined token economy. Its 54.58 tokens per second make it interactive enough on the test system without unnecessarily straining the 24 GB memory limit. The absence of timeouts is almost more important than any headline number.

The weaknesses are clearly nameable. In security, the final eye for critical chains and hidden escalation paths is sometimes missing. In reasoning, there is no substantive embarrassment, but an unnecessary tendency to violate format or language requirements. In text tasks the model is frequently good, but rarely brilliant. It delivers. It does not always elevate. That is precisely why it is well suited for day-to-day operational work, technical assistance, initial audit passes, CLI-adjacent agents, and structured transformation tasks. For highly critical security reviews, precise German-language compliance workflows, or linguistically polished external communications, a human or a more specialized model should remain the final authority.

Across all tests, no noteworthy hallucinations — the model prefers to produce insufficient depth rather than embarrass itself with nonsense. On provenance: locally pleasantly self-contained, but carrying a traceable medium risk through the community abliteration. This is not a showboat. It is a tool. And as a tool it is quite good, as long as you know where the edge ends.

This evaluation was generated automatically based on the benchmark data. Model used: GPT 4.5 by OpenAI. The raw data and full methodology are documented in the GitHub project.