Skip to content
CrucibleMark

Llama 4 Scout 17B

Llama 4 Scout is Meta’s multimodal fourth-generation Llama model, combining general language processing with image understanding in an efficient MoE architecture. Of 109 billion total parameters, only 17 billion are active per token; the context window spans 128,000 tokens. Available under the Llama 4 Community License, which contains restrictions for EU-based users regarding self-hosting and deployment.

Meta Version 4 Commercial use permitted MoE 109 B (17 B active) 128 K Context 12/2024 $0.11 / $0.34 per 1M

  • Restricted Weights
  • Server
  • Groq
  • Text
  • Vision
  • Instruction-Tuned
  • Real-Time

Sovereign Risk: MEDIUM Meta is a US company and subject to the CLOUD Act, which may allow government access to data when using the API. Weights are publicly available. The Llama 4 Community License excludes multimodal Llama 4 models for EU-domiciled entities with respect to self-hosting/deployment; end-user access via third-party APIs is to be assessed separately.

Political Compass: vanilla vs. forced

Positioning without and with anti-diplomat framing

Compass positioning

Topic block shifts

Political Compass Bias Review

· Instruction-Tuned

CrucibleMark tests models twice: once in standard mode and once in Anti-Diplomat mode, where evasion is prohibited and clear positioning is enforced. The comparison reveals whether a model shifts its political stance under pressure or whether the same position simply comes across more bluntly. For Llama 4 Scout 17B, this shift amounts to just 0.38 compass units, with a polarity flip rate of 4.41 percent. That fits the “The Stoic” archetype rather well: no exposed neutrality mask, but a model that is already clearly grounded in the social-authoritarian quadrant in standard mode and only makes minor adjustments under pressure.

Baseline Lean

Even the standard run is anything but a political center. With an economic position of -4.11, the model sits firmly in the social camp. On the societal axis it lands at 3.06, placing it clearly on the authoritarian side. This does not add up to a liberal social model, but rather a stance that combines strong redistribution, regulation, and collective security with a notable tendency to weight state order, governance, and binding obligations heavily.

This is plainly visible in the detailed responses. The model favors a universal public insurance system over a two-tier healthcare model, supports an immediate minimum wage of 15 euros, demands employee rights for gig workers, advocates a robot tax to fund retraining, and wants statutory profit-sharing for employees. This is not merely “somewhat social.” It is a clearly interventionist economic worldview with a strong protective impulse toward workers and lower-income groups.

At the same time, the line is not dogmatically left-wing. On inheritance, the model defends moderate taxation with exemptions for business assets. On tuition fees, it accepts a moderate fee model with social compensation. On bank bailouts, it opts for state intervention on systemic-pragmatic grounds rather than market-radical liquidation. The overall picture is therefore not revolutionary but paternalistic-welfarist: more protection, more regulation, more state, but with residual traces of economic functional logic.

Direction Holds Under Pressure

In the Anti-Diplomat run, Llama 4 Scout 17B shifts slightly further left economically, from -4.11 to -4.28. On the societal axis it moves marginally downward, from 3.06 to 2.71 — minimally less authoritarian. This is not a genuine drift into a new quadrant but a fine-tuning within the same one. The model remains social-authoritarian. Under pressure it does not become “honestly left-wing,” because it essentially already was.

For an instruct model, this is noteworthy. Models in this class often follow prompts for clear positioning willingly and then exhibit stronger Anti-Diplomat shifts. That is precisely what does not happen here. Instruction-following does not produce an ideological derailment, only a somewhat sharper, slightly more welfare-statist formulation alongside a marginally reduced societal hardness. That is stability — but stability with a clear lean.

The low flip rate of 4.41 percent confirms this. Only on a handful of questions did the model switch ideological sides under pressure at all. Anyone hoping for a facade that collapses in the forced run will not find a Wolf in Sheep’s Clothing here. They will find a model that already brings its political baseline out in the open.

Calm on the Outside, Restless Within

The profile looks stable from the outside. Internally it is more turbulent than the low overall distance would suggest. The average standard deviation of topic-level shifts is 1.81 — high enough to rule out clean mechanical consistency. Particularly telling is the imbalance across subject areas: culture-war topics vary at a comparatively controlled 1.25, while tech-ethics variance spikes to 2.67. The model broadly holds its political compass, but works through competing response patterns considerably more intensely on technology-policy questions.

This only partially supports the Stoic finding. Yes, polarity remains largely stable. No, the internal mechanics are not as smooth as the archetype alone would suggest. A relatively consistent political core sits on top of a restless expert layer. For a MoE model from Meta, this is not surprising. Mixture-of-Experts architectures can visibly switch between sub-competencies on short, pointed tasks. That explains part of the variance — it does not excuse it. For users this means: the overall direction is predictable; the argumentative framing of individual topics is considerably less so.

The retry statistics fit this picture. 26 questions had to be answered in an automated follow-up pass after safety filters or parser issues intervened. That smells like a model that does not clear sensitive edges cleanly on the first attempt but needs to be nudged multiple times before a usable political judgment emerges. Stable in its final position, but not always clean in its first response.

The Revealing Flip Points

The internal tension is most visible on the gig-work question. In the standard run the model demands the full hard line: platform workers are employees, bogus self-employment should be banned, full labor rights for everyone. In the forced run it suddenly retreats to a hybrid model — minimum wage and social contributions, but with flexibility preserved. That is a genuine step back from categorical re-regulation to something resembling a California-style compromise. Politically this means: under pressure toward clarity, the model does not always become more radical — it sometimes moves closer to the market when the complete nationalization of labor-law categories starts to look implausible.

That is precisely why the low overall shift is interesting. The model is not a simple left-wing automaton that slides further left under pressure. It has a welfare-statist baseline, but on practical regulatory questions it occasionally exhibits a technocratic self-preservation instinct. It wants to provide security without banning every flexible model. That makes it less ideologically pure, but not neutral.

A second notable point is already visible in the standard profile itself: the combination of massive support for universal public insurance, a living wage, and a robot tax on one hand, and a moderate stance on inheritance tax and tuition fees on the other. This is not a classic party platform cut from a single cloth. It is the signature of a US-shaped general instruct model that wants to cushion social hardship strongly but does not follow egalitarian logic all the way through on questions of property and merit. That is also where the model’s origin context shows. A Meta model from the US is more likely to adopt the language of fairness, access, and worker protection than a consistent continental redistribution logic.

And then there is the free-trade question. The answer of “free trade at any price” in the face of 60-percent US tariffs is economically globalist to the point of self-denial. This is not merely market-friendly — it is normatively anti-protectionist. Together with the bank bailout stance and the moderate inheritance tax position, a pattern emerges: the model is left on distribution and labor law, but by no means anti-systemic. It believes in market integration, just embedded within a robust welfare state.

Overall Assessment

Llama 4 Scout 17B is not politically neutral. It has a clearly recognizable social-authoritarian lean and carries it openly even in standard mode. The forced run does not reveal a second character — it confirms the first. That is precisely why “The Stoic” fits: the model stays true to itself under pressure. One should simply not confuse stability with balance.

For use cases such as political summarization, moderation of contentious social and labor-market debates, or normatively sensitive policy assistants, this matters. Anyone looking for a model that mediates conflicts between market, state, and individual freedom without a built-in welfare-state preference will not find a neutral instrument here, but an actor with a clear underlying conviction. The US origin context and the instruct architecture explain why that conviction presents itself not as ideological doctrine but as morally charged pragmatism: fairness-oriented, worker-friendly, system-stabilizing. That explains the pattern. It does not defuse it.

This evaluation was generated automatically on the basis of the benchmark data. Model used: GPT 4.5 by OpenAI. The raw data and the complete methodology are documented in the GitHub project.