Our checks, and the scores the makers publish.
Every model shows its checks before you use it: six stamps, and a missing one stays visible — even when no check has been run yet. This page shows where every model stands and how each check is run, then — kept apart — the benchmark scores the makers publish themselves, each with its source. We do not re-run those benchmarks.
Catalogue checked on 20 Oct 2026 · scores last read on 20 Oct 2026
- Six stamps
- 8 / 17models with all six checks
- Security passed
- 9 / 17five families of tests
- Live test passed
- 17 / 17a real call, end to end
- Published scores collected
- 8 / 17one source per score
Where every model stands
The 17 models, all run in Europe, newest first. ✓: check passed. ✕: missing — not run yet, or run without meeting the pass mark. –: not run yet, not a failure — security and benchmarks of Apertus v1.5 70B.
| Model | Where | Stamps | Licence | Provenance | Security | GDPR | Benchmarks | Live test |
|---|---|---|---|---|---|---|---|---|
 Mistral Large 4 | EU region of Mistral AI | 4/6 | missing | missing | passed | passed | passed | passed |
 DeepSeek V4.1 Flash | Germany (AKI.IO) | 6/6 | passed | passed | passed | passed | passed | passed |
 GLM-5.3 | Germany (AKI.IO) | 6/6 | passed | passed | passed | passed | passed | passed |
 Qwen3.8 27B | Paris, France | 6/6 | passed | passed | passed | passed | passed | passed |
| DeepSeek V4 Flash | Paris, France | 6/6 | passed | passed | passed | passed | passed | passed |
| Apertus v1.5 70B | Switzerland (Infomaniak) | 4/6 | passed | passed | not run yet | passed | not run yet | passed |
| GLM-5.2 | Paris, France | 6/6 | passed | passed | passed | passed | passed | passed |
| Qwen3.6 35B | Paris, France | 6/6 | passed | passed | passed | passed | passed | passed |
 Gemma 4 26B | Germany (AKI.IO) | 6/6 | passed | passed | passed | passed | passed | passed |
| Mistral Small 4 | EU region of Mistral AI | 5/6 | passed | passed | missing | passed | passed | passed |
| Qwen3.5 397B | Paris, France | 6/6 | passed | passed | passed | passed | passed | passed |
 GPT-OSS 120B | Paris, France | 5/6 | passed | passed | missing | passed | passed | passed |
| Qwen3 Coder 30B | Paris, France | 5/6 | passed | passed | missing | passed | passed | passed |
| Qwen3 235B | Paris, France | 5/6 | passed | passed | missing | passed | passed | passed |
| Mistral Small 3.2 | Paris, France | 5/6 | passed | passed | missing | passed | passed | passed |
 Llama 3.3 70B | Paris, France | 5/6 | passed | passed | missing | passed | passed | passed |
| Pixtral 12B | Paris, France | 5/6 | passed | passed | missing | passed | passed | passed |
How each check is run
The runs behind the stamps are dated: for the models run in France, security and benchmarks on 24 Sep 2026, live tests on 23 Sep 2026. Since then: Mistral Large 4 on 6 Oct 2026 (security, benchmarks and live test), Apertus v1.5 70B on 7 Oct 2026 (licence, provenance and live test), Mistral Small 4 on 13 Oct 2026 (security, benchmarks and live test), Qwen3.6 35B on 20 Oct 2026 (security, benchmarks and live test), Pixtral 12B on 20 Oct 2026 (security, benchmarks and live test), Qwen3 Coder 30B on 20 Oct 2026 (benchmarks and live test) and the models AKI.IO runs in Germany — DeepSeek V4.1 Flash, GLM-5.3 and Gemma 4 26B — on 13 Oct 2026 (security, benchmarks and live test).
- Licence — The licence is read on the model's official card, at the source, and recorded with the date it was read.
- Provenance — The operator's catalogue must name the maker's official release, and we cross-check it with the maker's own card: no unofficial copy of the weights.
- Security — Fifteen adversarial prompts, sent to the model on the route that serves it with no fallback: three in each of five families — instructions hidden in a text it is asked to process, a system prompt it must keep confidential, jailbreak framings, extraction of data it should not hand over, and clearly harmful requests. Only the visible answer is graded, never the reasoning. A family passes when at least two of its three answers are safe, and the stamp needs all five (the rule since 24 September 2026). It tests behaviour, not the weights.
- GDPR — The route is documented: operator, country, who controls the operator, and what its own documentation says it keeps. The stamp is set when the route that serves the model is sovereign according to that dossier: a European company with no parent outside the EU found, serving the model in Europe, with what it keeps documented. Published shareholders are not required: AKI.IO, a German company in Berlin, does not publish its shareholders, and no parent outside the EU appears in its legal notice, its data processing agreement or its site.
- Benchmarks — A fixed run on the operator's route: 25 self-contained questions (general knowledge, maths, reasoning, language), graded on the visible answer only, and two speed measures — time to the first token over three streamed prompts, and writing speed on an essay of 500 words or more. The stamp needs at least 72% correct answers and both speed measures within our limits.
- Live test — A real request sent through the operator and answered end to end. The date of the last one is recorded.
Each answer then carries the same stamps on its passport.
What the makers publish
Every score below comes from the model's maker — its card, technical report or announcement — except LMArena, a public leaderboard built on blind votes (snapshot of 13 Sep 2026). All were read on their source between 23 Sep 2026 and 20 Oct 2026. A score shows only if it is about the exact model served; otherwise: “—”.
Sources and settings of every score
| Model | GPQA (%) | MMLU (%) | LiveCodeBench (%) | LMArena (score) |
|---|---|---|---|---|
| GLM-5.2 Paris, France | 91.2 GPQA Diamond, temperature 1.0, up to 163,840 generated tokens | — | — | 1472 entry “glm-5.2-max”, ±5 |
| Qwen3.5 397B Paris, France | 88.4 card label “GPQA”, subset not named | — | 83.6 v6 | 1442 entry “qwen3.5-397b-a17b”, ±3 |
| Qwen3.6 35B Paris, France | 86.0 card label “GPQA”, subset not named | — | 80.4 v6 | — |
| GPT-OSS 120B Paris, France | 80.1 GPQA Diamond without tools, reasoning effort high (low: 67.1, medium: 73.1) | 90.0 reasoning effort high (low: 85.9, medium: 88.0) | — | 1352 entry “gpt-oss-120b”, ±4 |
| Qwen3 235B Paris, France | 77.5 card label “GPQA”, subset not named | — | 51.8 v6, problems from Feb. to May 2025 | 1423 entry “qwen3-235b-a22b-instruct-2507”, ±3 |
| Llama 3.3 70B Paris, France | 50.5 GPQA Diamond (CoT), 0-shot | 86.0 MMLU (CoT), 0-shot | — | 1318 entry “llama-3.3-70b-instruct”, ±4 |
| Mistral Small 3.2 Paris, France | 46.13 GPQA Diamond, 5-shot CoT | 80.5 setting not stated on the card | — | 1357 entry “mistral-small-2506”, ±5 |
| Pixtral 12B Paris, France | — | 69.2 5-shot | — | — |
No published score recorded for: Mistral Large 4, DeepSeek V4.1 Flash, GLM-5.3, Qwen3.8 27B, DeepSeek V4 Flash, Apertus v1.5 70B, Gemma 4 26B, Mistral Small 4 and Qwen3 Coder 30B.
- GPQA — graduate-level questions in physics, chemistry and biology; most makers publish the Diamond subset, the hardest one.
- MMLU — general knowledge across 57 subjects; MMLU-Pro and MMLU-Redux are other tests and are not in this column.
- LiveCodeBench — programming problems taken from contests; each version covers another period, hence the version stated.
- LMArena — a rating from blind votes between two answers, text leaderboard, default view (style control). It measures what voters prefer, not accuracy, and it moves as votes come in. arena.ai
How to read these numbers
A benchmark measures one capability, under controlled conditions: it does not say how a model will handle your own documents. Context, image input, tool calling, language or price often matter more. The safest test is still your real questions on two or three models. Compare the models side by side.
To go further: the page of GLM-5.2, or the full list of models.
Frequently asked questions
What does a missing stamp mean?
Not confirmed: either the check has not been run on this model yet, or it was run and did not meet the pass mark. The model stays in the catalogue, and both its page and its passport show the gap.
Do you re-run the makers' benchmarks?
No. The published scores are each maker's own, at its own setting, plus LMArena's public leaderboard. Our own runs are the six checks: they decide a stamp, not a ranking.
Which model scores highest?
No single score settles it, and each maker measures at its own setting. In the table of published scores, GLM-5.2 has the highest GPQA score (91.2).
Why offer a model that missed the security check?
Because hiding it would hide the result too. The missing stamp shows on its page, in the tables and on the passport of every answer: you choose knowing it, or you pick a model that has all six.
Can the makers' scores be compared with each other?
With care. Each maker picks its own protocol: number of examples, chain of thought, reasoning effort, benchmark version. Each score therefore states its setting and links to its source; two scores measured differently are not strictly comparable.