How to read benchmarks · Guide

Text and reasoning

For writing, analysis and reasoning, I look at the Arena text board with the rank range, at HLE when I want a knowledge ceiling, and at a test of my own in Portuguese. MMLU, MMLU-Pro and GPQA stay out of a choice among frontier models: they have already saturated at the top, and a gap of 1 to 2 points is usually noise. In Qwen's table, Qwen3.8-Max scores 92.6 on GPQA Diamond, in the same cut where the rivals on that board sit between 92.0 and 94.1.

On the Arena text board with style control, in the latest version we checked, Gemini 4 Carbon (High) led with an Elo of 1526. Qwen3.8-Max sat 23rd, at 1484, with a range from 13 to 37, in the same statistical pack as dozens of models. On the same board, GLM-5.3 (Max) came in at 1483 under an MIT license, and Qwen3.8-27B, Apache 2.0, at 1433.

Two caveats complete that reading. None of the major rankings measures Brazilian Portuguese in a central way; the test that matters most, for my use, is with text and tasks from here. And the list price per million tokens does not close the bill. Qwen3.8-max, on Model Studio in Singapore, lists at US$2 for input and US$6 for output per million. In a one-off citation from Artificial Analysis, on Index v5, Qwen3.8 Max (0902) comes out at US$10.02 per task and takes 56.8 minutes, with 114 seconds to the first token in the 0902 setup. Source: Artificial Analysis (artificialanalysis.ai). That cost depends on the provider and the configuration they used; a low price per token, with a lot of reasoning, can still make the work expensive.

There is also the episode of an index that changes scale. In the Apsara release, Alibaba said Qwen3.8-Max had moved from 40 to 45 on Artificial Analysis. Qwen3.8 Max (0902) sat at 34 on the current Index (v5). Source: Artificial Analysis (artificialanalysis.ai). The change of Index version is enough to refuse any "Index" figure cited without the version next to it.

Open source board

GPQA Diamond, best run per model. Source: Epoch AI, CC BY 4.0 license.

GPQA Diamond, best run per model. Source: Epoch AI, CC BY 4.0 license.
Model Value Lab Measured by Source
Gemini 4 Argon 96.4% Google DeepMind Epoch AI Source
GPT-6 Astra 95.8% OpenAI Epoch AI Source
Claude Sonnet 5.5 95.6% Anthropic Epoch AI Source
Gemini 3.8 Flash 95.4% Google DeepMind Epoch AI Source
Gemini 3.7 Flash 94.8% Google DeepMind Epoch AI Source
GPT-5.4 Pro 94.6% OpenAI Epoch AI Source
Gemini 3.1 Pro 94.4% Google DeepMind Epoch AI Source
Kimi K3.1 Pro 94.2% Moonshot Epoch AI Source
Claude Opus 5 93.9% Anthropic Epoch AI Source
GPT-5.4 93.3% OpenAI Epoch AI Source
DeepSeek V4.1 Flash 1017 93.2% DeepSeek Epoch AI Source
Kimi K3 93.1% Moonshot Epoch AI Source
Qwen 3.8 Max 92.7% Alibaba Epoch AI Source
Gemini 3 Pro 92.6% Google DeepMind Epoch AI Source
Qwen3.8 Max (0902) 92.3% Alibaba Epoch AI Source
GLM-5.2 91.9% Z.ai (Zhipu AI) Epoch AI Source
DeepSeek V4 Pro 0813 91.7% DeepSeek Epoch AI Source
GPT-5.2 91.4% OpenAI Epoch AI Source
Claude Opus 4.8 91% Anthropic Epoch AI Source
DeepSeek V4 Flash 0731 91% DeepSeek Epoch AI Source
DeepSeek-V4-Pro 90.9% DeepSeek Epoch AI Source
GLM-5.3 90.9% Z.ai (Zhipu AI) Epoch AI Source
Qwen3.7-Max 90.9% Alibaba Epoch AI Source
Kimi K2.6 90.8% Moonshot Epoch AI Source
Claude Opus 4.6 90.5% Anthropic Epoch AI Source
GLM-5.3-Flash 90.2% Z.ai (Zhipu AI) Epoch AI Source
GLM-5.1 89.9% Z.ai (Zhipu AI) Epoch AI Source
DeepSeek V4.1 Flash 89.8% DeepSeek Epoch AI Source
Qwen 3.6 Plus 88.4% Alibaba Epoch AI Source
Kimi K2.7 Code 87.9% Moonshot Epoch AI Source
GLM-5 87.8% Z.ai (Zhipu AI) Epoch AI Source
GPT-5.1 87.6% OpenAI Epoch AI Source
Kimi K2.5 87.6% Moonshot Epoch AI Source
Qwen 3.6 Max (Preview) 87.4% Alibaba Epoch AI Source
Qwen3.5 397B-A17B 86.4% Alibaba Epoch AI Source
GPT-5 86.2% OpenAI Epoch AI Source
Claude Opus 4.5 86% Anthropic Epoch AI Source
Gemini 2.5 Pro (Jun 2025) 85.3% Google DeepMind Epoch AI Source
Kimi K2 Thinking 84.2% Moonshot Epoch AI Source
Gemini 2.5 Pro (Mar 2025) 83.8% Google DeepMind Epoch AI Source
DeepSeek-V3.2 83.4% DeepSeek Epoch AI Source
GLM-4.7 83.3% Z.ai (Zhipu AI) Epoch AI Source
Claude Sonnet 4.5 82.3% Anthropic Epoch AI Source
o3 81.8% OpenAI Epoch AI Source
Qwen3-235B-A22B (Jul 2025) 80.1% Alibaba Epoch AI Source
Claude 3.7 Sonnet 79.7% Anthropic Epoch AI Source
o4-mini 79.6% OpenAI Epoch AI Source
o3-mini 77% OpenAI Epoch AI Source
o1 76.8% OpenAI Epoch AI Source
DeepSeek-R1 (May 2025) 76.3% DeepSeek Epoch AI Source
Qwen3-Max 72.6% Alibaba Epoch AI Source
DeepSeek-R1 71.7% DeepSeek Epoch AI Source
Qwen3-235B-A22B 70.7% Alibaba Epoch AI Source
DeepSeek-V3 (Mar 2025) 67.6% DeepSeek Epoch AI Source
Gemini 2.0 Pro 65.7% Google DeepMind Epoch AI Source
Qwen3-32B 65.7% Alibaba Epoch AI Source
QWQ-Plus 65.4% Alibaba Epoch AI Source
QwQ-32B 65.3% Alibaba Epoch AI Source
DeepSeek-R1-Distill-Qwen-32B 64.1% DeepSeek Epoch AI Source
o1-mini 62.4% OpenAI Epoch AI Source
Gemini 1.5 Pro 57.2% Google DeepMind Epoch AI Source
DeepSeek-V3 56.5% DeepSeek Epoch AI Source
Qwen2.5-Max 56.1% Alibaba Epoch AI Source
Claude 3.5 Sonnet (October 2024) 55.3% Anthropic Epoch AI Source
Claude 3.5 Sonnet 54% Anthropic Epoch AI Source
o1-preview 50.3% OpenAI Epoch AI Source
GPT-4o 49.2% OpenAI Epoch AI Source
Qwen2.5-72B 49.1% Alibaba Epoch AI Source
Gemini 1.5 Flash 47.3% Google DeepMind Epoch AI Source
Claude 3 Opus 47.2% Anthropic Epoch AI Source
GPT-4 Turbo (Apr 2024) 46.6% OpenAI Epoch AI Source
Qwen2.5-32B 46.1% Alibaba Epoch AI Source
Qwen2-72B 40.8% Alibaba Epoch AI Source
Claude 3 Sonnet 40.6% Anthropic Epoch AI Source
Gemma 2 27B 36.5% Google DeepMind Epoch AI Source
Qwen1.5-32B 30.7% Alibaba Epoch AI Source
Qwen1.5-72B 28.8% Alibaba Epoch AI Source
Gemma 2 9B 27.5% Google DeepMind Epoch AI Source