RSS

How to read benchmarks

Evolution

Chinese families on GPQA Diamond, from Epoch, CC BY 4.0 license.

Chinese families on GPQA Diamond

Chinese families on GPQA Diamond, from Epoch, CC BY 4.0 license.

506070809010020252026saturation band above 90Qwen2.5-Max: 56.1%.Qwen3-235B-A22B: 70.7%.Qwen3-235B-A22B (Jul 2025): 80.1%.Qwen3-Max: 72.6%.Qwen3.5 397B-A17B: 86.4%.Qwen 3.6 Max (Preview): 87.4%.Qwen3.7-Max: 90.9%.Qwen 3.8 Max: 92.7%.Qwen3.8 Max (0902): 92.3%.DeepSeek-R1: 71.7%.DeepSeek-V3 (Mar 2025): 67.6%.DeepSeek-R1 (May 2025): 76.3%.DeepSeek-V3.2: 83.4%.DeepSeek-V4-Pro: 90.9%.DeepSeek V4 Flash 0731: 91%.DeepSeek V4 Pro 0813: 91.7%.DeepSeek V4.1 Flash: 89.8%.DeepSeek V4.1 Flash 1017: 93.2%.Kimi K2 Thinking: 84.2%.Kimi K2.5: 87.6%.Kimi K2.6: 90.8%.Kimi K2.7 Code: 87.9%.Kimi K3: 93.1%.Kimi K3.1 Pro: 94.2%.GLM-4.7: 83.3%.GLM-5: 87.8%.GLM-5.1: 89.9%.GLM-5.2: 91.9%.GLM-5.3: 90.9%.GLM-5.3-Flash: 90.2%.
  • ● Qwen (linha Max)
  • ● DeepSeek
  • ● Kimi (Moonshot)
  • ● GLM (Z.ai)

The axis marks the year only. Each point detail carries the model and the value. The table repeats the same numbers, and the link goes to the source.

Chart data table
GPQA Diamond, best run per model. Source: Epoch AI, CC BY 4.0 license.
Model Value Lab Measured by License Source
Qwen2.5-Max 56.1% Alibaba stale ● Epoch AI CC-BY-4.0 Source
Qwen3-235B-A22B 70.7% Alibaba stale ● Epoch AI CC-BY-4.0 Source
Qwen3-235B-A22B (Jul 2025) 80.1% Alibaba stale ● Epoch AI CC-BY-4.0 Source
Qwen3-Max 72.6% Alibaba stale ● Epoch AI CC-BY-4.0 Source
Qwen3.5 397B-A17B 86.4% Alibaba ● Epoch AI CC-BY-4.0 Source
Qwen 3.6 Max (Preview) 87.4% Alibaba ● Epoch AI CC-BY-4.0 Source
Qwen3.7-Max 90.9% Alibaba ● Epoch AI CC-BY-4.0 Source
Qwen 3.8 Max 92.7% Alibaba ● Epoch AI CC-BY-4.0 Source
Qwen3.8 Max (0902) 92.3% Alibaba ● Epoch AI CC-BY-4.0 Source
DeepSeek-R1 71.7% DeepSeek stale ● Epoch AI CC-BY-4.0 Source
DeepSeek-V3 (Mar 2025) 67.6% DeepSeek stale ● Epoch AI CC-BY-4.0 Source
DeepSeek-R1 (May 2025) 76.3% DeepSeek stale ● Epoch AI CC-BY-4.0 Source
DeepSeek-V3.2 83.4% DeepSeek stale ● Epoch AI CC-BY-4.0 Source
DeepSeek-V4-Pro 90.9% DeepSeek ● Epoch AI CC-BY-4.0 Source
DeepSeek V4 Flash 0731 91% DeepSeek ● Epoch AI CC-BY-4.0 Source
DeepSeek V4 Pro 0813 91.7% DeepSeek ● Epoch AI CC-BY-4.0 Source
DeepSeek V4.1 Flash 89.8% DeepSeek ● Epoch AI CC-BY-4.0 Source
DeepSeek V4.1 Flash 1017 93.2% DeepSeek ● Epoch AI CC-BY-4.0 Source
Kimi K2 Thinking 84.2% Moonshot stale ● Epoch AI CC-BY-4.0 Source
Kimi K2.5 87.6% Moonshot stale ● Epoch AI CC-BY-4.0 Source
Kimi K2.6 90.8% Moonshot stale ● Epoch AI CC-BY-4.0 Source
Kimi K2.7 Code 87.9% Moonshot ● Epoch AI CC-BY-4.0 Source
Kimi K3 93.1% Moonshot ● Epoch AI CC-BY-4.0 Source
Kimi K3.1 Pro 94.2% Moonshot ● Epoch AI CC-BY-4.0 Source
GLM-4.7 83.3% Z.ai (Zhipu AI) stale ● Epoch AI CC-BY-4.0 Source
GLM-5 87.8% Z.ai (Zhipu AI) stale ● Epoch AI CC-BY-4.0 Source
GLM-5.1 89.9% Z.ai (Zhipu AI) ● Epoch AI CC-BY-4.0 Source
GLM-5.2 91.9% Z.ai (Zhipu AI) stale ● Epoch AI CC-BY-4.0 Source
GLM-5.3 90.9% Z.ai (Zhipu AI) ● Epoch AI CC-BY-4.0 Source
GLM-5.3-Flash 90.2% Z.ai (Zhipu AI) ● Epoch AI CC-BY-4.0 Source

GPQA Diamond record by lab

Each lab record on the same test, as a step line, from Epoch, CC BY 4.0 license.

2030405060708090100202420252026saturation band above 90Qwen1.5-72B: 28.8%.Qwen1.5-32B: 30.7%.Qwen2-72B: 40.8%.Qwen2.5-32B: 46.1%.Qwen2.5-72B: 49.1%.Qwen2.5-Max: 56.1%.QwQ-32B: 65.3%.QWQ-Plus: 65.4%.Qwen3-235B-A22B: 70.7%.Qwen3-32B: 65.7%.Qwen3-235B-A22B (Jul 2025): 80.1%.Qwen3.5 397B-A17B: 86.4%.Qwen 3.6 Plus: 88.4%.Qwen3.7-Max: 90.9%.Qwen 3.8 Max: 92.7%.DeepSeek-V3: 56.5%.DeepSeek-R1: 71.7%.DeepSeek-R1-Distill-Qwen-32B: 64.1%.DeepSeek-R1 (May 2025): 76.3%.DeepSeek-V3.2: 83.4%.DeepSeek-V4-Pro: 90.9%.DeepSeek V4 Flash 0731: 91%.DeepSeek V4 Pro 0813: 91.7%.DeepSeek V4.1 Flash 1017: 93.2%.Kimi K2 Thinking: 84.2%.Kimi K2.5: 87.6%.Kimi K2.6: 90.8%.Kimi K3: 93.1%.Kimi K3.1 Pro: 94.2%.GLM-4.7: 83.3%.GLM-5: 87.8%.GLM-5.1: 89.9%.GLM-5.2: 91.9%.GPT-4 Turbo (Apr 2024): 46.6%.GPT-4o: 49.2%.o1-mini: 62.4%.o1-preview: 50.3%.o1: 76.8%.o3-mini: 77%.o3: 81.8%.o4-mini: 79.6%.GPT-5: 86.2%.GPT-5.1: 87.6%.GPT-5.2: 91.4%.GPT-5.4: 93.3%.GPT-5.4 Pro: 94.6%.GPT-6 Astra: 95.8%.Claude 3 Opus: 47.2%.Claude 3 Sonnet: 40.6%.Claude 3.5 Sonnet: 54%.Claude 3.5 Sonnet (October 2024): 55.3%.Claude 3.7 Sonnet: 79.7%.Claude Sonnet 4.5: 82.3%.Claude Opus 4.5: 86%.Claude Opus 4.6: 90.5%.Claude Opus 4.8: 91%.Claude Opus 5: 93.9%.Claude Sonnet 5.5: 95.6%.Gemma 2 27B: 36.5%.Gemma 2 9B: 27.5%.Gemini 1.5 Flash: 47.3%.Gemini 1.5 Pro: 57.2%.Gemini 2.0 Pro: 65.7%.Gemini 2.5 Pro (Mar 2025): 83.8%.Gemini 2.5 Pro (Jun 2025): 85.3%.Gemini 3 Pro: 92.6%.Gemini 3.1 Pro: 94.4%.Gemini 3.7 Flash: 94.8%.Gemini 3.8 Flash: 95.4%.Gemini 4 Argon: 96.4%.
  • ● Alibaba
  • ● DeepSeek
  • ● Moonshot
  • ● Z.ai (Zhipu AI)
  • ● OpenAI
  • ● Anthropic
  • ● Google DeepMind

The axis marks the year only. Each point detail carries the model and the value. The table repeats the same numbers, and the link goes to the source.

Chart data table
GPQA Diamond, best run per model. Source: Epoch AI, CC BY 4.0 license.
Model Value Lab Measured by License Source
Qwen1.5-72B 28.8% Alibaba stale ● Epoch AI CC-BY-4.0 Source
Qwen1.5-32B 30.7% Alibaba stale ● Epoch AI CC-BY-4.0 Source
Qwen2-72B 40.8% Alibaba stale ● Epoch AI CC-BY-4.0 Source
Qwen2.5-32B 46.1% Alibaba stale ● Epoch AI CC-BY-4.0 Source
Qwen2.5-72B 49.1% Alibaba stale ● Epoch AI CC-BY-4.0 Source
Qwen2.5-Max 56.1% Alibaba stale ● Epoch AI CC-BY-4.0 Source
QwQ-32B 65.3% Alibaba ● Epoch AI CC-BY-4.0 Source
QWQ-Plus 65.4% Alibaba stale ● Epoch AI CC-BY-4.0 Source
Qwen3-235B-A22B 70.7% Alibaba stale ● Epoch AI CC-BY-4.0 Source
Qwen3-32B 65.7% Alibaba ● Epoch AI CC-BY-4.0 Source
Qwen3-235B-A22B (Jul 2025) 80.1% Alibaba stale ● Epoch AI CC-BY-4.0 Source
Qwen3.5 397B-A17B 86.4% Alibaba ● Epoch AI CC-BY-4.0 Source
Qwen 3.6 Plus 88.4% Alibaba ● Epoch AI CC-BY-4.0 Source
Qwen3.7-Max 90.9% Alibaba ● Epoch AI CC-BY-4.0 Source
Qwen 3.8 Max 92.7% Alibaba ● Epoch AI CC-BY-4.0 Source
DeepSeek-V3 56.5% DeepSeek stale ● Epoch AI CC-BY-4.0 Source
DeepSeek-R1 71.7% DeepSeek stale ● Epoch AI CC-BY-4.0 Source
DeepSeek-R1-Distill-Qwen-32B 64.1% DeepSeek ● Epoch AI CC-BY-4.0 Source
DeepSeek-R1 (May 2025) 76.3% DeepSeek stale ● Epoch AI CC-BY-4.0 Source
DeepSeek-V3.2 83.4% DeepSeek stale ● Epoch AI CC-BY-4.0 Source
DeepSeek-V4-Pro 90.9% DeepSeek ● Epoch AI CC-BY-4.0 Source
DeepSeek V4 Flash 0731 91% DeepSeek ● Epoch AI CC-BY-4.0 Source
DeepSeek V4 Pro 0813 91.7% DeepSeek ● Epoch AI CC-BY-4.0 Source
DeepSeek V4.1 Flash 1017 93.2% DeepSeek ● Epoch AI CC-BY-4.0 Source
Kimi K2 Thinking 84.2% Moonshot stale ● Epoch AI CC-BY-4.0 Source
Kimi K2.5 87.6% Moonshot stale ● Epoch AI CC-BY-4.0 Source
Kimi K2.6 90.8% Moonshot stale ● Epoch AI CC-BY-4.0 Source
Kimi K3 93.1% Moonshot ● Epoch AI CC-BY-4.0 Source
Kimi K3.1 Pro 94.2% Moonshot ● Epoch AI CC-BY-4.0 Source
GLM-4.7 83.3% Z.ai (Zhipu AI) stale ● Epoch AI CC-BY-4.0 Source
GLM-5 87.8% Z.ai (Zhipu AI) stale ● Epoch AI CC-BY-4.0 Source
GLM-5.1 89.9% Z.ai (Zhipu AI) ● Epoch AI CC-BY-4.0 Source
GLM-5.2 91.9% Z.ai (Zhipu AI) stale ● Epoch AI CC-BY-4.0 Source
GPT-4 Turbo (Apr 2024) 46.6% OpenAI stale ● Epoch AI CC-BY-4.0 Source
GPT-4o 49.2% OpenAI stale ● Epoch AI CC-BY-4.0 Source
o1-mini 62.4% OpenAI stale ● Epoch AI CC-BY-4.0 Source
o1-preview 50.3% OpenAI stale ● Epoch AI CC-BY-4.0 Source
o1 76.8% OpenAI stale ● Epoch AI CC-BY-4.0 Source
o3-mini 77% OpenAI stale ● Epoch AI CC-BY-4.0 Source
o3 81.8% OpenAI stale ● Epoch AI CC-BY-4.0 Source
o4-mini 79.6% OpenAI stale ● Epoch AI CC-BY-4.0 Source
GPT-5 86.2% OpenAI stale ● Epoch AI CC-BY-4.0 Source
GPT-5.1 87.6% OpenAI stale ● Epoch AI CC-BY-4.0 Source
GPT-5.2 91.4% OpenAI stale ● Epoch AI CC-BY-4.0 Source
GPT-5.4 93.3% OpenAI stale ● Epoch AI CC-BY-4.0 Source
GPT-5.4 Pro 94.6% OpenAI stale ● Epoch AI CC-BY-4.0 Source
GPT-6 Astra 95.8% OpenAI ● Epoch AI CC-BY-4.0 Source
Claude 3 Opus 47.2% Anthropic stale ● Epoch AI CC-BY-4.0 Source
Claude 3 Sonnet 40.6% Anthropic stale ● Epoch AI CC-BY-4.0 Source
Claude 3.5 Sonnet 54% Anthropic stale ● Epoch AI CC-BY-4.0 Source
Claude 3.5 Sonnet (October 2024) 55.3% Anthropic stale ● Epoch AI CC-BY-4.0 Source
Claude 3.7 Sonnet 79.7% Anthropic stale ● Epoch AI CC-BY-4.0 Source
Claude Sonnet 4.5 82.3% Anthropic stale ● Epoch AI CC-BY-4.0 Source
Claude Opus 4.5 86% Anthropic stale ● Epoch AI CC-BY-4.0 Source
Claude Opus 4.6 90.5% Anthropic stale ● Epoch AI CC-BY-4.0 Source
Claude Opus 4.8 91% Anthropic stale ● Epoch AI CC-BY-4.0 Source
Claude Opus 5 93.9% Anthropic ● Epoch AI CC-BY-4.0 Source
Claude Sonnet 5.5 95.6% Anthropic ● Epoch AI CC-BY-4.0 Source
Gemma 2 27B 36.5% Google DeepMind stale ● Epoch AI CC-BY-4.0 Source
Gemma 2 9B 27.5% Google DeepMind stale ● Epoch AI CC-BY-4.0 Source
Gemini 1.5 Flash 47.3% Google DeepMind stale ● Epoch AI CC-BY-4.0 Source
Gemini 1.5 Pro 57.2% Google DeepMind stale ● Epoch AI CC-BY-4.0 Source
Gemini 2.0 Pro 65.7% Google DeepMind stale ● Epoch AI CC-BY-4.0 Source
Gemini 2.5 Pro (Mar 2025) 83.8% Google DeepMind stale ● Epoch AI CC-BY-4.0 Source
Gemini 2.5 Pro (Jun 2025) 85.3% Google DeepMind stale ● Epoch AI CC-BY-4.0 Source
Gemini 3 Pro 92.6% Google DeepMind stale ● Epoch AI CC-BY-4.0 Source
Gemini 3.1 Pro 94.4% Google DeepMind ● Epoch AI CC-BY-4.0 Source
Gemini 3.7 Flash 94.8% Google DeepMind ● Epoch AI CC-BY-4.0 Source
Gemini 3.8 Flash 95.4% Google DeepMind ● Epoch AI CC-BY-4.0 Source
Gemini 4 Argon 96.4% Google DeepMind ● Epoch AI CC-BY-4.0 Source

How to read these charts

  • Each point is that model best run in the Epoch export.
  • The band above 90 marks the saturation zone.
  • The table below repeats model, value, lab, who measured, license and the source link.

Full guide Qwen3.8 Max JSON CSV