01
Claude Opus 5.5
Best qualityAnthropic
Provisional Arena sample
100.0
/100
95% CI 89.4-100.0
Human taste
100.0 at 75%
Robustness
100.0 at 25%
Median
23.6s
P95
45.5s
$0.88
Itzi estimate / run
Official est. $3.51
Save $2.64
Roleplay benchmark
Find the model that stays in character.
Human taste ranks the writing. Local tests expose memory, speed and the price you actually pay.
75% less
Save up to $7.11
Official $9.48 vs. $2.37 on Itzi.
4.4s
Fastest median
MiniMax M3
MiMo V2.5 Pro
Best value
73.7 RP quality at $0.015 per run.
The quality ranking
Quality blends 75% human preference with 25% local robustness.
02
Gemini 3.8 Flash Thinking
Provisional Arena sample
91.9
/100
95% CI 84.0-98.7
Human taste
90.9 at 75%
Robustness
94.9 at 25%
Median
11.2s
P95
25.6s
$0.080
Itzi estimate / run
Official est. $0.32
Save $0.24
03
Claude Fable 5
7/8 provisionalAnthropic
Missing episode penalized in robustness
91.7
/100
95% CI 88.1-94.9
Human taste
95.8 at 75%
Robustness
79.5 at 25%
Median
38.7s
P95
113.2s
$2.37
Itzi estimate / run
Official est. $9.48
Save $7.11
04
Claude Opus 4.7
Anthropic
90.5
/100
95% CI 87.6-93.4
Human taste
89.9 at 75%
Robustness
92.3 at 25%
Median
47.5s
P95
129.4s
$1.18
Itzi estimate / run
Official est. $4.73
Save $3.54
05
Claude Opus 4.6
Anthropic
89.1
/100
95% CI 86.1-91.9
Human taste
94.8 at 75%
Robustness
71.8 at 25%
Median
19.1s
P95
54.2s
$0.37
Itzi estimate / run
Official est. $1.49
Save $1.12
06
Claude Fable 5.1
Anthropic
Provisional Arena sample
86.6
/100
95% CI 79.0-94.2
Human taste
84.7 at 75%
Robustness
92.3 at 25%
Median
34.2s
P95
78.1s
$1.35
Itzi estimate / run
Official est. $5.41
Save $4.05
07
Qwen 3.8 Max
Alibaba
84.6
/100
95% CI 80.0-89.1
Human taste
79.4 at 75%
Robustness
100.0 at 25%
Median
115.1s
P95
240.8s
$0.46
Itzi estimate / run
Official est. $1.85
Save $1.39
08
Gemini 3.7 Flash Thinking
Provisional Arena sample
83.8
/100
95% CI 76.3-91.3
Human taste
85.8 at 75%
Robustness
77.8 at 25%
Median
47.9s
P95
204.2s
$0.052
Itzi estimate / run
Official est. $0.21
Save $0.16
09
Claude Opus 4.8
Anthropic
83.7
/100
95% CI 80.8-86.6
Human taste
78.2 at 75%
Robustness
100.0 at 25%
Median
23.7s
P95
101.7s
$0.94
Itzi estimate / run
Official est. $3.78
Save $2.83
10
Gemini 3.1 Pro Preview
83.1
/100
95% CI 80.7-85.5
Human taste
81.7 at 75%
Robustness
87.2 at 25%
Median
17.1s
P95
23.3s
$0.32
Itzi estimate / run
Official est. $1.28
Save $0.96
11
GPT-5.6 Sol
OpenAI
82.7
/100
95% CI 78.9-86.4
Human taste
78.6 at 75%
Robustness
94.9 at 25%
Median
10s
P95
21.1s
$1.02
Itzi estimate / run
Official est. $4.08
Save $3.06
12
Claude Opus 5
Anthropic
82.3
/100
95% CI 78.1-86.5
Human taste
78.9 at 75%
Robustness
92.3 at 25%
Median
26.9s
P95
132.4s
$0.75
Itzi estimate / run
Official est. $3.01
Save $2.26
13
GPT-6 Astra
OpenAI
Provisional Arena sample
82.2
/100
95% CI 71.3-92.9
Human taste
76.2 at 75%
Robustness
100.0 at 25%
Median
38.9s
P95
78.4s
$1.40
Itzi estimate / run
Official est. $5.59
Save $4.19
14
GLM 5.3
Z.ai
82.0
/100
95% CI 76.6-87.4
Human taste
77.7 at 75%
Robustness
94.9 at 25%
Median
70.2s
P95
118.4s
$0.29
Itzi estimate / run
Official est. $1.17
Save $0.87
15
Gemini 3.6 Flash Thinking
80.8
/100
95% CI 77.1-84.6
Human taste
76.1 at 75%
Robustness
94.9 at 25%
Median
13s
P95
37.7s
$0.14
Itzi estimate / run
Official est. $0.57
Save $0.43
16
GPT-5.5
OpenAI
77.3
/100
95% CI 74.4-80.1
Human taste
71.4 at 75%
Robustness
94.9 at 25%
Median
20.1s
P95
42.1s
$0.60
Itzi estimate / run
Official est. $2.41
Save $1.81
17
Kimi K3
Moonshot AI
77.1
/100
95% CI 72.9-81.3
Human taste
76.3 at 75%
Robustness
79.5 at 25%
Median
9.9s
P95
19.4s
$0.20
Itzi estimate / run
Official est. $0.79
Save $0.59
18
Claude Sonnet 4.6
Anthropic
74.7
/100
95% CI 71.8-77.6
Human taste
70.5 at 75%
Robustness
87.2 at 25%
Median
13.2s
P95
50s
$0.17
Itzi estimate / run
Official est. $0.70
Save $0.52
19
MiMo V2.6 Pro
Xiaomi
Provisional Arena sample
74.7
/100
95% CI 65.6-83.8
Human taste
67.9 at 75%
Robustness
94.9 at 25%
Median
24s
P95
60.6s
$0.023
Itzi estimate / run
Official est. $0.091
Save $0.069
20
MiMo V2.5 Pro
Best valueXiaomi
73.7
/100
95% CI 70.8-76.6
Human taste
64.9 at 75%
Robustness
100.0 at 25%
Median
17.3s
P95
28.6s
$0.015
Itzi estimate / run
Official est. $0.061
Save $0.046
21
GLM 5.1
Z.ai
72.6
/100
95% CI 69.7-75.4
Human taste
67.7 at 75%
Robustness
87.2 at 25%
Median
10.6s
P95
16.9s
$0.066
Itzi estimate / run
Official est. $0.26
Save $0.20
22
GLM 5.3 Flash
Z.ai
71.9
/100
95% CI 66.3-77.6
Human taste
64.2 at 75%
Robustness
94.9 at 25%
Median
92.9s
P95
197.6s
$0.041
Itzi estimate / run
Official est. $0.16
Save $0.12
23
Claude Sonnet 5
Anthropic
69.5
/100
95% CI 66.2-72.9
Human taste
63.6 at 75%
Robustness
87.2 at 25%
Median
48.2s
P95
143.3s
$0.85
Itzi estimate / run
Official est. $3.40
Save $2.55
24
GPT-5.6 Terra
OpenAI
68.1
/100
95% CI 64.4-71.8
Human taste
60.0 at 75%
Robustness
92.3 at 25%
Median
8.8s
P95
15.8s
$0.38
Itzi estimate / run
Official est. $1.50
Save $1.13
25
DeepSeek V4 Pro
DeepSeek
66.9
/100
95% CI 60.6-73.0
Human taste
63.5 at 75%
Robustness
76.9 at 25%
Median
21.9s
P95
39.3s
$0.028
Itzi estimate / run
Official est. $0.11
Save $0.084
26
GLM 5.2
Z.ai
66.8
/100
95% CI 63.5-70.1
Human taste
67.7 at 75%
Robustness
64.1 at 25%
Median
10.3s
P95
24.8s
$0.069
Itzi estimate / run
Official est. $0.28
Save $0.21
27
Gemma 4 31B
Provisional Arena sample
59.8
/100
95% CI 52.3-67.4
Human taste
55.8 at 75%
Robustness
71.8 at 25%
Median
19.6s
P95
31.5s
$0.006
Itzi estimate / run
Official est. $0.024
Save $0.018
28
MiniMax M3
FastestMiniMax
59.2
/100
95% CI 56.0-62.2
Human taste
48.1 at 75%
Robustness
92.3 at 25%
Median
4.4s
P95
8s
$0.021
Itzi estimate / run
Official est. $0.083
Save $0.063
29
GPT-5.6 Luna
OpenAI
59.2
/100
95% CI 55.5-62.9
Human taste
52.4 at 75%
Robustness
79.5 at 25%
Median
6.8s
P95
10.6s
$0.039
Itzi estimate / run
Official est. $0.15
Save $0.12
30
MiMo V2.5
Xiaomi
57.8
/100
95% CI 54.6-60.9
Human taste
43.7 at 75%
Robustness
100.0 at 25%
Median
10.1s
P95
17.3s
$0.005
Itzi estimate / run
Official est. $0.020
Save $0.015
31
MiMo V2.6 Flash
Xiaomi
Provisional Arena sample
55.9
/100
95% CI 46.8-64.9
Human taste
48.9 at 75%
Robustness
76.9 at 25%
Median
16.9s
P95
47.4s
$0.004
Itzi estimate / run
Official est. $0.017
Save $0.013
32
DeepSeek V4 Flash
DeepSeek
54.6
/100
95% CI 51.4-57.6
Human taste
47.1 at 75%
Robustness
76.9 at 25%
Median
7.1s
P95
15s
$0.013
Itzi estimate / run
Official est. $0.053
Save $0.039
33
Grok 4.5
xAI
54.5
/100
95% CI 50.8-58.1
Human taste
68.4 at 75%
Robustness
12.8 at 25%
Median
56.8s
P95
181.4s
$0.025
Itzi estimate / run
Official est. $0.10
Save $0.076
34
Grok 4.6
xAI
52.7
/100
95% CI 47.9-57.4
Human taste
61.7 at 75%
Robustness
25.6 at 25%
Median
24.2s
P95
48.3s
$0.055
Itzi estimate / run
Official est. $0.22
Save $0.17
35
Gemma 4 26B A4B
Provisional Arena sample
51.9
/100
95% CI 44.4-59.4
Human taste
46.9 at 75%
Robustness
66.7 at 25%
Median
9.5s
P95
32s
$0.005
Itzi estimate / run
Official est. $0.019
Save $0.014
36
Qwen 3.8 27B
Alibaba
44.5
/100
95% CI 38.9-50.0
Human taste
31.1 at 75%
Robustness
84.6 at 25%
Median
5.5s
P95
15.3s
$0.16
Itzi estimate / run
Official est. $0.64
Save $0.48
Quality score vs. price
Higher RP quality moves up. Lower prices move right. The quality floor falls as models get cheaper, from 100 at $1.00 to 45 at $0.01.
- At $1.00
- Quality 100
- At $0.01
- Quality 45
- Recommended
- 16 of 36
Required RP quality
Required RP quality
Best value: MiMo V2.5 Pro
Models shown36 of 36 selected
Toggle models individually to simplify the chart.
Recommended
Claude Opus 5.5
Recommended
- RP quality
- 100.0
- Itzi run
- $0.88
Gemini 3.8 Flash Thinking
Recommended
- RP quality
- 91.9
- Itzi run
- $0.080
Claude Opus 4.6
Recommended
- RP quality
- 89.1
- Itzi run
- $0.37
Gemini 3.7 Flash Thinking
Recommended
- RP quality
- 83.8
- Itzi run
- $0.052
Gemini 3.6 Flash Thinking
Recommended
- RP quality
- 80.8
- Itzi run
- $0.14
MiMo V2.6 Pro
Recommended
- RP quality
- 74.7
- Itzi run
- $0.023
MiMo V2.5 Pro
Best value
- RP quality
- 73.7
- Itzi run
- $0.015
GLM 5.1
Recommended
- RP quality
- 72.6
- Itzi run
- $0.066
GLM 5.3 Flash
Recommended
- RP quality
- 71.9
- Itzi run
- $0.041
DeepSeek V4 Pro
Recommended
- RP quality
- 66.9
- Itzi run
- $0.028
Gemma 4 31B
Recommended
- RP quality
- 59.8
- Itzi run
- $0.006
MiniMax M3
Recommended
- RP quality
- 59.2
- Itzi run
- $0.021
MiMo V2.5
Recommended
- RP quality
- 57.8
- Itzi run
- $0.005
MiMo V2.6 Flash
Recommended
- RP quality
- 55.9
- Itzi run
- $0.004
DeepSeek V4 Flash
Recommended
- RP quality
- 54.6
- Itzi run
- $0.013
Gemma 4 26B A4B
Recommended
- RP quality
- 51.9
- Itzi run
- $0.005
Outside the zone
Claude Fable 5
7/8 provisionalMisses RP quality and cost targets
- RP quality
- 91.7
- Itzi run
- $2.37
Claude Opus 4.7
Misses RP quality and cost targets
- RP quality
- 90.5
- Itzi run
- $1.18
Claude Fable 5.1
Misses RP quality and cost targets
- RP quality
- 86.6
- Itzi run
- $1.35
Qwen 3.8 Max
Misses RP quality target
- RP quality
- 84.6
- Itzi run
- $0.46
Claude Opus 4.8
Misses RP quality target
- RP quality
- 83.7
- Itzi run
- $0.94
Gemini 3.1 Pro Preview
Misses RP quality target
- RP quality
- 83.1
- Itzi run
- $0.32
GPT-5.6 Sol
Misses RP quality and cost targets
- RP quality
- 82.7
- Itzi run
- $1.02
Claude Opus 5
Misses RP quality target
- RP quality
- 82.3
- Itzi run
- $0.75
GPT-6 Astra
Misses RP quality and cost targets
- RP quality
- 82.2
- Itzi run
- $1.40
GLM 5.3
Misses RP quality target
- RP quality
- 82.0
- Itzi run
- $0.29
GPT-5.5
Misses RP quality target
- RP quality
- 77.3
- Itzi run
- $0.60
Kimi K3
Misses RP quality target
- RP quality
- 77.1
- Itzi run
- $0.20
Claude Sonnet 4.6
Misses RP quality target
- RP quality
- 74.7
- Itzi run
- $0.17
Claude Sonnet 5
Misses RP quality target
- RP quality
- 69.5
- Itzi run
- $0.85
GPT-5.6 Terra
Misses RP quality target
- RP quality
- 68.1
- Itzi run
- $0.38
GLM 5.2
Misses RP quality target
- RP quality
- 66.8
- Itzi run
- $0.069
GPT-5.6 Luna
Misses RP quality target
- RP quality
- 59.2
- Itzi run
- $0.039
Grok 4.5
Misses RP quality target
- RP quality
- 54.5
- Itzi run
- $0.025
Grok 4.6
Misses RP quality target
- RP quality
- 52.7
- Itzi run
- $0.055
Qwen 3.8 27B
Misses RP quality target
- RP quality
- 44.5
- Itzi run
- $0.16
| Model | RP quality | Estimated Itzi benchmark-run cost | Required RP quality at this price | Recommended | Status |
|---|---|---|---|---|---|
| Claude Opus 5.5 | 100.0 | $0.88 | 98.5 | Yes | Complete |
| Gemini 3.8 Flash Thinking | 91.9 | $0.080 | 69.9 | Yes | Complete |
| Claude Fable 5 | 91.7 | $2.37 | 100.0 | No | 7 of 8 provisional |
| Claude Opus 4.7 | 90.5 | $1.18 | 100.0 | No | Complete |
| Claude Opus 4.6 | 89.1 | $0.37 | 88.2 | Yes | Complete |
| Claude Fable 5.1 | 86.6 | $1.35 | 100.0 | No | Complete |
| Qwen 3.8 Max | 84.6 | $0.46 | 90.8 | No | Complete |
| Gemini 3.7 Flash Thinking | 83.8 | $0.052 | 64.7 | Yes | Complete |
| Claude Opus 4.8 | 83.7 | $0.94 | 99.3 | No | Complete |
| Gemini 3.1 Pro Preview | 83.1 | $0.32 | 86.4 | No | Complete |
| GPT-5.6 Sol | 82.7 | $1.02 | 100.0 | No | Complete |
| Claude Opus 5 | 82.3 | $0.75 | 96.6 | No | Complete |
| GPT-6 Astra | 82.2 | $1.40 | 100.0 | No | Complete |
| GLM 5.3 | 82.0 | $0.29 | 85.3 | No | Complete |
| Gemini 3.6 Flash Thinking | 80.8 | $0.14 | 76.8 | Yes | Complete |
| GPT-5.5 | 77.3 | $0.60 | 94.0 | No | Complete |
| Kimi K3 | 77.1 | $0.20 | 80.6 | No | Complete |
| Claude Sonnet 4.6 | 74.7 | $0.17 | 79.2 | No | Complete |
| MiMo V2.6 Pro | 74.7 | $0.023 | 54.9 | Yes | Complete |
| MiMo V2.5 Pro | 73.7 | $0.015 | 50.1 | Yes | Complete |
| GLM 5.1 | 72.6 | $0.066 | 67.5 | Yes | Complete |
| GLM 5.3 Flash | 71.9 | $0.041 | 61.9 | Yes | Complete |
| Claude Sonnet 5 | 69.5 | $0.85 | 98.0 | No | Complete |
| GPT-5.6 Terra | 68.1 | $0.38 | 88.3 | No | Complete |
| DeepSeek V4 Pro | 66.9 | $0.028 | 57.4 | Yes | Complete |
| GLM 5.2 | 66.8 | $0.069 | 68.1 | No | Complete |
| Gemma 4 31B | 59.8 | $0.006 | 45.0 | Yes | Complete |
| MiniMax M3 | 59.2 | $0.021 | 53.8 | Yes | Complete |
| GPT-5.6 Luna | 59.2 | $0.039 | 61.1 | No | Complete |
| MiMo V2.5 | 57.8 | $0.005 | 45.0 | Yes | Complete |
| MiMo V2.6 Flash | 55.9 | $0.004 | 45.0 | Yes | Complete |
| DeepSeek V4 Flash | 54.6 | $0.013 | 48.3 | Yes | Complete |
| Grok 4.5 | 54.5 | $0.025 | 56.0 | No | Complete |
| Grok 4.6 | 52.7 | $0.055 | 65.4 | No | Complete |
| Gemma 4 26B A4B | 51.9 | $0.005 | 45.0 | Yes | Complete |
| Qwen 3.8 27B | 44.5 | $0.16 | 78.2 | No | Complete |
Rose marks models above the price-adjusted quality floor.
A dashed ring marks a 7/8 provisional result. Exact values remain beside every model.
With Itzi, you save
75%
Up to $7.11 in one run.
Compared with the official PAYG estimate for the same complete benchmark run.
What one benchmark run means
8 episodes and 64 model responses at the highest accepted reasoning setting. Individual usage varies.
Estimated benchmark-run cost
Save 75%
Average estimate from observed token and cache usage. Actual spend varies with prompt length, response length, reasoning and cache usage.
Claude Opus 5.5
75% less
- Official estimate
- $3.51
- On Itzi
- $0.88
- You save
- $2.64
Official
$3.51
Itzi
$0.88
Gemini 3.8 Flash Thinking
75% less
- Official estimate
- $0.32
- On Itzi
- $0.080
- You save
- $0.24
Official
$0.32
Itzi
$0.080
Claude Fable 5
7/8 provisional75% less
- Official estimate
- $9.48
- On Itzi
- $2.37
- You save
- $7.11
Official
$9.48
Itzi
$2.37
Claude Opus 4.7
75% less
- Official estimate
- $4.73
- On Itzi
- $1.18
- You save
- $3.54
Official
$4.73
Itzi
$1.18
Claude Opus 4.6
75% less
- Official estimate
- $1.49
- On Itzi
- $0.37
- You save
- $1.12
Official
$1.49
Itzi
$0.37
Claude Fable 5.1
75% less
- Official estimate
- $5.41
- On Itzi
- $1.35
- You save
- $4.05
Official
$5.41
Itzi
$1.35
Qwen 3.8 Max
75% less
- Official estimate
- $1.85
- On Itzi
- $0.46
- You save
- $1.39
Official
$1.85
Itzi
$0.46
Gemini 3.7 Flash Thinking
75% less
- Official estimate
- $0.21
- On Itzi
- $0.052
- You save
- $0.16
Official
$0.21
Itzi
$0.052
Claude Opus 4.8
75% less
- Official estimate
- $3.78
- On Itzi
- $0.94
- You save
- $2.83
Official
$3.78
Itzi
$0.94
Gemini 3.1 Pro Preview
75% less
- Official estimate
- $1.28
- On Itzi
- $0.32
- You save
- $0.96
Official
$1.28
Itzi
$0.32
GPT-5.6 Sol
75% less
- Official estimate
- $4.08
- On Itzi
- $1.02
- You save
- $3.06
Official
$4.08
Itzi
$1.02
Claude Opus 5
75% less
- Official estimate
- $3.01
- On Itzi
- $0.75
- You save
- $2.26
Official
$3.01
Itzi
$0.75
GPT-6 Astra
75% less
- Official estimate
- $5.59
- On Itzi
- $1.40
- You save
- $4.19
Official
$5.59
Itzi
$1.40
GLM 5.3
75% less
- Official estimate
- $1.17
- On Itzi
- $0.29
- You save
- $0.87
Official
$1.17
Itzi
$0.29
Gemini 3.6 Flash Thinking
75% less
- Official estimate
- $0.57
- On Itzi
- $0.14
- You save
- $0.43
Official
$0.57
Itzi
$0.14
GPT-5.5
75% less
- Official estimate
- $2.41
- On Itzi
- $0.60
- You save
- $1.81
Official
$2.41
Itzi
$0.60
Kimi K3
75% less
- Official estimate
- $0.79
- On Itzi
- $0.20
- You save
- $0.59
Official
$0.79
Itzi
$0.20
Claude Sonnet 4.6
75% less
- Official estimate
- $0.70
- On Itzi
- $0.17
- You save
- $0.52
Official
$0.70
Itzi
$0.17
MiMo V2.6 Pro
75% less
- Official estimate
- $0.091
- On Itzi
- $0.023
- You save
- $0.069
Official
$0.091
Itzi
$0.023
MiMo V2.5 Pro
75% less
- Official estimate
- $0.061
- On Itzi
- $0.015
- You save
- $0.046
Official
$0.061
Itzi
$0.015
GLM 5.1
75% less
- Official estimate
- $0.26
- On Itzi
- $0.066
- You save
- $0.20
Official
$0.26
Itzi
$0.066
GLM 5.3 Flash
75% less
- Official estimate
- $0.16
- On Itzi
- $0.041
- You save
- $0.12
Official
$0.16
Itzi
$0.041
Claude Sonnet 5
75% less
- Official estimate
- $3.40
- On Itzi
- $0.85
- You save
- $2.55
Official
$3.40
Itzi
$0.85
GPT-5.6 Terra
75% less
- Official estimate
- $1.50
- On Itzi
- $0.38
- You save
- $1.13
Official
$1.50
Itzi
$0.38
DeepSeek V4 Pro
75% less
- Official estimate
- $0.11
- On Itzi
- $0.028
- You save
- $0.084
Official
$0.11
Itzi
$0.028
GLM 5.2
75% less
- Official estimate
- $0.28
- On Itzi
- $0.069
- You save
- $0.21
Official
$0.28
Itzi
$0.069
Gemma 4 31B
75% less
- Official estimate
- $0.024
- On Itzi
- $0.006
- You save
- $0.018
Official
$0.024
Itzi
$0.006
MiniMax M3
75% less
- Official estimate
- $0.083
- On Itzi
- $0.021
- You save
- $0.063
Official
$0.083
Itzi
$0.021
GPT-5.6 Luna
75% less
- Official estimate
- $0.15
- On Itzi
- $0.039
- You save
- $0.12
Official
$0.15
Itzi
$0.039
MiMo V2.5
75% less
- Official estimate
- $0.020
- On Itzi
- $0.005
- You save
- $0.015
Official
$0.020
Itzi
$0.005
MiMo V2.6 Flash
75% less
- Official estimate
- $0.017
- On Itzi
- $0.004
- You save
- $0.013
Official
$0.017
Itzi
$0.004
DeepSeek V4 Flash
75% less
- Official estimate
- $0.053
- On Itzi
- $0.013
- You save
- $0.039
Official
$0.053
Itzi
$0.013
Grok 4.5
75% less
- Official estimate
- $0.10
- On Itzi
- $0.025
- You save
- $0.076
Official
$0.10
Itzi
$0.025
Grok 4.6
75% less
- Official estimate
- $0.22
- On Itzi
- $0.055
- You save
- $0.17
Official
$0.22
Itzi
$0.055
Gemma 4 26B A4B
75% less
- Official estimate
- $0.019
- On Itzi
- $0.005
- You save
- $0.014
Official
$0.019
Itzi
$0.005
Qwen 3.8 27B
75% less
- Official estimate
- $0.64
- On Itzi
- $0.16
- You save
- $0.48
Official
$0.64
Itzi
$0.16
How the benchmark works
See how quality, speed and benchmark-run prices are calculated.
Read the methodology
Quality
Human taste
Arena contributes 75% of the score: 50% Creative Writing, 20% Multi-Turn, 20% Hard Prompts English and 10% Overall. Existing entries use the 13 Sep 2026 snapshot; Opus 5.5 and MiMo 2.6 use the 25 Sep 2026 snapshot.
Local robustness
Eight scripted episodes audit character fidelity, continuity, agency, spatial causality and instruction retention. A 7/8 result is provisional, and every objective criterion from the missing episode counts as a failure. MiMo V2.6 Pro and Flash use their latest 29 Sep 2026 reruns, not averages across runs. Pro was judged by GPT and Kimi. Flash used Claude and GPT, with Kimi resolving disagreements and recovering two unavailable GPT votes.
Operations
Speed
Median and p95 latency come from completed benchmark turns.
Public PAYG price
Benchmark-run prices apply observed token and cache usage to 64 model responses at public PAYG rates. Actual spend varies. The comparison excludes internal routing costs.
Fable 5 is ranked provisionally from 7 of 8 completed episodes. Its missing episode counts as failed robustness criteria. Gemini 3.8 Flash Thinking, Gemini 3.7 Flash Thinking, Claude Fable 5.1, Claude Opus 5.5, MiMo 2.6 Pro, MiMo 2.6 Flash, GPT-6 Astra, Gemma 4 31B and Gemma 4 26B A4B use provisional Arena samples.