Roleplay benchmark
Human taste ranks the writing. Local tests expose memory, speed and the price you actually pay.
75% less
Save up to $7.11
Official $9.48 vs. $2.37 on Itzi.
6.8s
Fastest median
GPT-5.6 Luna
Gemini 3.7 Flash Thinking
Best value
84.0 RP quality at $0.052 per run.
Quality blends 75% human preference with 25% local robustness.
01
Anthropic
Missing episode penalized in robustness
91.6
/100
95% CI 87.9-94.9
Human taste
95.6 at 75%
Robustness
79.5 at 25%
Median
38.7s
P95
113.2s
$2.37
Itzi estimate / run
Official est. $9.48
Save $7.11
02
Anthropic
90.5
/100
95% CI 87.6-93.4
Human taste
89.9 at 75%
Robustness
92.3 at 25%
Median
47.5s
P95
129.4s
$1.18
Itzi estimate / run
Official est. $4.73
Save $3.54
03
Anthropic
89.1
/100
95% CI 86.1-91.9
Human taste
94.8 at 75%
Robustness
71.8 at 25%
Median
19.1s
P95
54.2s
$0.37
Itzi estimate / run
Official est. $1.49
Save $1.12
04
Provisional Arena sample
84.0
/100
95% CI 76.5-91.5
Human taste
86.0 at 75%
Robustness
77.8 at 25%
Median
47.9s
P95
204.2s
$0.052
Itzi estimate / run
Official est. $0.21
Save $0.16
05
OpenAI
83.6
/100
95% CI 79.5-87.6
Human taste
79.8 at 75%
Robustness
94.9 at 25%
Median
10s
P95
21.1s
$1.02
Itzi estimate / run
Official est. $4.08
Save $3.06
06
Anthropic
83.3
/100
95% CI 80.4-86.2
Human taste
77.7 at 75%
Robustness
100.0 at 25%
Median
23.7s
P95
101.7s
$0.94
Itzi estimate / run
Official est. $3.78
Save $2.83
07
Z.ai
83.1
/100
95% CI 76.6-89.4
Human taste
79.1 at 75%
Robustness
94.9 at 25%
Median
70.2s
P95
118.4s
$0.29
Itzi estimate / run
Official est. $1.17
Save $0.87
08
82.8
/100
95% CI 80.4-85.2
Human taste
81.3 at 75%
Robustness
87.2 at 25%
Median
17.1s
P95
23.3s
$0.32
Itzi estimate / run
Official est. $1.28
Save $0.96
09
Anthropic
81.6
/100
95% CI 77.0-86.2
Human taste
78.1 at 75%
Robustness
92.3 at 25%
Median
26.9s
P95
132.4s
$0.75
Itzi estimate / run
Official est. $3.01
Save $2.26
10
81.0
/100
95% CI 77.0-85.0
Human taste
76.4 at 75%
Robustness
94.9 at 25%
Median
13s
P95
37.7s
$0.14
Itzi estimate / run
Official est. $0.57
Save $0.43
11
OpenAI
77.9
/100
95% CI 75.0-80.7
Human taste
72.2 at 75%
Robustness
94.9 at 25%
Median
20.1s
P95
42.1s
$0.60
Itzi estimate / run
Official est. $2.41
Save $1.81
12
Moonshot AI
77.8
/100
95% CI 73.3-82.3
Human taste
77.2 at 75%
Robustness
79.5 at 25%
Median
9.9s
P95
19.4s
$0.20
Itzi estimate / run
Official est. $0.79
Save $0.59
13
Anthropic
74.5
/100
95% CI 71.6-77.3
Human taste
70.2 at 75%
Robustness
87.2 at 25%
Median
13.2s
P95
50s
$0.17
Itzi estimate / run
Official est. $0.70
Save $0.52
14
Z.ai
72.5
/100
95% CI 69.5-75.4
Human taste
67.6 at 75%
Robustness
87.2 at 25%
Median
10.6s
P95
16.9s
$0.066
Itzi estimate / run
Official est. $0.26
Save $0.20
15
Z.ai
Provisional Arena sample
71.6
/100
95% CI 63.3-79.8
Human taste
63.8 at 75%
Robustness
94.9 at 25%
Median
92.9s
P95
197.6s
$0.041
Itzi estimate / run
Official est. $0.16
Save $0.12
16
Anthropic
69.6
/100
95% CI 66.0-73.2
Human taste
63.7 at 75%
Robustness
87.2 at 25%
Median
48.2s
P95
143.3s
$0.85
Itzi estimate / run
Official est. $3.40
Save $2.55
17
OpenAI
68.0
/100
95% CI 64.2-71.8
Human taste
59.9 at 75%
Robustness
92.3 at 25%
Median
8.8s
P95
15.8s
$0.38
Itzi estimate / run
Official est. $1.50
Save $1.13
18
Z.ai
67.0
/100
95% CI 63.7-70.4
Human taste
68.0 at 75%
Robustness
64.1 at 25%
Median
10.3s
P95
24.8s
$0.069
Itzi estimate / run
Official est. $0.28
Save $0.21
19
DeepSeek
Provisional Arena sample
64.7
/100
95% CI 57.1-72.3
Human taste
60.6 at 75%
Robustness
76.9 at 25%
Median
21.9s
P95
39.3s
$0.028
Itzi estimate / run
Official est. $0.11
Save $0.084
20
OpenAI
58.6
/100
95% CI 54.5-62.6
Human taste
51.6 at 75%
Robustness
79.5 at 25%
Median
6.8s
P95
10.6s
$0.039
Itzi estimate / run
Official est. $0.15
Save $0.12
21
xAI
54.8
/100
95% CI 51.0-58.6
Human taste
68.8 at 75%
Robustness
12.8 at 25%
Median
56.8s
P95
181.4s
$0.025
Itzi estimate / run
Official est. $0.10
Save $0.076
22
DeepSeek
54.8
/100
95% CI 51.6-57.9
Human taste
47.4 at 75%
Robustness
76.9 at 25%
Median
7.1s
P95
15s
$0.013
Itzi estimate / run
Official est. $0.053
Save $0.039
23
xAI
Provisional Arena sample
54.6
/100
95% CI 45.0-64.2
Human taste
64.3 at 75%
Robustness
25.6 at 25%
Median
24.2s
P95
48.3s
$0.055
Itzi estimate / run
Official est. $0.22
Save $0.17
Higher RP quality moves up. Lower prices move right. The quality floor falls as models get cheaper, from 100 at $1.00 to 45 at $0.01.
Required RP quality
Required RP quality
Best value: Gemini 3.7 Flash Thinking
Recommended
Best value
Recommended
Recommended
Recommended
Recommended
Recommended
Misses RP quality and cost targets
Misses RP quality and cost targets
Misses RP quality and cost targets
Misses RP quality target
Misses RP quality target
Misses RP quality target
Misses RP quality target
Misses RP quality target
Misses RP quality target
Misses RP quality target
Misses RP quality target
Misses RP quality target
Misses RP quality target
Misses RP quality target
Misses RP quality target
Misses RP quality target
Rose marks models above the price-adjusted quality floor.
A dashed ring marks a 7/8 provisional result. Exact values remain beside every model.
| Model | RP quality | Estimated Itzi benchmark-run cost | Required RP quality at this price | Recommended | Status |
|---|---|---|---|---|---|
| Claude Fable 5 | 91.6 | $2.37 | 100.0 | No | 7 of 8 provisional |
| Claude Opus 4.7 | 90.5 | $1.18 | 100.0 | No | Complete |
| Claude Opus 4.6 | 89.1 | $0.37 | 88.2 | Yes | Complete |
| Gemini 3.7 Flash Thinking | 84.0 | $0.052 | 64.7 | Yes | Complete |
| GPT-5.6 Sol | 83.6 | $1.02 | 100.0 | No | Complete |
| Claude Opus 4.8 | 83.3 | $0.94 | 99.3 | No | Complete |
| GLM 5.3 | 83.1 | $0.29 | 85.3 | No | Complete |
| Gemini 3.1 Pro Preview | 82.8 | $0.32 | 86.4 | No | Complete |
| Claude Opus 5 | 81.6 | $0.75 | 96.6 | No | Complete |
| Gemini 3.6 Flash Thinking | 81.0 | $0.14 | 76.8 | Yes | Complete |
| GPT-5.5 | 77.9 | $0.60 | 94.0 | No | Complete |
| Kimi K3 | 77.8 | $0.20 | 80.6 | No | Complete |
| Claude Sonnet 4.6 | 74.5 | $0.17 | 79.2 | No | Complete |
| GLM 5.1 | 72.5 | $0.066 | 67.5 | Yes | Complete |
| GLM 5.3 Flash | 71.6 | $0.041 | 61.9 | Yes | Complete |
| Claude Sonnet 5 | 69.6 | $0.85 | 98.0 | No | Complete |
| GPT-5.6 Terra | 68.0 | $0.38 | 88.3 | No | Complete |
| GLM 5.2 | 67.0 | $0.069 | 68.1 | No | Complete |
| DeepSeek V4 Pro | 64.7 | $0.028 | 57.4 | Yes | Complete |
| GPT-5.6 Luna | 58.6 | $0.039 | 61.1 | No | Complete |
| Grok 4.5 | 54.8 | $0.025 | 56.0 | No | Complete |
| DeepSeek V4 Flash | 54.8 | $0.013 | 48.3 | Yes | Complete |
| Grok 4.6 | 54.6 | $0.055 | 65.4 | No | Complete |
With Itzi, you save
75%
Compared with the official PAYG estimate for the same complete benchmark run.
What one benchmark run means
8 episodes and 64 model responses at maximum reasoning. Individual usage varies.
Estimated benchmark-run cost
Save 75%
Average estimate from observed token and cache usage. Actual spend varies with prompt length, response length, reasoning and cache usage.
75% less
75% less
75% less
75% less
75% less
75% less
75% less
75% less
75% less
75% less
75% less
75% less
75% less
75% less
75% less
75% less
75% less
75% less
75% less
75% less
75% less
75% less
75% less
See how quality, speed and benchmark-run prices are calculated.
Arena contributes 75% of the score: 50% Creative Writing, 20% Multi-Turn, 20% Hard Prompts English and 10% Overall from a snapshot captured across 1-2 Sep 2026.
Eight scripted episodes audit character fidelity, continuity, agency, spatial causality and instruction retention. A 7/8 result is provisional, and every objective criterion from the missing episode counts as a failure.
Median and p95 latency come from completed benchmark turns.
Benchmark-run prices apply observed token and cache usage to 64 model responses at public PAYG rates. Actual spend varies. The comparison excludes internal routing costs.
Fable 5 is ranked provisionally from 7 of 8 completed episodes. Its missing episode counts as failed robustness criteria. GLM 5.3, GLM 5.3 Flash, DeepSeek V4 Pro and Grok 4.6 use provisional Arena samples.