Paleo Bench

55 model configurations, 7 Byzantine Greek folios

Run 24 October 2026

Image & transcription

1 of 55 by mean score
Character error
0.4%
Word error
2.8%
Cost
$0.043
Time
52 seconds
Output tokens
944
Model: Gemini 4 Argon (multi-step)
Folio: GA 2523, 190r, Romans 5:18–6:13
99

Scoring the transcription…

  • Red pen and margin deductions: word alignment against the human committee transcription, approximate
  • Circled score: 100 minus character error rate
  • Score: 100 minus character error rate, rounded down, minimum 0
  • Scoring ignores accents, punctuation and capitals
  • Every sigma form counts as σ; ϑ, ϐ and similar variants as their usual letters
  • ϛ, ȣ and ϗ count as στ, ου and και
  • Latin or Cyrillic look-alikes inside Greek words count as Greek
  • Means skip failed folios
  • Ground truth keeps rubrics on GA L60 and GA L587 but omits a faded one on GA L2010
  • Time: summed request time, timeouts included
  • Cost: US dollars; OpenRouter figures estimate from listed token prices