Paleo Bench

55 model configurations, 7 Byzantine Greek folios

Run 24 October 2026

By folio

  • Scores out of 100, one per folio
  • Circled: under 50; failed: timeout, token limit or request error
  • Highlighted: 1st and 2nd on each folio by character error
  • Median: middle score among models that ran the folio
  • Select a score to open its transcription, a heading to sort
ModelColour imagesMicrofilm
Gemini 4 Argon (multi-step)9999969699999998
Gemini 4 Argon (high)9999949699999998
Gemini 4 Argon (low)9998949699999697
Gemini 3 Flash (low)9999899399889795
Gemini 3 Pro (high)9898919496929595
Gemini 3.8 Flash (low)9897909398949495
Opus 5.5 (high)9797919499909495
Gemini 3.5 Flash (low)9899879297929694
Gemini 3 Pro (low)9897909399849494
Gemini 3.5 Flash (high)9896909497859594
Gemini 3 Flash (high)9893909297869293
Gemini 3.1 Pro (low)9696779299899592
Gemini 3.1 Pro (high)9694779299829491
Opus 5.5 (low)9998879598896590
GPT-6.1 Sol (high)9894failed9397failed9785 of 7 run
Sonnet 5.5 (low)9797829196362375
Opus 4.6 (thinking)8896667994512972
Sonnet 5.5 (high)9798469196551872
Opus 4.79797619294273171
Muse Spark Open9598548790343871
Opus 4.7 (thinking)9897609292232970
Opus 4.69188617995423270
GPT-6.1 Sol (low)989864929331669
Sonnet 4.68493568083352765
Kimi K3 (no thinking)909762838834165
Kimi K3969748failed943215646 of 7 run
Kimi K2.6failed954886912828636 of 7 run
Haiku 5.5 (low)8895248480283161
Haiku 5.5 (high)9295328085162761
Sonnet 4.6 (thinking)7192556683322861
DeepSeek V4.1 Flash 10179495failed3752failed12585 of 7 run
Gemini 3.8 Flash (high)171791919718958
GPT-6 Astra (low)9894095980055
Qwen 3.6 Plus859507475232954
GPT-5.2 (low)89982870850053
GPT-5.2 (high)92964935768351
GPT-5.5 (low)229650532579046
DeepSeek V4.1 Flash 1017 (no thinking)859318227526046
Qwen 3.6 35B A3B4085225257262844
Grok 4.3598039495111041
GPT-4.14394324334261441
GPT-5.4 Mini (low)478645512727040
Qwen 3.8 27B397785256262440
GPT-5 Mini (low)375042485227337
Haiku 4.5484945354328736
GPT-5 Mini (high)48514240530033
Haiku 4.5 (thinking)495040332228633
Llama 4 Maverick313737045292729
Llama 4 Scout3129372644281029
Nemotron 3.5 Super VLfailed7failed4425failedfailed253 of 7 run
GLM 5.3 Flash71891341124
GPT-5.4 Mini (high)449101784023
Nemotron 3.5 Super VL (no thinking)036000005
GPT-6 Luna (low)701011203
GPT-6 Luna (high)20000501
Median9195508088292761
  • Score: 100 minus character error rate, rounded down, minimum 0
  • Scoring ignores accents, punctuation and capitals
  • Every sigma form counts as σ; ϑ, ϐ and similar variants as their usual letters
  • ϛ, ȣ and ϗ count as στ, ου and και
  • Latin or Cyrillic look-alikes inside Greek words count as Greek
  • Means skip failed folios
  • Ground truth keeps rubrics on GA L60 and GA L587 but omits a faded one on GA L2010
  • Time: summed request time, timeouts included
  • Cost: US dollars; OpenRouter figures estimate from listed token prices