Can AI Read Cuneiform?

Source Library processes 16th–18th century printed manuscripts: Latin, German, Arabic, Hebrew, Sanskrit. Our AI pipeline reads page images with Gemini, transcribes the text, and translates it into English. But printed books are only 600 years old. Writing itself is 5,000 years old. What happens if we point the same pipeline at cuneiform?

We selected four tablets spanning 1,500 years of Mesopotamian history, wrote a custom OCR prompt, and asked Gemini 3 Flash to read them. The model correctly identified Law 196 of the Code of Hammurabi, independently detected a 2,500-year-old forgery — and also fabricated an entire Sumerian document with 0.95 confidence.

We ran eight experiments. Experiment 1 asked Gemini to produce scholarly ATF transliterations — the standard format used by Assyriologists. Experiment 2 asked a simpler question: can you just identify the individual cuneiform signs? Experiment 3 scaled up to 107 tablets and tested whether the model is reading from its training data or genuinely analyzing the photographs. Experiment 4 compared four Gemini models. Experiment 5 tested whether cropping and image preprocessing could help the model see better. Experiment 6 tested edge detection, raking light simulation, multi-pass prompting, and thinking models. Experiment 7 brought in Claude to test whether the ~15% F1 ceiling was Gemini-specific. Experiment 8 discovered that temperature settings were confounding all prior results.

How cuneiform works

Cuneiform is the oldest known writing system, invented in southern Mesopotamia around 3200 BCE. Scribes pressed a reed stylus into wet clay at different angles to make wedge-shaped impressions. The word “cuneiform” itself comes from the Latin cuneus (“wedge”). There are four basic wedge types: a vertical wedge, a horizontal wedge, a diagonal wedge, and a corner wedge called a Winkelhaken. Every cuneiform sign is a specific arrangement of these wedges.

Unlike an alphabet, where each letter represents a sound, cuneiform signs can work in multiple ways simultaneously. A single sign might be:

• A syllable: the sign an represents the syllable “an”
• A word (logogram): the same sign AN means “heaven” or “sky”
• A determinative: placed before a word to indicate its category — {d} before a name marks it as a god, {ki} marks a place

The same sign can have multiple readings depending on context. The script was used for over a dozen languages — Sumerian, Akkadian, Elamite, Hittite, Urartian — each assigning their own values to the signs. Reading cuneiform requires knowing which language the text is in, which period it comes from (sign forms evolved over 3,000 years), and often which genre of text you're looking at.

What transliteration means

When Assyriologists “read” a cuneiform tablet, they produce a transliteration — converting the wedge impressions into a standardized Roman-letter encoding called ATF (ASCII Transliteration Format). Here is what ATF looks like for a simple Sumerian administrative text:

@obverse
1. 2(disz) udu niga       2 fattened sheep
2. u4 7(disz)-kam          day 7
3. ki ab-ba-sa6-ga-ta      from Abbasaga
4. a-hu-we-er              (personal name) Ahuwer
@reverse
1. i3-dab5                 received
2. iti ezem-{d}nin-a-zu    month: Festival of Ninazu
3. mu en-mah-gal-an-na ... year: Enmahgalanna was installed...

Every convention carries meaning. The @obverse and @reverse markers indicate the physical face of the tablet. Curly-brace determinatives like {d} (divine name) aren't pronounced — they're a scribal classifier. Lowercase readings (udu, “sheep”) represent Sumerian syllabic values; UPPERCASE (LUGAL, “king”) represents Sumerian logograms in Akkadian text. Square brackets mark broken or missing signs: [x]. Half-brackets mark partially visible signs: ⸢x⸣.

This is what we asked Gemini to produce — not a freeform description of the tablet, but a precise ATF transliteration that could be compared sign-by-sign against the published scholarship in the Cuneiform Digital Library Initiative (CDLI), the central repository for cuneiform tablet data.

The test tablets

We selected four tablets from CDLI, each with published ATF transliterations (our ground truth) and high-quality photographs:

Each tablet was imported into Source Library as a “book” with one page per photograph. All experiments used Gemini 3 Flash Preview.

Experiment 1: ATF transliteration

Our first experiment asked Gemini to do what Assyriologists do: produce a full ATF transliteration from a tablet photograph. This is the hardest possible ask — it requires reading individual wedge impressions, knowing which signs they form, determining the correct reading in context, and encoding the result in a precise scholarly format.

We wrote a custom “Cuneiform OCR” prompt with structured metadata output (script type, period, genre, condition, confidence score) and compared each output line-by-line against CDLI's published ATF.

The sheep receipt (Ur III tablet)

The Ur III administrative tablet (P102318) was our “easy” test case — the kind of tablet we need AI to handle if cuneiform OCR is going to matter. It's a small, well-preserved clay tablet from Drehem (ancient Puzrish-Dagan), a livestock management center near Nippur, dating to the Third Dynasty of Ur (ca. 2100 BCE). Thousands of nearly identical tablets survive from this period, recording the daily flow of animals, grain, and other commodities through the Ur III state bureaucracy.

The photograph shows both faces of the tablet. The text is seven lines total. Here is what this tablet actually says, according to the published CDLI transliteration:

@obverse
1. 2(disz) udu niga          → 2 fattened sheep
2. u4 7(disz)-kam             → day 7
3. ki ab-ba-sa6-ga-ta         → from Abbasaga
4. a-hu-we-er                 → (received by) Ahuwer
@reverse
1. i3-dab5                    → received
2. iti ezem-{d}nin-a-zu       → month: Festival of Ninazu
3. mu en-mah-gal-an-na ...   → year: Enmahgalanna was installed...

It's a sheep receipt. Two fattened sheep, received by Ahuwer from Abbasaga on the 7th day of the Festival of Ninazu. Simple, clear, seven lines.

What Gemini produced

Gemini correctly identified the script (Neo-Sumerian), the language (Sumerian), the genre (administrative), and the period (Ur III, reign of Shu-Suen, ca. 2037–2029 BCE). It even noted the vertical crack running through both faces. Then it produced this transliteration:

@obverse
1. 1(barig) 1(ban2) {gesz}haszhur duru5  → fresh apples
2. ki-szum2-ma-ta                          → from the onion field
3. 1(barig) 3(ban2) {gesz}haszhur duru5  → fresh apples
4. e2-gal-sze3                             → to the palace
@reverse
1. de6-a                                   → delivered
2. giri3 lu2-{d}nin-szubur                → via Lu-Ninshubur
3. iti szu-numun                           → month: Sowing
4. mu {d}szu-{d}suen lugal                → year: Shu-Suen (became) king

The format is perfect. The ATF conventions are correct — @obverse, @reverse, determinatives, capacity measures, date formula. The Sumerian vocabulary is real and appropriate to an Ur III administrative context. It even added a scholarly note: “In Umma, apples were frequently intercropped in onion gardens.”

But the content is entirely fabricated.

Line-by-line comparison

Zero lines match. Not a single sign reading is correct. The real tablet records sheep (udu niga); the AI invented apples (haszhur duru5). The real tablet names Abbasaga and Ahuwer; the AI invented Lu-Ninshubur and an “onion field.” The real month is the Festival of Ninazu; the AI chose the Sowing month. Even the year name is wrong: the tablet dates to Enmahgalanna's installation, not Shu-Suen's accession.

What this tells us

This is a textbook hallucination, but a revealing one. The model knows exactly what Ur III administrative tablets look like. It knows the format (commodity, date, personnel, transaction verb, month name, year name). It knows the right kind of vocabulary (capacity measures, agricultural commodities, temple personnel). It can produce a document that would fool a non-specialist.

But it's generating from distributional knowledge, not reading the wedge impressions in this photograph. The model has internalized the statistical structure of Ur III administrative texts without developing the ability to visually discriminate the specific signs on this specific tablet. It reported 0.95 confidence.

The other three tablets

Code of Hammurabi: accurate reading of a famous text

Gemini identified this as Law 196 and produced a correct transliteration:

1. šum-ma a-wi-lum       → If a citizen
2. i-in                   → the eye
3. DUMU a-wi-lim          → of a fellow citizen
4. uh-tap-pí-id           → has destroyed,
5. i-in-šu                → his eye
6. ú-ha-⸢ap-pa-du⸣       → they shall destroy.

“If a citizen destroys the eye of a fellow citizen, they shall destroy his eye.”

The model correctly read the Akkadian vocabulary, identified the specific column and line numbers on the stele (Column XXV, reverse, lines 40–44), and even noted the class distinction: this “eye for an eye” principle applied only between members of the awīlum (upper) class. It described the relief sculpture — King Hammurabi receiving the rod and ring from the sun god Shamash. Confidence: 1.0.

But: Law 196 is the most reproduced cuneiform text in the world. It appears in every introductory textbook, every museum label, every popular article about ancient Mesopotamia. This is the easiest possible test — recognizing the context may be sufficient to produce the correct reading without actually discriminating individual signs.

Manishtushu monument: detecting a forgery

The most surprising result. This black diorite monument claims to be an inscription of King Manishtushu of Akkad (ca. 2270 BCE), recording land purchases and temple offerings. But scholars have long suspected it's a pious fraud — a forgery created by Neo-Babylonian temple priests around 550 BCE to “discover” ancient royal grants justifying their tax exemptions.

From the main view, Gemini correctly read the king's name and title: Manishtushu, LUGAL KIŠ (“King of Kish”). From the detail view, it went further:

<script>other (Archaizing Neo-Babylonian mimicking Old Akkadian)</script>
<period>ca. 550 BC (imitating ca. 2270-2255 BC)</period>
<genre>royal-inscription (Pseudo-Old Akkadian "Pious Fraud")</genre>

The model classified the script as “archaizing Neo-Babylonian mimicking Old Akkadian” and the genre as a “Pious Fraud.” It explained its reasoning: the sign forms are “extremely regular” for an ostensibly archaic text, and noted grammatical features (like the -ni verb suffix) that Neo-Babylonian scribes believed were characteristic of ancient Akkadian. It then transliterated the daily offerings the “ancient” king supposedly established for Shamash at Sippar — 20 GUR of wheat, 20 mina of honey, 20 mina of ghee.

This is the standout result. The model performed paleographic analysis, dating the script style independently of the claimed date, and arrived at the same conclusion as modern Assyriologists. Whether it's truly reasoning from visual features or pattern-matching from its training data is an open question, but the output is scholarly-grade.

Neo-Assyrian medical tablet: wrong text, right library

The Neo-Assyrian tablet (P394421) is catalogued as a medical prescription text (BAM 6, 555). Gemini identified it as the final section of Tablet VII of Enūma Eliš, the Babylonian Creation Epic — a different text entirely. However, it correctly identified the fragment numbers (K.2421, K.2511, K.16765), the provenance (Library of Ashurbanipal, Nineveh), and the period (ca. 668–631 BCE).

Interpretation: The model correctly reads the museum labels and fragment numbers visible in the photograph. Its period identification is accurate. But the sign readings appear to be influenced by its strong prior about what K-numbered Nineveh fragments typically contain — Enūma Eliš is one of the most famous texts from Ashurbanipal's library. The model matched the context correctly but the content incorrectly.

Knowledge versus vision

Across all three experiments, the same pattern emerges. Gemini has extensive knowledge about cuneiform — it knows ATF format, Sumerian and Akkadian vocabulary, administrative tablet conventions, royal inscription formulae, sign names, Unicode code points, and the scholarly literature. But its ability to visually discriminate individual cuneiform signs from a photograph remains limited.

When knowledge and vision align (Hammurabi), the output is correct. When the model can reason from context without needing sign-level vision (Manishtushu forgery detection), the output is impressive. But when the task requires reading unfamiliar signs in an unfamiliar text (Ur III sheep receipt), the model generates from its distributional knowledge rather than reading what's in front of it — and this happens whether we ask for ATF transliteration (Experiment 1) or simple sign identification (Experiment 2).

Experiment 3's contamination test adds an important nuance: the model isn't simply reciting memorized transliterations. The Pearson correlation between training data overlap and vision performance is effectively zero (r = −0.076). Whatever the model is doing when it looks at a cuneiform photograph, it's doing it the same way regardless of whether it has the text memorized. The problem isn't memory contamination — it's that the visual analysis itself is limited.

The honest summary: Gemini 3 Flash is a cuneiform commentator, not a cuneiform reader. It can discuss cuneiform tablets with scholarly precision. It cannot reliably read them. But it is genuinely trying to read them — the contamination test proves that. Sign detection is real but low-accuracy (~37% of expected signs), making it a starting point for future improvement rather than a fundamental dead end.

Model comparison: 2.5 Flash vs. 3 Flash

We initially ran this experiment with Gemini 2.5 Flash before switching to Gemini 3 Flash Preview. The difference was stark:

Gemini 2.5 Flash produced more text (34 lines for the Ur III tablet vs. 11) but with lower accuracy, suggesting it was generating more freely from its training distribution. The 3 Flash model was more constrained and closer to the right structure, even when the sign readings were wrong.

Experiment 2: Sign transcription

Experiment 1 asked the model to do everything at once: see the signs, read the language, produce correct ATF. We decided to decompose the problem. Experiment 2 asks only: what cuneiform signs do you see?

Instead of ATF transliteration, we asked Gemini to identify individual signs and give three things for each: the Unicode cuneiform character (from the Unicode block U+12000–U+1236E), the standard sign name, and a wedge count (horizontal, vertical, diagonal). No language reading. No transliteration. Just: what shapes are on this clay?

Example output format:
Line 1: 𒀭 DINGIR (1 vertical + 3 diagonal) | 𒈾 NA (2 horizontal + 1 vertical)
Line 2: 𒈗 LUGAL (1 horizontal + 3 vertical) | 𒆧 KIS (5 horizontal + 3 vertical)

The rationale: wedge counts are the most objective visual measure. You can look at a photo, count wedges, and compare to the model's claim without knowing Sumerian. If the model says a sign has “3 horizontal + 1 vertical” and you can see 3 horizontal + 1 vertical in the photo, it's actually reading. If the wedge counts don't match, it's generating from knowledge.

Results

We ran the sign transcription prompt on the same four tablets. The outputs now contain Unicode cuneiform characters — actual glyphs you can compare visually against the photographs.

Ur III tablet: plausible structure, unverifiable content

Line 1: 𒐏 4(u) | 𒈠 MA | 𒈾 NA | 𒊺 ŠE
Line 2: 𒂆 GIN2 | 𒈠 MA | 𒈾 NA | 𒊺 ŠE
Line 3: 𒍪 ZU  | 𒈠 MA | 𒈾 NA | 𒊺 ŠE
Line 4: 𒇲 LA2 | 𒈠 MA | 𒈾 NA | 𒊺 ŠE
--- REVERSE ---
Line 5: 𒋗 ŠU  | 𒁀 BA | 𒋾 TI
Line 6: 𒈬 MU  | 𒂗 EN | 𒀭 DINGIR
Line 7: 𒆠 KI  | 𒉘 AG2 | 𒂗 EN

The model found 7 lines with 25 signs, confidence 0.85. It correctly identifies a repeating pattern on the obverse (MA-NA-ŠE, which could refer to a mana weight of grain) and the standard Ur III receipt formula on the reverse (šu ba-ti, “received”). The structure is plausible. But the ground truth is sheep (udu niga), not grain — the same content mismatch as Experiment 1, expressed through different signs.

Code of Hammurabi: reading a modern rendering

Line 1: 𒋳 ŠUM | 𒈠 MA | 𒀀 A | 𒉿 PI | 𒈝 LUM | 𒄿 I | 𒈾 NA
   →  šum-ma a-wi-lum i-na  →  "If a citizen..."
Confidence: 0.98

The model correctly identified the signs spelling šumma awīlum ina — the opening of the “eye for an eye” law. It also noted that the bottom portion of the image is a “modern artistic graphic” rather than the actual stele surface. Confidence: 0.98. But this is a modern digital rendering with vector-sharp signs, not ancient clay. The test doesn't tell us about vision on real tablets.

Neo-Assyrian tablet: honest about damage

Line 1: [DAMAGED]
Line 2: [DAMAGED] | 𒀸 AŠ | 𒀸 AŠ | [DAMAGED]
Line 3: [DAMAGED] | 𒀀 A  | 𒀸 AŠ | [DAMAGED]
Line 4: [DAMAGED] | 𒀸 AŠ | 𒀭 DINGIR | [DAMAGED]
...
40 lines total, 52 signs identified
Confidence: 0.65

A notable improvement over Experiment 1. Where the ATF prompt produced 8 lines of confident (but wrong) Enūma Eliš, the sign transcription prompt produced 40 lines that are mostly [DAMAGED]. The model defaulted to AŠ (the simplest cuneiform sign — a single horizontal wedge) for most readable positions, which is honest if not useful. The lower confidence (0.65 vs. 0.9) is better calibrated to the actual difficulty.

Manishtushu monument: formulaic patterns

Line 1: 𒈠 MA | 𒉌 NI | 𒅖 ISH | 𒌅 TU | 𒋢 SU
Line 2: 𒈗 LUGAL | 𒆧 KIS
Line 3: 𒀀 A  | 𒈾 NA
Line 4: 𒌷 URU | 𒀭 DINGIR | 𒂗 EN | 𒆤 KID/LIL
...
Detail view (Side D, Column I):
Line 1: 𒀭 DINGIR | 𒂗 EN | 𒆤 LIL2
Line 2: 𒈗 LUGAL  | 𒆳 KUR | 𒆳 KUR
...62 signs total, 0.9 confidence

The model correctly identified Manishtushu, LUGAL KIŠ (“King of Kish”) using individual Unicode signs. The detail view identified repetitive formulaic patterns: DINGIR EN-LIL (the god Enlil), LUGAL KISH (King of Kish). The repetition is consistent with what we know about this monument — it lists land purchases in formulaic blocks.

But the detail view repeats the same block of signs (lines 1–10 and 11–20 are nearly identical), which could mean either: (a) the text genuinely repeats its formulaic header, or (b) the model generated one plausible block and repeated it. We can't distinguish these without expert sign-by-sign verification against the actual stone.

What Experiment 2 reveals

The sign transcription approach has three advantages over ATF transliteration:

1. Visual verifiability. Unicode cuneiform characters are rendered as actual glyphs. A non-specialist can compare the glyph shape to what they see in the photograph. This doesn't require knowing Sumerian.
2. Better calibration. The Neo-Assyrian tablet went from 0.9 confidence (Experiment 1, wrong text) to 0.65 confidence (Experiment 2, honestly marking damage). The model produces more calibrated confidence when it's not trying to read language.
3. Separation of concerns. If sign identification is reasonably accurate, translation can be handled as a separate downstream step — potentially by a specialized model or rule-based system, rather than asking one model to do everything.

But the core question remains: is the model seeing the signs or knowing what should be there? The Ur III tablet produces different content from Experiment 1 but the same pattern of plausible-but-unverifiable output. The Manishtushu monument produces correct royal titles that the model has certainly seen in its training data. Only the damaged Neo-Assyrian tablet — where the model can't fall back on knowledge — gives us a clean signal, and there the output is mostly [DAMAGED].

Experiment 3: Is the model reading or remembering?

Experiments 1 and 2 tested four tablets. That's a proof of concept, not evidence. The core ambiguity remained: when Gemini produces a correct reading, is it seeing the cuneiform signs or remembering the text from its training data? The Code of Hammurabi is in every textbook. The Manishtushu monument is extensively published. Even the Ur III sheep receipt — while the model got it wrong — comes from a corpus of thousands of nearly identical tablets.

Experiment 3 tests this directly. We built a corpus of 107 tablets from CDLI spanning 11 historical periods, then ran two separate tests on each one: a contamination test (is this tablet in the model's training data?) and a vision test (can the model identify signs from the photograph?). Cross-referencing the results tells us whether training data contamination predicts vision performance.

The corpus

We selected 107 tablets from CDLI with published ATF transliterations and photographs, stratified across periods:

The contamination test

For each tablet, we gave Gemini only the P-number and catalog metadata — no photograph. We asked: “Can you reproduce the ATF transliteration of this tablet from memory?” If the model can reproduce the text without seeing it, the tablet's transliteration was in its training data.

We compared the model's text-only output against the CDLI ground truth, extracting sign readings from both and computing the overlap percentage. A tablet with >30% sign overlap was classified as contaminated (in training data). Below 30% was classified as clean.

The results:

Contamination varied dramatically by period. 46% of Ur III tablets were contaminated — unsurprising, since Ur III administrative texts are the most heavily published cuneiform corpus. Zero Old Akkadian tablets were contaminated, making them ideal clean benchmarks. Old Babylonian tablets (50% contaminated) are also well-published.

A striking detail: 104 out of 107 tablets, the model claimed to know the text — but only 31 could actually reproduce it. The model confabulates knowledge it doesn't have.

The cross-analysis

Now the key question. We ran the Experiment 2 sign-identification prompt on the same 107 tablets (102 completed; 5 failed due to API errors). For each tablet, we had both a contamination score (how much of the ATF is in training data) and a vision performance score (confidence, signs detected, accuracy). If the model is “reading from memory,” contaminated tablets should show higher vision performance.

Pearson correlation between training data overlap and vision confidence: r = −0.076.

That's essentially zero. There is no meaningful relationship between whether a tablet's text appears in the model's training data and how well the model performs on the vision task. If anything, the model performs slightly better on clean tablets — the opposite of what you'd expect from memorization.

To control for period confounds (contaminated tablets are mostly Ur III; clean tablets include harder Old Akkadian), we compared within the Ur III period specifically: contaminated Ur III average confidence was 0.68, clean Ur III was 0.69. Identical.

What this means

The model is doing genuine visual analysis, not reading from memory. When Gemini looks at a cuneiform photograph, it is attempting to process the visual features of the clay surface — not retrieving a memorized transliteration. This is the most important finding of the three experiments, because it tells us that the model's cuneiform vision, while limited, is real.

The 20 Old Akkadian tablets with 0% training data contamination are particularly valuable: they represent a gold-standard test set where we can be confident the model has never seen the content. These tablets scored an average confidence of 0.79 — higher than the overall average — suggesting that Old Akkadian sign forms (larger, more distinct wedge impressions) may actually be easier for vision models than the cramped Ur III administrative script.

The limitation is that the model detects roughly 37% of ground-truth signs regardless of contamination status. It's genuinely looking, but it's not seeing very well yet. This is the gap that future models — or fine-tuning on cuneiform data — would need to close.

The infrastructure works

The most practically useful outcome: Source Library's existing pipeline handles cuneiform with zero code changes. The two-step architecture maps naturally:

• OCR step = ATF transliteration (cuneiform signs → ASCII text)
• Translation step = Sumerian/Akkadian → English

Custom prompts are stored as named families in the prompts collection, and the reader, search, and translation percentage tracking all work without modification. The four test tablets are browsable in Source Library:

• Ur III Administrative Tablet
• Cruciform Monument of Manishtushu
• Code of Hammurabi
• Neo-Assyrian Tablet (K.2421+)

Experimental design for a systematic evaluation

Our proof of concept tested four tablets. A proper evaluation needs a controlled experiment with statistical power. Here is the design we propose.

Research question

Can general-purpose vision-language models produce accurate ATF transliterations of cuneiform tablets from photographs, and if so, under what conditions?

Hypotheses

1. H1 (Knowledge contamination): Models will perform significantly better on tablets whose transliterations appear in their training data (published CDLI corpus, textbook examples) than on unpublished or recently published tablets. We predict >50% sign accuracy on “known” tablets vs. <20% on “unknown” tablets.
2. H2 (Script period effect): Accuracy will vary by script period, with Neo-Assyrian (most standardized signs) outperforming Old Babylonian and Old Akkadian (more variant forms). We predict Neo-Assyrian > Old Babylonian > Ur III > Old Akkadian.
3. H3 (Genre effect): Administrative tablets (short, formulaic) will produce higher structural accuracy than literary or scholarly texts (complex vocabulary, rare signs). We predict >70% line-count accuracy on administrative vs. <50% on literary texts.
4. H4 (Few-shot improvement): Providing 3–5 example transliterations from the same corpus in the prompt will significantly improve accuracy on novel tablets of the same type. We predict >2x improvement in sign accuracy over zero-shot.
5. H5 (Model scaling): Larger models (Gemini Pro, GPT-4o, Claude Opus) will show higher sign-level accuracy than smaller models (Gemini Flash, Claude Haiku), controlling for the same prompt and tablets.

Sample

100 tablets selected from CDLI, stratified by:

• Period (25 each): Ur III, Old Babylonian, Middle Babylonian/Assyrian, Neo-Assyrian
• Genre (balanced within each period): administrative, literary, legal, letters, ritual
• Publication date: 50 tablets with transliterations published before 2020 (“known” — likely in training data) and 50 published 2023–2026 (“unknown” — likely not in training data)
• Photograph quality: all tablets must have CDLI photographs rated “good” or “excellent” by the cataloguer
• Size: 5–30 lines per face (to control for length effects)

Independent variables

Dependent variables (metrics)

• Sign accuracy: Percentage of signs correctly identified (Levenshtein distance between AI sign sequence and ground truth sign sequence, normalized). This is the primary metric.
• Line accuracy: Percentage of lines where the AI reading matches ground truth after normalizing whitespace and formatting differences.
• Structural accuracy: Correct identification of surface markers (@obverse/@reverse), line count (within ±1), and column breaks.
• Metadata accuracy: Period correct (within ±100 years), language correct, genre correct. Scored as 3 binary variables.
• Hallucination rate: Percentage of AI-generated lines that contain no overlap with any ground truth line (overlap < 0.1). This measures pure fabrication.
• Calibration: Correlation between model confidence score and actual sign accuracy. Well-calibrated models should report low confidence on fabricated readings.

Controls

• Baseline 1 (random): Randomly generated ATF from a Markov model trained on CDLI corpus, to establish floor accuracy
• Baseline 2 (retrieval): TF-IDF matching against CDLI corpus using only the photograph metadata (museum number, period) — measures how far pure knowledge retrieval can go without vision
• Human expert: 20 of the 100 tablets independently transliterated by a professional Assyriologist, providing a ceiling and inter-rater reliability baseline
• Repeated runs: Each model×prompt×tablet combination run 3 times to measure variance (temperature 0.0 for deterministic baseline, 0.3 for variance measurement)

Analysis plan

1. Report mean sign accuracy with 95% confidence intervals, stratified by all independent variables
2. Two-way ANOVA: model × publication date (tests H1, knowledge contamination)
3. One-way ANOVA: accuracy by script period (tests H2)
4. Paired t-test: zero-shot vs. few-shot accuracy on matched tablets (tests H4)
5. Reliability plot: model confidence vs. actual accuracy, with expected calibration error (tests calibration)
6. Qualitative error taxonomy: categorize wrong readings as (a) plausible sign misread, (b) genre-appropriate hallucination, (c) unrelated fabrication, (d) retrieval of known text

What this would settle

This design separates four capabilities that our PoC tangled together: (1) visual sign discrimination, (2) knowledge of cuneiform conventions, (3) document-level pattern matching, and (4) training data memorization. Experiment 3 already settled question (4) — memorization does not drive vision performance. A full factorial experiment like this would settle the remaining three, telling us whether a model that scores high on “unknown” tablets with no context hints has genuine cuneiform vision, or is doing sophisticated retrieval. Both are useful, but for different purposes — and only the first would transform the field.

Experiment 5: Can we help the model see better?

If the model is genuinely trying to read cuneiform but struggling, maybe the problem is the input image. CDLI composite photographs pack six views of a tablet — obverse, reverse, top edge, bottom edge, left edge, right edge — into a single image on a black background. That's a lot of visual noise. What if we cropped the image to show just the inscribed face? Or enhanced the contrast to make wedge impressions more visible?

We tested two strategies: spatial cropping (reducing what the model sees) and image preprocessing (changing how it looks). Both were tested with Gemini 2.5 Pro, our best-performing model from Experiment 4.

Strategy 1: Cropping

We tested three levels of cropping on tablet P250675 (Ur III, 1949×3053px — a high-resolution image):

• Full composite: The original CDLI image with all six views
• Face crop: Just the obverse or reverse face, black background removed
• Line strips: Individual horizontal strips, one per text line

On this high-resolution tablet, line strips won decisively. The full composite produced zero parseable signs. The face crop produced only 8 signs, but with 100% precision — every sign it identified was correct. The line strips produced 74 signs with 20 correct sign types (KU, MA, NA, SAG, KI, DU, GA, DA, LUGAL, KA, AN, UD, DUB, GI, DUG, MU, and others). Narrowing the visual field helped the model focus.

But does this pattern hold? We ran the same experiment on P100500 (also Ur III, but only 1039×1487px — roughly half the resolution):

The opposite pattern. On this lower-resolution image, the full composite won with 28.3% F1, while the obverse crop produced zero correct signs and line strips scored only 14.9%. The reason: cropping a 1039px-wide image to a 470px-wide face crop leaves too few pixels per cuneiform sign for the model to work with. Each wedge impression is only a handful of pixels across.

The key variable is image resolution, not visual complexity. High-resolution photos benefit from cropping (less visual noise, enough pixels per sign). Low-resolution photos are hurt by it (too few pixels survive the crop). This means the optimal preprocessing strategy depends on the input image.

Strategy 2: Image preprocessing

Can we enhance the image to make wedge impressions more visible? We tested seven preprocessing techniques on the P100500 obverse crop — the condition that scored 0.0% F1 with no preprocessing:

Upscale + CLAHE was the clear winner, rescuing a completely dead crop (0.0% F1) to 16.7% F1 with 42.9% precision. It correctly identified six sign types: MA, LA, DA, BA, MU, and BI. The technique works by first adding pixel density (2x Lanczos3 upscale), then enhancing local contrast (CLAHE) so wedge impressions stand out from the clay surface. To human eyes the result looks like a stippled mess, but the model can apparently extract useful signal from the enhanced micro-texture.

The other techniques offered marginal improvements at best. CLAHE alone (4.7%) and inversion (4.9%) showed some promise, but without the upscaling step the pixel density was too low for meaningful enhancement. The combined pipeline (grayscale + CLAHE + sharpen + normalize) performed worst of all — too many transformations degraded the image rather than enhancing it.

What Experiment 5 tells us

Image preprocessing helps — but modestly and conditionally. The optimal strategy depends on the input image:

• High-resolution photos (>1500px wide): Crop to individual line strips. On P250675, line strips achieved 16.3% F1 vs. 0% for the full composite.
• Low-resolution photos (<1500px wide): Keep the full composite. On P100500, the uncropped image achieved 28.3% F1 vs. 0% for the face crop.
• Upscale + CLAHE as a rescue technique: When a crop is necessary but resolution is low, 2x upscaling with CLAHE can partially compensate, though it cannot match a naturally high-resolution source.

The fundamental ceiling remains. Even with optimal cropping and preprocessing, accuracy stays in the 15–30% F1 range. The bottleneck is the model's cuneiform sign recognition capability, not image quality. Better photographs would help, but the real breakthrough will require models trained or fine-tuned on cuneiform data.

Experiment 6: Edge detection, raking light, and multi-pass prompting

Experiments 4 and 5 established that model choice and cropping both matter, but accuracy stayed in the 15–30% F1 range. Experiment 6 explored two remaining ideas: could image processing techniques borrowed from archaeology and epigraphy help the model see wedge impressions more clearly? And could a multi-pass prompting strategy — surveying the tablet first, then identifying signs, then reviewing against the image — improve accuracy over a single-pass approach?

We tested 21 conditions across two sub-experiments on the P250675 obverse crop — the same Ur III tablet from Experiment 5 (208 ground truth signs). All image processing used the sharp library in Node.js. All conditions were evaluated using the same bag-of-signs F1 metric.

Part A: image processing techniques (13 conditions)

Archaeologists use raking light — a low-angle light source that casts shadows along surface relief — to make wedge impressions visible on clay tablets. We simulated this digitally, along with several other image processing techniques:

• Sobel edge detection: Highlights gradients (the borders of wedge impressions)
• Pseudo-heightmap: Inverts brightness to approximate depth, then simulates directional lighting
• Edge overlay: Darkens the original image where edges are strong, emphasizing wedge boundaries
• Raking light: Simulates a low-angle light source from the left or from above, casting directional shadows along the horizontal or vertical gradient
• Multi-image input: Sending two or three image variants together so the model can cross-reference
• Sign reference chart: A lookup table of 50 common signs (Unicode glyph + name) sent alongside the photograph

All conditions used Gemini 2.5 Pro, which had the best accuracy in Experiment 4.

The most striking result: the raw photograph — which Experiment 5 had already shown was unreadable as a face crop — produced zero signs. But Sobel edge detection on the same image matched the best results from Experiment 5 (15.1% F1, 43.2% precision). Edge detection converts the photograph into a map of brightness gradients, which corresponds directly to the physical wedge impressions on clay.

Raking light from the left — the digital simulation of the technique archaeologists use in the field — achieved the highest precision of any condition (53.3%), though with very few signs. This makes sense: the technique is highly selective, emphasizing only the strongest wedge impressions while losing fainter signs.

Multi-image input consistently performed worse than single processed images. Sending the model three image variants (raw + raking + sobel) produced zero signs. The sign reference chart — a lookup table of 50 common cuneiform signs — was also useless. More information confused the model rather than helping it.

Part B: multi-pass prompting and model comparison (8 conditions)

The second sub-experiment tested whether asking the model to make multiple passes over the same image could improve accuracy. The multi-pass strategy mimicked how a human epigrapher works:

1. Survey pass: Count the lines, describe the tablet's physical condition, note the script style
2. Identification pass: Using the survey as context, go line by line and identify each sign
3. Review pass: Look at the image again, check each identification against what you see, correct any errors

We compared this against single-pass identification across three models: Gemini 2.0 Flash (non-thinking), Gemini 2.5 Pro (thinking), and Gemini 3 Flash Preview (thinking).

Three clear findings:

1. Model choice dominates everything else. Gemini 2.5 Pro single-pass with the raw photograph (F1 = 15.2%) beat every Flash condition regardless of technique. The cheapest Pro run outperformed the most elaborate Flash pipeline by 3–10x.

2. Multi-pass hurt accuracy. For both Flash and Pro, the single-pass approach outperformed multi-pass. Pro single-pass achieved 15.2% F1 vs. 11.0–12.0% for multi-pass. The review step introduced new errors while losing some correct identifications — the model second-guessed itself into worse performance. The only multi-pass advantage was precision: Pro multi-pass with dual images achieved 53.8% precision (highest of any condition), but at the cost of fewer total signs and lower F1.

3. Thinking tokens don't help for cuneiform. Gemini 3 Flash Preview is a “thinking” model that spends tokens on internal deliberation before responding. On the P250675 photograph, it burned 15,587 thinking tokens (compared to zero for 2.0 Flash), took 102 seconds instead of 4 seconds, and produced 74 signs — but at only 12.2% precision and 6.4% F1. It confidently hallucinated a repeating field survey pattern (“GAN SHA LA KU MU”), reported 0.95 confidence, and performed worse than the non-thinking 2.5 Pro. Extended reasoning doesn't compensate for limited visual capability.

What Experiment 6 tells us

The strongest single finding is that Sobel edge detection on an otherwise-unreadable face crop matches the best accuracy of any technique we tested (15.1% F1). This is practically useful: many CDLI photographs are composite views (obverse + reverse in one image) that produce zero results when cropped to a single face. Running a 3×3 Sobel kernel on the crop — a few milliseconds of computation — can rescue these images entirely.

But the ceiling remains. Across all 21 conditions, no technique exceeded 15.2% F1. More compute (multi-pass), more images (dual/triple input), more deliberation (thinking tokens), and specialized image processing all failed to break through the ~15% barrier established in Experiments 4–5. The bottleneck is neither image quality nor prompting strategy nor compute — it is the model's ability to visually discriminate cuneiform signs.

The optimal pipeline is now clear: crop to face, apply Sobel edge detection if the raw crop fails, use the best available model (Gemini 2.5 Pro), single-pass prompt. Everything else is wasted effort until models improve their cuneiform sign recognition.

But we had only tested Gemini models. What about Claude?

Experiment 7: Claude vs. Gemini

Experiments 1–6 used only Gemini models. To test whether the ~15% F1 ceiling was Gemini-specific or a general limitation of current vision models, we tested Anthropic's Claude Opus 4.6 and Claude Sonnet 4.6 on the same P250675 tablet photograph. Each condition was run 3 times to measure variance.

Claude Opus matched the best Gemini result at 16.4% F1 on the raw photograph — the same level as Gemini 2.5 Pro in Experiment 4. But the Gemini models showed unexpected failures: Pro 2.5 produced empty output despite spending 8,000 thinking tokens, and Pro 3 entered an infinite repetition loop, outputting the same sign 400+ times. Only Flash 3 with Sobel edge detection produced usable output among the Gemini models.

Opus also identified 17 of 27 unique sign types in the ground truth (MA, NA, E, LA, BA, UR, KI, GA, DA, LUGAL, AN, UD, UM, LIL, GI, RA, MU) and correctly inferred the tablet's genre (administrative), period (Ur III), and institutional context (real estate sale with witnesses). It reported 0.25 confidence — a realistic self-assessment compared to Gemini 3 Flash's 0.95 confidence on worse results.

The precision pattern was particularly striking. Opus with Sobel edge detection achieved 55.9% precision — the highest of any condition across all experiments. More than half of the signs it identified were correct. But like every other condition, recall remained low (6.3%) because the model identified far fewer signs than exist on the tablet.

The Gemini failures were puzzling. Pro 2.5 had been our best performer in Experiment 4, and Pro 3 had produced results (if hallucinated) in Experiment 6. Both were now failing completely. This suggested a parameter issue rather than a model capability problem.

Experiment 8: Temperature

The Gemini failures in Experiment 7 led us to check the temperature parameter. We discovered that all Gemini experiments (1–7) had been run at temperature: 0.1 — very low, pushing the model toward deterministic output. Claude, meanwhile, had been using its API default of temperature: 1.0 (no explicit setting).

Low temperature makes sense for tasks with clear right answers (code, math, factual recall). But cuneiform sign identification from photographs is inherently uncertain — the model needs room to explore multiple possible readings. At t=0.1, thinking models may get locked into repetition loops or exhaust their output budget on deliberation.

We re-ran each model at t=0.5 and t=1.0 on the same P250675 raw photograph.

Temperature had a dramatic effect:

1. Claude Opus at t=0.3 set a new high-water mark: 18.8% F1. Lower temperature improved accuracy for Claude, likely because the task benefits from more deterministic sign identification once the model has genuine visual signal. The 43.1% precision means nearly half of all identified signs were correct.

2. Higher temperature rescued the Gemini thinking models. Pro 2.5, which produced empty output at t=0.1 and t=0.5, finally generated 17 signs at t=1.0 — with 47.1% precision, its highest ever. Pro 3 broke out of its repetition loop and produced usable (if inaccurate) output. Flash 3 went from 0% F1 at t=0.5 to 8.5% at t=1.0.

3. The Gemini failures in earlier experiments were partly a temperature artifact. When Gemini 2.5 Pro achieved 15.2% F1 in Experiment 4, it was likely a different API version or deployment — the same model ID at t=0.1 now produces nothing. This suggests that the Experiment 4 results, while real, may not be reproducible with the current API. Temperature sensitivity is a confound that affects all historical comparisons.

4. Claude and Gemini have opposite temperature preferences. Claude improved from 15.6% to 18.8% as temperature dropped from 1.0 to 0.3. Gemini improved from 0% to 7–8% as temperature rose from 0.1 to 1.0. This may reflect architectural differences: Claude is a non-thinking model that benefits from deterministic output on tasks where it has signal, while Gemini's thinking models need randomness to escape deliberation traps on uncertain tasks.

Updated leaderboard

Combining results across all eight experiments:

Claude Opus 4.6 at t=0.3 is the new overall best at 18.8% F1 — breaking through the ~15% ceiling that had held across all six Gemini-only experiments. The top 5 all cluster between 13–19% F1, with precision consistently between 41–56%. The highest-precision condition remains Opus with Sobel (55.9%), meaning more than half of identified signs are correct, but recall stays stubbornly low.

What 18.8% F1 Looks Like

Statistics only go so far. Here is the actual output from our best condition — Claude Opus 4.6 at t=0.3 — alongside the tablet photograph it was reading. You can compare each cuneiform sign the model produced against the clay surface yourself.

Claude Opus 4.6 output (t=0.3, 22 seconds)

1: 𒌋 [UNCLEAR] 𒂠 SHE₃ 𒄀 GI 𒈬 MU [UNCLEAR]

2: 𒋗 SHU 𒃻 GAR 𒊏 RA 𒈾 NA 𒀀 A

3: 𒆳 KUR 𒁀 BA 𒀭 AN 𒂗 EN 𒆤 LIL₂

4: 𒇷 LI 𒆷 LA 𒊑 RI 𒅎 IM 𒄀 GI

5: 𒈬 MU 𒌓 UD 𒁲 DU₃ 𒀀 A 𒁀 BA

6: 𒊮 SHA₃ 𒂍 E₂ 𒀭 AN 𒈾 NA 𒋗 SHU

7: 𒅆 IGI 𒃲 GAL 𒈗 LUGAL 𒌷 URU 𒆠 KI

8: 𒄿 I 𒋾 TI 𒅗 KA 𒊏 RA 𒁀 BA

9: 𒈬 MU 𒌑 U₂ 𒊕 SAG 𒂠 SHE₃ [UNCLEAR]

10: 𒋛 SI 𒀀 A 𒈾 NA 𒆕 [UNCLEAR] 𒋃 [UNCLEAR]

58 signs extracted · 25 matched ground truth · 43% precision · 12% recall

Self-reported confidence: 0.28 — “low due to the photographic angle, surface damage, and the difficulty of resolving individual wedge impressions from photographs rather than direct examination.”

Some observations from reading the output against the photograph:

The model correctly reads the tablet as multi-column and identifies many real sign shapes — 𒀭 AN, 𒈗 LUGAL, 𒂍 E₂, 𒆠 KI are all signs that appear in the ground truth ATF. Line 7's 𒅆 IGI 𒃲 GAL 𒈗 LUGAL reads as “before the king” — a witness formula common in Old Akkadian legal texts, which is exactly what this tablet is (a real estate sale). The model inferred the document genre correctly.

But the errors are equally visible. The model reads 10 lines where the ground truth has three columns of 10+ lines each. It is reading one column (or parts of multiple columns) while missing most of the tablet. Many of the cuneiform characters it outputs are valid Unicode codepoints that look plausible but don't match the actual wedge patterns on the clay surface. This is the fundamental challenge: the model has learned what cuneiform signs look like in general, but cannot reliably match specific wedge impressions in a photograph to specific signs.

The 0.28 self-reported confidence is refreshingly honest. The model knows it is struggling.

Conclusion

Can AI read cuneiform? Not yet — but the picture is more nuanced than “no.” Across 107 tablets and eight experiments spanning two model families, these models correctly identify the Code of Hammurabi, detect a Neo-Babylonian forgery from its script style, produce excellent scholarly commentary, and — crucially — perform genuine visual analysis that is independent of their training data. They are not reading from memory.

The contamination test (Experiment 3) settles the most important methodological question: the model's vision performance is the same whether or not it has the text memorized (Pearson r = −0.076). This means there is a real, if limited, visual signal being extracted from cuneiform photographs. The model detects ~37% of expected signs at ~0.7 confidence — not enough for production use, but enough to build on.

The models have cuneiform knowledge without reliable cuneiform vision. But they have some cuneiform vision, and that vision is genuine. Experiments 4–6 systematically tested what helps: better models double accuracy (Experiment 4), image cropping matters for high-resolution photographs (Experiment 5), and Sobel edge detection can rescue otherwise-unreadable crops (Experiment 6). Multi-pass prompting, thinking tokens, multi-image input, sign reference charts, and raking light simulations all failed to move the needle. Experiments 7–8 brought in Claude Opus and discovered that temperature settings are a critical confound: Claude Opus at t=0.3 broke through the ~15% F1 ceiling to reach 18.8%, while Gemini thinking models need higher temperature (t=1.0) to avoid repetition loops and empty output.

The best result so far — 18.8% F1 with 43% precision — means that when the model identifies a sign, it is correct nearly half the time. But it only identifies a small fraction of the signs on the tablet. The bottleneck is not image quality, prompting, or compute: it is the model's ability to reliably distinguish individual cuneiform wedge patterns.

For Source Library, this means cuneiform support is infrastructure-ready but model-limited. The prompts, import pipeline, evaluation corpus, and contamination testing framework are built. The optimal processing pipeline is established: crop to face, apply Sobel edge detection as a fallback, use the most capable available model (currently Claude Opus 4.6 at t=0.3), single pass. When a model can reliably read wedge impressions — whether through fine-tuning on CDLI's 300,000 tablet photographs, improved vision capabilities, or the next generation of foundation models — Source Library can process cuneiform tablets with the same pipeline it uses for Renaissance printed books.

The oldest writing system in the world is waiting.

Technical details: Experiments 1–3: Gemini 3 Flash Preview. Experiment 1 (4 tablets): ATF transliteration prompt. Experiment 2 (4 tablets): sign identification prompt requesting Unicode characters and sign names. Experiment 3 (107 tablets): contamination test (text-only memory probe, >30% sign overlap threshold) and cross-analysis against vision performance. Experiment 4: model comparison across Gemini 3 Flash, 2.5 Flash, 2.5 Pro, and 3 Pro on 4 tablets. Experiment 5: cropping (composite vs. face crop vs. line strips) and image preprocessing (CLAHE, sharpen, upscale 2x, invert, combined, upscale+CLAHE) using Gemini 2.5 Pro on P250675 and P100500. Experiment 6: 13 image processing conditions (Sobel edge detection, pseudo-heightmap, raking light simulation, edge overlay, multi-image input, sign reference chart) using Gemini 2.5 Pro on P250675 obverse crop; 8 multi-pass prompting conditions (single-pass vs. 3-pass survey/identify/review) across Gemini 2.0 Flash, 2.5 Pro, and 3 Flash Preview (thinking model). Experiment 7: cross-vendor comparison using Claude Opus 4.6, Claude Sonnet 4.6 (Anthropic Messages API, anthropic-version 2023-06-01) and Gemini 2.5 Pro, 3 Pro, 3 Flash (Gemini REST API) on P250675 obverse raw photograph; 3 trials per condition, standard deviation reported; Opus additionally tested with Sobel edge detection. Experiment 8: temperature sweep — Claude Opus at t=0.3 and t=1.0; Gemini 2.5 Pro, 3 Pro, and 3 Flash each at t=0.1, t=0.5, and t=1.0; single trial per condition on P250675 raw photograph. Corpus: 107 tablets from CDLI spanning 11 periods. Tablet photographs sourced from CDLI. Ground truth ATF from CDLI published transliterations. Preprocessing: sharp library (Node.js). Evaluation: bag-of-signs F1, precision, recall, Jaccard. Full evaluation reports, corpus data, and analysis scripts available on request.