Source Library can read printed Latin, German, Arabic, Hebrew, and Sanskrit with Gemini 3 Flash. We’ve OCR’d over 300,000 pages. After testing cuneiform, the natural next question was: what about Egyptian hieroglyphs?
We selected three pages from E.A. Wallis Budge’s Egyptian Reading Book for Beginners (1896), a standard text where hieroglyphic passages are printed alongside Latin-character transliteration — a built-in answer key. Each page has 5–8 lines of hieroglyphs in the upper portion, with the corresponding transliteration below.
We tried four different approaches. All four failed. But they failed in different ways, and the pattern of failures tells us something useful about where visual language models actually break down on non-Latin scripts.
Why hieroglyphs are hard
Egyptian hieroglyphs present a fundamentally different challenge than cuneiform, printed Latin, or even Chinese characters:
- ~1,071 distinct signs in the Unicode Egyptian Hieroglyphs block (U+13000–U+1342F), far more than any alphabet
- Dense visual similarity — many signs differ by tiny details (the owl 𓅓 vs. the vulture 𓄿, the mouth 𓂋 vs. the forearm 𓂝)
- No word boundaries — hieroglyphs are arranged in visual groups, read right-to-left or left-to-right depending on which way the animals face
- Stacking — signs can be arranged vertically within a single group, not just sequentially
- Determinatives — some signs are semantic classifiers that aren’t pronounced, making the mapping from text to transliteration non-obvious
The test pages are from a 1896 printed book, not hand-carved stone, so the signs are clear and consistent. This is about as easy as hieroglyph OCR can get. If the model can’t read clean typeset hieroglyphs, it certainly can’t read weathered temple inscriptions.
The four approaches
Direct Unicode OCR
Ask Gemini to look at the page and output Unicode hieroglyphs directly, the same way it reads Latin or Arabic.
The simplest possible approach: give the model the image and ask it to transcribe each line of hieroglyphs using Unicode characters. This is exactly what works for our Latin, German, and Arabic OCR pipeline.
The model does produce Unicode hieroglyphs. But comparing the output to the original page, the signs are mostly wrong. The model generates plausible-looking sequences of hieroglyphic characters, but they don’t match what’s actually on the page.
What Gemini produced (line 1, Tale of the Two Brothers)
What the transliteration says line 1 should contain
er tet unu neket er-a emtutu her tat-nek uā en ṯebu en heqt her tet-k
The problem is that the model appears to be pattern-matching visual features to plausible hieroglyphic sequences rather than carefully identifying each sign. It’s the hieroglyphic equivalent of a student who can write Chinese-looking characters but doesn’t actually know Chinese.
Verdict: produces hieroglyphs, but the wrong ones. The model doesn’t have enough training data on hieroglyphic Unicode to map visual signs to correct codepoints.
Gardiner Sign List codes
Instead of Unicode, ask for Gardiner codes (A1, M17, N35...) — the standard Egyptological catalog — then convert to Unicode with a lookup table.
The hypothesis: maybe the model knows the Gardiner Sign List better than Unicode codepoints. The Gardiner list assigns memorable codes to each hieroglyph — A1 is “seated man,” M17 is “reed,” N35 is “water ripple.” Egyptology textbooks use these codes extensively, so they should be well-represented in training data.
We built a comprehensive Gardiner-to-Unicode mapping table (750+ signs, A1 through Aa32) and asked Gemini to OCR in Gardiner codes instead of Unicode. The conversion step worked perfectly — 100% of the Gardiner codes the model produced mapped to valid Unicode codepoints.
But the Gardiner codes themselves were fantastically wrong. Adding an intermediate abstraction layer gave the model two chances to fail instead of one: it had to both correctly identify the visual sign and recall the correct catalog number. It turns out Gemini is better at pattern-matching images to Unicode than at recalling catalog metadata.
Verdict: adding a symbolic intermediate layer made everything worse. The model can visually recognize hieroglyphic shapes better than it can recall Gardiner catalog numbers.
Two-pass self-correction
Run direct OCR, then show the model its own output alongside the original image and ask it to correct errors.
Self-correction works well for text generation — show a model its own output and ask it to fix mistakes. We tried a two-pass pipeline: Pass 1 does direct Unicode OCR, Pass 2 gets the image plus the Pass 1 output and is asked to carefully re-examine each sign and correct errors.
The prompt specifically called out common error types: confused similar-looking signs, wrong sign order, missing small signs, duplicated signs. The model dutifully produced a “corrected” version with a log of changes.
The corrected version was virtually identical to the original. The model made the same mistakes both times, because it lacks the underlying knowledge to distinguish correct from incorrect readings. Self-correction only works when the model knows what “correct” looks like — and for hieroglyphic Unicode, it doesn’t.
Correction log from pass 2 (typical)
“Line 1: Minor corrections to sign order. Line 2: Replaced 𓁶 with 𓁷 (face orientation). Line 3: No changes needed.”
The model performed cosmetic edits while leaving the fundamental errors untouched.
Verdict: self-correction requires knowing what correct looks like. The model makes cosmetic adjustments but the same structural errors persist.
Describe each glyph individually
Force the model to slow down: describe each sign visually (“seated man,” “owl,” “water ripple”) before mapping it to Unicode.
The most promising hypothesis: maybe the model fails because it tries to read entire lines at once, the way it reads Latin text. What if we force it to examine each glyph individually, describing what it sees before committing to a Unicode character?
This is chain-of-thought reasoning applied to visual recognition. Instead of “read this line of hieroglyphs,” we asked: “for each sign, describe what you see, then identify it.”
Glyph-by-glyph output (Tale of the Two Brothers, line 1)
| Description | Unicode |
|---|---|
| mouth | 𓂋 |
| cobra | 𓆓 |
| loaf | 𓏏 |
| hare | 𓃹 |
| water ripple | 𓈖 |
| quail chick | 𓅱 |
| vulture | 𓄿 |
| loaf | 𓏏 |
| seated man | 𓀀 |
| owl | 𓅓 |
| face | 𓁷 |
| basket | 𓎡 |
12 of ~35 signs shown. Full reasoning chain available on the eval page.
This produced the most interesting results. The visual descriptions are often correct — the model really can see “a seated man,” “an owl,” “a water ripple.” The breakdown happens at the mapping step: knowing you’re looking at an owl doesn’t always produce the correct Unicode codepoint for “owl” (𓅓, Gardiner G17).
It also revealed an inconsistency problem: the model sometimes described 35 glyphs for a line that actually contains 20, inserting signs that aren’t there. The visual grounding is better than the other approaches, but the model still hallucinates signs between the ones it correctly identifies.
Verdict: the most informative failure. Vision works; the bottleneck is the Unicode mapping and sign-counting, not the visual recognition itself.
Summary
| Approach | Hypothesis | Result | What it revealed |
|---|---|---|---|
| Direct Unicode | Same pipeline as Latin OCR | Wrong signs | Insufficient hieroglyphic Unicode in training data |
| Gardiner codes | Catalog numbers better than Unicode | Much worse | Visual recognition > catalog recall |
| Self-correction | Second pass catches errors | No change | Can’t correct what you can’t evaluate |
| Describe-each | Slow down, examine one at a time | Best failure | Vision works; Unicode mapping doesn’t |
What we learned
The four failures point to a consistent diagnosis. The bottleneck is not vision — the model can see hieroglyphic signs and describe them correctly. The bottleneck is the mapping from visual recognition to the correct Unicode codepoint. This is a training data problem, not an architecture problem.
Consider the contrast with Latin script OCR. When Gemini sees the letter “A,” it has seen millions of examples of that letter mapped to Unicode codepoint U+0041. When it sees the hieroglyph 𓅓 (an owl), it has seen perhaps a handful of examples mapping that visual shape to U+13153. The Noto Sans Egyptian Hieroglyphs font only has about 1,000 users. Hieroglyphic Unicode text barely exists on the internet.
This means the problem is likely solvable, but not with prompting alone. It would require either:
- Fine-tuning on a hieroglyphic dataset — a few hundred pages of aligned image/Unicode pairs would probably be enough, given that the vision component already works.
- A sign-level classifier — crop individual signs from the page, classify each one against the ~750 Gardiner signs using a dedicated image classifier, then assemble into text. This sidesteps the Unicode mapping problem entirely.
- Few-shot prompting with a sign chart — include a reference sheet of all ~100 common signs with their Unicode codepoints in the prompt, so the model can look up signs rather than recall them from memory.
For now, Source Library will hold the hieroglyphic books in their original scanned form without AI transcription. The six Egyptian texts we imported — Budge’s Reading Book, Book of the Dead, and four others — are accessible as page images through the Internet Archive scans. When the Unicode mapping problem is solved, either through model improvements or a dedicated classifier, we’ll run the pipeline again.
How this compares to cuneiform
Our cuneiform experiments hit a ~15% F1 accuracy ceiling across most models, with Claude Opus reaching ~23%. Hieroglyphs are qualitatively different: we can’t even compute a meaningful accuracy score because the output sequences don’t align well enough with the ground truth for character-level comparison.
But the failure mode is also different. Cuneiform failed because the 3D wedge shapes are hard to see in 2D photographs — a genuine vision problem. Hieroglyphs fail because the model can’t map what it sees to the right Unicode codepoints — a knowledge problem. The cuneiform bottleneck is upstream (seeing); the hieroglyph bottleneck is downstream (naming).
This means hieroglyphs are likely to be solved first. Vision improvements won’t help cuneiform until cameras can capture the three-dimensional impression of a stylus in clay. But the hieroglyph Unicode mapping problem is purely a training data gap that could be closed with a relatively small labeled dataset.
Technical details
- Model: Gemini 3 Flash Preview (gemini-3-flash-preview)
- Source: Budge, Egyptian Reading Book for Beginners (1896), Internet Archive
- Test pages: 80 (Tale of the Two Brothers), 200 (Battle of Kadesh), 270 (Stele of Pi-Ankhi-Meri-Amen)
- Image resolution: 1500px wide IIIF crops
- Interactive eval: Full side-by-side comparison page