How IIIF Helped Us Translate the Renaissance

The hero image above is the engraved frontispiece of Michael Maier’s Atalanta Fugiens (Oppenheim, 1617), an alchemical book of fifty emblems set to music. It reached Source Library as a single line in a manifest published by e-rara, the Swiss rare-books platform run from the ETH Library in Zurich — the same kind of manifest published by the Bodleian, by Gallica, by the Vatican, by the Bavarian State Library. That shared format is IIIF, the International Image Interoperability Framework, and it is the reason a small team could take rare books from a dozen of the world’s great libraries and turn them into readable, translated, citable English editions at a scale that would otherwise be impossible.

Source Library is on a mission to translate the Renaissance. We started inside the Bibliotheca Philosophica Hermetica (the Embassy of the Free Mind) in Amsterdam, and we now hold roughly 30,000 publicly visible works in Latin, German, Hebrew, Arabic, Sanskrit, Chinese and more — with over 6,000 first-ever English translations you can read today and nearly 143,000 illustrations extracted from their pages. This is the story of how IIIF made that possible, told from both sides of the standard: as a consumer, and as a publisher.

A universal input layer

Before IIIF, ingesting a book from each new institution meant writing a new scraper against a new bespoke viewer, with its own URL scheme, its own tiling quirks, its own login walls. The promise of IIIF is that the shape of the data is the same everywhere. Point the same importer at a Gallica manifest or a Bodleian manifest and the page images come out the same way. The institution changes; the code does not.

That uniformity is what let us treat thirteen-plus archives as a single faucet feeding one downstream pipeline. Everything past the manifest — OCR, translation, illustration detection, scholarly publishing — is identical regardless of who digitized the book.

Here is what that machine consumer actually looks like. Asked to explore the historical libraries discussed in the collection, Claude works through Source Library, surfaces the relevant works, and — on a follow-up to find quotes — returns exact passages from Gabriel Naudé’s Advice on Establishing a Library (1627), each with a page number and a citation link back to the original:

From manifest to readable book

When a book enters the library, its IIIF manifest is the seed. Our importer detects the version — IIIF Presentation 2.x stores canvases under sequences[0].canvases[], while 3.0 puts them under items[] — walks every canvas, and resolves each page to a canonical Image API URL. From there a single pipeline carries the book the rest of the way.

A principle we hold firmly: every page image is mirrored to our own storage (Cloudflare R2) as soon as the book is imported. About 99.8% of pages now serve from our mirror. We credit and link the source institution on every book, but a reader who lands here never depends on an upstream endpoint that might rate-limit, move, or disappear.

The books arrive hidden. Nothing becomes publicly visible until it has been through OCR, translation, and a quality pass — the visible flag flips only at the end. The whole sequence, from a Gallica manifest to a readable, translated English edition, runs unattended; minting a citable DOI through Zenodo is a separate, deliberate publishing step on top.

The line that does the work

The quiet hero of the whole arrangement is the IIIF Image API URL. It is a small grammar that lets you request any region of any page at any size, by rotation and quality, just by editing the path. Here is a real one — how a viewer asks the Wellcome Collection’s image server for a full page at 1600 pixels wide:

That grammar is why thumbnails, reading-size pages, and the multi-thousand-pixel crops our illustration detector needs are all the same request with three characters changed. It is also why, for the small tail of books we still serve from an upstream IIIF server, external viewers get true deep-zoom for free: we hand the viewer the provider’s image service and it tiles on demand.

The parts that are hard

The clean story above hides a lot of friction. IIIF standardizes the shape of the data, but not the politics of getting it — and a project that fetches millions of images has to be a good citizen or it gets (rightly) blocked.

Rate limits, and being polite

Library and museum image servers throttle aggressively, and they should. We keep a per-host token bucket with deliberately conservative limits and exponential backoff on every fetch. These are real numbers from our fetcher — requests per second, per host:

Those small numbers add up. A single corpus-wide re-archive job to pull full-resolution illustrations from the British Library’s Endangered Archives meant roughly 240,000 pages × 9 image tiles each — about 2.16 million requests. Done impatiently, that’s an accidental denial-of-service against a cultural institution. Done at fifteen requests a second behind their cache, it’s invisible. The rate limit isn’t a nuisance to route around; it’s the terms of the relationship.

The datacenter problem (and an accidental gift from IIIF)

Some institutions — Harvard, and apparently Gallica — serve residential IP addresses happily but rate-limit datacenter IPs hard. Our import server runs on Vercel, in a datacenter, so its requests get 429’d while the same manifest opens fine in a browser at home. The workaround is to fetch the manifest from a residential connection and insert the book directly.

Here IIIF’s architecture quietly saves us. Because the manifest is separate from the images — it merely references image URLs that are then served straight to the reader’s browser — we only need to fetch the manifest once. The facsimile renders for every reader afterwards even if our own servers never successfully fetch those images datacenter-side. Separating presentation metadata from pixels turns a hard block into a one-time, low-volume fetch.

Browser gates, version drift, and the things we won’t do

• •Browser gates. Some viewers hide the manifest behind a JavaScript or Cloudflare challenge (the International Dunhuang Programme is one). There is no clean API call; you have to drive a real browser to get the manifest at all.
• •Version drift. Presentation 2.x and 3.0 nest their canvases differently, and plenty of “IIIF” endpoints are partial — a valid manifest with no Image API service (so no deep zoom), metadata in a dozen idiosyncratic shapes, the occasional canvas pointing at a missing image.
• •Terms we honor. Some sources prohibit automated download outright (ctext.org, the National Library of China). We don’t ingest them, even where it would be technically trivial. “Can” is not “may,” and a library that didn’t respect that wouldn’t deserve the name.

When there is no manifest

Not everything publishes IIIF. A great many digitized books live behind a bespoke viewer, an OAI-PMH feed, a discovery layer, or nothing more than a download link. IIIF is our preferred on-ramp, not our only one — so when there’s no manifest, we fall back, roughly in this order:

1. 1.Another catalog API. OAI-PMH, SRU, or a discovery API like Primo VE (which we sometimes drive through a real browser session) gives us a candidate list and enough metadata to dedupe and locate the page images, which we then assemble ourselves. This is how we harvested KU Leuven’s digitized holdings, where there is no clean IIIF-only facet.
2. 2.The fallback chain. To find a digitized copy at all, we try Internet Archive → Gallica → HathiTrust → Project Gutenberg in turn. If one source blocks or 403s, we move on rather than fight it.
3. 3.Direct partner ingest. The founding collection — the Embassy of the Free Mind’s own rare books — comes straight off the institution’s scanning archive, never via IIIF. (Those scans bring their own puzzles: many are two-page spreads that have to be detected and split before the pipeline can read a single page.)

The design choice that makes this manageable: every path normalizes into the same book and page document shape — same fields, same content fingerprint for dedup, same provenance record — before anything downstream runs. So the pipeline neither knows nor cares whether a book arrived as a pristine IIIF v3 manifest or was hand-assembled from a browser session. IIIF is the widest, cleanest door into one shared room.

We give it back

IIIF is not just how books come in. It is also how they go back out. Every book in the library publishes its own Presentation 3.0 manifest:

Live example: the manifest for Atalanta Fugiens — load it in a viewer and you can search and read its translation.

Point Mirador, Universal Viewer, Clover, or Theseus at that URL and you get the page images — but also something the original manifest never had: our AI-generated OCR and English translation, delivered as Web Annotations that overlay the original folio. The transcription and the translation sit on the very pixels they describe, so you can read the Latin and the English against the page itself, in a viewer the institution already trusts.

Serving annotations from a machine carries an obligation, and we treat it as a hard invariant. AI text is never presented as a faithful human transcription: every annotation carries a Web-Annotation generator labeled “machine-generated, not human-verified,” with the model name attached. And we are scrupulous never to leak the AI’s editorial notes — the little summaries and keyword blocks our pipeline writes about a page — into the transcription, because those describe the page rather than quote it. The whole point of a source library is that what you quote is really on the leaf in front of you.

Two more disciplines that matter to a IIIF audience: the digitizing institution is always listed first in the manifest’s provider array and named in a required-statement (“Digitized by {institution}. Re-hosted with AI-generated OCR and translation by Source Library”); and canvas dimensions use the real digitized pixel size whenever the archiving step recorded it, so a region citation lands on the words it claims. There is also a Content Search 2.0 endpoint, so a viewer can search across a book’s OCR and translation and get back highlighted hits as annotations — a single book’s manifest can return dozens of matches with the exact phrase highlighted on each folio.

Provenance is the whole point

Everything above is in service of one idea, and it is worth saying plainly. We take a book, run it through machine OCR, run that through machine translation, and add machine-written editorial notes. By the time a reader sees an English sentence, it has passed through three layers of automation. The single thing that keeps this honest — that makes it a library rather than a content farm — is an unbroken chain of provenance back to the original leaf.

IIIF is unusually good at carrying that chain, and we lean on it at both ends:

• •On the way in, every book records who digitized it, the URL of the original catalog record, the institution’s own attribution and rights statement, and a content fingerprint (often the IIIF identifier itself) used for deduplication. We mirror the page images to our own storage for speed and reliability — but we never sever the link back. Every book page keeps a live link to the source record and to the upstream IIIF manifest, so a skeptical reader can go check the original for themselves. Ad fontes.
• •On the way out, our manifest credits the digitizing institution first; every AI annotation carries a machine-generated marker with the model’s name; canvas dimensions are the real digitized pixel sizes (when known) so a region citation lands on the words it claims; and we rigorously strip the AI’s editorial descriptions of a page so they can never be quoted as if they were the text on it.

That last point is not theoretical. Early on, a page-description note our pipeline had written about a page — mentioning mercury discussed on a neighboring leaf — leaked into a search snippet and produced a confident citation to words that were not on the page. The fix was to treat the boundary between the author’s words and the machine’s words as sacred, everywhere those words are served. For a human reader that boundary is a matter of trust; for an AI consuming the library through these same manifests, it is the difference between grounded scholarship and fluent invention. Provenance is what lets either one tell the two apart.

Where this could go next

A few directions we’re thinking about — some prompted by the question that started this post: do we really have to hit an endpoint for every single book?

Appreciations

None of this exists without the institutions that did the patient, expensive work of digitizing these books and — crucially — chose to publish them as IIIF rather than locking them behind a bespoke viewer. Every English translation we’ve made rests on their decision to be interoperable. A standard is only as generous as the people who adopt it, and these libraries have been generous.

First among them is the Bibliotheca Philosophica Hermetica / Embassy of the Free Mind in Amsterdam, where this project began and whose collection remains its heart. And then, in gratitude, the archives whose IIIF manifests have become books you can now read in English:

Counts are of imported works by source; many more institutions contribute manuscripts, artworks, and open-access images through IIIF and adjacent APIs (the Rijksmuseum, the Met, the National Gallery, Wikimedia Commons, e-codices, Chester Beatty, and others). To every digitization team, cataloguer, and standards contributor behind these manifests: thank you. You built the rails. We just ran a train on them.