A bibliography that only existed as formatting
The starting point for this project was a personal bibliography maintained for years by a teacher: hundreds of children's and young adult books she had come across and thought would be useful or interesting for students. It lived in a LibreOffice document, not a database. There were no identifiers, tags, or explicit data model. The structure existed entirely in the formatting: categories were level-one headings, publishers level-two headings, and each book was a paragraph with the title underlined, the author in plain text, and the summary in italics.
That worked for a long time. Eventually the document grew to hundreds of pages, became slow enough to make the computer struggle, and was cumbersome to navigate. The information was still valuable, but the document was no longer a useful way of accessing it.
The question was whether its formatting could be turned back into something queryable.
Extraction
The first step runs inside the document itself. LibreOffice exports the file as HTML, and extract.js is a small script meant to be pasted into that export and executed in the browser. It walks the document's structure, reconstructs the records, and replaces the page with the JSON it produced. The resulting file becomes the starting point for the rest of the pipeline.
The parser does not attempt to understand arbitrary LibreOffice documents. It encodes the conventions of this particular one: An underlined element at the beginning of a paragraph is treated as a title. The text next to it becomes the author. An italic element following the title is interpreted as the summary, including summaries that span several paragraphs. Categories and publishers are recovered from the nearest preceding headings. Author names are split on conventions such as &, et, and line breaks, with a few normalizations applied along the way.
It is an explicit encoding of habits that had previously existed only as formatting. When those habits are violated, the parser breaks loudly. That is intentional: rather than pretending to have discovered a universal document structure, the script makes the assumptions of this particular document visible and executable.
The extraction is also deliberately one-shot. The original document was not intended to become a maintained source database; it needed to be converted once. The resulting JSON, however, became a reproducible input for the subsequent stages of the pipeline.
Enrichment
The extracted data is thin: roughly 866 books, each with a title, authors and a short summary. enrich.js turns those records into something more useful by querying four external sources in parallel: the BnF's SRU service, Google Books, Babelio, and Decitre.
The hard part is not retrieving records. It is deciding when records from different systems describe the same book: Each source has its own conventions for titles, author names, publishers and identifiers. A book that appears in one catalogue as Dahl, Roald (1916–1990) may appear elsewhere simply as Roald Dahl. Page counts, publishers and even titles can differ. There is no single record that can simply be treated as the canonical truth.
Candidate records are therefore scored using fuzzy string matching, with source-specific thresholds and a series of fallback strategies for common mismatches: reducing the author query, trimming titles, stripping punctuation and separators, or splitting titles on a dash. The BnF and Google Books use different acceptance thresholds, while Decitre is queried by EAN once another source has supplied an ISBN. Requests are rate-limited, and records that cannot be enriched are kept in an error log rather than silently discarded.
The pipeline is therefore less about scraping metadata than about making explicit decisions about when two imperfect descriptions can reasonably be treated as descriptions of the same object.
Reconciling authors
The most interesting part of the pipeline is process_authors.js, which turns the many ways an author can be mentioned across the dataset into a single set of people. The BnF provides authority records with identifiers, ISNI numbers, dates and INTERMARC role codes. Babelio provides display names and profile links. These identifiers and names rarely line up directly. Deduplication therefore proceeds in stages of decreasing confidence.
Shared identifiers come first: an ISNI, BnF identifier or Babelio profile link is strong evidence that two records refer to the same person. Exact matches follow, using normalized names with diacritics removed. Finally, highly similar names are matched using Jaro–Winkler similarity. Roles are reconciled in a similar way. Free-text descriptions such as illustrateur or traducteur are matched against the INTERMARC vocabulary and mapped to standardized function codes. The resulting author records contain a preferred name, aliases, external identifiers, and a list of contributions with their roles.
The result is more than a cleaned-up bibliography: The original document described books as formatted pieces of text; the final dataset describes relationships between people and books, with identities and roles attached to those relationships. The 1,014 people recovered from the 866 books form a small graph of who made what, reconstructed from several catalogues that were never designed to agree with each other.
From formatting to data
The frontend is deliberately small: a Vue 2 application loaded from a CDN, backed by a single minified JSON file. A lastupd timestamp allows the browser to cache the dataset locally and invalidate it when the data changes.
The enrichment and reconciliation stages, unlike the initial extraction, are designed to be run repeatedly. A small CLI makes it possible to add books to the dataset and rerun the relevant stages without going back to the original LibreOffice document.
What started as a document maintained by one person has therefore acquired several explicit layers of structure:
formatting → records → external metadata → entities → relationships
None of the individual technologies is particularly exotic. The difficult work lies in reconstructing an implicit data model, deciding how heterogeneous sources should be reconciled, and making those decisions explicit enough to be repeated.