Making MARC legible
The premise
A MARC record is a string of bytes segmented with non-printable characters. Push one through a terminal and you get ordinary ASCII. Except some of the whitespace are places where the record separator (\u001d), the field separator (\u001e), and the subfield delimiter (\u001f) sit. Those three are C0 control codes, relics of encoding schemes that predates the web and was designed for magnetic tape, when "machine readable" meant, literally, that a machine could read it and a person mostly could not. And this is still the format underneath a huge part of the world's library infrastructure.
I came to it sideways. I was asking a concrete question: how does the data a library actually stores get into a browser? Underneath every catalog search box is a pile of these records. National union catalogs contain tens of millions of them. If you want to display one, search one, transform one, or compare two editions of the same book, you first have to get past a byte format that almost no web-facing tooling is written to understand.
The project that came out of that question, JsMarc, is a small TypeScript library for parsing MARC records, together with a layer that tries to make those records legible to a human. The interesting part turned out not to be parsing the bytes. It was understanding what the bytes mean.
The format
MARC is the cataloguing structure carried by ISO 2709, a record format designed in the 1960s. A record has three parts, laid end to end.
The Leader is 24 fixed character positions. The first five hold the record's own length as a zero-padded number (01208 means the record is 1,208 bytes); the rest carry coded status, record type, and bibliographic level, ending with a base address that points at where the data body begins. Nothing is delimited here. Position is the meaning.
Then comes the Directory: a table of 12-character entries, one per field, terminated by a field separator. Each entry is a triple of tag, length, and byte offset. In the parser this becomes a simple template:
"@directory": {
code: [0, 3],
length: [3, 7],
position: [7, 12],
}
So 001000900000 means: tag 001, field is 0009 bytes long, starting at byte offset 00000.
Three details matter downstream.
- the tag lives in the directory, not in the field itself. The field is just bytes with no self-description.
- fields are located by byte offset rather than by scanning for delimiters. The directory therefore has to be read and trusted before a single field can be cut out.
- duplicate tags are legal and common. A record can contain several
650subject headings or several020ISBNs. A field is distinguished not just by its tag, but by its position in the directory.
Finally comes the Body: the actual fields, each the length its directory entry promised.
There are two kinds. Control fields (00X) have neither indicators nor subfield codes; they are raw data up to the next field separator, and in MARC21 are often themselves position-coded. Data fields open with two indicator characters, followed by subfields, each introduced by the delimiter \u001f and a single data-element identifier.
Here is a fragment of a real record (the Library of Congress catalog entry for Just for Fun, by Linus Torvalds and David Diamond):
...12252055\u001e \u001fa0066620724 (hc)\u001fc...
The 020 field contains two blank indicators, then subfield a containing the ISBN. Parsed, it becomes:
{
"code": "020",
"indicator": " ",
"subfields": [{ "code": "a", "value": "0066620724 (hc)" }]
}
None of this is especially complicated. It is just old. And almost nothing about it matches the assumptions a modern parser wants to make.
Parsing is not understanding
There is a second problem hiding behind the byte structure. MARC is not one format with one vocabulary. ISO 2709 provides the anatomy, but different MARC dialects assign different meanings to the same positions. MARC21 and UNIMARC share the same underlying structure. A directory entry is a directory entry, a control field is a control field. But the meaning of every tag changes. The canonical example is the title: field 245 in MARC21, field 200 in UNIMARC. Same bytes, same structural position, entirely different field.
UNIMARC's history explains why. It emerged from IFLA's idea of Universal Bibliographic Control: a document should ideally be catalogued once, in its country of origin, and that record should then be shareable internationally. By the early 1970s, however, individual countries had developed their own MARC dialects (INTERMARC, RUMARC, CANMARC and others) and they did not necessarily agree. UNIMARC was conceived as a "switching format": an intermediary that records could be converted into and out of.
The dialects even disagree on how fields are grouped. UNIMARC arranges them into functional blocks: 0XX identification, 1XX coded information, 2XX descriptive data, 3XX notes, 4XX linking entries, 5XX related titles, 6XX subject analysis, and 7XX responsibility. MARC21 instead uses 0XX for control information and standard numbers, 1XX for main entries, 2XX for titles and edition, 6XX for subjects, and 7XX for added entries. Two maps drawn over the same territory. This is where parsing stops being enough.
Parsing the bytes gets you tags, not meaning. A 200 with no context could be a title or something else entirely depending on which dialect produced the record. To explain a record, the parser therefore needs another layer: definitions that connect structural codes to human concepts.
Making the definitions data
The explanation layer is backed by JSON definition files: MARC21 definitions from the Library of Congress and UNIMARC definitions from ABES. Rather than hard-coding descriptions into the parser, I extracted the institutions' online documentation into structured data using small scripts. The result is a separation between the format and its vocabulary:
raw bytes
↓
ISO 2709 parser
↓
MARC fields
↓
dialect definitions
↓
human-readable meaning
That separation turned out to be useful in its own right. Some subfield meanings depend not just on the subfield code but on its value. The schema therefore allows either a wildcard description or a map from values to descriptions. The definitions are also searchable in both directions. You can start with a field code and find its meaning, but you can also start with a concept. Searching for the French word "auteur", for example, returns the UNIMARC fields and subfields used to represent authorship, including 200$c and 701$4. That reverse lookup changes the role of the definitions. They are no longer merely documentation attached to a parser. They become a small queryable representation of the cataloguing scheme itself.
Getting the data
A parser needs real records. Most real library catalogs, however, expose those records through Z39.50, a pre-web protocol that browsers cannot speak directly. So alongside JsMarc I built a small companion, web-z3950: an intentionally minimal Node.js HTTP-to-Z39.50 gateway with no npm dependencies, using IndexData's yaz-client underneath. It only does ISBN lookups. It is not intended as a service (no authentication, no rate limiting, a new process per request), its job is simply to put a live library record in front of the parser. It can query real catalogs, including the Library of Congress and the French SUDOC union catalog.
The whole loop becomes:
curl "https://z3950.nibroc.fr/?server=lx2.loc.gov:210/LCDB&isbn=0066620724&format=usmarc" \
| npx jsmarc display - --format=marc21
A record comes off a decades-old protocol and lands, explained, in a modern terminal. That was the point at which the two halves of the project connected: old infrastructure on one side, modern tooling on the other, with the parser sitting between them.
What building it revealed
The format itself is simple enough. The difficult parts appeared where its assumptions collided with those of modern software.
Bytes are not characters
The directory stores byte offsets, but JavaScript strings are sequences of UTF-16 code units. A single multibyte character is therefore enough to make naive string offsets diverge from the positions stored in the record. Read byte offset n as JavaScript character index n, and eventually you are slicing into the wrong field. The parser consequently has a small binary layer that measures and slices according to byte positions. It builds a lookup from byte positions to string indices in one pass, applying UTF-8 width rules as it goes.
The first implementation recomputed this information for every slice. That was perfectly adequate for a single field, but became quadratic when processing batches of records. Rebuilding it as one linear pass turned a seemingly insignificant implementation detail into the difference between a parser that works on examples and one that can process real collections.
The format does not tell you everything
MARC does not explicitly label a field as "control" or "data". The parser infers the shape from the field's contents: the presence of a subfield delimiter means indicators and subfields; its absence means raw control data. The format's apparent lack of explicit typing is therefore not necessarily a flaw. It is part of its design. Once the structural assumptions are understood, a single byte can be enough to determine the parsing strategy.
Real records are the test fixture
The parser is tested against genuine .mrc files from the Library of Congress, SUDOC and OpenEdition. The files are decoded so that their original byte positions survive intact, split into individual records, and compared against checked-in JSON representations. This matters because a synthetic MARC record is easy to make behave. A real library record contains the accumulated conventions, edge cases and historical residue of an actual cataloguing system. A real record is a far more hostile test fixture than one I would have invented.
Open threads
The long-term direction everyone points to is BIBFRAME and linked data: a possible move away from self-contained records toward a web of connected entities. But MARC is not going away simply because a successor exists. There are decades of cataloguing infrastructure behind it and billions of existing records. Migration, where it happens, is a long process through intermediate representations, conversion tools and systems that still need to understand the old format. MARC will probably remain legible long after it has ceased to be the format people choose for new systems. That is not really a contradiction. It is how infrastructure ages.
JsMarc is one small artifact of that process: taking a format that almost everyone in a particular domain depends on and almost nobody outside it can read, and making it legible enough to inspect, query and experiment with.