The tree you never build: a JSON parsing deep read
Arthur Jamet's 25 September post parses JSON straight into a record whose fields start out undefined and finishes the job with a bit-set counter, skipping the JSON tree entirely and running about three times faster in his own benchmark.
Arthur Jamet published Parsing JSON Objects without intermediate ASTs on his blog on 25 September 2026. 1 The post does what most serialisation guides spend their first section warning you against: it builds the record before there is anything to put in it, leaves the fields as
undefined on purpose, and lets the parser fill them in as it walks past. On the author's own benchmark the result is about three times faster than the same code going through a JSON tree, and it still reports cleanly when a required field never arrives.The detour through the tree
Almost every JSON library in a typed language works in two stages. A parser turns the bytes into a representation of JSON itself, and a second function walks that representation and produces the value the program actually wants. In Haskell the first stage looks like this — a data type whose constructors mirror the six kinds of JSON value, with small wrapper types for the payloads:
data JSON
= Number Number
| Bool Bool
| List [JSON]
| Null Null
| String String
| Object ObjectThe second stage converts it, either by hand or through a deriving mechanism such as Template Haskell's
deriveFromJSON or aeson's generic instances:data Album = Album
{ title :: T.Text
, artistName :: Maybe T.Text
, songCount :: Int
}
fromJSON :: JSON -> Maybe Album
fromJSON = \case
Object obj -> do
title <- obj !? "title"
artistName <- obj !? "artist_name"
songCount <- obj !? "song_count"
return Album{..}
_ -> NothingThe middle value is an intermediate representation, and it costs something. Jamet's accounting is blunt: an allocation at least as large as the final object, plus a second pass to convert it. What the detour buys is real, and the post says so. Keeping parsing and validation apart is what lets a library derive the conversion for you, and it is why a generated
fromJSON reads easily and tests easily. 1A value that is half-built on purpose
The idea is to delete the middle. Instead of parsing into JSON and converting, parse straight into
Album — and accept that the value is broken for most of the parse. Laziness makes that expressible. The record can start with its fields left as undefined, or be built with no fields filled in at all, which GHC accepts while warning about the fields left out. Nothing forces those fields while the parser is still working, so nothing crashes. 1The remaining question is when the value stopped being broken. Evaluating
undefined to find out is the one thing that must not happen, so the post threads a second value alongside the record: a bit set, one bit per field, starting at maxBound and clearing a bit each time the matching field is set.parseAlbum :: Parser Album
parseAlbum = case runParser (go Album{} maxBound) of
(a, 0) -> return a
(_, bits) -> fail ...
where
go :: Album -> BitSet -> Parser (Album, BitSet)A bit set of zero at the end means every field arrived. Anything else means the input omitted fields, and the failure branch turns that into an error message naming them, in place of a crash somewhere downstream. The matching step underneath reads like this:
go'' :: Album -> BitSet -> Parser (Album, BitSet)
go'' album bitset = do
fieldName <- parseString
char ':'
case fieldName of
"title" -> do
title <- parseString
return (album{title = title}, bitset `clearBit` 0)
"artist_name" -> do
artistName <- parseString
return (album{artistName = artistName}, bitset `clearBit` 1)
"song_count" -> do
songCount <- parseInt
return (album{songCount = songCount}, bitset `clearBit` 2)Optional fields need one more rule. When the bits still set at the end all belong to fields wrapped in
Maybe, the parser writes Nothing into them and returns the value as complete. 1Two constraints come with the trick, and both are easy to trip over later. The record's fields have to stay lazy: GHC's
StrictData extension makes constructor fields strict by default for a whole module, which would force the undefined fields during the parse and defeat the approach. 2 And the parser keeps its completion state on the side rather than inside the record, which is where the returned (Album, BitSet) pair comes from.
Let the compiler write the parser
The case expression in
go'' holds one branch per field, and each branch names a field, a type and a bit position. That structure follows directly from the data type, which is why Jamet generates it with Template Haskell, GHC's facility for compile-time meta-programming, instead of writing it out by hand. 3 The parsing engine underneath is flatparse, a high-performance parser over strict bytestrings, and the implementation sits in a package that describes itself as compile-time generated de/serialisers in Haskell. 45Background, and the author's own earlier work rather than part of this post: writing meta-programs against Template Haskell's AST types gets verbose, so Jamet built a small DSL that uses lenses and the
State monad to assemble those ASTs, and he explains the reasoning in a February 2026 post. 6What the benchmarks say
The measurement compares three configurations over two object shapes, using the
Criterion benchmarking harness on one machine: two Xeon Gold 6244 CPUs at 3.60 GHz, 32 GB of RAM, Ubuntu 22.04. 1| Library / object | Book | Author with a list of books |
|---|---|---|
| aeson (with an intermediate representation) | 1.051 µs | 2.901 µs |
| Jamet's library (with an intermediate representation) | 922.7 ns | 2.813 µs |
| Jamet's library (without one) | 314.2 ns | 1.027 µs |
Every row is from the same Criterion run in the post. 1
The middle row is what makes the table readable. It runs the author's own parser through a generic JSON value and then through a hand-written
fromJSON, so its gap to the third row isolates one thing: the cost of the intermediate representation. That gap is the roughly three-fold difference he reports. The gap to aeson in the first row measures something else, because aeson is different software with options the prototype never implements. 7Everyone involved calls these micro-benchmarks. They cover objects the size of an
Album on a single machine, and the author says they may not reflect performance in real applications. 1What the approach gives up
The bit set answers one safety question: was a required field missing. The
Maybe sweep answers the optional version of the same question. The third case the post names — fields present in the input but absent from the data type — falls to the same name matching, where each member's name is read and only the known names lead anywhere. 1 Everything else about the parse is the parser's ordinary business.The costs are the ones the design implies. Parsing and validation are now a single step, so the code is harder to follow and harder to maintain. It also cannot offer the customisation a real library needs: field-name casing, configurable behaviour on missing fields, handling for sum types. The prototype's scalar parsing falls short of full compliance as well — escaped characters in strings are only partly handled, and only some number formats are supported. The author calls the whole thing a proof of concept. 1
He also declines the conclusion the benchmark invites. His stated target is the runtime cost of intermediate representations, and he says outright that the representations are useful and that his approach trades maintainability for speed. 1
The portable part of the idea is the shape of the move rather than its Haskell details. A strict language would need an empty value, such as a zeroed region, where this code relies on
undefined. Rust's serde already does the compile-time version of the same move, generating its implementations with a derive macro instead of consulting a runtime data model. 18Where that leaves a reader deciding: the technique earns its complexity when deserialisation of a fixed shape sits on a hot path and the record is safe to build in place. It loses that argument when the schema varies, when the options matter, or when the code passes to other hands.
Verbatim quotes
"In this post, I will show a way to parse JSON objects without using an IR, by using partially-initialised data.""I know, this sounds horrible and unbearably unsafe. But I believe this approach can help the compiler during its optimisation pass.""The problem is thatPartialAlbumis an IR, so we want to avoid that."— Arthur Jamet, Parsing JSON Objects without intermediate ASTs
"The goal of this post is not to shame libraries likeaesonfor relying on IRs. IRs are useful and allow decoupling the parsing process from the validation step. Our approach merges both, and makes the code more complex and less maintainable but significantly faster."— Arthur Jamet, Parsing JSON Objects without intermediate ASTs
References
- 1Parsing JSON Objects without intermediate ASTs
arthi-chaud.github.io
- 2GHC User's Guide — Bang patterns and Strict Haskell
ghc.gitlab.haskell.org
- 3GHC User's Guide — Template Haskell
ghc.gitlab.haskell.org
- 4flatparse
hackage.haskell.org
- 5
- 6From AST definition to monadic DSL
arthi-chaud.github.io
- 7aeson
hackage.haskell.org
- 8Serde — Overview
serde.rs
This story was produced automatically by a channel. One sentence is all it takes for Neodrop to keep producing for you.