Celebrating ten years of jsonparser by taking back the “fastest” title
From 'fastest' to 'one of the fastest' and back, with a formal proof along the way.
Ten years ago I claimed it was the fastest JSON parser for Go. The ecosystem caught up, and “the fastest” quietly became “one of the fastest.” This summer I went back to make the claim true again, and to formally prove the library correct while I was at it. The technical appendix at the bottom has every fix, every failed experiment, and the code.
TL;DR:
jsonparser turned ten: 5,600+ stars, used inside Grafana, Tyk, Keybase, lux, and the Sentry, Solana, and even indirectly in Docker and Istio
Six releases in one summer: 50 open issues and 12 open PRs down to zero, 12 real bugs fixed, 25+ new APIs, zero breaking changes
It became mine first Go library with a full formal proof case: 123 formally prooved requirements, 100% MC/DC coverage, zero audit findings
It is the fastest parser in the benchmark again, ahead of gjson and sonic, and still the only one that allocates nothing
Ten years ago, I was building a company dashboard. It needed to analyze gigabytes of JSON data (server logs, API responses, analytics events), extract insights, and visualize them in real time.
I had a choice. I could set up a proper database pipeline: ingest, transform, index, query. I could build aggregation layers and ETL jobs and wait minutes for each dashboard refresh. That’s what everyone did. That’s what you’re supposed to do.
I didn’t want to do that.
I wanted real-time analytics from plain files, with no database and no ingestion pipeline to maintain. Just read the JSON, pull out the fields I needed, and show the numbers. The problem was that Go’s standard encoding/json was too slow for that. Deserializing gigabytes of JSON into structs or map[string]interface{} took too long and allocated too much memory.
Performance is a feature. That conviction started the whole thing.
I didn’t need a full parser. I needed three fields out of a 1MB JSON document, so there was no reason to deserialize the entire thing or build a tree. If I just walked the bytes and extracted the path I wanted, I could skip 90% of the work.
So on March 20, 2016, I released buger/jsonparser, a path-based, zero-allocation JSON parser that worked directly on bytes. The original README made a wonderfully restrained claim:
“Alternative JSON parser for Go (so far fastest)”
It was 10x faster than encoding/json and allocated nothing. And it was enough. The real-time dashboard worked, reading gigabytes of JSON from plain files with no database in sight. People like to overengineer. I like plain files and fast code.
The library caught on. Over the next decade it accumulated 5,600+ stars and 450+ forks. The ecosystem moved, as ecosystems do. Generated-code parsers like easyjson and ffjson became serious competitors. Go’s own encoding/json improved. The claim gradually softened from “the fastest” to “one of the fastest.”
Somewhere along the way, the dashboard itself was retired. I moved on to other projects. The library kept working without me, which is both the best and the strangest fate for code you write: at some point, it stops needing you.
When I finally sat down to see who actually depends on jsonparser, I expected a few personal projects and some CLI tools. What I found it is now used by Grafana, Lux, Tyk, Keybase, Solana, Sentry, and even indirectly by projects like Docker and Itsio, and much more! A nice thought, right up until you realize it also makes my bugs other people’s problems. And I have a few public CVEs which freaked out everyones scanners (nothing THAT serious thou! - but read the small text when when you join Google OSS Fuzz program 😅)
In July 2026, I went back in
I received Anthropic’s Claude for Open Source program, which gives maintainers of qualifying projects six months of the $200-a-month Max plan for free; jsonparser qualified on stars alone. This comeback is part of what that grant bought. If you maintain something old and widely used, apply. The backlog you have been avoiding for years is suddenly a summer project!
The repository had 50 open issues and 12 open pull requests. But most important we no longer were the fastest!
For its tenth birthday, someone did.
Six releases shipped: v1.1.0 through v1.6.0. The scoreboard:
50 open GitHub issues → zero
12 open pull requests → zero
4 formally tracked known issues → zero
12 real bugs found and fixed
25+ new backward-compatible APIs added
Zero breaking changes
And after a decade of competition, jsonparser was once again the fastest parser in the benchmark, across all three payload sizes.
The scoreboard matters less than what we found on the way there, though.
Formal verification, and what it did not magically solve
The comeback began with putting jsonparser through Proof, which made it the public Go library with a full L3 assurance case. Concretely, that means 123 formal requirements, full traceability across the public API, 100% Modified Condition/Decision Coverage, and an audit result of zero errors and zero warnings. The proof artifacts live with the code rather than in a detached report.
MC/DC is stronger than ordinary line or branch coverage: it demands evidence that each condition in a decision can independently flip the outcome, which is why safety-critical software relies on it.
But 100% MC/DC does not mean “there are no bugs.” We proved that distinction almost immediately.
The scalar-array corruption path was covered. Both sides of its main decision were exercised. The problem was that the wrong input category entered the wrong branch. MC/DC knew the branches were reachable; it did not know which output was semantically correct. The code ran everywhere, and nobody had ever written down what it was supposed to do. That is the verification gap.
That led to a blameless proof-gap postmortem. Every escaped defect forced the assurance model to get stronger. For the aliasing bug, we added an explicit “must not mutate the input buffer” obligation. For the EachKey inconsistency, we added a cross-API consistency gate. For the benchmark mistake (more on that shame in a moment), we added a benchmark-honesty lint.
Why ordinary fuzzing missed it
jsonparser had already been through OSS-Fuzz. It found a real Delete panic. It still missed several of the bug classes above.
The reason was reachability. The fuzz harness mutated JSON bytes, but used fixed key paths like "test". No amount of byte mutation can discover a panic that requires an empty path component if the path is never mutated. Likewise, a fuzzer cannot reach an out-of-range array-index mutation when it only generates ordinary object keys.
So I built probelabs/json-fuzz, a structure-aware JSON fuzzer. It generates valid JSON from a grammar, then applies mutations at meaningful structural boundaries: after a colon, inside a Unicode escape, before a closing delimiter, or in an adversarial key path. It runs at roughly 250,000 inputs per second, and a crash is not the only failure it looks for: its gates include output validity, round trips, numeric differential tests against encoding/json, offset bounds, determinism, aliasing, and input preservation.
That fuzzer found bugs years of normal use and blind mutation had missed.
The benchmark had been lying since 2017
The funniest and most painful discovery was sitting in issue #126, opened in October 2017.
The benchmark’s payload types had ffjson-generated MarshalJSON and UnmarshalJSON methods. When the supposed encoding/json benchmark received those types, Go correctly called their custom methods. So the “encoding/json” column was not really measuring encoding/json. It was measuring ffjson-generated code through the standard-library interface.
The issue was right. It stayed open for almost nine years. To the person who filed it: you were correct the entire time, and I am sorry it took a decade and a birthday to say so. However, benchmark was wrong on the other side - our numbers were even better then we originally thought!
For sure comparing with encoding/json is not fully fair, as it does not only parsing. And Go team encoding/json got 2.4x faster over 10 years.
We fixed the benchmark by introducing plain payload types, adding the competitors people would actually ask about (tidwall/gjson and ByteDance’s sonic), updating every library to its latest version, recording the hardware and Go version, and taking results as the median of five runs.
Path to regain the fastests badge
To get back to the “fastests” first place, I started with “ADHD” skill: a structured process that looks at the same problem through multiple independent cognitive frames (hardware engineer, competitor, speedrunner, assumption-remover, biologist), deliberately producing divergent hypotheses before converging.
Five frames generated 30 optimization ideas. Fifteen of those became isolated worktree experiments. Most did not win, which is fine. The job is to make many claims cheap to falsify, not to have one clever idea.
The biggest win came from an almost absurd piece of repeated work. stringEndConfig found the first quote, then searched the entire remaining parent document for a backslash. On a 24KB payload, it could walk tens of kilobytes for every string, looking for a character that wasn’t there.
Bounding that scan to the actual string body took the large benchmark from roughly 128µs to 22µs. A 5.8x improvement from a one-line fix.
Two more wins followed: a single 8-byte SWAR (SIMD-Within-A-Register) pass that fuses the quote and backslash scans (another eight percent), and a fast-skip trick I found by reading gjson’s source, which skips every byte above 0x5C in one unsigned comparison because all JSON structural characters sit at or below the backslash. That one bought 11 to 19 percent on the small and medium payloads, exactly where gjson had been winning. The final margin owes something to the runner-up’s own technique. Credit where it is due: tidwall writes fast code.
The final large-payload results on an Apple M4 Max with Go 1.26.3, all libraries at latest versions:
That is 13% ahead of gjson, the other path-based parser and the obvious head-to-head comparison. It is twice as fast as ByteDance’s SIMD-accelerated sonic, and 6.5x faster than encoding/json, while remaining the only parser in the table that allocates nothing. The medium and small payload benchmarks land the same way, with jsonparser variants at the top of both tables; the full results are in the README.
Ten years later, the “fastest” claim came back, this time on a corrected benchmark, against the strongest competition available, with a guard against repeating the old mistake.
Ten years of feature requests, without breaking ten years of users
Closing the backlog did not mean mechanically closing old tickets. Many represented legitimate gaps in the API, each one somebody’s real workflow.
The highlights: wildcard paths, a JSONPath compiler, error-returning iteration, a Config type with opt-in lenient parsing, and a streaming ReaderParser for inputs of 10GB and beyond. More than 25 new APIs in total; the release notes have the full list.
The compatibility policy was non-negotiable. Six releases is aggressive enough; forcing thousands of downstream users through a migration at the same time would have been irresponsible. Docker was not going to migrate for me.
What a comeback really means
The repository is now at v1.6.1. Zero open issues, zero open PRs, zero known issues, 123 formal requirements, zero audit findings.
“Zero known issues” does not mean “zero bugs forever.” If this session proved anything, it is that mature software can hide incorrect assumptions behind green tests, full coverage, widespread production use, and confident benchmark labels.
What changed is that jsonparser now has better ways to turn the next surprise into a permanent improvement. And it settled something for me personally: AI is what made six releases in one summer feasible for a single maintainer. The proof case is what made them safe to ship. Neither would have been enough alone.
There is a fashionable take right now that AI is killing open source. Agents strip-mine repositories for vulnerabilities, cloning a project takes an afternoon, so why maintain anything at all?
This project is my answer. The same economics that let an agent hunt for bugs in anyone’s code let me formally prove a ten-year-old library correct, in my free time, at a depth that used to require a safety-critical budget. I would not have believed that sentence a year ago.
The win is not that AI makes the work cheap. The win is that expertise and a clear idea of what should exist are now enough to ship it.
That does not look like open source dying to me. It looks like leverage finally landing on the maintainer’s side of the table.
In 2016, the project was an experiment in whether performance alone could replace an entire infrastructure category. Could you skip the database and just read files fast enough?
Ten years later, the answer is still yes. And the project picked up a second question along the way: how far can one maintainer take an old, widely used library if he is willing to reopen every assumption?
Including the flattering ones.
The dashboard I originally built is long gone. But the library outlived its reason to exist and grew into a hundred reasons I never planned. It is now inside more software than I can track, carries a formal proof case, and is faster than it has ever been.
So, to everyone who filed an issue over these ten years, sent a pull request, argued with me in a comment thread, or shipped jsonparser without ever saying hello: thank you. It turns out a library’s tenth birthday is less about the code than about the people, the slow and mostly invisible collaboration between one maintainer and thousands of strangers he will never meet.
That feels like a better anniversary than a cake.
jsonparser is on GitHub. The formal verification tooling is Proof. The fuzzer is probelabs/json-fuzz.
Appendix: the performance story, fix by fix
For readers who want the exact mechanics behind “the fastest claim came back,” this is the full account: three fixes that worked, ten experiments that did not, and why.
The starting point
The main story covers how the benchmark had been mismeasuring encoding/json since 2017. With that fixed, the honest starting point on an Apple M4 Max with Go 1.26.3 was roughly 128µs on the large payload. Not the fastest anymore, by a wide margin.
Fix 1: bound the backslash scan (5.8x)
stringEndConfig found the closing quote via bytes.IndexByte(data, '"'), then checked for escapes via bytes.IndexByte(data, '\\'). But data was the entire remaining parent slice, not the string body, so the escape check could walk 24,000 bytes for a single short string.
// Before (scans entire remaining document):
firstBackslash := bytes.IndexByte(data, '\\')
// After (scans only the string body):
if bytes.IndexByte(data[:firstQuote], '\\') == -1 {
return firstQuote + 1, false
}
Result: 128µs to 22µs. One line changed.
Fix 2: one SWAR pass instead of two SIMD calls (8% more)
After fix 1, the CPU profile still showed 46% of time in bytes.IndexByte. The function made two SIMD calls per string, one for the quote and one for the backslash, and each call carries roughly 5 to 10ns of function-call overhead before any scanning happens. The replacement is a single inline 8-byte SWAR loop that checks for both characters at once:
const swarLsb = 0x0101010101010101
const swarMsb = 0x8080808080808080
broadcastQuote := uint64(quote) * swarLsb
broadcastBackslash := uint64('\\') * swarLsb
for i+8 <= n {
w := binary.LittleEndian.Uint64(data[i:])
xq := w ^ broadcastQuote
xb := w ^ broadcastBackslash
quoteHit := (xq - swarLsb) & ^xq & swarMsb
bsHit := (xb - swarLsb) & ^xb & swarMsb
if quoteHit|bsHit == 0 { i += 8; continue }
break
}
The SWAR formula (x - 0x0101...01) & ^x & 0x8080...80 detects zero bytes (matching bytes after the XOR broadcast) in a 64-bit word using pure ALU operations. Go’s NEON bytes.IndexByte is faster per byte, but for the “find quote, then find backslash” pattern the call overhead dominates, and the fused loop wins. Result: 22µs to 21µs.
Fix 3: gjson’s fast-skip (11 to 19% on small and medium)
gjson was still 1.8x faster on the medium payload. Its inner loops carry a deceptively simple trick:
for ; i < len(c.json); i++ {
if c.json[i] > '\\' { continue } // skip bytes > 0x5C
// Only structural characters reach here (all <= 0x5C)
}
Every JSON structural character sits at or below \ (0x5C): the quote is 0x22, the colon 0x3A, the comma 0x2C, the open bracket 0x5B, the backslash itself 0x5C. A single unsigned comparison therefore skips the vast majority of content bytes in one branch. We applied it to three hot loops: the per-byte tail of stringEndConfig, and the main loops of blockEndConfig and searchKeysConfig. For the latter two, {, }, and ] are explicitly excluded from the skip, since those structural characters sit above 0x5C.
Payload Before After Improvement Small (190B) 382 ns 339 ns 11.3% Medium (2.4KB) 3,899 ns 3,141 ns 19.4% Large (24KB) 20,788 ns 20,114 ns 3.2%
Medium benefits most because it does the most sibling skipping, navigating past large nested objects to reach the target key.
What didn’t work: ten failed experiments
The lesson from the failures: on Apple Silicon, Go’s compiler and runtime are already extremely well optimized. NEON bytes.IndexByte is hand-tuned assembly, and the branch predictor chews through per-byte switches at about a cycle per byte. The wins came only from eliminating work entirely (the bounded scan), fusing operations (one SWAR pass instead of two calls), or reducing per-byte branch cost (the fast-skip). Micro-optimizations to dispatch and depth logic returned under 2% because they targeted the wrong 33%.
The cumulative journey
Stage Large payload What changed Before any fix ~128µs stringEnd scanned the entire parent document Fix 1: bounded scan ~22µs (5.8x) Scan only the string body Fix 2: SWAR string scan ~21µs (+8%) Fuse quote and backslash into one 8-byte pass Fix 3: gjson fast-skip 20µs (+5%) Skip non-structural bytes in one comparison Total 128µs → 20µs 6.4x, and the top of the leaderboard
If you read all the way past the appendix: thank you. You just finished roughly four thousand words about a JSON parser, which statistically makes you one of my people. The issue tracker is at zero right now, so if you find something, you know where to file it. I promise the response time will beat nine years.