json_parser

A hand-written, to-spec JSON parser in Rust — built to reinforce recursive-descent parsing and trait design, and to have a real object to profile and optimize.

It ships two interchangeable lexer backends behind one Lexer trait: a streaming byte-by-byte reader that works over anything implementing std::io::Read, and a zero-copy reader backed by a memory-mapped file. The mmap backend is measurably faster (see Performance) at the cost of needing a real file on disk.

Quickstart

Requires a Rust toolchain new enough for the 2024 edition (1.85+).

git clone <this-repo>
cd json_parser
cargo build --release

# parse and time sample.json with the default engine
./target/release/json_parser

# parse a specific file with the zero-copy mmap engine and print the result
./target/release/json_parser --filepath path/to/data.json --engine mmap --display

Run the test suite:

cargo test

Features

• Full JSON grammar coverage: objects, arrays, strings (including \uXXXX escapes and surrogate pairs), numbers (fractions, exponents, overflow detection), and literals.
• Streaming-friendly: the default lexer works over any Read source, including sockets — not just files on disk. (See the simulated-TCP-stream test in Testing.)
• Zero-copy fast path: the mmap lexer borrows unescaped strings directly out of the mapped file instead of allocating; only strings containing escapes fall back to an owned buffer.

Architecture: the Lexer trait

Both lexer backends implement one trait. Three low-level methods are required:

Method	Purpose
next_byte	Advance to and return the next byte.
peek_byte	Look at the next byte without advancing.
return_byte	Push a byte back, undoing the last next_byte call.

Everything else — read_string, read_number, next_token, escape and surrogate-pair handling — is a default method built on top of those three, shared by both backends. The mmap lexer overrides read_string specifically to take advantage of having the whole buffer in memory at once (a memchr-based scan for the closing quote), falling back to the shared byte-by-byte implementation the moment it sees a \.

CLI usage

Flag	Default	Description
-f, --filepath <PATH>	sample.json	File to parse.
-e, --engine <normal|mmap>	normal	Which Lexer implementation to use.
-d, --display	false	Print each parsed value to stdout (otherwise only timing is printed).

json_parser --filepath data.json --engine mmap --display

Performance

Benchmarked with hyperfine against a 25MB JSON file (5 warmup runs, 15+ measured):

Engine	Mean	Range
normal (streaming)	232.2ms ± 1.5ms	230.1 – 236.7ms
mmap (zero-copy)	151.4ms ± 0.5ms	150.7 – 152.6ms

mmap is ~1.53× faster, driven by two changes made during profiling with samply: threading Cow<'a, str> through the token/value types so unescaped strings skip allocation entirely, and swapping the default HashMap hasher for ahash to cut hashing cost on object keys.

This isn't attempting to compete with serde_json — it doesn't have a custom float parser or SIMD string scanning, and building a generic JsonValue tree is inherently more expensive than serde's typed deserialization, which skips the tree entirely.

Design decisions

• Duplicate object keys are last-wins, and key order isn't preserved. RFC 8259 explicitly leaves this implementation-defined; JsonValue::Object is a plain hash map, not an ordered map.
• Numbers are f64. Values that would overflow to infinity are rejected with a parse error rather than silently returned as inf.

Testing

Unit tests plus quickcheck-driven property tests covering strings, numbers, and escape sequences. Notably includes a test that simulates an external server feeding the parser JSON in small chunks over a real TCP socket, to exercise the streaming lexer's buffer-refill logic — and a test that hits a live public JSON API to validate against real-world payloads (requires network access).

Roadmap

• read_number doesn't yet have the same fast-path treatment read_string got (still byte-by-byte via the shared trait default).
• Numbers with multiple '.' and / or 'e'/'E' won't error until the final f64 parse call. This could be caught sooner.
• Doc comments and // SAFETY: comments on the unsafe mmap-creation calls are still thin.
• No lib.rs yet — the parser isn't currently importable as a dependency from another crate.