Benchmark: block-memo × incremental parse

This page preserves the in-browser comparison study for block memoization and incremental prefix-freeze parsing. It separates pipeline stage time from React commit time and keeps the measured payload sizes, run counts, and noise bands so the reported savings can be interpreted and reproduced.

The tables are a 2026-07-15 snapshot on 1.x development builds. At that time the switches were config.blockMemoEnabled and config.incrementalParseEnabled; their current names are blockMemo and incrementalParse, both enabled by default. The “v2 regimes” label below refers to the second iteration of the incremental engine in that study, not a claim that package 2.0 was benchmarked on that date.

Current releases have changed preprocessing, boundary detection, planning, registry subscriptions, and code presentation. The old measurements remain evidence for the tested implementations, not current latency promises. Read streaming and performance for the implementation now in the repository.

There are three measurement tools with different scopes: Storybook comparison stories generated the manual tables here; pnpm bench:unit runs the Vitest LaTeX microbenchmark; and benchmarks/ contains the production browser harness invoked by pnpm bench:web. None automatically regenerates another tool’s historical tables.

Methodology

• Harness: the Storybook A/B stories under React → Performance Lab → Streaming Comparisons — BlockMemoCompare, IncrementalParseCompare, BoostCompare (plus *Isolated process-isolated variants, not used for this run — same-page is the fairest JS-layer A/B since both sides share one main thread and receive the same stream in the same commit).
• Scenario: randomTokens — 2–8-char chunks at randomized 15–60 ms intervals, seeded PRNG, the closest shape to real LLM token streams.
• Payload: the Storybook stress payload (headings, prose, code fences, $$ math, tables, blockquotes, lists) at 4× (2,968 chars / 29 blocks) and 16× (11,872 chars) scales.
• Config: spies OFF (clean timing), registry OFF (standalone), defs OFF.
• Runs: Run ×3 at 4× (the noise-band machinery needs same-config runs); single runs at 16× (each run is ~90 s of wall-clock streaming).
• Environment: Apple M3 Pro · Chrome · Storybook dev server · React dev build — absolute milliseconds run large; only the relative gap between the two columns of a comparison is meaningful.
• Date / version: 2026-07-15, on the commit series introducing incremental parsing (post-v1.5.1, pre-1.6.0). Recalibrated the same day after the review-hardening pass (six added detector blockers, checkpoint incremental scanning, inline code-span masking): headline percentages were unchanged within noise — the tightened blockers cost nothing on these corpora — while the scan stage itself dropped from 9.2 ms to 3.9 ms at 4× and from 84 ms to 13 ms at 16× (the checkpoint scanner’s O(document) → O(tail) effect).
• v2 re-run (same day, single runs, after footnote injection replay + cross-chunk phantom suffixes + stripped-node alignment landed): the plain 4× headline is unchanged within noise (385 ms vs 2,254 ms → 83%), and two previously-fallback regimes now splice — see “Results — v2 regimes” below.

Each comparison also runs the built-in per-frame DOM-equality verifier (clobber prefixes normalized): both sides’ live innerHTML must be byte-identical on every streamed frame.

---

Results — 4× payload (2,968 chars, ~560 frames, Run ×3)

Comparison	Sides	Headline metric	Result
BlockMemoCompare	memo vs legacy	commit-total Δ	+50 / +199 / +151 ms (mean +133) — within the ±203 ms noise band → tie
IncrementalParseCompare	incremental vs full parse (both memo)	pipeline (scan+parse+transform)	353 ms vs 2,174 ms → 84% saved
IncrementalParseCompare	〃	commit Δ across runs	+1,740 / +1,676 / +1,603 ms — stable, well beyond noise
BoostCompare	memo+incremental vs legacy	commit total	2,956 ms vs 5,196 ms → 2,240 ms saved (43%); runs +2,196 / +2,255 / +2,240

Per-side stage breakdown (incremental axis, sums over ~560 frames):

Stage	incremental ON	incremental OFF
scan	9.2 ms	—
parse	202.5 ms	907.1 ms
transform	141.4 ms	1,266.4 ms

Note the transform (remark/rehype plugin chain) saving exceeds the parse saving itself — the tail-only run skips the plugin chain over the frozen prefix too, which the pre-implementation estimates did not count.

Results — v2 regimes (footnotes, cross-chunk; 2026-07-15, single runs)

Comparison	Regime	Headline metric	Result
IncrementalParseCompare, 4×, defs ON	footnote/link defs tail — previously the [^ fallback	pipeline	421 ms vs 2,675 ms → 84% saved (658 frames, 0 mismatches)
IncrementalParseCompare, 4×, defs OFF	plain (regression check vs v1’s 353 / 2,174 → 84%)	pipeline	385 ms vs 2,254 ms → 83% — unchanged within noise
BoostCompare, 4×	end-to-end (v1: 43%)	commit total	2,026 ms vs 4,004 ms → 49% saved
CrossChunkIncrementalCompare, 1×	coordinated, 3 chunks/side, phantom churn — previously ineligible	pipeline	104 ms vs 141 ms → 26% saved (190 frames, 0 mismatches)

The headline v2 fact: a defs-bearing payload now saves the SAME 84% as a plain one — in v1 that toggle measured the [^ full-parse fallback (stage numbers converged). The cross-chunk number is modest by construction: per-chunk documents are short (the 1× payload splits into ~330-char chunks) and every cross-chunk reference pins the boundary below it (the taint IS the correctness mechanism) — the win grows with chunk length, exactly like the standalone axis.

Results — 16× payload (11,872 chars, ~2,220 frames, single runs)

Comparison	Headline metric	Result
BlockMemoCompare	commit-total Δ	4,094 ms saved (7.0%), beyond the ±2,347 ms noise band; p95 commit −12.4 ms
IncrementalParseCompare	pipeline	1,706 ms vs 28,455 ms → 94% saved; commit Δ +24,468 ms
BoostCompare	commit total	25,563 ms vs 57,353 ms → 31,790 ms saved (55%)
BoostCompare	p50 commit	7.5 ms vs 32.4 ms (4.3×) — typical React commit below 16.7 ms; total frame includes other work
BoostCompare	p95 commit	71 ms vs 105 ms

Correctness (all runs)

Every run’s DOM-equality verifier reported 0 mismatches (4×: 597 frames; 16×: 2,372 frames). The boost axis crosses all three output contracts at once: legacy ≡ block-memo ≡ spliced.

Cross-check

At 16×: boost (31.5 s) ≈ block-memo alone (4.1 s) + incremental alone (24.3 s) = 28.4 s — the axes close to within single-run noise (±2.3 s) plus baseline-interaction effects (the incremental axis is measured on a memo-enabled baseline).

---

Interpretation

1. Incremental parsing is the dominant win for long streaming documents, and it scales. Pipeline savings grow from 84% (4×) to 94% (16×) because a full parse revisits the document while a successful splice parses and transforms the unfrozen tail. This does not establish tail-only complexity for the complete renderer: planning and other document-wide work remain. The measurement-study estimate (70–89%, parse-only) is exceeded in the real pipeline because the transform plugin chain is skipped over the frozen prefix too.
2. Block-memo alone ties at small/medium payloads and wins clearly at large ones (7% commit total, −12 ms p95 at 16×) — exactly the regime split its documentation describes. Its role is the render-layer guardrail and the host for incremental parsing.
3. The user-perceivable number is boost p50: 32.4 ms → 7.5 ms per commit at 16× — a substantial reduction in measured React work. A commit below 16.7 ms still shares a frame with layout, painting, other scripts, and browser work; it does not establish sustained 60 fps.

Footguns — reading these numbers

• Dev-build milliseconds are inflated. React dev mode adds bookkeeping everywhere. Never quote the absolute values; quote the relative gap, and re-measure any production claim in a production build.
• Respect the noise band. The stories widen/tighten it from same-config run history; a delta inside the band is a tie no matter how green it looks. This page once mis-read small-payload noise as a regression — twice (see BlockMemoComparison.tsx header).
• Spies exaggerate. The component-count spies cost time proportional to render count, dragging whichever side renders more. All numbers above are spy-OFF.
• Same-page vs isolated. Same-page shares one main thread — fps / long tasks are page-wide, only React JS work is per-side. The *Isolated stories trade stream-delivery symmetry for genuinely per-side browser-level signals. Disagreement between the two is itself signal.
• Stage panels vs commit totals. On the incremental axis the stage table is the attribution-clean signal and commit Δ is noisy garnish; on the boost axis the legacy side emits no stage timings at all, so the commit total IS the headline.
• The defs toggle no longer measures a fallback. Since v2, footnote payloads splice (injection replay) — flip it to measure the replay’s overhead against a plain payload (≈nil: 84% vs 83% above). What still pins the boundary is an UNRESOLVED reference: content that opens with a ref whose def arrives much later re-parses everything after the ref until the def settles.

Reproducing

Manual (the comparison stories drive the real renderer in a browser):

pnpm storybook   # → http://localhost:6006

Open React → Performance Lab → Streaming Comparisons → BlockMemoCompare / IncrementalParseCompare / BoostCompare / CrossChunkIncrementalCompare; set payload scale, turn spies off, hit Run ×3; read the verdict banner (block-memo axis) or the summary strip (incremental/boost/cross-chunk axes). For per-side browser-level metrics use the *Isolated variants from a loopback hostname on the dev machine.

Re-measure a current change

Choose the question first. For an incremental-parser change, compare parse and transform stages with identical plugin selections and input snapshots. For a custom renderer or code-highlighting change, use a production browser app and include the same CSS and providers an application uses. For coordinated references, include multiple mounted chunks and definition changes; standalone numbers cannot estimate registry fanout.

Record the commit, package versions, browser, device, build mode, payload, delivery schedule, and repetitions with the result. Preserve correctness checking alongside timing. Normalize generated document prefixes only where the equality harness requires it; do not erase a meaningful href, missing footer, or reordered node merely to make a comparison pass.

Compare each scenario with itself before and after the change. Report medians or repeated runs with their spread, identify timeouts explicitly, and separate warm startup from steady streaming. A large relative parser saving can coexist with a small end-to-end improvement when highlighting or layout dominates; that is a useful attribution result, not a contradiction.