docs(sync): convict the loose-write residual — FAT linear dir scans

sd_bench dir-entry scaling: stat-miss is ~0.1 ms/entry and the
loose-object composite hits p50 402.9 ms at 256 siblings, reproducing
the field ~360-400 ms/loose-write from primitives. The objects/
fan-out is ~256 dirs, so every path resolution under it pays the scan.
Bounded and accepted; levers (fewer resolutions, pack-not-loose) go to
a future perf pass.
This commit is contained in:
Julien Calixte
2026-07-14 09:44:47 +02:00
parent 7378d30450
commit fe62b47eee
2 changed files with 55 additions and 14 deletions

View File

@@ -433,6 +433,38 @@ walk FAT directories linearly; consistent with the orphan-creep signal) — is
one instrumentation pass for later (log each `p_mmap` miss + bench dir ops in
`sd_bench`), not a prerequisite for the plumbing.
**Run 6 (2026-07-14, `sd_bench` directory-entry scaling): the residual is
CONVICTED — FAT linear directory scans.** Two independent signals, both from
the real card at 10 MHz:
1. `stat` on a **missing** name (the freshen probe's shape — no early exit,
the whole parent is scanned) is linear in sibling count: 4.7 ms at 8
siblings → 12.8 ms at 64 → 28.9 ms at 256 ≈ **~0.1 ms per directory entry
per scan**. (`stat` on a *present* name plateaued at ~8 ms, but that's the
bench's shape — 20 iterations only ever touch the first 20 entries, which
sit early in the scan.)
2. The **loose-object composite** (freshen stat-miss + temp create + write +
remove + rename, libgit2's exact sequence) scales the same way and lands
exactly on the field number: mean 95.5 ms at 8 siblings → 163.6 at 64 →
**366.8 ms (p50 402.9) at 256** — the observed ~360400 ms/loose-write,
reproduced from primitives. The multiplier over a single scan is the
sequence's ~56 full parent scans (every miss probe, free-slot search, and
rename-target check walks all N entries; 38-hex LFN names burn several
entries each, steepening the slope).
The real repo's `.git/objects/` holds up to 256 two-hex fan-out dirs plus
`pack/`/`info/`, so every path resolution under it pays the ~256-entry scan —
that's the whole residual. It composes with the earlier finding that the
splice does **0 pack reads** during these writes (run-1 `p_mmap` counters):
nothing here is data I/O. Consequences: (a) the cost is *bounded* — ~0.4 s ×
loose objects per commit (typically ≤ 5) ≈ ≤ 2 s, already inside the shipped
splice numbers, so nothing burns; (b) the only real levers are *fewer
path-resolutions per object* (libgit2 surgery) or *not writing loose objects
at all* (commit straight into a small pack) — both out of scope until a
future perf pass; (c) orphan-creep is now mechanically explained: each
leftover loose object adds entries to a fan-out dir and slows every later
scan of it.
### The walk is ~1.4 s even at N ≈ 2
Mostly fixed cost — the worktree-diff setup and the second (`update_all`) pass —
@@ -613,10 +645,12 @@ toy — the toy pack understates everything by ~2 orders of magnitude.
### Still open (none block the plumbing)
- The residual ~360 ms/loose-write ≈ 8 unique small SD round-trips; next
suspect is FAT **directory-op** cost in the freshen/refresh path. One
instrumentation pass for later (log each `p_mmap` miss + bench dir ops in
`sd_bench`).
- ~~The residual ~360 ms/loose-write~~ **CLOSED 2026-07-14** — convicted as
FAT linear directory scans of the ~256-entry `.git/objects/` fan-out (run 6
above): `sd_bench`'s dir-entry-scaling section reproduced the field number
from primitives (loose composite p50 402.9 ms at 256 siblings). Bounded and
accepted; the levers (fewer path resolutions, or pack-not-loose commits)
are a future perf pass.
- Ref/reflog-update cost on the real repo — the bench's `commit(None)` writes
no ref, so the shipping commit's last leg is unmeasured.
- The push's ~6.5 s network floor

View File

@@ -114,16 +114,23 @@ Sync performance — inherited from the
warm `:sync` before hunting further. Run-3 data point: a rejected-push
`:sync` (worst case, three connections) totalled 24.0 s vs 59 s in
run 1; a clean warm publish is still unmeasured.
- [~] Instrument the residual ~360 ms/loose-write (suspect: FAT directory-op
cost in the freshen/refresh path) — **instrumentation BUILT 2026-07-14,
measurement run pending**: `sd_bench` grew a directory-entry-scaling
section (stat hit/miss + the loose-object composite at 8/64/256
siblings — FAT resolves paths by linear scan, so cost climbing with N
convicts dir scans; flash with `just flash-bench`), and `stage_and_commit`
now brackets the splice with the `p_mmap` counters, logging
`N mmaps / M KB read` per commit. First data point (run 1, 2026-07-14):
both real-repo splices logged **0 mmaps / 0 KB read**the residual is
FAT ops, not pack reads.
- [x] Instrument the residual ~360 ms/loose-write — **measured 2026-07-14,
CONVICTED: FAT linear directory scans** (full analysis in the
[tradeoff doc](tradeoff-curves/sync-commit-staging.md), run 6).
`sd_bench`'s dir-entry-scaling section shows stat-miss cost linear in
sibling count (~0.1 ms/entry/scan) and the loose-object composite
reproducing the field number exactly: p50 402.9 ms at 256 siblings —
the `.git/objects/` fan-out is ~256 dirs, and every path resolution
under it scans them. Composes with the run-1 splice counters (**0
mmaps / 0 KB read** — no pack I/O in the residual). Bounded (≤ ~2 s per
typical commit) and accepted; the levers — fewer path resolutions in
libgit2, or committing straight into a small pack instead of loose
objects — are a future perf pass, not v0.x work. (The bench run also
caught a cleanup bug: std's `remove_dir_all` mis-types directories via
the generic-unix DT constants and dies on FatFS `FR_DENIED`; `sd_bench`
now removes its scratch tree itself, same decoding as the palette
walk. The aborted run left `/sd/sdbench` on the card — the next bench
run's setup sweep removes it.)
- [ ] The images-off-card lever
([notes/git-sync-images-and-repo-size.md](notes/git-sync-images-and-repo-size.md))
composes with the splice — decide whether to take it here (user's call).