Compare commits

...

4 Commits

Author SHA1 Message Date
Julien Calixte
dade5e6bb3 docs(sync): record real-repo commit bench and TreeBuilder handoff
The real jcalixte/notes clone (570 MB pack, 1179 files) proved both index
strategies are O(N_tree) and unshippable: index.write re-hashes the tree
(611 s), and the index-free read_tree is 77 s cold and OOMs the mmap cache
(zlib crash). Record the run in the tradeoff curve, revise the verdict to an
O(depth) TreeBuilder walk, and add a handoff note with the design, firmware
call sites, and bench steps for the next session.
2026-07-12 15:04:59 +02:00
Julien Calixte
a6f52df8b6 perf(sync): bench the index-free commit path and mmap cache
Replace the index.write()/write_tree ops (which re-hash the whole working
tree on a fresh FAT clone -- up to 611 s on the real repo) with the
index-free candidate: Index::new + read_tree(HEAD) + blob + write_tree_to.
Add per-op mmap-cache + free-heap logging (via esp_map_stats), announce each
op before it runs so a hang reveals the culprit, mount_for_git for the FD
budget, tune the mwindow limits, and drop N 10 -> 3 for the slow real clone.
2026-07-12 15:04:52 +02:00
Julien Calixte
6c5e666f4b feat(persistence): add mount_for_git with a larger open-file budget
libgit2 keeps the pack, .idx and commit-graph descriptors open for the
repo's lifetime and opens loose objects on top, so a read_tree walk overruns
the editor's tight 4-FD budget with "no free file descriptors". Add
mount_for_git (16 FDs, matching the flash-FAT git binaries) alongside the
editor's mount; both route through a shared mount_with_max_files.
2026-07-12 15:04:44 +02:00
Julien Calixte
c5e119f6be perf(git): cache emulated pack mmap to avoid re-reading the pack
p_mmap emulated a mapping by malloc+read()ing the range from SD on every
call. libgit2 re-hits the pack idx/windows on every object write (via
git_odb__freshen -> git_odb_refresh), so a commit re-read pack bytes over
SPI repeatedly -- ~500ms-1.1s/op on a small repo, far worse on the real
570 MB pack.

Cache read-only mappings >= 64 KB (pack idx/windows, commit-graph, midx,
packed-refs -- all immutable on this device) in a refcounted, PSRAM-backed
slot table keyed by (dev, ino, size, mtime, offset, len), LRU-evicted under
a soft cap. Small mutable working-tree maps (diff_file.c) fall below the
floor and are never served stale.
2026-07-12 15:04:37 +02:00
6 changed files with 598 additions and 67 deletions

View File

@@ -12,3 +12,4 @@
| [`git-sync-images-and-repo-size.md`](git-sync-images-and-repo-size.md) | Why we don't shrink the notes repo — its 153 MB of media is remanso's image CDN, so rewriting history to slim the on-device clone breaks the web app. |
| [`boot-time-budget.md`](boot-time-budget.md) | Where the ~4.3 s to cursor goes — and why the ≤ 3 s v1.0 target is hard: one ~1.9 s full refresh is unavoidable at cold boot, so the splash is nearly free. |
| [`sync-latency.md`](sync-latency.md) | Where the ~16 s cold `:sync` goes — Wi-Fi + SNTP + one TLS push; why optimistic-retry cut a whole handshake, and why the rest is close to the protocol floor. |
| [`sync-commit-handoff.md`](sync-commit-handoff.md) | **Handoff:** the on-device commit must become an O(depth) TreeBuilder walk — `add_all`/`index.write` re-hash the whole 570 MB-pack tree (611 s / OOM) and the real repo has never synced. Design + firmware plumbing + bench steps for the next session. |

View File

@@ -0,0 +1,138 @@
# Handoff — the on-device commit must become an O(depth) TreeBuilder walk
> **Start here if you're picking up `:sync` performance.** Written 2026-07-12 after
> benching `git_bench` on a full clone of the real `jcalixte/notes` repo. Full
> analysis + numbers: [`../tradeoff-curves/sync-commit-staging.md`](../tradeoff-curves/sync-commit-staging.md).
> Latency memory: `sync-timing`. Why the repo can't shrink:
> [`git-sync-images-and-repo-size.md`](git-sync-images-and-repo-size.md).
## TL;DR
The current `firmware::git_sync::stage_and_commit` (`add_all``index.write`
`write_tree`) **cannot commit the real repo** — it is O(N_tree) and the real repo is
1179 files / 158 dirs / 570 MB pack / 150 MB of images. Measured on device:
- `index.write()` re-hashes the whole working tree → **up to 611 s** (10-min freeze).
- The index-free alternative (`Index::new` + `read_tree(HEAD)` + `write_tree_to`)
still reads the whole tree cold → **77 s**, and drove the `esp_map.c` mmap cache
to 7.4 MB, starving zlib so `repo.blob()` crashed (`zlib (5)`, 508 KB heap left).
**The real repo has almost certainly never completed a `:sync` on device** — only
the toy `typoena-test` (`notes.md`) has. The repo cannot be shrunk (the 150 MB of
images serve another app — see the images note). So the fix is a **new commit
mechanism**, not a tuning knob.
## The fix — O(depth) TreeBuilder walk
Rebuild only the edited file's ancestor subtree chain onto HEAD's tree. Never
materialise the 1179-entry index; never `index.write()`; never `read_tree` the whole
tree. Cost is O(depth × dirty_files), flat in repo size, tiny heap, and it carries
every unchanged entry (all 260 images, the other 1176 files) forward untouched —
which means **the device doesn't even need the images in its working tree.**
Sketch (git2 0.20 API — all present: `Repository::treebuilder(Option<&Tree>)`,
`TreeBuilder::{insert,remove,write}`, `Tree::{get_name,get_path}`,
`TreeEntry::{to_object,filemode}`):
```rust
/// Return a new tree OID = `base` with `path` set to `new` (Some(blob_oid) to
/// add/replace, None to delete). Reads ~depth subtree objects, writes ~depth trees.
fn splice(repo: &Repository, base: &Tree, path: &[&str], new: Option<Oid>) -> Result<Oid> {
let (head, rest) = path.split_first().unwrap();
if rest.is_empty() {
// leaf level: patch this tree directly
let mut tb = repo.treebuilder(Some(base))?;
match new {
Some(oid) => tb.insert(head, oid, 0o100644)?, // FileMode::Blob
None => { tb.remove(head).ok(); } // delete
};
return Ok(tb.write()?);
}
// descend: find (or synthesize) the subtree, recurse, re-insert its new OID
let sub = base.get_name(head)
.and_then(|e| e.to_object(repo).ok())
.and_then(|o| o.peel_to_tree().ok());
let empty; let sub_ref = match &sub { Some(t) => t, None => { empty = repo.treebuilder(None)?; /* ... */ unreachable!() } };
let new_sub = splice(repo, sub_ref, rest, new)?;
let mut tb = repo.treebuilder(Some(base))?;
// if new_sub is the empty tree and this was a delete, remove the dir entry instead
tb.insert(head, new_sub, 0o040000)?; // FileMode::Tree
Ok(tb.write()?)
}
```
Then, for the dirty set, fold each change through `splice` (thread the running root
tree), and `commit(Some("HEAD"), …, &final_tree, &[&parent])`. Edge cases to handle:
new files where an intermediate dir doesn't exist yet (build subtree from `None`),
deletes that empty a directory (remove the dir entry rather than insert an empty
tree), and path splitting on `/` (paths are repo-relative POSIX).
**Bench it first** (per the established discipline): add a `treebuilder splice→tree`
op to `firmware/src/bin/git_bench.rs` alongside the existing ones and confirm it's
sub-second cold on the real repo, with heap staying healthy (no OOM). Only then wire
it into the firmware.
## Firmware plumbing (after the bench validates)
1. **`firmware/src/git_sync.rs`**
- Rewrite `stage_and_commit` (currently ~L271332) to the `splice`-walk above.
Drop `add_all`/`update_all`/`index.write`/`index.write_tree`. Keep the
`commit split —` timing log, the `tree unchanged → nothing to publish` check,
and the signature/message code.
- `PublishRequest` (~L79) is currently an **empty struct** — it must carry the
dirty set: `{ changed: Vec<(String /*repo-rel path*/, ...)>, deleted: Vec<String> }`.
The commit needs the blob content or a way to read it; simplest is to pass the
paths and let the git thread `repo.blob_path(abs)` / read `/sd/repo/<path>`.
- `reconcile_onto_origin` (~L377/L394) uses `repo.reset(Mixed)` — with an
index-free commit there's no index to reset; switch to `ResetType::Soft` (move
HEAD only) or drop the reset and just re-`splice` onto the new origin tip.
- The macOS-cruft filter (`skip_macos_cruft`) is no longer needed — the walk only
ever touches paths the editor explicitly hands it, so `._*`/`.DS_Store` can't
sneak in. (Keep a note; don't silently lose the Spike-14 lesson.)
2. **Dirty-set source — `firmware/src/persistence.rs` + `main.rs`**
- Writes funnel through `Storage::save_path` (~L296) and deletes through
`Storage::delete_path` (~L332), both `&self`. Accumulate a dirty/deleted set
(needs `RefCell` interior mutability, or move the set up to `main.rs`).
- `Effect::Publish` handler in `main.rs` (~L222) builds the `PublishRequest` from
that set and clears it on a successful `Pushed`/`UpToDate` outcome.
3. **`esp_map.c` cache fix (same pass) — `firmware/components/libgit2/esp_map.c`**
- Bug: cached buffers are freed only lazily in `p_mmap`'s `evict_for`, so a
window libgit2 has `p_munmap`'d stays resident until the *next* map — defeating
`MWINDOW_MAPPED_LIMIT` and starving non-mmap `git__malloc` (zlib).
- Fix: on `p_munmap` when refcount hits 0, evict down to a low-water mark (or free
outright); lower `ESP_MAP_CACHE_CAP` (4 MB → ~1.52 MB) and/or `SLOTS`.
- ⚠️ Editing this `.c` needs the fingerprint dance or the change won't rebuild —
see the `esp-idf-component-rebuild` memory:
`rm -rf firmware/target/xtensa-esp32s3-espidf/release/.fingerprint/esp-idf-sys-*`
then rebuild.
## How to bench / flash
`git_bench.rs` runs git ops on the 96 KB `GIT_STACK` thread (the main task stack
overflows on these ops — that's why the real service has a dedicated thread). It's
Rust-only, so a plain rebuild picks it up (no fingerprint dance unless you also
touched `esp_map.c`).
```
just flash-gitbench
# = . ~/export-esp.sh && LIBGIT2_SRC=<repo>/firmware/components/libgit2/vendor \
# LIBGIT2_NO_VENDOR=1 PKG_CONFIG_ALLOW_CROSS=1 \
# PKG_CONFIG_LIBDIR=<repo>/firmware/pkgconfig \
# cargo run --release --bin git_bench --features git
```
Bench on the **real repo** clone (`/sd/repo` = full `jcalixte/notes`), not the toy —
the toy pack understates everything by ~2 orders of magnitude.
## What's proven vs open
**Proven (2026-07-12, real repo):** `odb.write` 142 ms (mmap cache holds);
`index.write` 611 s (whole-tree re-hash via `truncate_racily_clean`, index.c:822 /
index.h:117); index-free `read_tree` 77 s cold; mmap cache OOM at 7.4 MB → zlib
crash. `Repository::open` 88 ms, odb-open ~6 s cold (maps 1.7 MB `.idx`).
**Open:** the TreeBuilder walk is **designed but not yet benched or built.** Confirm
its cold-real-repo latency and heap before wiring. Push (network half, ~6.5 s) is a
separate floor, untouched here.

View File

@@ -1,12 +1,17 @@
# Commit-staging cost vs working-tree size
> **Decision (pending measurement):** keep the file-agnostic `add_all(["*"])`
> staging, or switch to explicit-path staging (`add_path` over the editor's dirty
> set)? The fork is worth taking **only if the FAT working-tree walk dominates the
> ~4 s commit** — which the split-timer added to
> [`../../firmware/src/git_sync.rs`](../../firmware/src/git_sync.rs)
> (`stage_and_commit`, the `commit split —` log line) resolves. This note records
> the cost model and the rule the measurement decides.
> **Decision (RESOLVED 2026-07-12, real-repo bench):** neither `add_all(["*"])`
> nor an index-free `read_tree`+`write_tree` is viable on the real
> `jcalixte/notes` clone — **both are O(N_tree)** and blow up on the 570 MB pack /
> 1179-file tree (611 s hash one end, 77 s tree-read + OOM the other; see
> [Real-repo run](#real-repo-run-2026-07-12-jcalixtenotes-570 mb-pack--the-index-is-the-wrong-primitive)
> below). The commit must be rebuilt with an **O(depth) TreeBuilder walk** —
> patch only the edited file's ancestor subtree chain onto HEAD's tree, never
> materialise all 1179 entries. Shrinking the repo is **not** an option
> ([`../notes/git-sync-images-and-repo-size.md`](../notes/git-sync-images-and-repo-size.md):
> the images are load-bearing for another app), which is exactly why the O(depth)
> mechanism is the only lever left. This note records the cost model and the full
> measurement trail that got here.
>
> Tradeoff-curves index: [`README.md`](README.md). Docs index:
> [`../README.md`](../README.md). Where the whole sync goes:
@@ -124,9 +129,97 @@ write amplification / better card / SPI-clock" framing is refuted: **the SD card
not the bottleneck.** fsync is still confirmed off; the extra ~600 ms/op is CPU or
repeated `.git` I/O *inside* libgit2 (candidates: ODB refresh scanning
`objects/`, the treebuilder's per-entry `git_odb_exists`, ref-lock + reflog writes,
config/attributes re-reads). `git_bench` (`firmware/src/bin/git_bench.rs`) times
`odb.write` / `index.write` / `write_tree` in isolation to localize it — **run
pending.**
config/attributes re-reads). `git_bench` (`firmware/src/bin/git_bench.rs`) localizes
it — see below.
### It's the pack, read through an un-caching emulated mmap (`git_bench`, 2026-07-12)
`git_bench` times the git2 primitives in isolation on `/sd/repo` (git ops on the
same 96 KB thread the real service uses — the main-task stack overflows on
`index.write`, which is itself the reason the service has a dedicated thread):
| git2 op | p50 | note |
| --- | ---: | --- |
| `Repository::open` | 100 ms | one-time |
| `odb.write(blob)` (unique) | **45 ms** | writes a fresh object; touches no existing object |
| `repo.index()` open | ~0 ms | cached |
| `index.write()` | 376 ms | index + `index.lock` rename + tree-cache |
| `write_tree` [unchanged] | ~0 ms | tree exists → freshen-skips the write |
| **`write_tree` [changed]** | **1136 ms** | writes ONE 45 ms object |
| **`commit(None)` orphan obj** | **563 ms** | writes ONE 45 ms object, no ref/reflog |
Writing a fresh object is 45 ms; the ops that wrap one are 825×. The cause, from
the vendored source: `git_odb_write` calls `git_odb__freshen` (odb.c:1011), which
on a not-found object runs **`git_odb_refresh`** (re-reads the pack dir + reloads
pack indexes), and existence checks (`freshen(tree)` in `commit.c:169`, base-object
lookups in `write_tree`) hit the **pack**. Pack access goes through our
`p_mmap` (`esp_map.c`), which **`malloc`s and `read()`s the mapped range from the
card on every call — no cache** — with a 32 MB window on this 32-bit target. So
each write re-reads pack bytes from SD; `odb.write` of a fresh blob is 45 ms only
because it touches no packed object.
**This scales with pack size.** The toy repo's pack is tiny; the real
`jcalixte/notes` clone has a **570 MB pack**, and provisioning rsyncs a full clone
onto the card — so a real-repo commit has **never been benchmarked** and, on this
mechanism, will be far worse than the ~3.3 s toy number. That is the single biggest
open risk in sync.
### Real-repo run (2026-07-12, `jcalixte/notes`, 570 MB pack) — the index is the wrong primitive
`git_bench` was finally run against a full clone of the real repo (1179 files, 158
dirs, 570 MB pack). It settles the design: **any index-based commit is O(N_tree)
and does not fit this device.** Two independent walls:
| op | result | reading |
| --- | ---: | --- |
| `Repository::open` | 88 ms | fine |
| odb open (implicit) | ~6 s cold | maps the 1.7 MB pack `.idx` once (16 miss / 1790 KB) |
| `odb.write(blob)` | **142 ms** p50 | the mmap cache win **holds** (was 862 ms uncached) ✅ |
| `repo.index()` load (1179 entries) | 514 ms max | the on-disk index we were trying to avoid |
| `index.write()` | **min 360 ms / p50 12.8 s / max 611 s** | ⚠️ hangs — see root cause |
| **seed `read_tree(HEAD)` (cold, 1×)** | **~77 s** | ⚠️ reads all ~158 tree objects, 22.7 MB of pack windows |
| `Index::new + read_tree` (warm) | 447 ms p50 | windows still mapped → pure CPU |
| **index-free `stage→tree`** | **💥 crash** | `zlib (5)`: `deflateInit` failed, **508 KB heap left** |
**Wall 1 — `index.write()` hashes the whole working tree (up to 611 s).**
`git_index_write` unconditionally calls `truncate_racily_clean` (index.c:822),
which runs `git_diff_index_to_workdir` over **every** entry flagged "racy" and
re-hashes its file. On a fresh FAT clone the mtime granularity is 2 s and
`index.stamp.mtime <= entry.mtime` for ~all 1179 entries (index.h:117), so the
whole tree looks racy → it re-hashes ~170 MB (mostly the 150 MB of images) over
10 MHz SPI. The 611 s → 12.8 s → 360 ms decay across three iterations is the
signature: each write bumps the index mtime, shrinking the racy set. **Implication:
the shipping `stage_and_commit` calls `index.write()`, so `:sync` on the real repo
effectively bricks on the first commit** — the user sees a 10-minute freeze,
resets, the index mtime never advances, and it re-hashes forever. The real repo has
almost certainly never completed a sync on device (only the toy `typoena-test` has).
**Wall 2 — the index-free path is still O(N_tree), and the mmap cache OOMs.**
Skipping `index.write()` entirely (fresh `Index::new()`, stamp = 0, so
`truncate_racily_clean` can never fire) removes Wall 1. But to seed the in-memory
index, `read_tree(HEAD)` materialises all 1179 entries and reads every tree object
from the 570 MB pack — **77 s cold** (447 ms only once the windows are resident).
`write_tree_to` is O(changed), but you pay O(N_tree) to build the cache it needs, so
the index-free path only trades a 611 s hash for a 77 s tree-read. Worse, that
`read_tree` drove the `esp_map.c` cache to **7.4 MB resident** — past its own 4 MB
soft cap — which left 508 KB of heap and made `repo.blob()`'s zlib `deflateInit`
fail. **The one write we cannot skip crashed.** Root cause of the OOM: our cache
holds pack windows *after* libgit2 `p_munmap`s them (refcount 0, freed only lazily
on the next `p_mmap`), which **defeats `GIT_OPT_SET_MWINDOW_MAPPED_LIMIT`**
libgit2 thinks it released the memory; we didn't.
**Conclusion — use an O(depth) TreeBuilder walk.** "Replace K files in a 1179-file
tree" should touch `O(depth × K)` objects, not `O(N_tree)`. Walk HEAD's tree down
the edited path (`tree.get_name`/`get_path` → read ~depth subtree objects), then
rebuild bottom-up with `repo.treebuilder(Some(&subtree))``insert`/`remove`
`write()`, and `commit` the new root. That never materialises the 1179 entries,
never re-hashes anything, never visits the 150 MB of images, reads only a handful
of tree windows (so the cache stays small and zlib keeps its heap), and — crucially
— **carries the image entries forward untouched from HEAD's tree, so the device
does not even need the images in its working tree.** The `esp_map.c` cache still
needs an evict-on-`munmap` fix (drop the cap, free past a low-water mark) so it can
never again starve a downstream `git__malloc`, but with the TreeBuilder walk the
pressure it was under largely disappears.
### The walk is ~1.4 s even at N ≈ 2
@@ -139,27 +232,42 @@ For orientation: `publish(commit+push)` was 9846 ms cold, so the **network half
~6.5 s** — still the biggest single block of a warm sync (10.1 s total), a separate
floor ([`../notes/sync-latency.md`](../notes/sync-latency.md)).
## The verdict (provisional — pending `git_bench`)
## The verdict
Two things are now settled and one is open:
The real-repo run (above) overturned the earlier ranking. Both index strategies
are O(N_tree) and fail on the 570 MB-pack clone, and the repo cannot be shrunk. The
work, ranked:
- **Settled: the card is fast.** The SD-clock and better-card levers are off the
table — they target I/O that costs ~86 ms, not the ~700 ms we see. Do not spend
the PCB's 20 MHz budget expecting a commit-latency win here.
- **Settled: explicit-path staging is still worth doing** — but on *design +
big-repo* grounds, not toy-repo latency (its measured payoff there is ~0.7 s). It
**caps the O(N) walk on the 1179-file target**, **never visits the ~260 images**
(150 MB it would otherwise scan), lets us **drop the macOS-cruft filter**, and
aligns the git layer with what the editor changed.
- **Open: the ~600 ms/op libgit2 overhead** is now the largest single mystery in
the commit and likely the highest-value fix — if it's ODB refresh or reflog
writes, it may be a cheap config/flag change that speeds up *every* commit
regardless of repo size or staging. `git_bench` decides. **Localize it before
committing effort to (a).**
1. **Rewrite the commit as an O(depth) TreeBuilder walk (the fix — build this).**
Rebuild only the edited path's ancestor subtree chain onto HEAD's tree; never
materialise the 1179-entry index, never `index.write()`, never `read_tree` the
whole tree. This is the ONLY mechanism that fits: O(depth × dirty) reads/writes,
flat in repo size, small heap, images carried forward untouched. Replaces
`stage_and_commit`'s `add_all`/`update_all`/`index.write`/`write_tree`. Needs the
editor's dirty set (+ deleted set) plumbed to the git service — the editor
already knows both. **Bench the walk on the real repo before wiring it in.**
2. **Fix the `esp_map.c` cache so it can't OOM (do it alongside #1).** It grew to
7.4 MB resident past its 4 MB cap and starved zlib. Evict on `p_munmap` (not
only lazily on the next `p_mmap`) down to a low-water mark, and lower the cap, so
a released window's memory is actually returned to `git__malloc`. The cache's
`odb.write` win (862 → 142 ms) is real and worth keeping — this is a
memory-discipline fix, not a removal.
3. **Retired: `add_all`/explicit-path *index* staging.** Explicit-path `add_path`
still goes through the index and `index.write``truncate_racily_clean`, so it
hits Wall 1 just the same. The TreeBuilder walk supersedes it entirely; the
"explicit-path staging" idea survives only as "the editor's dirty set feeds the
walk."
4. **Retired: SD clock / better card.** The card does a full object write in
~86 ms; raw I/O is not the bottleneck. Do not spend the PCB's 20 MHz budget
expecting a commit-latency win.
5. **Kept: the mmap cache + mwindow tuning** (`GIT_OPT_SET_MWINDOW_*`, 256 KB
window / 4 MB mapped limit). It fixed `odb.write` and the push read path; #2 just
makes it well-behaved under memory pressure.
**Recommendation:** run `git_bench` to pin the libgit2 overhead; then implement
explicit-path staging for the design + big-repo reasons; the SD/card levers are
retired.
**Recommendation:** build #1 (O(depth) TreeBuilder walk) with #2 (cache
evict-on-munmap) in the same pass. See
[`../notes/sync-commit-handoff.md`](../notes/sync-commit-handoff.md) for the
concrete next-session plan (bench design, firmware plumbing, exact call sites).
## Adjacent lever: should the images be on the card at all?

View File

@@ -7,9 +7,24 @@
* Allocations go through git__malloc, so with PSRAM in the heap they land in
* the 8 MB external RAM rather than the ~340 KB internal DRAM.
*
* Limitation: writable/shared mappings are not written back (libgit2 does not
* mmap for writing in the paths we exercise). If that ever changes it will
* surface at runtime, not here.
* CACHE (2026-07-12): the emulation reads the range from SD on every call, so
* libgit2's repeated pack access — `git_odb_write` → `git_odb__freshen` →
* `git_odb_refresh` re-reads the pack idx/windows on *every* object write —
* turned a 45 ms object write into 500 ms1.1 s on a small repo, and would be far
* worse on the real 570 MB-pack clone (see
* docs/tradeoff-curves/sync-commit-staging.md). We cache the read buffers so a
* given file region is read from the card once and reused.
*
* Correctness: cache ONLY read-only mappings >= ESP_MAP_CACHE_MIN. libgit2 maps
* pack idx/data, the commit-graph, midx and packed-refs — all immutable on this
* device (only loose objects/refs/index are written, none via mmap). The one
* mutable mmap is diff_file.c on small working-tree files (notes.md), which the
* size floor excludes, so a mutable file is never served stale. The writable
* mapping the pack indexer uses (fetch/clone) is excluded by the prot check.
* Identity is (dev, ino, size, mtime, offset, len); for an immutable pack any of
* size/mtime already differs if the file is ever replaced.
*
* Limitation: writable/shared mappings are not written back (and not cached).
*/
#include "git2_util.h"
@@ -18,6 +33,8 @@
#include <unistd.h>
#include <string.h>
#include <errno.h>
#include <sys/stat.h>
#include <stdint.h>
int git__page_size(size_t *page_size)
{
@@ -31,43 +48,154 @@ int git__mmap_alignment(size_t *alignment)
return 0;
}
int p_mmap(git_map *out, size_t len, int prot, int flags, int fd, off64_t offset)
{
/* Only cache mappings at least this large: covers pack idx/windows, excludes the
* small mutable working-tree files diff_file.c maps. */
#define ESP_MAP_CACHE_MIN (64 * 1024)
/* Cached-buffer budget in PSRAM. Entries with no live ref are LRU-evicted to
* stay under this; pinned entries may exceed it transiently. */
#define ESP_MAP_CACHE_CAP (4 * 1024 * 1024)
#define ESP_MAP_CACHE_SLOTS 24
struct map_entry {
unsigned char *data;
size_t len;
off64_t offset;
dev_t dev;
ino_t ino;
off_t size;
time_t mtime;
int refcount;
uint32_t used; /* LRU stamp; 0 = empty slot */
};
static struct map_entry g_cache[ESP_MAP_CACHE_SLOTS];
static uint32_t g_clock;
static size_t g_cached_bytes;
/* Diagnostics, read from the bench via esp_map_stats(). */
static uint32_t g_hits, g_misses;
static uint64_t g_read_bytes;
void esp_map_stats(uint32_t *hits, uint32_t *misses, uint32_t *read_kb, uint32_t *cached_kb)
{
if (hits) *hits = g_hits;
if (misses) *misses = g_misses;
if (read_kb) *read_kb = (uint32_t)(g_read_bytes / 1024);
if (cached_kb) *cached_kb = (uint32_t)(g_cached_bytes / 1024);
}
static int read_range(int fd, off64_t offset, size_t len, unsigned char *data)
{
size_t got = 0;
GIT_UNUSED(prot);
GIT_UNUSED(flags);
GIT_MMAP_VALIDATE(out, len, prot, flags);
out->data = NULL;
out->len = 0;
data = git__malloc(len);
GIT_ERROR_CHECK_ALLOC(data);
if (lseek(fd, offset, SEEK_SET) < 0) {
git_error_set(GIT_ERROR_OS, "failed to seek for mmap emulation");
git__free(data);
return -1;
}
while (got < len) {
ssize_t n = read(fd, data + got, len - got);
if (n < 0) {
if (errno == EINTR)
continue;
git_error_set(GIT_ERROR_OS, "failed to read for mmap emulation");
git__free(data);
return -1;
}
if (n == 0)
break; /* short file: zero-fill the tail, like a real mapping */
got += (size_t)n;
}
if (got < len)
memset(data + got, 0, len - got);
return 0;
}
/* Best-effort: evict unreferenced LRU entries until `need` more bytes fit the
* cap. Pinned entries (refcount > 0) can't be evicted, so the cap is soft. */
static void evict_for(size_t need)
{
while (g_cached_bytes + need > ESP_MAP_CACHE_CAP) {
int victim = -1;
uint32_t oldest = 0xFFFFFFFFu;
int i;
for (i = 0; i < ESP_MAP_CACHE_SLOTS; i++) {
if (g_cache[i].used && g_cache[i].refcount == 0 &&
g_cache[i].used < oldest) {
oldest = g_cache[i].used;
victim = i;
}
}
if (victim < 0)
break;
git__free(g_cache[victim].data);
g_cached_bytes -= g_cache[victim].len;
memset(&g_cache[victim], 0, sizeof(g_cache[victim]));
}
}
int p_mmap(git_map *out, size_t len, int prot, int flags, int fd, off64_t offset)
{
unsigned char *data;
struct stat st;
int cacheable, i;
GIT_UNUSED(flags);
GIT_MMAP_VALIDATE(out, len, prot, flags);
out->data = NULL;
out->len = 0;
/* Cache only large, read-only mappings whose file we can identify. */
cacheable = (len >= ESP_MAP_CACHE_MIN) && !(prot & GIT_PROT_WRITE);
if (cacheable && fstat(fd, &st) != 0)
cacheable = 0;
if (cacheable) {
for (i = 0; i < ESP_MAP_CACHE_SLOTS; i++) {
struct map_entry *e = &g_cache[i];
if (e->used && e->len == len && e->offset == offset &&
e->dev == st.st_dev && e->ino == st.st_ino &&
e->size == st.st_size && e->mtime == st.st_mtime) {
e->refcount++;
e->used = ++g_clock;
g_hits++;
out->data = e->data;
out->len = len;
return 0;
}
}
}
data = git__malloc(len);
GIT_ERROR_CHECK_ALLOC(data);
if (read_range(fd, offset, len, data) < 0) {
git__free(data);
return -1;
}
g_misses++;
g_read_bytes += len;
if (cacheable) {
evict_for(len);
for (i = 0; i < ESP_MAP_CACHE_SLOTS; i++) {
if (!g_cache[i].used) {
g_cache[i].data = data;
g_cache[i].len = len;
g_cache[i].offset = offset;
g_cache[i].dev = st.st_dev;
g_cache[i].ino = st.st_ino;
g_cache[i].size = st.st_size;
g_cache[i].mtime = st.st_mtime;
g_cache[i].refcount = 1;
g_cache[i].used = ++g_clock;
g_cached_bytes += len;
out->data = data;
out->len = len;
return 0;
}
}
/* All slots pinned: return uncached (freed directly at munmap). */
}
out->data = data;
out->len = len;
@@ -76,7 +204,20 @@ int p_mmap(git_map *out, size_t len, int prot, int flags, int fd, off64_t offset
int p_munmap(git_map *map)
{
int i;
GIT_ASSERT_ARG(map);
/* Cached buffer: drop a ref, keep it for reuse. Otherwise free it. */
for (i = 0; i < ESP_MAP_CACHE_SLOTS; i++) {
if (g_cache[i].used && g_cache[i].data == map->data) {
if (g_cache[i].refcount > 0)
g_cache[i].refcount--;
map->data = NULL;
map->len = 0;
return 0;
}
}
git__free(map->data);
map->data = NULL;
map->len = 0;

View File

@@ -22,8 +22,9 @@ use firmware::persistence::{Storage, REPO_DIR};
const BUILD_TAG: &str = concat!("build ", env!("BUILD_TIME"), " @", env!("BUILD_GIT"));
/// Iterations per op. Small — some ops write to the card, and the first vs rest
/// spread (min vs max) is itself the signal (e.g. write vs freshen-skip).
const N: usize = 10;
/// spread (min vs max) is itself the signal (e.g. write vs freshen-skip). Kept
/// low (3) on the real 570 MB-pack clone so a slow op still finishes in seconds.
const N: usize = 3;
fn main() -> Result<()> {
esp_idf_svc::sys::link_patches();
@@ -49,7 +50,18 @@ fn main() -> Result<()> {
}
fn run() -> Result<()> {
let _sd = Storage::mount().context("mounting SD")?;
// libgit2 holds the pack + idx (+ commit-graph) fds open and reads loose
// objects on top; the editor's default 4-FD budget can't cover read_tree.
let _sd = Storage::mount_for_git().context("mounting SD")?;
// A 32 MB default mwindow window (mwindow.c) would git__malloc > PSRAM on the
// real 570 MB pack; small windows keep each p_mmap read cheap, and the
// esp_map cache keeps them from being re-read on every freshen→refresh.
// SAFETY: process-global libgit2 options, set once before any repo work.
unsafe {
git2::opts::set_mwindow_size(256 * 1024).ok();
git2::opts::set_mwindow_mapped_limit(4 * 1024 * 1024).ok();
}
// Repository open — one-time, but shows the cost of scanning .git (config,
// refs, ODB backends/packs) which every later op may implicitly refresh.
@@ -57,40 +69,151 @@ fn run() -> Result<()> {
let repo = Repository::open(REPO_DIR)
.with_context(|| format!("opening git repo at {REPO_DIR}"))?;
log::info!("Repository::open {:.1} ms", t.elapsed().as_micros() as f64 / 1000.0);
log_map_stats("open");
// 1) odb.write(blob) in isolation — unique content each iter forces a real
// write (no freshen-skip). This is the single number that localizes it: if
// ~86 ms the ODB write path is fine and the cost is in the tree/ref layer;
// if ~700 ms the cost is inside the ODB write itself (deflate/sha/freshen).
let odb = repo.odb().context("opening odb")?;
summarize("odb.write(blob)", time_each(|i| {
bench("odb.write(blob)", |i| {
let data = format!("typoena git_bench orphan blob #{i} — unique so the write is real\n");
odb.write(ObjectType::Blob, data.as_bytes())
.map(|_| ())
.context("odb.write")
})?);
})?;
log_map_stats("odb.write");
// 2) repo.index() — cost of loading the index from the card each time.
summarize("repo.index() open", time_each(|_| {
// 2) on-disk index LOAD (no write). Times loading all ~1179 entries from the
// card and prints the count. We deliberately do NOT bench index.write() any
// more: it calls truncate_racily_clean, which diffs the whole working tree
// against the index and — because a fresh FAT clone makes every entry look
// "racy" (2 s mtime granularity) — re-hashes ~170 MB over SPI, up to ~10 min
// on this repo (proven 2026-07-12, index.write max 611 s). The fix below
// never writes the on-disk index, so that path never runs.
bench("repo.index() load", |_| {
repo.index().map(|_| ()).context("index open")
})?);
})?;
let n_entries = repo.index().map(|i| i.len()).unwrap_or(0);
log::info!("on-disk index has {n_entries} entries");
log_map_stats("index load");
// 3) index.write() — serialize + checksum + filebuf (index.lock → rename).
let mut idx = repo.index().context("opening index")?;
summarize("index.write()", time_each(|_| {
idx.write().context("index.write")
})?);
// 3) THE PROPOSED FIX — index-free commit tree (what git_sync::stage_and_commit
// will do). Build the new tree from HEAD + one changed file in a FRESH
// in-memory index: read_tree(HEAD) leaves stamp=0 so truncate_racily_clean
// can NEVER fire; the changed file is written as a blob and added by OID;
// write_tree_to writes to the odb WITHOUT touching the on-disk index. Because
// read_tree seeds the tree cache and add invalidates only the changed path,
// write_tree rebuilds just that path's subtrees — O(changed), not O(1179).
// If this is sub-second on the real repo, the fix is validated.
let head_tree = repo
.head()?
.peel_to_commit()
.context("HEAD → commit")?
.tree()
.context("HEAD tree")?;
// A real path already in the tree, so the add REPLACES it (a realistic edit) and
// write_tree rebuilds its ancestor subtrees — not just the cheap root case.
let edit_path: Vec<u8> = {
let mut seed = git2::Index::new().context("seed index")?;
seed.read_tree(&head_tree).context("seed read_tree")?;
seed.get(0)
.map(|e| e.path)
.unwrap_or_else(|| b"notes.md".to_vec())
};
log::info!(
"index-free: editing existing path {}",
String::from_utf8_lossy(&edit_path)
);
// 4) index.write_tree() — build tree(s) from the index and write to the ODB.
// First call writes the tree; later calls find it exists (freshen-skip), so
// min≈build+exists and max≈build+write — the spread separates the two.
summarize("index.write_tree()", time_each(|_| {
idx.write_tree().map(|_| ()).context("write_tree")
})?);
// read_tree alone: populating the in-memory index from HEAD's tree (reads tree
// objects through the mmap cache; NO working-file hashing).
bench("Index::new + read_tree", |_| {
let mut idx = git2::Index::new().context("Index::new")?;
idx.read_tree(&head_tree).context("read_tree")?;
Ok(())
})?;
log_map_stats("read_tree");
// Full index-free staging → tree — this REPLACES add_all + index.write +
// write_tree (the ~10-min hang) with an O(changed) path.
bench("index-free stage→tree", |i| {
let mut idx = git2::Index::new().context("Index::new")?;
idx.read_tree(&head_tree).context("read_tree")?;
let data = format!("typoena index-free bench edit #{i}\n");
let oid = repo.blob(data.as_bytes()).context("write blob")?;
idx.add(&blob_entry(&edit_path, oid)).context("index.add")?;
idx.write_tree_to(&repo).map(|_| ()).context("write_tree_to")
})?;
log_map_stats("index-free");
// 6) commit(None, …) — create a commit OBJECT without moving HEAD or writing a
// reflog (update_ref = None → an orphan commit, gc-able). Isolates commit-
// object creation from the ref-update + reflog cost. Reuses the parent's
// tree (no new tree needed); unique message each iter forces a real write.
let parent = repo.head()?.peel_to_commit().context("HEAD → commit")?;
let tree = parent.tree().context("parent tree")?;
let sig = Signature::now("typoena-bench", "bench@typoena.local").context("sig")?;
bench("commit(None) orphan obj", |i| {
let msg = format!("typoena git_bench orphan commit #{i}");
repo.commit(None, &sig, &sig, &msg, &tree, &[&parent])
.map(|_| ())
.context("commit(None)")
})?;
log_map_stats("commit");
Ok(())
}
unsafe extern "C" {
/// Counters from the p_mmap cache in `components/libgit2/esp_map.c`.
fn esp_map_stats(hits: *mut u32, misses: *mut u32, read_kb: *mut u32, cached_kb: *mut u32);
}
/// Log the p_mmap cache counters — hits vs misses (SD reads avoided), total KB
/// read from the card, and KB currently resident. If the pack-read hypothesis is
/// right, hits climb and `KB read` stops growing across the write ops.
fn log_map_stats(label: &str) {
let (mut hits, mut misses, mut read_kb, mut cached_kb) = (0u32, 0u32, 0u32, 0u32);
unsafe { esp_map_stats(&mut hits, &mut misses, &mut read_kb, &mut cached_kb) };
// Free heap spans PSRAM here; a drop toward 0 during write_tree/commit on the
// real repo would point at mwindow/idx allocation pressure (or thrash) as the
// cause of an apparent hang, not CPU.
let free_kb = unsafe { esp_idf_svc::sys::esp_get_free_heap_size() } / 1024;
log::info!(
"mmap cache @ {label:<11} {hits} hit / {misses} miss, {read_kb} KB read, {cached_kb} KB resident, {free_kb} KB heap free"
);
}
/// Announce, time, and summarize an op. The `→ label …` line prints BEFORE the op
/// runs, so if an op hangs on the real 570 MB-pack repo we can see which one it
/// entered — a bare `summarize` prints only after all N iters, hiding the culprit.
fn bench<F: FnMut(usize) -> Result<()>>(label: &str, op: F) -> Result<()> {
log::info!("→ {label} …");
summarize(label, time_each(op)?);
Ok(())
}
/// A minimal index entry pointing at an already-written blob — for `index.add`,
/// which (unlike `add_frombuffer`) needs no repo owner, so it works on a bare
/// in-memory index. Only `id`, `path` and `mode` feed the tree write.
fn blob_entry(path: &[u8], oid: Oid) -> IndexEntry {
IndexEntry {
ctime: IndexTime::new(0, 0),
mtime: IndexTime::new(0, 0),
dev: 0,
ino: 0,
mode: 0o100644,
uid: 0,
gid: 0,
file_size: 0,
id: oid,
flags: 0,
flags_extended: 0,
path: path.to_vec(),
}
}
/// Run `op(i)` for `i in 0..N`, returning each call's wall time in microseconds.
fn time_each<F: FnMut(usize) -> Result<()>>(mut op: F) -> Result<Vec<u64>> {
let mut times = Vec::with_capacity(N);

View File

@@ -79,6 +79,15 @@ pub const MAX_FILE_BYTES: u64 = 256 * 1024;
/// The C mount point (`/sd\0`) for the esp-idf FFI calls.
const MOUNT_C: &std::ffi::CStr = c"/sd";
/// VFS open-file budget for the editor path: it opens only a note and its
/// `*.tmp`, so a tight budget keeps FatFS's per-file buffers off the heap.
const MAX_FILES_EDITOR: i32 = 4;
/// VFS open-file budget for the git tooling. libgit2 keeps the pack + `.idx`
/// (and commit-graph) descriptors open for the repo's lifetime and opens loose
/// objects on top, so a `read_tree` walk overruns [`MAX_FILES_EDITOR`] with a
/// "no free file descriptors" error. Matches the flash-FAT git binaries' 16.
const MAX_FILES_GIT: i32 = 16;
/// A mounted SD card. Holds the live card handle for its lifetime; v0.1 never
/// unmounts (the card stays up for the whole power session). Not `Send` — the
/// handle lives on the task that mounted it (the ui/main task). The git thread
@@ -113,6 +122,17 @@ impl Storage {
/// (The Spike 3 bench binary sets it true for convenience on blank cards;
/// this path must not.)
pub fn mount() -> Result<Self> {
Self::mount_with_max_files(MAX_FILES_EDITOR)
}
/// Like [`Storage::mount`], but with the larger [`MAX_FILES_GIT`] open-file
/// budget the git tooling (bench / sync) needs — libgit2 holds several
/// descriptors open at once, which the editor's default budget can't cover.
pub fn mount_for_git() -> Result<Self> {
Self::mount_with_max_files(MAX_FILES_GIT)
}
fn mount_with_max_files(max_files: i32) -> Result<Self> {
// 1) SPI3 with the SD's four lines. Dedicated bus (ADR-012) — no EPD
// deselect needed: the panel is on SPI2 and can't contend here.
// SAFETY: a zeroed spi_bus_config_t is valid (all pins default 0); we
@@ -177,7 +197,7 @@ impl Storage {
// 4) Mount config. format_if_mount_failed = FALSE — see method docs.
let mount = sys::esp_vfs_fat_mount_config_t {
format_if_mount_failed: false,
max_files: 4,
max_files,
allocation_unit_size: 16 * 1024,
disk_status_check_enable: false,
use_one_fat: false,