diff --git a/docs/v0.7-search-and-git.md b/docs/v0.7-search-and-git.md index f2c7ccc..60c31e9 100644 --- a/docs/v0.7-search-and-git.md +++ b/docs/v0.7-search-and-git.md @@ -3,54 +3,98 @@ > Part of the [Typoena macro plan](macroplan.md). Requirements and targets: > [qfd.md](qfd.md). Load-bearing decisions: [adr.md](adr.md). -**Status:** **`/` search + `n N` done in core** (2026-07-13, host-tested — 207 -editor tests). **`:gl` pull BUILT end-to-end 2026-07-14** (firmware 0.7.0, -compiles clean; editor refresh hooks host-tested, 209 tests) — **on-device -verification pending** (needs Wi-Fi + a remote that moved). +**Status: feature-complete, one gate left.** `/` search is done and +panel-verified (case-insensitive since 2026-07-14; `n`/`N` confirmed +on-device). `:gl` is built end-to-end and three of its four shapes are +on-device verified; the **fast-forward (Pulled) shape is the closing gate** — +its first attempt (2026-07-14, run 2) crashed in libgit2's `checkout_tree` +(O(tree) workdir walk + internal-DRAM exhaustion + an esp-idf spi_master +NULL-deref on its own failed-alloc path), and the checkout has been rebuilt as +an O(changed) tree-diff apply plus three memory fixes (file-list interning, +64 KB DMA reserve, TLS session reuse). **Re-run the Mac-push → device `:gl` +test to close the version.** Run-1 bonus: the first on-device +rejected-push → reconcile → replay → push success (replay commit 59444a94). -- [x] `/` forward search, `n N` — done in core 2026-07-13 (host-tested). - Literal, case-sensitive substring (no regex on-device, no smartcase - surprises); `/` reuses the `:` command line's editing with a `/` prompt - (one command-line mode, as in vim), and the jump happens on **Enter - only** — no incremental caret-chasing, for the same e-ink-refresh reason - snippets have no completion popup. Wraps around with a "wrapped" notice; - "not found: pat" leaves the caret alone; a bare `/`+Enter repeats the - last pattern, which is editor-global (survives buffer switches, like - vim's search register). `n`/`N` are motions (counts work, Visual extends - over them); deliberately **not** operator targets (`dn` out of scope). +- [x] `/` forward search, `n N` — core 2026-07-13, panel-verified 2026-07-14 + (including `n`/`N`). Literal, **case-insensitive** substring (Unicode + lowercase fold — `É` matches `é`; no regex on-device; deliberately not + smartcase: a capital silently flipping to exact-match is a surprise, not + a feature, on a writing appliance). `/` reuses the `:` command line's + editing with a `/` prompt (one command-line mode, as in vim), and the + jump happens on **Enter only** — no incremental caret-chasing, for the + same e-ink-refresh reason snippets have no completion popup. Wraps + around with a "wrapped" notice; "not found: pat" leaves the caret alone; + a bare `/`+Enter repeats the last pattern, which is editor-global + (survives buffer switches, like vim's search register). `n`/`N` are + motions (counts work, Visual extends over them); deliberately **not** + operator targets (`dn` out of scope). - [~] `:gl` — pull: fetch + **fast-forward only**, refuse on divergence and - surface it (renamed from the planned `:Gpull`). Editor command + - `Effect::Pull` done 2026-07-11; **the git-thread fetch/fast-forward is - BUILT 2026-07-14** — on-device verification pending. How it works: + surface it (renamed from the planned `:Gpull`). On-device 2026-07-14: + **UpToDate shape, dirty-journal gate, and the reconcile/replay path + verified**; the **Pulled (fast-forward) shape is rebuilt after the run-2 + crash and awaits its on-device pass** — the closing gate. How it works: - The git thread takes a `GitRequest::Pull` on the same channel as publish - (shared `ensure_online` Wi-Fi/clock/TLS preamble). `pull_once` fetches - origin (the fetch also refreshes the remote-tracking ref, keeping - publish's radio-free up-to-date check honest), then maps the four shapes: - up to date / **LocalAhead** (stranded commit — `:sync`'s job) / clean - fast-forward / **Diverged** (refused, no merge on the device). - - The fast-forward is checkout-then-ref-move with a **SAFE** checkout: it - refuses to overwrite a file whose content differs from HEAD (belt; the UI - gate is the braces). FAT caveat: the splice never updates the index, so - SAFE re-hashes each file the pull changes — O(changed), never O(tree). + (shared `ensure_online` Wi-Fi/clock/TLS preamble). `pull_once` does + **ls-refs first**: the ref advertisement alone answers "anything new?", so + up to date and LocalAhead (stranded commit — `:sync`'s job) return without + entering pack negotiation (run 1 paid a 9.7 s fetch to learn "up to + date"); a needed download rides the same open connection. Divergence is + refused — no merge on the device. Tracking-ref updates keep publish's + radio-free up-to-date check honest. + - The fast-forward is an **O(changed) tree-diff apply, not `checkout_tree`** + (`apply_tree_diff`, the splice philosophy on the read side): + `diff_tree_to_tree` skips identical subtree OIDs in memory, then exactly + the changed paths are written (unlink+tmp+rename — FAT `f_rename` won't + overwrite) or unlinked, and the ref moves last. libgit2's SAFE checkout + instead iterates the whole working directory (readdir over SPI on all + ~1100 files) — run 2 died there: internal DRAM was exhausted mid-walk and + esp-idf's spi_master NULL-derefs on its own failed-DMA-alloc path + (LoadProhibited in setup/uninstall_priv_desc). Belt kept O(changed): any + to-be-clobbered file whose content no longer hashes to the old tree's blob + aborts the pull before the first write (edits made behind git's back — + e.g. desktop edits directly on the card). A half-applied working copy with + the ref unmoved self-heals: the next `:gl` re-applies idempotently. - UI gate: `:gl` is refused while the dirty journal is non-empty - ("unsynced changes - :sync first") — an unpublished save would fight the - checkout, and sync-then-pull is the single-writer appliance's natural - order. A RAM-dirty (never-saved) buffer does *not* gate: its edits simply - win over the pulled state (last-writer-wins, like the reconcile). + ("unsynced changes - :sync first", also logged) — an unpublished save + would fight the apply, and sync-then-pull is the single-writer appliance's + natural order. A RAM-dirty (never-saved) buffer does *not* gate: its edits + simply win over the pulled state (last-writer-wins, like the reconcile). - After a `Pulled` outcome the UI drops clean parked buffers (next switch re-reads the disk), re-reads the clean active buffer in place (`Editor::refresh_active` — boot posture, undo cleared), keeps a - RAM-dirty active buffer untouched, and re-walks the palette file list. - Snackbar: `pulled ` / `up to date` / `ahead - :sync to publish` / - `diverged - resolve on a computer` / `pull: `. + RAM-dirty active buffer untouched, and re-walks the palette file list in + the background. Snackbar: `pulled ` / `up to date` / + `ahead - :sync to publish` / `diverged - resolve on a computer` / + `pull: `. + +Memory + connection work landed here (2026-07-14, forced by the run-2 crash; +all three compose): + +- [x] **File-list interning** — 1099 palette paths as individual `String`s + measured **182 KB of internal DRAM** (each alloc under the 16 KB + SPIRAM-malloc threshold), a main contributor to the DMA-pool exhaustion. + The walk now emits one newline-joined blob (seeded at 64 KB → PSRAM); + the editor stores blob + sorted spans (`set_file_list_joined`) and the + palette reads slices. Expected walk log line now: ~0 KB internal + consumed — confirm on the next boot. +- [x] **DMA reserve 32 → 64 KB** (`CONFIG_SPIRAM_MALLOC_RESERVE_INTERNAL`) — + the esp-idf SPI driver crashes rather than errors on a failed DMA alloc, + so the floor must be generous enough that it never sees one. +- [x] **TLS session resumption + one-connection pull** — every git op paid a + full multi-second handshake (a rejected-push `:sync` runs three). + `esp_mbedtls_stream.c` (second vendor delta) caches the mbedTLS session + at stream close and offers it on the next connect to the same host; + measure as first-op vs second-op `ls-refs`/handshake time in a session. Sync performance — inherited from the [real-repo-sync kaizen](kaizen/real-repo-sync.md) (closed 2026-07-13 at 24.1 s end-to-end; target ≤ ~10 s): -- [ ] Close the publish gap: 19.8 s today — the splice's 5.0 s and the ~7 s of - repo-open/negotiation overhead are the next curves - ([tradeoff analysis](tradeoff-curves/sync-commit-staging.md)). +- [~] Close the publish gap: 19.8 s at the kaizen close — the splice's 5.0 s + and the ~7 s of repo-open/negotiation overhead are the next curves + ([tradeoff analysis](tradeoff-curves/sync-commit-staging.md)). TLS + session resumption (above) attacks the negotiation half; re-measure a + warm `:sync` before hunting further. - [~] Instrument the residual ~360 ms/loose-write (suspect: FAT directory-op cost in the freshen/refresh path) — **instrumentation BUILT 2026-07-14, measurement run pending**: `sd_bench` grew a directory-entry-scaling @@ -58,8 +102,9 @@ Sync performance — inherited from the siblings — FAT resolves paths by linear scan, so cost climbing with N convicts dir scans; flash with `just flash-bench`), and `stage_and_commit` now brackets the splice with the `p_mmap` counters, logging - `N mmaps / M KB read` per commit so the next real `:sync` splits the - residual into pack-read I/O vs everything else. + `N mmaps / M KB read` per commit. First data point (run 1, 2026-07-14): + both real-repo splices logged **0 mmaps / 0 KB read** — the residual is + FAT ops, not pack reads. - [ ] The images-off-card lever ([notes/git-sync-images-and-repo-size.md](notes/git-sync-images-and-repo-size.md)) - composes with the splice — decide whether to take it here. + composes with the splice — decide whether to take it here (user's call).