# v0.7 — Search + better git

> Part of the [Typoena macro plan](macroplan.md). Requirements and targets:
> [qfd.md](qfd.md). Load-bearing decisions: [adr.md](adr.md).

**Status: CLOSED 2026-07-14.** `/` search is done and panel-verified
(smartcase + accent-folded; `n`/`N` confirmed on-device). `:gl` is verified
on-device in all four shapes: the **fast-forward (Pulled) closing gate passed
on run 3** — `main 17840720 → e738a6a1`, `apply_tree_diff` wrote the one
changed file in 1271 ms, internal DRAM never dropped below ~48 KB, and the
palette re-walk picked the new file up in the background (0 KB internal
consumed — interning confirmed). Run 2 had crashed there in libgit2's
`checkout_tree` (O(tree) workdir walk + internal-DRAM exhaustion + an esp-idf
spi_master NULL-deref on its own failed-alloc path); the rebuild as an
O(changed) tree-diff apply plus three memory fixes (file-list interning,
64 KB DMA reserve, TLS session reuse) is what run 3 verified. Run-1 bonus:
the first on-device rejected-push → reconcile → replay → push success; run 3
repeated that cycle at 24 s total (was 59 s — the reconcile fetch fell from
~30 s to ~5 s with session resumption). Residual watch item from run 3:
boot loaded 1 byte from notes.md but `:w` saved 0 — possible trailing-newline
round-trip asymmetry on an empty-note buffer (host-test it; harmless here,
the tree was unchanged).

- [x] `/` forward search, `n N` — core 2026-07-13, panel-verified 2026-07-14
      (including `n`/`N`). Literal substring (no regex on-device), matched
      **smartcase + accent-folded** (2026-07-14, user decision — supersedes
      the same-day plain-insensitive version): an all-lowercase pattern is
      case-insensitive, one capital makes it exact (vim smartcase), and
      diacritics always fold both ways — `/ete` finds `été`, `/été` finds
      `ete` (Latin-1 repertoire; ligatures `œ`/`æ` excluded — they fold to
      two chars). `n`/`N` recompute smartcase from the remembered pattern, so
      repeats behave like the original search. `/` reuses the `:` command line's
      editing with a `/` prompt (one command-line mode, as in vim), and the
      jump happens on **Enter only** — no incremental caret-chasing, for the
      same e-ink-refresh reason snippets have no completion popup. Wraps
      around with a "wrapped" notice; "not found: pat" leaves the caret alone;
      a bare `/`+Enter repeats the last pattern, which is editor-global
      (survives buffer switches, like vim's search register). `n`/`N` are
      motions (counts work, Visual extends over them); deliberately **not**
      operator targets (`dn` out of scope).
- [x] `:gl` — pull: fetch + **fast-forward only**, refuse on divergence and
      surface it (renamed from the planned `:Gpull`). On-device 2026-07-14:
      UpToDate shape, dirty-journal gate, and the reconcile/replay path
      verified (runs 1 and 3); **Pulled (fast-forward) verified run 3**
      (ls-refs 2792 ms, download 5136 ms, apply 1271 ms / 1 file, `:gl` total
      16.0 s — ~3 s of the total is still unaccounted connect/negotiation,
      tracked with the publish gap below). How it works:
  - The git thread takes a `GitRequest::Pull` on the same channel as publish
    (shared `ensure_online` Wi-Fi/clock/TLS preamble). `pull_once` does
    **ls-refs first**: the ref advertisement alone answers "anything new?", so
    up to date and LocalAhead (stranded commit — `:sync`'s job) return without
    entering pack negotiation (run 1 paid a 9.7 s fetch to learn "up to
    date"); a needed download rides the same open connection. Divergence is
    refused — no merge on the device. Tracking-ref updates keep publish's
    radio-free up-to-date check honest.
  - The fast-forward is an **O(changed) tree-diff apply, not `checkout_tree`**
    (`apply_tree_diff`, the splice philosophy on the read side):
    `diff_tree_to_tree` skips identical subtree OIDs in memory, then exactly
    the changed paths are written (unlink+tmp+rename — FAT `f_rename` won't
    overwrite) or unlinked, and the ref moves last. libgit2's SAFE checkout
    instead iterates the whole working directory (readdir over SPI on all
    ~1100 files) — run 2 died there: internal DRAM was exhausted mid-walk and
    esp-idf's spi_master NULL-derefs on its own failed-DMA-alloc path
    (LoadProhibited in setup/uninstall_priv_desc). Belt kept O(changed): any
    to-be-clobbered file whose content no longer hashes to the old tree's blob
    aborts the pull before the first write (edits made behind git's back —
    e.g. desktop edits directly on the card). A half-applied working copy with
    the ref unmoved self-heals: the next `:gl` re-applies idempotently.
  - UI gate: `:gl` is refused while the dirty journal is non-empty
    ("unsynced changes - :sync first", also logged) — an unpublished save
    would fight the apply, and sync-then-pull is the single-writer appliance's
    natural order. A RAM-dirty (never-saved) buffer does _not_ gate: its edits
    simply win over the pulled state (last-writer-wins, like the reconcile).
  - After a `Pulled` outcome the UI drops clean parked buffers (next switch
    re-reads the disk), re-reads the clean active buffer in place
    (`Editor::refresh_active` — boot posture, undo cleared), keeps a
    RAM-dirty active buffer untouched, and re-walks the palette file list in
    the background. Snackbar: `pulled <oid>` / `up to date` /
    `ahead - :sync to publish` / `diverged - resolve on a computer` /
    `pull: <reason>`.
  - **Superseded (v0.9):** divergence is no longer refused — `:gl` now rebases
    the device's local commit(s) onto origin (`rebase_local_onto`,
    last-writer-wins per note, no content merge) and ends `LocalAhead` for
    `:gp` to publish. The `diverged - resolve on a computer` snackbar is
    replaced by `rebased <oid> - :gp to publish`. See
    [v0.9-robustness.md](v0.9-robustness.md).

Memory + connection work landed here (2026-07-14, forced by the run-2 crash;
all three compose):

- [x] **File-list interning** — 1099 palette paths as individual `String`s
      measured **182 KB of internal DRAM** (each alloc under the 16 KB
      SPIRAM-malloc threshold), a main contributor to the DMA-pool exhaustion.
      The walk now emits one newline-joined blob (seeded at 64 KB → PSRAM);
      the editor stores blob + sorted spans (`set_file_list_joined`) and the
      palette reads slices. **Confirmed run 3**: the post-pull re-walk logged
      `0 KB consumed, blob 51 KB`. (The boot walk's "125 KB consumed" is a
      measurement artifact — that window runs concurrently with USB attach and
      other boot allocations; the quiescent re-walk is the clean reading.)
- [x] **DMA reserve 32 → 64 KB** (`CONFIG_SPIRAM_MALLOC_RESERVE_INTERNAL`) —
      the esp-idf SPI driver crashes rather than errors on a failed DMA alloc,
      so the floor must be generous enough that it never sees one.
- [x] **TLS session resumption + one-connection pull** — every git op paid a
      full multi-second handshake (a rejected-push `:sync` runs three).
      `esp_mbedtls_stream.c` (second vendor delta) caches the mbedTLS session
      at stream close and offers it on the next connect to the same host.
      First same-session evidence (run 3): the reconcile fetch — always the
      second connection of a rejected-push `:sync` — fell from ~30 s (run 1)
      to ~5 s; a controlled first-op vs second-op `ls-refs` comparison is
      still worth logging when convenient.

Sync performance — inherited from the
[real-repo-sync kaizen](kaizen/real-repo-sync.md) (closed 2026-07-13 at
24.1 s end-to-end; target ≤ ~10 s):

- [~] Close the publish gap: 19.8 s at the kaizen close — the splice's 5.0 s
  and the ~7 s of repo-open/negotiation overhead are the next curves
  ([tradeoff analysis](tradeoff-curves/sync-commit-staging.md)). TLS
  session resumption (above) attacks the negotiation half; re-measure a
  warm `:sync` before hunting further. Run-3 data point: a rejected-push
  `:sync` (worst case, three connections) totalled 24.0 s vs 59 s in
  run 1. **Warm clean publish measured 2026-07-14 (run 4, as `:gp` after
  the rename): 19.1 s** — but the mix shifted entirely. The push leg
  collapsed to **5.9 s** (6 objects; TLS-resumed connect 2.4 s vs 4.0 s
  for the session's first; post-repack marking cost invisible), and the
  splice grew to **10.3 s** because the committed file sat four
  directories deep: O(depth) means ~5 loose tree writes instead of 1
  (× the ~0.4 s dir-scan cost above) plus the session's first pack touch
  (5 mmaps / 1808 KB — the 1740 KB .idx map dominates). Same run:
  up-to-date `:gl` is now **4.7 s** git-side via the ls-refs fast path
  ("no fetch" — was 9.7 s in run 1). So the residual curve is now
  **splice depth × loose-write cost**, which is the same FAT-dir-scan
  story as above and shares its levers (pack-not-loose foremost) — a
  future perf pass; a root-level file's warm `:gp` should sit ≈ 12–13 s.
- [x] Instrument the residual ~360 ms/loose-write — **measured 2026-07-14,
      CONVICTED: FAT linear directory scans** (full analysis in the
      [tradeoff doc](tradeoff-curves/sync-commit-staging.md), run 6).
      `sd_bench`'s dir-entry-scaling section shows stat-miss cost linear in
      sibling count (~0.1 ms/entry/scan) and the loose-object composite
      reproducing the field number exactly: p50 402.9 ms at 256 siblings —
      the `.git/objects/` fan-out is ~256 dirs, and every path resolution
      under it scans them. Composes with the run-1 splice counters (**0
      mmaps / 0 KB read** — no pack I/O in the residual). Bounded (≤ ~2 s per
      typical commit) and accepted; the levers — fewer path resolutions in
      libgit2, or committing straight into a small pack instead of loose
      objects — are a future perf pass, not v0.x work. (The bench run also
      caught a cleanup bug: std's `remove_dir_all` mis-types directories via
      the generic-unix DT constants and dies on FatFS `FR_DENIED`; `sd_bench`
      now removes its scratch tree itself, same decoding as the palette
      walk. The aborted run left `/sd/sdbench` on the card — the next bench
      run's setup sweep removes it.)
- [x] The images-off-card lever — **decided + landed 2026-07-14
      (skip-media-in-apply)**: the repo stays untouched (its images are
      remanso's blob-served CDN), and the device-side half shipped as media
      paths invisible to `apply_tree_diff` (never written/deleted/belt-hashed
      — a pulled 16 MB image was `:gl`'s last OOM path). Accepted trade: the
      card's media go stale vs HEAD, so a computer sees phantom
      modified/deleted images in `git status` — device-owned card, refresh
      with `just load`, never hand-commit from it. Full write-up in
      [notes/git-sync-images-and-repo-size.md](notes/git-sync-images-and-repo-size.md).