julien/typewriter
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(v0.7): record run 2, the checkout rebuild, and the closing gate

Browse Source
This commit is contained in:
Julien Calixte
2026-07-14 09:23:04 +02:00
parent 44c65e9ea5
commit 11545efb0a
1 changed files with 84 additions and 39 deletions
Download Patch File Download Diff File Expand all files Collapse all files

123
docs/v0.7-search-and-git.md
View File

@@ -3,54 +3,98 @@
> Part of the [Typoena macro plan](macroplan.md). Requirements and targets:
> [qfd.md](qfd.md). Load-bearing decisions: [adr.md](adr.md).
**Status:** **`/` search + `n N` done in core** (2026-07-13, host-tested — 207
editor tests). **`:gl` pull BUILT end-to-end 2026-07-14** (firmware 0.7.0,
compiles clean; editor refresh hooks host-tested, 209 tests) — **on-device
verification pending** (needs Wi-Fi + a remote that moved).
**Status: feature-complete, one gate left.** `/` search is done and
panel-verified (case-insensitive since 2026-07-14; `n`/`N` confirmed
on-device). `:gl` is built end-to-end and three of its four shapes are
on-device verified; the **fast-forward (Pulled) shape is the closing gate** —
its first attempt (2026-07-14, run 2) crashed in libgit2's `checkout_tree`
(O(tree) workdir walk + internal-DRAM exhaustion + an esp-idf spi_master
NULL-deref on its own failed-alloc path), and the checkout has been rebuilt as
an O(changed) tree-diff apply plus three memory fixes (file-list interning,
64 KB DMA reserve, TLS session reuse). **Re-run the Mac-push → device `:gl`
test to close the version.** Run-1 bonus: the first on-device
rejected-push → reconcile → replay → push success (replay commit 59444a94).
- [x] `/` forward search, `n N` — done in core 2026-07-13 (host-tested).
Literal, case-sensitive substring (no regex on-device, no smartcase
surprises); `/` reuses the `:` command line's editing with a `/` prompt
(one command-line mode, as in vim), and the jump happens on **Enter
only** — no incremental caret-chasing, for the same e-ink-refresh reason
snippets have no completion popup. Wraps around with a "wrapped" notice;
"not found: pat" leaves the caret alone; a bare `/`+Enter repeats the
last pattern, which is editor-global (survives buffer switches, like
vim's search register). `n`/`N` are motions (counts work, Visual extends
over them); deliberately **not** operator targets (`dn` out of scope).
- [x] `/` forward search, `n N` — core 2026-07-13, panel-verified 2026-07-14
(including `n`/`N`). Literal, **case-insensitive** substring (Unicode
lowercase fold — `É` matches `é`; no regex on-device; deliberately not
smartcase: a capital silently flipping to exact-match is a surprise, not
a feature, on a writing appliance). `/` reuses the `:` command line's
editing with a `/` prompt (one command-line mode, as in vim), and the
jump happens on **Enter only** — no incremental caret-chasing, for the
same e-ink-refresh reason snippets have no completion popup. Wraps
around with a "wrapped" notice; "not found: pat" leaves the caret alone;
a bare `/`+Enter repeats the last pattern, which is editor-global
(survives buffer switches, like vim's search register). `n`/`N` are
motions (counts work, Visual extends over them); deliberately **not**
operator targets (`dn` out of scope).
- [~] `:gl` — pull: fetch + **fast-forward only**, refuse on divergence and
surface it (renamed from the planned `:Gpull`). Editor command +
`Effect::Pull` done 2026-07-11; **the git-thread fetch/fast-forward is
BUILT 2026-07-14** — on-device verification pending. How it works:
surface it (renamed from the planned `:Gpull`). On-device 2026-07-14:
**UpToDate shape, dirty-journal gate, and the reconcile/replay path
verified**; the **Pulled (fast-forward) shape is rebuilt after the run-2
crash and awaits its on-device pass** — the closing gate. How it works:
- The git thread takes a `GitRequest::Pull` on the same channel as publish
(shared `ensure_online` Wi-Fi/clock/TLS preamble). `pull_once` fetches
origin (the fetch also refreshes the remote-tracking ref, keeping
publish's radio-free up-to-date check honest), then maps the four shapes:
up to date / **LocalAhead** (stranded commit — `:sync`'s job) / clean
fast-forward / **Diverged** (refused, no merge on the device).
- The fast-forward is checkout-then-ref-move with a **SAFE** checkout: it
refuses to overwrite a file whose content differs from HEAD (belt; the UI
gate is the braces). FAT caveat: the splice never updates the index, so
SAFE re-hashes each file the pull changes — O(changed), never O(tree).
(shared `ensure_online` Wi-Fi/clock/TLS preamble). `pull_once` does
**ls-refs first**: the ref advertisement alone answers "anything new?", so
up to date and LocalAhead (stranded commit — `:sync`'s job) return without
entering pack negotiation (run 1 paid a 9.7 s fetch to learn "up to
date"); a needed download rides the same open connection. Divergence is
refused — no merge on the device. Tracking-ref updates keep publish's
radio-free up-to-date check honest.
- The fast-forward is an **O(changed) tree-diff apply, not `checkout_tree`**
(`apply_tree_diff`, the splice philosophy on the read side):
`diff_tree_to_tree` skips identical subtree OIDs in memory, then exactly
the changed paths are written (unlink+tmp+rename — FAT `f_rename` won't
overwrite) or unlinked, and the ref moves last. libgit2's SAFE checkout
instead iterates the whole working directory (readdir over SPI on all
~1100 files) — run 2 died there: internal DRAM was exhausted mid-walk and
esp-idf's spi_master NULL-derefs on its own failed-DMA-alloc path
(LoadProhibited in setup/uninstall_priv_desc). Belt kept O(changed): any
to-be-clobbered file whose content no longer hashes to the old tree's blob
aborts the pull before the first write (edits made behind git's back —
e.g. desktop edits directly on the card). A half-applied working copy with
the ref unmoved self-heals: the next `:gl` re-applies idempotently.
- UI gate: `:gl` is refused while the dirty journal is non-empty
("unsynced changes - :sync first") — an unpublished save would fight the
checkout, and sync-then-pull is the single-writer appliance's natural
order. A RAM-dirty (never-saved) buffer does *not* gate: its edits simply
win over the pulled state (last-writer-wins, like the reconcile).
("unsynced changes - :sync first", also logged) — an unpublished save
would fight the apply, and sync-then-pull is the single-writer appliance's
natural order. A RAM-dirty (never-saved) buffer does *not* gate: its edits
simply win over the pulled state (last-writer-wins, like the reconcile).
- After a `Pulled` outcome the UI drops clean parked buffers (next switch
re-reads the disk), re-reads the clean active buffer in place
(`Editor::refresh_active` — boot posture, undo cleared), keeps a
RAM-dirty active buffer untouched, and re-walks the palette file list.
Snackbar: `pulled <oid>` / `up to date` / `ahead - :sync to publish` /
`diverged - resolve on a computer` / `pull: <reason>`.
RAM-dirty active buffer untouched, and re-walks the palette file list in
the background. Snackbar: `pulled <oid>` / `up to date` /
`ahead - :sync to publish` / `diverged - resolve on a computer` /
`pull: <reason>`.
Memory + connection work landed here (2026-07-14, forced by the run-2 crash;
all three compose):
- [x] **File-list interning** — 1099 palette paths as individual `String`s
measured **182 KB of internal DRAM** (each alloc under the 16 KB
SPIRAM-malloc threshold), a main contributor to the DMA-pool exhaustion.
The walk now emits one newline-joined blob (seeded at 64 KB → PSRAM);
the editor stores blob + sorted spans (`set_file_list_joined`) and the
palette reads slices. Expected walk log line now: ~0 KB internal
consumed — confirm on the next boot.
- [x] **DMA reserve 32 → 64 KB** (`CONFIG_SPIRAM_MALLOC_RESERVE_INTERNAL`) —
the esp-idf SPI driver crashes rather than errors on a failed DMA alloc,
so the floor must be generous enough that it never sees one.
- [x] **TLS session resumption + one-connection pull** — every git op paid a
full multi-second handshake (a rejected-push `:sync` runs three).
`esp_mbedtls_stream.c` (second vendor delta) caches the mbedTLS session
at stream close and offers it on the next connect to the same host;
measure as first-op vs second-op `ls-refs`/handshake time in a session.
Sync performance — inherited from the
[real-repo-sync kaizen](kaizen/real-repo-sync.md) (closed 2026-07-13 at
24.1 s end-to-end; target ≤ ~10 s):
- [ ] Close the publish gap: 19.8 s today — the splice's 5.0 s and the ~7 s of
repo-open/negotiation overhead are the next curves
([tradeoff analysis](tradeoff-curves/sync-commit-staging.md)).
- [~] Close the publish gap: 19.8 s at the kaizen close — the splice's 5.0 s
and the ~7 s of repo-open/negotiation overhead are the next curves
([tradeoff analysis](tradeoff-curves/sync-commit-staging.md)). TLS
session resumption (above) attacks the negotiation half; re-measure a
warm `:sync` before hunting further.
- [~] Instrument the residual ~360 ms/loose-write (suspect: FAT directory-op
cost in the freshen/refresh path) — **instrumentation BUILT 2026-07-14,
measurement run pending**: `sd_bench` grew a directory-entry-scaling
@@ -58,8 +102,9 @@ Sync performance — inherited from the
siblings — FAT resolves paths by linear scan, so cost climbing with N
convicts dir scans; flash with `just flash-bench`), and `stage_and_commit`
now brackets the splice with the `p_mmap` counters, logging
`N mmaps / M KB read` per commit so the next real `:sync` splits the
residual into pack-read I/O vs everything else.
`N mmaps / M KB read` per commit. First data point (run 1, 2026-07-14):
both real-repo splices logged **0 mmaps / 0 KB read** — the residual is
FAT ops, not pack reads.
- [ ] The images-off-card lever
([notes/git-sync-images-and-repo-size.md](notes/git-sync-images-and-repo-size.md))
composes with the splice — decide whether to take it here.
composes with the splice — decide whether to take it here (user's call).