Step 6 evaluated: infinity (never completes) to 24.1s measured
snackbar-to-snackbar, with the measured after-bar charted in step 6 (the
step-1 chart stays potential-only, per the format). Documents the second
localization loop the verification runs forced on the push half: UI death
by shared heap fate, odb cache exonerated, telemetry axis bug, mmaps named
as the eater, the impossible fixed-maps budget, the keep-alive race, and
pack layout as a device-performance variable.
Runs 7-9 on the real repo (2026-07-13): esp_map live-bytes wired into
log_push_heap plus a 5s sibling heartbeat thread proved the push OOM was
libgit2's mmaps — ~3.4MB of whole-file .idx/multi-pack-index maps living
OUTSIDE the mwindow budget, plus windows at the (working) 4MB soft limit,
an impossible budget on the 8MB pool. Shrinking to 64KB windows / 1.5MB
mapped limit left 4.5MB min-ever free and cut marking reads 4x; with the
card repacked to a single pack, the first-ever real-repo push landed
(6713ea6e, :sync 24.1s end-to-end).
The heartbeat stays for now (the marking phase fires no libgit2 callbacks,
so it is the only heap trace through it); its 8KB internal stack is part
of the min-ever-2099B-internal tightness flagged for follow-up.
Run 4 (2026-07-13): remote.push() consumed ~6 MB of PSRAM over 66 s and
the UI thread aborted on Frame::new_white's per-draw vec![0xFF; 26928].
Run 5 confirmed the UI now survives, but the push still exhausts both
pools (a ~7 KB inflateInit fails inside the pack build) and the heap
telemetry never fired — hence the telemetry rework.
- display/editor: Editor::draw_into() renders into a caller-owned
Frame; main.rs keeps two boot-time frames (shown/back) and mem::swaps
on success, so steady-state repaints never allocate.
- git_sync: odb cache capped at 1 MB via raw libgit2-sys opts (git2
0.20 doesn't wrap the total cap); log_push_heap at pre-push,
post-push and push-failure with largest-PSRAM-block and per-pool
min-evers; pack_progress logging is now time-gated (2 s) — the
count gate (+256 objects) never fired because AddingObjects reports
total=0 and a small push inserts only dozens of objects.
- editor: empty/whitespace snippets file parses as 0 snippets instead
of a JSON error at boot.
- build.rs: refuse a git-feature build with unset TW_* publish vars —
a bare `cargo build --features git` baked empty creds and produced a
firmware whose :sync could never work.
- Palette Ctrl-N/P (and half-page keys) wrap around the result list
instead of clamping at the ends; tests updated to the new semantics.
- try_push maps libgit2's negotiation-time NotFastForward error to
PushFailure::Rejected so the reconcile path runs instead of failing
as a transport error.
- walk_files decodes esp-idf d_type values (DT_REG=1/DT_DIR=2) that
libc 0.2.178's generic unix table misreads as fifo/chardev — the
palette walk was silently dropping every file on the card.
- `just load` refuses to mirror over unpublished device edits listed
in the card's .typoena-dirty journal (backup+discard via
TW_DISCARD_UNPUBLISHED=1), gains --force for a from-scratch copy,
and a flash-only recipe flashes the already-built ELF.
The tradeoff-curves doc stays the chronological data layer; the kaizen
is the story layer — value model, 8-factor hypothesis table, ideas
ranked by verified cause, and an evaluation left PENDING until the
end-to-end on-device :sync lands.
"free heap 5.8MB" in the crash log was PSRAM and masked the actual
internal-RAM exhaustion. git_sync's publish/commit/push lines now
carry the MALLOC_CAP_INTERNAL reading, and boot brackets the palette
file-list build with it — small Strings are forced internal by the
16 KB SPIRAM malloc threshold, so the 1098-path list is a suspected
~60-70 KB DRAM consumer competing with Wi-Fi/TLS; the log decides
whether interning into one PSRAM buffer is worth it.
esp-idf's default is internal-RAM-only, and a TLS connection needs
~33 KB of it (two ~17 KB I/O buffers + contexts) at the exact moment
:sync pushes — with Wi-Fi, USB host, the editor and libgit2 resident,
mbedtls_ssl_setup failed there on the first real-repo push (and its
failure path then hit the stream double-free fixed in the previous
commit). TLS buffers are CPU-only data, so PSRAM is safe; the
handshake is network-bound.
Upstream libgit2 v1.9.4 bug: mbedtls_stream_wrap's ssl_setup error
path closes and frees the caller's socket stream, then
git_mbedtls_stream_new frees it again — and wrap's other error paths
don't free it at all, so callers can't compensate either way. When
ssl_setup failed on the device (internal-RAM exhaustion during the
first real-repo push), the double git__free tripped tlsf ("block
already marked as free") and reset the chip mid-:sync.
Fixed the way esp_map.c replaces map.c: the vendored streams/mbedtls.c
is excluded from the component and esp_mbedtls_stream.c (verbatim copy
+ one-hunk fix: out_err leaves st->io to the caller, and frees the
leaked ssl struct) takes its place. Keep the copy in lockstep on
submodule bumps; worth reporting upstream.
The per-entry metadata() call makes FatFS re-walk the directory by
path every time — ~32ms/file on the card, 35s for a 1098-file tree.
esp-idf's FAT VFS always fills d_type (DT_DIR/DT_REG from the FILINFO
readdir already holds, never unknown) and Rust std maps file_type()
onto it stat-free, so the walk is now one readdir pass per directory.
With the file list now a recursive walk of the whole card, an unranked
short query pages through too much. Below PALETTE_MIN_QUERY (2) the
result list is the MRU only, keeping Cmd-P + Enter quick-switch one
keystroke away; two chars reveal the full fuzzy-ranked list. Commands
(>) and snippets ($) are short curated lists and keep matching from
the first char.
A nested repo (jcalixte/notes) showed only its top-level files in the
palette. Dot entries are skipped at every level so .git is never
descended; each directory is read fully before recursing so a single
FatFS dir handle is open at a time; depth capped at 8. The boot walk
logs its file count and duration to keep the FAT dir-IO cost visible.
The merged plan section now records the shipped design: single path
set (working tree as truth) instead of {changed, deleted}, the dirty
journal, stranded-commit recovery, radio-free up-to-date, and the
soft-reset reconcile with its carried-files side win.
The device's libgit2 has no SSH transport (HTTPS+PAT over mbedTLS
only), but _load-repo copied the desktop clone's origin verbatim — an
SSH-shaped URL fails every on-device push/fetch with "unsupported URL
protocol" (first real-repo :sync, 2026-07-13). Derive the HTTPS
equivalent for the card copy (git@host:path and ssh://git@host/path);
the source clone keeps its own URL. Warns on non-github hosts, since
the embedded trust store carries GitHub roots only.
The index pipeline (add_all → index.write → write_tree) is O(N_tree)
and cannot commit the real 1179-file / 570 MB-pack clone (index.write
measured up to 611 s on FAT's racy-clean re-hash). stage_and_commit is
now an O(depth) TreeBuilder splice of exactly the paths the editor
saved or deleted — ~2-2.8 s on the real clone — with the working tree
as source of truth (existing file → insert, missing → remove).
Storage records those repo-relative paths on every save/delete and
journals them to /sd/.typoena-dirty (atomic, only on growth), so a
power pull can't strand a saved-but-unpublished note now that nothing
walks the tree. take_dirty → publish_succeeded/publish_failed settles
each publish's snapshot from the UI outcome handler.
Also required by / discovered with the splice:
- mwindow opts at git-service start (32-bit defaults would OOM PSRAM
on the first real-pack access; bench-proven 256 KB / 4 MB).
- 16-FD mount for git builds (libgit2 holds pack+idx descriptors open;
the editor's 4-FD budget overruns).
- reconcile is a soft reset (no index to reset anymore); side win: a
remote-only added file is carried by the replay instead of dropped.
- stranded-commit recovery: tree-unchanged now pushes anyway when
origin/<branch> lacks HEAD (a commit whose push failed used to be
silently never retried).
- radio-free up-to-date: empty dirty set + origin at HEAD answers
without bringing Wi-Fi up.
- try_push splits ref rejection (reconcilable) from transport failure
(surfaced directly) — the first on-device run burned a doomed
reconcile on "unsupported URL protocol" and hid the cause.
Deliberate behavior change: files changed on the card outside the
editor are never committed anymore (also retires the macOS-cruft
filter). Trail: docs/tradeoff-curves/sync-commit-staging.md.
The bench phase the handoff was written for is closed (splice benched,
fast-seek landed, cache removed, decision made), so the note's live half —
the firmware plumbing plan — moves into sync-commit-staging.md as "The fix —
wiring the O(depth) splice into the firmware", reconciled to the run-5 state.
The duplicated TL;DR/measurement summaries are dropped in favor of the
curve's own trail; inbound references (notes index, git_bench comments)
now point at the curve.
Run 4: memory discipline verified (1833 KB flat, no OOM) but 0 hits
with the small maps demonstrably retained — window-repetition theory
refuted, sub-second bar failed (splice 2.83 s cold / ~2 s steady).
Decision: wire the splice in anyway (~9-10 s cold real-repo :sync vs
611 s/OOM for every alternative). Run 5: cache removal confirmed
free; the ~1.85 MB "resident" was mwindow's live window set, which
makes the mwindow opts in shipping git_sync.rs load-bearing.
Four instrumented real-repo bench runs scored 0 cache hits: libgit2's
mwindow layer reuses its open windows, so only genuinely new
(offset, len) ranges ever reach p_mmap — there is no repetition to
cache. The 7.4 MB OOM the v2 discipline fixed was caused by the cache
itself holding buffers past p_munmap; plain free-at-munmap is honest
with MWINDOW_MAPPED_LIMIT by construction. Run 5 confirmed removal is
I/O-neutral on device (byte-identical read pattern, warm splice 1953
vs 1949 ms) and even 15 reads better — v2's low-water eviction was
fighting mwindow. Stats counters kept to spot any future workload
that does repeat ranges.
Admission keyed on file size (>= 1 MB) instead of map length: the hot
set is the small repeated maps (pack trailer, idx fanout, delta bases)
that the 64 KB floor excluded — 0 hits across three real-repo runs.
Small mutable working-tree files stay uncacheable. p_munmap now evicts
unreferenced entries to a 2 MB low-water mark so released windows are
actually returned to git__malloc (fixes the 7.4 MB resident / 508 KB
heap zlib OOM and keeps MWINDOW_MAPPED_LIMIT honest).
Without CONFIG_FATFS_USE_FASTSEEK every long/backward lseek in the
263 MB pack walks the FAT cluster chain over SPI (~190 ms), and libgit2
pays ~8 such seeks per loose-object write. The CLMT makes lseek O(1)
for read-mode files; verified on device: far seek 198.7 -> 20.4 ms,
splice commit 6.5 -> 2.8 s.
Reads 4 KB at the start vs the end of the repo's largest pack (skipping
macOS ._ sidecars). Proved the ~1.5 s/loose-object cost was FatFS
walking the FAT cluster chain on every long/backward lseek: 5.8 ms at
offset 0 vs 198.7 ms at the end of the 263 MB pack.
Adds the splice prototype (patch one path onto HEAD's tree, O(depth))
as git_bench's headline op, run first so its cold number survives the
index ops' OOM, plus odb.read_header/odb.exists probes and strict-off
re-benches that localized the residual cost and refuted the
strict-object-creation theory.
The real jcalixte/notes clone (570 MB pack, 1179 files) proved both index
strategies are O(N_tree) and unshippable: index.write re-hashes the tree
(611 s), and the index-free read_tree is 77 s cold and OOMs the mmap cache
(zlib crash). Record the run in the tradeoff curve, revise the verdict to an
O(depth) TreeBuilder walk, and add a handoff note with the design, firmware
call sites, and bench steps for the next session.
Replace the index.write()/write_tree ops (which re-hash the whole working
tree on a fresh FAT clone -- up to 611 s on the real repo) with the
index-free candidate: Index::new + read_tree(HEAD) + blob + write_tree_to.
Add per-op mmap-cache + free-heap logging (via esp_map_stats), announce each
op before it runs so a hang reveals the culprit, mount_for_git for the FD
budget, tune the mwindow limits, and drop N 10 -> 3 for the slow real clone.
libgit2 keeps the pack, .idx and commit-graph descriptors open for the
repo's lifetime and opens loose objects on top, so a read_tree walk overruns
the editor's tight 4-FD budget with "no free file descriptors". Add
mount_for_git (16 FDs, matching the flash-FAT git binaries) alongside the
editor's mount; both route through a shared mount_with_max_files.
p_mmap emulated a mapping by malloc+read()ing the range from SD on every
call. libgit2 re-hits the pack idx/windows on every object write (via
git_odb__freshen -> git_odb_refresh), so a commit re-read pack bytes over
SPI repeatedly -- ~500ms-1.1s/op on a small repo, far worse on the real
570 MB pack.
Cache read-only mappings >= 64 KB (pack idx/windows, commit-graph, midx,
packed-refs -- all immutable on this device) in a refcounted, PSRAM-backed
slot table keyed by (dev, ino, size, mtime, offset, len), LRU-evicted under
a soft cap. Small mutable working-tree maps (diff_file.c) fall below the
floor and are never served stale.
Break the stage+commit window into sub-phases (FAT working-tree walk vs
object writes) via `commit split —` log lines, and add two micro-benchmarks
(sd_bench for SD/FAT primitive ops, git_bench for libgit2 object overhead)
with justfile recipes. Documents the walk-vs-writes cost model in
tradeoff-curves/sync-commit-staging.md to decide whether explicit-path
staging over the editor's dirty set is worth replacing add_all(["*"]).
esp-idf-hal always compiles its i2c.rs, which binds the old driver/i2c.h
API, so ESP-IDF links the legacy driver's constructor into every image and
prints its deprecation warning at boot. Typoena uses no I2C (EPD/SD are SPI,
keyboard is USB), so the warning is pure noise; this flag compiles the
constructor (and its conflict-abort check) out.
`just load` / `just init` now fast-forward the source repo from its
origin before rsyncing to /sd/repo, so the card picks up notes the
device already pushed. Fast-forward-only and best-effort: dirty tree,
diverged history, detached HEAD, or offline warns and copies the
current on-disk state rather than aborting. TW_NO_PULL=1 skips it.
Default espflash baud (115200) made a full flash take ~2-3 min of
serial transfer. Bump the cargo runner and the two direct-espflash
recipes (flash-git-push, flash-git-sync) to 921600. Monitor recipes
keep 115200 to match the device UART log rate.
Mark v0.6 delivered (core, host-tested; on-device gate pending): the
snippet engine, the Cmd-P/>/$ palette split retiring :e, and the just
init catalog. Note the →/≠ ISO-8859-15 caveat.
Record the firmware boot-read (serde_json builds for xtensa) and the
just-init catalog as delivered, and replace the proposed French catalog
with the finalised 17-snippet English set (three groups). Note the
→/≠ base-font caveat and the no-clobber seeding guard.
Add a curated snippet catalog (firmware/snippets-catalog/, three opt-in
groups: Symbols, Structure, Prose — 17 snippets, English prefixes,
translated from the source Zed set) and a _seed-configs step in `just
init` that jq-merges the chosen groups into repo/.typoena.snippets.json
and writes a starter repo/.typoena.toml. Seeds only files that are
absent, so re-running init never clobbers a synced library.
Mirror the prefs boot-read: load the git-tracked snippet library from the
SD repo, parse it with Snippets::parse, and hand it to the editor before
the first render. Missing/unreadable/malformed is non-fatal (no snippets,
editor runs). Verified serde_json builds for xtensa (cargo check, 0.6.0).
Record the > command registry, one-shot/two-step dispatch, and the :e
retirement as done in core; note the ISO-8859-15 label constraints
(ASCII `...`, `»` hint marker).
Turn the > palette from a settings-only list into an action registry
dispatched by PaletteCmd::kind: toggles stay open, one-shots (format,
publish) run and close, and the parameterised `new file...` morphs the
palette into a filename input step. Share run_publish between :sync and
the publish command, and retire :e (bare Cmd-P opens files).
Snapshot the snippet name inline Tab would expand into a panel field in
refresh_stats, so it rides the typing-pause throttle rather than
repainting per keystroke. Draw it as `» name` on the row above the mode
line (Latin-9 has no tab glyph), sharing the slot with the NO KBD flag,
which can't co-occur since the hint means you're typing.
Parse the Zed-compatible .typoena.snippets.json (serde_json), strip
${n:label} stops to bare $n, and drive a forward-only tab-stop session.
Expand a prefix inline on Tab in Insert; browse and insert from the
Cmd-P palette via a $ sigil (parallel to the > command mode), landing
on $1 as one undo group.
Reshape v0.6 snippets from a hard-coded table into a git-synced,
Zed-compatible JSON library (.typoena.snippets.json) with an inline Tab
path and a $ palette launcher. Add the file-format reference and
document the Cmd-P/>/$ verb split and the just init catalog.
Reference doc gains a theme key, an Options column, and the e-paper dark-mode
ghosting caveat. The v0.5 spec gets an amendment note superseding the two
reversed "decide-before-build" calls (auto_sync now a set-ahead palette
command; theme shipped early) without rewriting the verified-on-device
history; macroplan updated to match.
Generalise the palette so Enter advances any pref to its next value and
wraps; a boolean toggle is the two-option case. Both string prefs share a
next_option(current, &OPTIONS) helper (off-list values snap to the head).
- theme (light/dark): rendered as a single whole-frame Frame::invert at the
end of draw(), so text, selection, caret, panel and palette flip together.
- auto_sync: cycles 2m/5m/10m/15m/30m. Set-ahead only — still read by
nothing until the v0.7 periodic push.
toggle_pref -> cycle_pref; palette hint "Enter toggle" -> "Enter change".
Record the d14d9e7 behaviour the doc didn't cover: format-on-save keeps at
most one trailing blank line, and load-verbatim + guarded-save render a
file's POSIX terminator as a visible trailing empty line.
Move each version's scope checklist and status out of the monolithic
macroplan into a dedicated docs/vX.Y-<slug>.md page; macroplan keeps the
source block, the rollup status, and a one-line summary + link per
version. v0.1 reuses its existing product/technical pages.
Also drop the "Optional column ruler at 80" requirement from v0.6.
