Compare commits

..

66 Commits

Author SHA1 Message Date
Julien Calixte
2660a3e9dd perf(palette): trust dirent d_type instead of a per-entry stat
The per-entry metadata() call makes FatFS re-walk the directory by
path every time — ~32ms/file on the card, 35s for a 1098-file tree.
esp-idf's FAT VFS always fills d_type (DT_DIR/DT_REG from the FILINFO
readdir already holds, never unknown) and Rust std maps file_type()
onto it stat-free, so the walk is now one readdir pass per directory.
2026-07-13 01:05:10 +02:00
Julien Calixte
79fad4689c docs(palette): amend v0.5 with the recursive walk and search threshold 2026-07-13 00:55:34 +02:00
Julien Calixte
98fc817b3f feat(palette): show recents only until the query reaches two chars
With the file list now a recursive walk of the whole card, an unranked
short query pages through too much. Below PALETTE_MIN_QUERY (2) the
result list is the MRU only, keeping Cmd-P + Enter quick-switch one
keystroke away; two chars reveal the full fuzzy-ranked list. Commands
(>) and snippets ($) are short curated lists and keep matching from
the first char.
2026-07-13 00:55:25 +02:00
Julien Calixte
d306caacf7 feat(palette): walk the card recursively for the file list
A nested repo (jcalixte/notes) showed only its top-level files in the
palette. Dot entries are skipped at every level so .git is never
descended; each directory is read fully before recursing so a single
FatFS dir handle is open at a time; depth capped at 8. The boot walk
logs its file count and duration to keep the FAT dir-IO cost visible.
2026-07-13 00:55:16 +02:00
Julien Calixte
2166b932b6 docs(sync): update the wiring section to how-it-landed
The merged plan section now records the shipped design: single path
set (working tree as truth) instead of {changed, deleted}, the dirty
journal, stranded-commit recovery, radio-free up-to-date, and the
soft-reset reconcile with its carried-files side win.
2026-07-13 00:53:50 +02:00
Julien Calixte
02b7ed88b6 fix(provisioning): rewrite the card origin to HTTPS at load
The device's libgit2 has no SSH transport (HTTPS+PAT over mbedTLS
only), but _load-repo copied the desktop clone's origin verbatim — an
SSH-shaped URL fails every on-device push/fetch with "unsupported URL
protocol" (first real-repo :sync, 2026-07-13). Derive the HTTPS
equivalent for the card copy (git@host:path and ssh://git@host/path);
the source clone keeps its own URL. Warns on non-github hosts, since
the embedded trust store carries GitHub roots only.
2026-07-13 00:53:50 +02:00
Julien Calixte
a5edaed810 feat(sync): commit by splicing journaled dirty paths onto HEAD
The index pipeline (add_all → index.write → write_tree) is O(N_tree)
and cannot commit the real 1179-file / 570 MB-pack clone (index.write
measured up to 611 s on FAT's racy-clean re-hash). stage_and_commit is
now an O(depth) TreeBuilder splice of exactly the paths the editor
saved or deleted — ~2-2.8 s on the real clone — with the working tree
as source of truth (existing file → insert, missing → remove).

Storage records those repo-relative paths on every save/delete and
journals them to /sd/.typoena-dirty (atomic, only on growth), so a
power pull can't strand a saved-but-unpublished note now that nothing
walks the tree. take_dirty → publish_succeeded/publish_failed settles
each publish's snapshot from the UI outcome handler.

Also required by / discovered with the splice:
- mwindow opts at git-service start (32-bit defaults would OOM PSRAM
  on the first real-pack access; bench-proven 256 KB / 4 MB).
- 16-FD mount for git builds (libgit2 holds pack+idx descriptors open;
  the editor's 4-FD budget overruns).
- reconcile is a soft reset (no index to reset anymore); side win: a
  remote-only added file is carried by the replay instead of dropped.
- stranded-commit recovery: tree-unchanged now pushes anyway when
  origin/<branch> lacks HEAD (a commit whose push failed used to be
  silently never retried).
- radio-free up-to-date: empty dirty set + origin at HEAD answers
  without bringing Wi-Fi up.
- try_push splits ref rejection (reconcilable) from transport failure
  (surfaced directly) — the first on-device run burned a doomed
  reconcile on "unsupported URL protocol" and hid the cause.

Deliberate behavior change: files changed on the card outside the
editor are never committed anymore (also retires the macOS-cruft
filter). Trail: docs/tradeoff-curves/sync-commit-staging.md.
2026-07-13 00:53:39 +02:00
Julien Calixte
e86a3b8254 docs(sync): merge the commit handoff into the staging tradeoff curve
The bench phase the handoff was written for is closed (splice benched,
fast-seek landed, cache removed, decision made), so the note's live half —
the firmware plumbing plan — moves into sync-commit-staging.md as "The fix —
wiring the O(depth) splice into the firmware", reconciled to the run-5 state.
The duplicated TL;DR/measurement summaries are dropped in favor of the
curve's own trail; inbound references (notes index, git_bench comments)
now point at the curve.
2026-07-13 00:29:07 +02:00
Julien Calixte
9309f3f239 docs(sync): record the esp_map v2 verdict and cache removal
Run 4: memory discipline verified (1833 KB flat, no OOM) but 0 hits
with the small maps demonstrably retained — window-repetition theory
refuted, sub-second bar failed (splice 2.83 s cold / ~2 s steady).
Decision: wire the splice in anyway (~9-10 s cold real-repo :sync vs
611 s/OOM for every alternative). Run 5: cache removal confirmed
free; the ~1.85 MB "resident" was mwindow's live window set, which
makes the mwindow opts in shipping git_sync.rs load-bearing.
2026-07-13 00:20:12 +02:00
Julien Calixte
1a5e1736f5 refactor(sync): remove the never-hit mmap window cache
Four instrumented real-repo bench runs scored 0 cache hits: libgit2's
mwindow layer reuses its open windows, so only genuinely new
(offset, len) ranges ever reach p_mmap — there is no repetition to
cache. The 7.4 MB OOM the v2 discipline fixed was caused by the cache
itself holding buffers past p_munmap; plain free-at-munmap is honest
with MWINDOW_MAPPED_LIMIT by construction. Run 5 confirmed removal is
I/O-neutral on device (byte-identical read pattern, warm splice 1953
vs 1949 ms) and even 15 reads better — v2's low-water eviction was
fighting mwindow. Stats counters kept to spot any future workload
that does repeat ranges.
2026-07-13 00:20:02 +02:00
Julien Calixte
ce3204a350 docs(sync): record fast-seek root cause and splice bench trail
Splice bench (6.5 s, O(depth) shape confirmed), FatFS cluster-chain
root cause with elm-chan/ESP-IDF references, on-device fast-seek
verification, second localization round (strict-creation refuted,
read_header 470 ms), esp_map v2 design, and the two firmware plumbing
gaps (mwindow opts + 16-FD mount).
2026-07-13 00:02:55 +02:00
Julien Calixte
dd3d70cbec perf(sync): cache small pack windows and evict mmap cache on munmap
Admission keyed on file size (>= 1 MB) instead of map length: the hot
set is the small repeated maps (pack trailer, idx fanout, delta bases)
that the 64 KB floor excluded — 0 hits across three real-repo runs.
Small mutable working-tree files stay uncacheable. p_munmap now evicts
unreferenced entries to a 2 MB low-water mark so released windows are
actually returned to git__malloc (fixes the 7.4 MB resident / 508 KB
heap zlib OOM and keeps MWINDOW_MAPPED_LIMIT honest).
2026-07-13 00:02:54 +02:00
Julien Calixte
2c1c590c9e perf(sync): enable FatFS fast seek for pack reads
Without CONFIG_FATFS_USE_FASTSEEK every long/backward lseek in the
263 MB pack walks the FAT cluster chain over SPI (~190 ms), and libgit2
pays ~8 such seeks per loose-object write. The CLMT makes lseek O(1)
for read-mode files; verified on device: far seek 198.7 -> 20.4 ms,
splice commit 6.5 -> 2.8 s.
2026-07-13 00:02:42 +02:00
Julien Calixte
2c24ece3a5 feat(bench): add packfile long-seek op to sd_bench
Reads 4 KB at the start vs the end of the repo's largest pack (skipping
macOS ._ sidecars). Proved the ~1.5 s/loose-object cost was FatFS
walking the FAT cluster chain on every long/backward lseek: 5.8 ms at
offset 0 vs 198.7 ms at the end of the 263 MB pack.
2026-07-13 00:02:42 +02:00
Julien Calixte
da8ca7d8d2 feat(bench): bench the O(depth) TreeBuilder splice and odb probes
Adds the splice prototype (patch one path onto HEAD's tree, O(depth))
as git_bench's headline op, run first so its cold number survives the
index ops' OOM, plus odb.read_header/odb.exists probes and strict-off
re-benches that localized the residual cost and refuted the
strict-object-creation theory.
2026-07-13 00:02:29 +02:00
Julien Calixte
dade5e6bb3 docs(sync): record real-repo commit bench and TreeBuilder handoff
The real jcalixte/notes clone (570 MB pack, 1179 files) proved both index
strategies are O(N_tree) and unshippable: index.write re-hashes the tree
(611 s), and the index-free read_tree is 77 s cold and OOMs the mmap cache
(zlib crash). Record the run in the tradeoff curve, revise the verdict to an
O(depth) TreeBuilder walk, and add a handoff note with the design, firmware
call sites, and bench steps for the next session.
2026-07-12 15:04:59 +02:00
Julien Calixte
a6f52df8b6 perf(sync): bench the index-free commit path and mmap cache
Replace the index.write()/write_tree ops (which re-hash the whole working
tree on a fresh FAT clone -- up to 611 s on the real repo) with the
index-free candidate: Index::new + read_tree(HEAD) + blob + write_tree_to.
Add per-op mmap-cache + free-heap logging (via esp_map_stats), announce each
op before it runs so a hang reveals the culprit, mount_for_git for the FD
budget, tune the mwindow limits, and drop N 10 -> 3 for the slow real clone.
2026-07-12 15:04:52 +02:00
Julien Calixte
6c5e666f4b feat(persistence): add mount_for_git with a larger open-file budget
libgit2 keeps the pack, .idx and commit-graph descriptors open for the
repo's lifetime and opens loose objects on top, so a read_tree walk overruns
the editor's tight 4-FD budget with "no free file descriptors". Add
mount_for_git (16 FDs, matching the flash-FAT git binaries) alongside the
editor's mount; both route through a shared mount_with_max_files.
2026-07-12 15:04:44 +02:00
Julien Calixte
c5e119f6be perf(git): cache emulated pack mmap to avoid re-reading the pack
p_mmap emulated a mapping by malloc+read()ing the range from SD on every
call. libgit2 re-hits the pack idx/windows on every object write (via
git_odb__freshen -> git_odb_refresh), so a commit re-read pack bytes over
SPI repeatedly -- ~500ms-1.1s/op on a small repo, far worse on the real
570 MB pack.

Cache read-only mappings >= 64 KB (pack idx/windows, commit-graph, midx,
packed-refs -- all immutable on this device) in a refcounted, PSRAM-backed
slot table keyed by (dev, ino, size, mtime, offset, len), LRU-evicted under
a soft cap. Small mutable working-tree maps (diff_file.c) fall below the
floor and are never served stale.
2026-07-12 15:04:37 +02:00
Julien Calixte
456c4c43e7 perf(sync): instrument and benchmark commit-staging latency
Break the stage+commit window into sub-phases (FAT working-tree walk vs
object writes) via `commit split —` log lines, and add two micro-benchmarks
(sd_bench for SD/FAT primitive ops, git_bench for libgit2 object overhead)
with justfile recipes. Documents the walk-vs-writes cost model in
tradeoff-curves/sync-commit-staging.md to decide whether explicit-path
staging over the editor's dirty set is worth replacing add_all(["*"]).
2026-07-12 12:24:50 +02:00
Julien Calixte
beb11eda5e chore(firmware): skip legacy-I2C conflict check
esp-idf-hal always compiles its i2c.rs, which binds the old driver/i2c.h
API, so ESP-IDF links the legacy driver's constructor into every image and
prints its deprecation warning at boot. Typoena uses no I2C (EPD/SD are SPI,
keyboard is USB), so the warning is pure noise; this flag compiles the
constructor (and its conflict-abort check) out.
2026-07-12 12:23:10 +02:00
Julien Calixte
88bb2b99cf feat(provisioning): pull source clone before copying to the card
`just load` / `just init` now fast-forward the source repo from its
origin before rsyncing to /sd/repo, so the card picks up notes the
device already pushed. Fast-forward-only and best-effort: dirty tree,
diverged history, detached HEAD, or offline warns and copies the
current on-disk state rather than aborting. TW_NO_PULL=1 skips it.
2026-07-12 10:59:05 +02:00
Julien Calixte
69843086f9 chore(firmware): flash at 921600 baud
Default espflash baud (115200) made a full flash take ~2-3 min of
serial transfer. Bump the cargo runner and the two direct-espflash
recipes (flash-git-push, flash-git-sync) to 921600. Monitor recipes
keep 115200 to match the device UART log rate.
2026-07-12 10:49:16 +02:00
Julien Calixte
45664fc956 feat: add glyphs 2026-07-12 10:43:41 +02:00
Julien Calixte
56f5d18bd8 docs(macroplan): record v0.6 markdown complete in core (0.6.0)
Mark v0.6 delivered (core, host-tested; on-device gate pending): the
snippet engine, the Cmd-P/>/$ palette split retiring :e, and the just
init catalog. Note the →/≠ ISO-8859-15 caveat.
2026-07-12 10:42:56 +02:00
Julien Calixte
767742ba12 docs(v0.6): mark boot-read + catalog done, finalise the catalog list
Record the firmware boot-read (serde_json builds for xtensa) and the
just-init catalog as delivered, and replace the proposed French catalog
with the finalised 17-snippet English set (three groups). Note the
→/≠ base-font caveat and the no-clobber seeding guard.
2026-07-12 10:36:58 +02:00
Julien Calixte
85cbeceea9 feat(provisioning): seed a snippet catalog + starter prefs on just init
Add a curated snippet catalog (firmware/snippets-catalog/, three opt-in
groups: Symbols, Structure, Prose — 17 snippets, English prefixes,
translated from the source Zed set) and a _seed-configs step in `just
init` that jq-merges the chosen groups into repo/.typoena.snippets.json
and writes a starter repo/.typoena.toml. Seeds only files that are
absent, so re-running init never clobbers a synced library.
2026-07-12 10:36:46 +02:00
Julien Calixte
8527f75bf8 feat(firmware): read .typoena.snippets.json at boot into set_snippets
Mirror the prefs boot-read: load the git-tracked snippet library from the
SD repo, parse it with Snippets::parse, and hand it to the editor before
the first render. Missing/unreadable/malformed is non-fatal (no snippets,
editor runs). Verified serde_json builds for xtensa (cargo check, 0.6.0).
2026-07-12 10:30:02 +02:00
Julien Calixte
5a076e3226 docs(v0.6): mark the palette generalisation delivered in core
Record the > command registry, one-shot/two-step dispatch, and the :e
retirement as done in core; note the ISO-8859-15 label constraints
(ASCII `...`, `»` hint marker).
2026-07-12 10:22:24 +02:00
Julien Calixte
c969d3e051 feat(editor): generalise the > palette into a command registry
Turn the > palette from a settings-only list into an action registry
dispatched by PaletteCmd::kind: toggles stay open, one-shots (format,
publish) run and close, and the parameterised `new file...` morphs the
palette into a filename input step. Share run_publish between :sync and
the publish command, and retire :e (bare Cmd-P opens files).
2026-07-12 10:22:17 +02:00
Julien Calixte
baf9715570 feat(editor): hint the matching snippet in the panel on typing pause
Snapshot the snippet name inline Tab would expand into a panel field in
refresh_stats, so it rides the typing-pause throttle rather than
repainting per keystroke. Draw it as `» name` on the row above the mode
line (Latin-9 has no tab glyph), sharing the slot with the NO KBD flag,
which can't co-occur since the hint means you're typing.
2026-07-12 10:09:43 +02:00
Julien Calixte
4aaf89d977 feat(editor): add snippet library, tab-stop engine, and $ palette
Parse the Zed-compatible .typoena.snippets.json (serde_json), strip
${n:label} stops to bare $n, and drive a forward-only tab-stop session.
Expand a prefix inline on Tab in Insert; browse and insert from the
Cmd-P palette via a $ sigil (parallel to the > command mode), landing
on $1 as one undo group.
2026-07-12 10:03:45 +02:00
Julien Calixte
221135cd9b docs(v0.6): specify the snippet library and $/> palette split
Reshape v0.6 snippets from a hard-coded table into a git-synced,
Zed-compatible JSON library (.typoena.snippets.json) with an inline Tab
path and a $ palette launcher. Add the file-format reference and
document the Cmd-P/>/$ verb split and the just init catalog.
2026-07-12 10:03:38 +02:00
Julien Calixte
93f3a43b34 docs(prefs): document theme + auto_sync presets, override v0.5 deferral
Reference doc gains a theme key, an Options column, and the e-paper dark-mode
ghosting caveat. The v0.5 spec gets an amendment note superseding the two
reversed "decide-before-build" calls (auto_sync now a set-ahead palette
command; theme shipped early) without rewriting the verified-on-device
history; macroplan updated to match.
2026-07-12 09:02:21 +02:00
Julien Calixte
f37da21462 feat(editor): add theme and auto_sync preset prefs with rotate-on-Enter
Generalise the palette so Enter advances any pref to its next value and
wraps; a boolean toggle is the two-option case. Both string prefs share a
next_option(current, &OPTIONS) helper (off-list values snap to the head).

- theme (light/dark): rendered as a single whole-frame Frame::invert at the
  end of draw(), so text, selection, caret, panel and palette flip together.
- auto_sync: cycles 2m/5m/10m/15m/30m. Set-ahead only — still read by
  nothing until the v0.7 periodic push.

toggle_pref -> cycle_pref; palette hint "Enter toggle" -> "Enter change".
2026-07-12 09:02:15 +02:00
Julien Calixte
fdd59093db docs(v0.5): document the trailing-newline display model
Record the d14d9e7 behaviour the doc didn't cover: format-on-save keeps at
most one trailing blank line, and load-verbatim + guarded-save render a
file's POSIX terminator as a visible trailing empty line.
2026-07-12 02:25:24 +02:00
Julien Calixte
55a8d21e0f docs(macroplan): split each version into its own page
Move each version's scope checklist and status out of the monolithic
macroplan into a dedicated docs/vX.Y-<slug>.md page; macroplan keeps the
source block, the rollup status, and a one-line summary + link per
version. v0.1 reuses its existing product/technical pages.

Also drop the "Optional column ruler at 80" requirement from v0.6.
2026-07-12 02:10:29 +02:00
Julien Calixte
d14d9e77a5 fix(persistence): show a saved note's trailing newline as an empty line
Read notes verbatim and insert the final newline only when it is missing,
instead of stripping the terminator on load and appending it unconditionally
on save. The editor's `rows = #\n + 1` model then renders a file's POSIX
terminator as a visible trailing blank line — what a writer expects: open a
note and see (and land the caret on) the empty line the newline stands for.

Supersedes the strip-on-load / unconditional-append handling that shipped
with the prefs work (c535864), which kept the buffer newline-free and hid the
terminator. Load + save are now an identity round-trip for any device-written
file (all end in '\n'); files stay git-clean (exactly one terminator); and a
trailing blank line the writer leaves is mirrored, never doubled.

- load_path: read verbatim (drop the strip)
- save_path: guarded final-newline (drop the unconditional append)
- Prefs::to_toml: ends in a newline again — the guarded save leaves exactly
  one, so the prefs file is byte-identical to before and its device-verified
  round-trip still holds
- sd_fat spike: payload ends in '\n' so its exact-equality round-trip holds
2026-07-12 02:02:46 +02:00
Julien Calixte
d46cdb7e7f docs(macroplan): mark v0.5 delivered, close slice-4 on-device gate
All four v0.5 slices are done and device-verified: the prefs round-trip is
confirmed both directions (boot-read + on-device palette edit -> push flipped
line_numbers on origin), and the slice-3 delete fix is recorded as confirmed on
device. Descoped items (buffer close, grey-Publish-in-Local cue, multi-file
publish count) noted, not blocking delivery.
2026-07-12 01:57:00 +02:00
Julien Calixte
c535864ee7 feat(editor): add .typoena.toml prefs and palette settings (v0.5 slice 4)
Git-tracked editor preferences read at boot and toggled live on-device:

- Prefs type (line-based TOML parse/serialize, no crate on xtensa) held on
  Editor; firmware reads /sd/repo/.typoena.toml before the first render and
  falls back to per-key defaults. Keys: save_on_idle, format_on_save,
  line_numbers (bool) + auto_sync (string, schema/default only until v0.7).
- line_numbers applied live (gutter_cols -> 0 when off).
- Palette > command mode toggles the three bools; the list stays open so
  several flip in one visit, and :settings opens it directly. Each toggle
  applies live and queues Effect::SavePrefs (host atomic-writes the file,
  which rides the next :sync).
- save_on_idle honoured host-side as a silent, unformatted idle auto-save.
- to_toml is newline-free; save_path now appends exactly one terminator
  unconditionally so buffers round-trip byte-for-byte (trailing blanks kept).
- Firmware 0.4.0 -> 0.5.0; new docs/typoena-toml.md reference. Also refreshes
  the slice-3 macroplan status (delete fix was confirmed on device).
2026-07-12 01:48:10 +02:00
Julien Calixte
82f305cea6 feat(persistence): end saved files with a trailing newline
Add the POSIX line terminator on save and strip it on load, so files
written by the editor no longer trip git's "No newline at end of file".
Done at the persistence choke point, not in :fmt: the editor buffer is
newline-free by design (rows = #\n + 1), so a trailing '\n' in the buffer
would render a phantom blank last line. save_path adds exactly one '\n'
(guarded against doubling); load_path strips one back off so the buffer
stays newline-free and round-trips byte-stable.

Update the sd_fat spike payload to be newline-free so its byte-identity
round-trip assertion holds under the new normalizing contract.
2026-07-12 01:11:28 +02:00
Julien Calixte
c9c07165e0 feat(editor): add :enew and :delete with real git-staging (v0.5 slice 3)
:enew <name> creates a new file (empty, dirty, added to the palette list);
:delete unlinks the current file via a new Effect::Delete and switches to a
parked buffer or scratch. Scope is read from the path (local/x, repo/x) rather
than a modal prompt, and the /sd prefix is now optional in resolve_path.

On-device testing showed deletions never reached the remote: add_all(["*"])
alone does not stage a removal on this libgit2, so the tree came back unchanged
and the push was a silent no-op. stage_and_commit now runs add_all then
update_all(["*"]) (git add -u) — together git add -A. The :delete snackbar now
confirms the scoped file and flags that a Tracked file is local until :sync.
2026-07-12 00:44:39 +02:00
Julien Calixte
e967773bd6 feat(editor): add v0.5 file palette (Cmd-P) with fuzzy match
Cmd-P opens a modal transient palette over the writing column: a bare
fuzzy-search input, the ranked file list, and the selected row in reverse
video. A pure host-testable fuzzy_score (subsequence + word-boundary and
consecutive-run bonuses) ranks results; an in-core MRU floats recently
opened files first and is shared with :e (both route through open_path).
The host feeds the file list once at boot (enumerate_files over /sd/repo
and /sd/local). Ctrl-n/Ctrl-p navigate the list; Enter opens via the same
park/evict path as :e; Esc closes.

Ctrl-n/Ctrl-p also become down/up line motions in Normal and View (vim
CTRL-N/CTRL-P, count-aware), which is why the palette opener is Cmd-P
alone. No `>` prefix on the file input — `>` is reserved for the command
palette (slice 4).

112 editor + 28 keymap tests; the no-git firmware binary builds clean.
2026-07-12 00:11:33 +02:00
Julien Calixte
2215da939d feat: add multi-file buffer foundation (v0.5 slice 1)
Rework the single Effect return into a drained effect queue
(Save{path,scope,contents} / Load / Publish / Pull) so one action can
ask the host for several ordered steps: opening a non-resident file
queues a Save of the outgoing dirty buffer then a Load of the target.

Keep the active buffer's fields inline on Editor and park inactive
buffers in a small LRU (<=3 resident = active + 2); switching back to a
resident buffer restores its caret/scroll/undo without touching the SD.
A dirty parked buffer is saved before eviction, so nothing leaves RAM
unsaved. `:e <path>` opens by prefix (/sd/repo -> Tracked, /sd/local ->
Local); `:sync` is refused in-core for a Local buffer.

Firmware drains the queue to empty each batch (a Load can cascade an
eviction Save) and persistence generalises the atomic save off the
hard-coded notes.md into load_path/save_path.

Also bump the side panel to FONT_9X15 and the `:` command line to
FONT_10X20 for legibility, word-wrapping the snackbar so a long notice
keeps its actionable tail.
2026-07-11 22:26:37 +02:00
Julien Calixte
fa0ea56e1a feat(editor): add Visual mode (v/V) with y/d/c, move View to gr
Charwise `v` and linewise `V` selection with yank/delete/change on the
span; motions and counts extend it. Read-only View moves off v/V to
`gr` (go-read). Selection renders reverse-video on the 1-bit panel.
Normal motions factored into a shared move_by. Firmware -> 0.4.0.
2026-07-11 20:50:16 +02:00
Julien Calixte
470a9d25d0 docs(macroplan): note v0.3 on-device smoke-test and paste-scroll fix 2026-07-11 20:26:15 +02:00
Julien Calixte
937868dd85 fix(editor): reveal the pasted block when it runs past the fold
A multi-line paste near the bottom of the screen left its later lines below
the viewport — adjust_scroll only kept the caret's (first) pasted line
visible. Add reveal(), which scrolls the end of the pasted block into view
while the caret stays on its first line (vim semantics unchanged).
2026-07-11 20:26:15 +02:00
Julien Calixte
884c8f2e48 docs(postmortems): record the SPI-DMA OOM editor freeze
Root cause, shipped safety net, and the specced eradication (a
persistent internal DMA scratch buffer in Epd so paints never allocate
mid-sync). Tracks hardware re-test and the eradication as follow-ups.
2026-07-11 20:21:49 +02:00
Julien Calixte
32b9bc6a15 fix(firmware): keep the editor alive when a panel paint fails
A screen refresh that ran while :sync had Wi-Fi + TLS up could fail to
allocate an internal DMA bounce buffer (ESP_ERR_NO_MEM); the error
propagated through ? out of main(), main_task returned from app_main(),
and the editor loop died while the USB and git threads kept running —
keys logged, panel frozen forever.

A paint is idempotent and retryable (the buffer is the source of truth),
so every editor-loop paint now logs and drops the frame instead of
propagating, leaves shown untouched so the next paint repaints the diff,
and forces a full refresh next to re-sync both RAM banks. Same contract
as save_note. The boot-time first render stays fatal on purpose.
2026-07-11 20:21:49 +02:00
Julien Calixte
f8ef9c821c Merge remote-tracking branch 'origin/main' 2026-07-11 20:04:37 +02:00
Julien Calixte
f950abdc4a docs(macroplan): mark v0.3 editing complete
Records v0.3 as delivered in core (host-tested, on-device smoke-test pending)
and notes the known dot-repeat limits. Also reorders the plan so the Status
block follows the macroplan source.
2026-07-11 20:03:59 +02:00
Julien Calixte
137a51eb76 chore(firmware): bump version to 0.3.0
v0.3 editing (register/yank/paste, undo/redo, dot-repeat) is complete, so the
device release version tracks the shipped feature set.
2026-07-11 20:03:59 +02:00
Julien Calixte
ba8f4e9a92 feat(editor): add v0.3 editing — register, undo/redo, and dot-repeat
Yank/paste via one unnamed register: `y` operator (yy/yw/yiw/y$), `p`/`P`,
with `x`/`d`/`c` also filling it so `dd`…`p` moves a line. Undo/redo (`u`,
`Ctrl-r`) as bounded full-buffer snapshots — one Insert session is one undo
group. `.` replays the last change from its recorded keystrokes, so it
repeats insert sessions like `ciwfoo<Esc>`.
2026-07-11 20:03:52 +02:00
Julien Calixte
8e4e3a7586 feat(keymap): add Ctrl-r redo intent
Decodes Ctrl+R (HID 0x15) to a new Key::Redo, the inverse of the editor's
`u`. Meaningful in Normal mode; ignored elsewhere.
2026-07-11 20:03:42 +02:00
657aba3cf3 Updating docs/qfd.md from Remanso 2026-07-11 18:46:27 +01:00
Julien Calixte
229c259e7c docs: mark v0.2 complete and align delivered-release bookkeeping
Spike 13 (line-number gutter) verified on the panel closes the last v0.2
gate. Record v0.2 as delivered, and give v0.2.5 a delivered date + learning
note to match v0.1/v0.2 (its source-block entry still read on-track).
2026-07-11 19:41:17 +02:00
Julien Calixte
f5a0b45f6f feat(editor): format the buffer on save/sync (format_on_save)
Run :fmt in-core before :w/:sync emit their effect, so :sync is
fmt -> save -> commit -> push and :w saves formatted. Gated on the
format_on_save field (default on); the v0.5 .typoena.toml key will drive it.
2026-07-11 19:32:16 +02:00
Julien Calixte
cb3160541d feat(editor,firmware): add :gl fast-forward pull command
Add Effect::Pull and the `:gl` command (fetch + fast-forward only, refuse on
divergence). The editor side is host-tested; the firmware arm is a stub posting
"pull: not wired yet (v0.7)" — the on-device fetch/fast-forward in git_sync is
v0.7 work (only push is wired today).
2026-07-11 19:32:02 +02:00
Julien Calixte
1d7448ba75 feat(editor): edit the : command line with Ctrl-W / Cmd-Backspace
Handle DeleteWord (Ctrl-W) and DeleteLine (Cmd-Backspace) in Command mode:
Ctrl-W drops the previous word, Cmd-Backspace clears the line. Neither exits to
Normal on empty (unlike Backspace), so command editing stays on the line.
2026-07-11 19:30:39 +02:00
Julien Calixte
5d9591e5ea docs: rename roadmap.md to macroplan.md and refresh the plan
The file is the macroplan (plus per-version scope), so rename it to match and
retitle to "Macroplan"; update all inbound links and friendly labels across the
docs. Refresh the plan while here: v0.2 gutter built, :gl pull recorded (v0.7),
command-line editing (v0.4), and the format_on_save pref (v0.5).
2026-07-11 19:27:27 +02:00
Julien Calixte
97216db0c0 docs: record line-number gutter and v0.5 line_numbers pref
Mark the v0.2 gutter built + host-tested (Spike 13 bench check still pending),
add the v0.5 .typoena.toml line_numbers toggle, and update CONTEXT.md screen
regions for the rebalanced 63-col writing region / 25-col side panel.
2026-07-11 18:59:52 +02:00
Julien Calixte
8b30658275 feat(editor): absolute line-number gutter
Reserve a left gutter for absolute line numbers, drawn on each logical line's
first display row and blank on wrapped continuation rows. Gutter width tracks
the buffer's line count (2 digits + separator, widening past 99 lines) and
steals its columns from the soft-wrap. Widen the writing region 60->63 cols so
the gutter doesn't narrow the text: a file up to 99 lines keeps a full 60-col
text column, at the cost of the side panel shrinking 30->25 cols.
2026-07-11 18:59:46 +02:00
Julien Calixte
54dc7a31d3 Merge remote-tracking branch 'origin/main' 2026-07-11 18:34:05 +02:00
Julien Calixte
fc83306a81 feat(keymap,editor): add Ctrl-d/u half-page scroll
Decode Ctrl+D/Ctrl+U as HalfPageDown/HalfPageUp intents (matching the existing
DeleteWord/DeleteLine chord pattern, so the editor stays ignorant of Ctrl).
They step display (soft-wrapped) rows, not logical lines, so half a page is
half the visible window regardless of how prose wraps: Normal moves the caret
and the viewport follows, View scrolls the viewport, Insert/Command are no-ops.
2026-07-11 18:32:05 +02:00
Julien Calixte
491dc57144 docs: drop relative line numbering from the v0.2 gutter
Relative numbering renumbers the whole gutter on every j/k, a tall partial
refresh per cursor move that eats the e-ink ghosting budget for no
proportionate gain. Gutter is now absolute-only; Spike 13 downgrades from a
genuine e-ink risk to a layout decision.
2026-07-11 18:31:33 +02:00
9f797d814c Updating docs/qfd.md from Remanso 2026-07-11 17:00:42 +01:00
49 changed files with 8967 additions and 802 deletions

View File

@@ -15,7 +15,7 @@ by user-facing weight. References the terms in this file as canonical.
surface, expressed in this vocabulary.
[`docs/v0.1-mvp-technical.md`](docs/v0.1-mvp-technical.md) — how v0.1 is
built.
[`docs/roadmap.md`](docs/roadmap.md) — per-version scope, where new terms
[`docs/macroplan.md`](docs/macroplan.md) — per-version scope, where new terms
(e.g. multi-file **Buffer** concepts at v0.5) will enter this glossary.
## Language
@@ -34,6 +34,25 @@ stay Local for their lifetime. Lives under `/sd/local/`.
_Avoid_: draft, private, untracked, scratch (these all imply impermanence or
promotability, which is not the model).
### Editing model
**Buffer**:
A **File** loaded into memory for editing, with its own caret, scroll position,
and undo history. Opening a file makes it the **active buffer** — the one the
**Writing column** shows. Up to three buffers stay **resident** at once (the
active one plus two parked in the background); switching back to a resident
buffer restores its caret and undo without re-reading the card. A fourth open
**evicts** the least-recently-used resident buffer — saved first if it has
unsaved edits, so nothing is lost.
_Avoid_: tab, window, document (a buffer is not a UI chrome element); "the file"
when you mean the in-memory copy rather than the bytes on the card.
**Open**:
Bringing a **File** into the **active buffer**, via `Cmd-P` (the file palette)
or `:e`. Scope is read from where the file lives (`/sd/repo`**Tracked**,
`/sd/local`**Local**), never chosen at open time.
_Avoid_: load (implementation talk for the disk read behind an Open).
### User-facing actions
**Save**:
@@ -60,13 +79,16 @@ into user-facing language).
**Writing column**:
The left region of the panel showing the text being edited — the _only_ region
that repaints per keystroke. ~60 columns, full panel height; straddles the
driver's `x = 396` seam invisibly.
that repaints per keystroke. A 63-col region split into a **line-number gutter**
(absolute numbers, 24 cols wide, sized to the buffer's line count) and the text
column it steals from (~60 cols for a file ≤ 99 lines). Full panel height;
straddles the driver's `x = 396` seam invisibly.
_Avoid_: edit area, text area, main pane (superseded — they named the old
full-width text region before the side panel carved out its right edge).
**Side panel**:
The right region (~150 px / ~20 cols, full height) holding all metadata:
The right region (~160 px / ~17 cols at its FONT_9X15 metadata font, full
height) holding all metadata:
filename + dirty dot, word count, elapsed time, clock, Wi-Fi,
keyboard-disconnect flag, publish state, and the mode indicator at its
bottom-left. Sits entirely in the master half

View File

@@ -9,7 +9,7 @@ runs on it.
> verified in spikes; SD mount and save are now wired into the app binary. No
> release has shipped yet — v0.1's remaining gate is the boot splash and wiring
> git publish (`Ctrl-G` → push) into the app binary. Live per-item status:
> [`docs/roadmap.md`](docs/roadmap.md) · failure write-ups:
> [`docs/macroplan.md`](docs/macroplan.md) · failure write-ups:
> [`docs/postmortems/`](docs/postmortems/README.md).
---
@@ -69,7 +69,7 @@ surface (mostly `usb_kbd.rs`) is in [`MEMORY_AUDIT.md`](MEMORY_AUDIT.md).
| HAL / runtime | `esp-idf-svc`, `esp-idf-hal` | std build: heap, threads, VFS, mbedtls, Wi-Fi stack. |
| Display | Custom SSD1683 driver (`src/epd.rs`) + `embedded-graphics` | Dual-controller 792×272 panel; dirty-rect partial refresh (~630 ms measured). |
| UI layer | Custom thin widget layer | Ratatui's API _shape_ without its char-grid terminal model ([ADR-002](docs/adr.md#adr-002-ui-strategy--custom-widgets-on-embedded-graphics-not-ratatui)). |
| Editor core | Custom, in-tree (`src/editor.rs`) | Modal (Normal / Insert / View / Command), motions, operators + text objects. Plain-ASCII buffer until the v0.2 UTF-8 work. |
| Editor core | Custom, in-tree (`src/editor.rs`) | Modal (Normal / Insert / Visual / VisualLine / View / Command), motions, operators + text objects. Plain-ASCII buffer until the v0.2 UTF-8 work. |
| USB host | `esp-idf` TinyUSB bindings | Boot-protocol HID; verified on hardware (Spike 4). |
| Git | **libgit2 via `git2`**, built as an esp-idf component with mbedTLS (`firmware/components/libgit2/`) | `gix` was the original pick but can't push over HTTPS — the [ADR-004](docs/adr.md#adr-004-git-implementation--gitoxide-gix) kill-switch fired ([postmortem](docs/postmortems/2026-07-05-spike7-gix-https-push.md)). On-device add → commit → push verified; ~16 s cold-`:sync` [latency breakdown](docs/notes/sync-latency.md). |
| TLS | `mbedtls` via `esp-idf` | GitHub HTTPS with the chain checked against embedded roots; ≈35 KB heap measured during handshake (Spike 6). |
@@ -98,22 +98,22 @@ around, not against:
Releases are frequent, and every version is a usable artifact rather than a
checkpoint. Per-version scope, current `[x]`/`[~]` marks, and the Macroplan
source live in [`docs/roadmap.md`](docs/roadmap.md).
source live in [`docs/macroplan.md`](docs/macroplan.md).
| Version | Theme | One-liner |
| ------------------------------------------------------- | ------------ | -------------------------------------------- |
| [v0.1](docs/roadmap.md#v01--mvp-it-writes-it-pushes--) | MVP | Boots, edits one file, `Ctrl-G` pushes. |
| [v0.2](docs/roadmap.md#v02--vim-navigation--) | Vim nav | Normal/Insert, motions, line numbers. |
| [v0.2.5](docs/roadmap.md#v025--international-input--) | Intl input | US-Intl dead keys: à é ê ç, `'`+space = `'`. |
| [v0.3](docs/roadmap.md#v03--vim-editing--) | Vim edit | `dd yy p`, undo/redo, counts. |
| [v0.4](docs/roadmap.md#v04--visual-mode--ex-commands--) | Visual + ex | `v V`, `:w :q :e` command line. |
| [v0.5](docs/roadmap.md#v05--file-palette--multi-file--) | Files | `Ctrl-P` over `/repo` + `/local`, buffers. |
| [v0.6](docs/roadmap.md#v06--markdown-affordances--) | Markdown | Headings, list continuation, soft-wrap. |
| [v0.7](docs/roadmap.md#v07--search--better-git--) | Search + git | `/` search, `:Gpull`. |
| [v0.8](docs/roadmap.md#v08--power-battery--sleep--) | Power | 18650 + sleep + lid switch. |
| [v0.9](docs/roadmap.md#v09--robustness--) | Robustness | Crash-safe writes, reconnect, settings. |
| [v1.0](docs/roadmap.md#v10--polish--) | Polish | Boot ≤ 3 s, fonts, themes, enclosure, guide. |
| [v1.x](docs/roadmap.md#v1x--stretch--nice-to-have) | Stretch | 10.3″ panel, multi-remote, stats, BLE. |
| [v0.1](docs/macroplan.md#v01--mvp-it-writes-it-pushes--) | MVP | Boots, edits one file, `Ctrl-G` pushes. |
| [v0.2](docs/macroplan.md#v02--vim-navigation--) | Vim nav | Normal/Insert, motions, line numbers. |
| [v0.2.5](docs/macroplan.md#v025--international-input--) | Intl input | US-Intl dead keys: à é ê ç, `'`+space = `'`. |
| [v0.3](docs/macroplan.md#v03--vim-editing--) | Vim edit | `dd yy p`, undo/redo, counts. |
| [v0.4](docs/macroplan.md#v04--visual-mode--ex-commands--) | Visual + ex | `v`/`V` select + `y d c`, `gr` read, `:` ex. |
| [v0.5](docs/macroplan.md#v05--file-palette--multi-file--) | Files | `Ctrl-P` over `/repo` + `/local`, buffers. |
| [v0.6](docs/macroplan.md#v06--markdown-affordances--) | Markdown | Headings, list continuation, soft-wrap. |
| [v0.7](docs/macroplan.md#v07--search--better-git--) | Search + git | `/` search, `:Gpull`. |
| [v0.8](docs/macroplan.md#v08--power-battery--sleep--) | Power | 18650 + sleep + lid switch. |
| [v0.9](docs/macroplan.md#v09--robustness--) | Robustness | Crash-safe writes, reconnect, settings. |
| [v1.0](docs/macroplan.md#v10--polish--) | Polish | Boot ≤ 3 s, fonts, themes, enclosure, guide. |
| [v1.x](docs/macroplan.md#v1x--stretch--nice-to-have) | Stretch | 10.3″ panel, multi-remote, stats, BLE. |
---

341
display/src/glyphs.rs Normal file
View File

@@ -0,0 +1,341 @@
//! Extra glyphs the ISO-8859-15 render font lacks.
//!
//! The buffer is UTF-8, so it can hold codepoints outside Latin-9. The
//! `embedded-graphics` `iso_8859_15` font draws those as a fallback box. This
//! module carries hand-authored 10×20 bitmaps for a curated set of common
//! single-width characters — typographic punctuation that arrives in imported
//! prose (curly quotes, en/em dashes, ellipsis, bullet) and the math symbols the
//! user's Markdown snippets insert (→ ≠ Σ). [`Editor::draw`] overlays these over
//! the fallback box after the font has laid out the line.
//!
//! Every glyph is exactly one 10×20 cell (matching the body `FONT_10X20`, i.e.
//! `editor::CW`×`editor::CH`), so the editor's 1-char = 1-cell layout invariant
//! is untouched. A row is a 10-bit pattern: the leftmost column (x = 0) is the
//! high bit, so a binary literal reads left-to-right as the pixels of that row.
//! Rows are top (y = 0) to bottom (y = 19); 1 = ink.
use crate::Frame;
use embedded_graphics::pixelcolor::BinaryColor;
use embedded_graphics::prelude::*;
const GLYPH_W: usize = 10;
const GLYPH_H: usize = 20;
/// A 10×20 1-bit glyph: one `u16` per row, low `GLYPH_W` bits used, high bit =
/// leftmost pixel.
pub type Glyph = [u16; GLYPH_H];
// → U+2192 RIGHTWARDS ARROW: a horizontal shaft with a `>` head at the right.
#[rustfmt::skip]
const ARROW_RIGHT: Glyph = [
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000010000,
0b0000001000,
0b0000000100,
0b0000000010,
0b0111111111,
0b0111111111,
0b0000000010,
0b0000000100,
0b0000001000,
0b0000010000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
];
// ≠ U+2260 NOT EQUAL TO: two equals bars with a slash through them.
#[rustfmt::skip]
const NOT_EQUAL: Glyph = [
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000010,
0b0000000100,
0b0000000100,
0b0000001000,
0b0111111110,
0b0000010000,
0b0000010000,
0b0000100000,
0b0111111110,
0b0001000000,
0b0001000000,
0b0010000000,
0b0010000000,
0b0000000000,
0b0000000000,
0b0000000000,
];
// Σ U+03A3 GREEK CAPITAL LETTER SIGMA: top & bottom bars with the diagonals
// meeting at a vertex on the centre-right.
#[rustfmt::skip]
const SIGMA: Glyph = [
0b0000000000,
0b0000000000,
0b0000000000,
0b0111111110,
0b0100000000,
0b0010000000,
0b0001000000,
0b0000100000,
0b0000010000,
0b0000100000,
0b0001000000,
0b0010000000,
0b0100000000,
0b0100000000,
0b0111111110,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
];
// • U+2022 BULLET: a filled dot at mid-height.
#[rustfmt::skip]
const BULLET: Glyph = [
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0001111000,
0b0011111100,
0b0011111100,
0b0011111100,
0b0001111000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
];
// … U+2026 HORIZONTAL ELLIPSIS: three dots on the baseline.
#[rustfmt::skip]
const ELLIPSIS: Glyph = [
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0110110110,
0b0110110110,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
];
// U+2013 EN DASH: a short mid-height bar.
#[rustfmt::skip]
const EN_DASH: Glyph = [
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0011111100,
0b0011111100,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
];
// — U+2014 EM DASH: a full-width mid-height bar.
#[rustfmt::skip]
const EM_DASH: Glyph = [
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b1111111111,
0b1111111111,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
];
// ' U+2018 LEFT SINGLE QUOTATION MARK: raised, tail up (opening).
#[rustfmt::skip]
const LEFT_SINGLE_QUOTE: Glyph = [
0b0000000000,
0b0000000000,
0b0000100000,
0b0001100000,
0b0001100000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
];
// ' U+2019 RIGHT SINGLE QUOTATION MARK / apostrophe: raised, tail down (closing).
#[rustfmt::skip]
const RIGHT_SINGLE_QUOTE: Glyph = [
0b0000000000,
0b0000000000,
0b0001100000,
0b0001100000,
0b0000100000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
];
// " U+201C LEFT DOUBLE QUOTATION MARK: two opening quotes.
#[rustfmt::skip]
const LEFT_DOUBLE_QUOTE: Glyph = [
0b0000000000,
0b0000000000,
0b0010010000,
0b0011011000,
0b0011011000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
];
// " U+201D RIGHT DOUBLE QUOTATION MARK: two closing quotes.
#[rustfmt::skip]
const RIGHT_DOUBLE_QUOTE: Glyph = [
0b0000000000,
0b0000000000,
0b0011011000,
0b0011011000,
0b0010010000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
0b0000000000,
];
/// The 10×20 bitmap for a character the ISO-8859-15 font can't draw, or `None`
/// if the base font already covers it (all ASCII and Latin-9, including `œ €`).
pub fn extra_glyph(c: char) -> Option<&'static Glyph> {
Some(match c {
'\u{2192}' => &ARROW_RIGHT, // →
'\u{2260}' => &NOT_EQUAL, // ≠
'\u{03A3}' => &SIGMA, // Σ
'\u{2022}' => &BULLET, // •
'\u{2026}' => &ELLIPSIS, // …
'\u{2013}' => &EN_DASH, //
'\u{2014}' => &EM_DASH, // —
'\u{2018}' => &LEFT_SINGLE_QUOTE, // '
'\u{2019}' => &RIGHT_SINGLE_QUOTE, // '
'\u{201C}' => &LEFT_DOUBLE_QUOTE, // "
'\u{201D}' => &RIGHT_DOUBLE_QUOTE, // "
_ => return None,
})
}
/// Paint a glyph over one 10×20 cell at `(x, y)`. Fills the whole cell — set
/// bits in `ink`, unset bits in `ink.invert()` — so it fully overwrites whatever
/// the base font drew there (e.g. the fallback box). Pass `ink = On` for a
/// normal black-on-white cell, `ink = Off` for a reverse-video cell (a white
/// glyph over the black selection/caret fill).
pub fn blit_glyph(f: &mut Frame, x: i32, y: i32, g: &Glyph, ink: BinaryColor) {
let bg = ink.invert();
let pixels = g.iter().enumerate().flat_map(move |(row, &bits)| {
(0..GLYPH_W).map(move |c| {
let on = (bits >> (GLYPH_W - 1 - c)) & 1 == 1;
Pixel(
Point::new(x + c as i32, y + row as i32),
if on { ink } else { bg },
)
})
});
// Frame's DrawTarget error is Infallible.
let _ = f.draw_iter(pixels);
}

View File

@@ -13,6 +13,9 @@ use embedded_graphics::prelude::*;
use embedded_graphics::primitives::{Circle, PrimitiveStyleBuilder};
use embedded_graphics::text::{Alignment, Baseline, Text, TextStyleBuilder};
mod glyphs;
pub use glyphs::{blit_glyph, extra_glyph, Glyph};
pub const WIDTH: u16 = 792;
pub const HEIGHT: u16 = 272;
@@ -76,6 +79,16 @@ impl Frame {
pub fn bytes(&self) -> &[u8] {
&self.buf
}
/// Flip every pixel black↔white across the whole framebuffer. The editor
/// draws its native black-ink-on-white-paper frame, then calls this once at
/// the end for the dark theme — so text, selection, caret, panel and palette
/// all invert together and each stays legible against the flipped ground.
pub fn invert(&mut self) {
for b in &mut self.buf {
*b = !*b;
}
}
}
impl OriginDimensions for Frame {

View File

@@ -14,7 +14,8 @@
| [`adr.md`](adr.md) | Architecture Decision Records — the load-bearing technical choices and why. |
| [`v0.1-mvp-product.md`](v0.1-mvp-product.md) | v0.1 product design — boot, type one file, `Ctrl-S` to save, `Ctrl-G` to publish. |
| [`v0.1-mvp-technical.md`](v0.1-mvp-technical.md) | v0.1 technical design — single Rust binary on `esp-idf-rs`, modules, threads, bring-up order. |
| [`roadmap.md`](roadmap.md) | Version-by-version plan; each release is a usable artifact, not a checkpoint. |
| [`macroplan.md`](macroplan.md) | Version-by-version plan; each release is a usable artifact, not a checkpoint. |
| [`typoena-toml.md`](typoena-toml.md) | `.typoena.toml` reference — the git-tracked editor preferences (auto-save, format-on-save, line numbers, auto-sync). |
| [`hardware.md`](hardware.md) | Part choices for the bench build and the rationale behind them. |
## Quality method

View File

@@ -12,7 +12,7 @@ Format inspired by Michael Nygard's ADR template, kept short on purpose.
[`../CONTEXT.md`](../CONTEXT.md) — project glossary: **Tracked**, **Local**,
**Save**, **Publish**, plus the principles ("writing tool, not sync engine")
that constrain [ADR-010] specifically.
[`roadmap.md`](roadmap.md) — per-version scope (v0.1 → v1.x).
[`macroplan.md`](macroplan.md) — per-version scope (v0.1 → v1.x).
[`v0.1-mvp-product.md`](v0.1-mvp-product.md) — what the v0.1 device must do.
[`v0.1-mvp-technical.md`](v0.1-mvp-technical.md) — how v0.1 is built.
[`qfd.md`](qfd.md) — Quality Function Deployment: requirements → functions →
@@ -180,7 +180,7 @@ as the price of those properties.
do not promise "instant feedback."
- Idle power on e-ink is structurally ~0, which makes the v0.8 battery
sizing exercise straightforward — see [ADR-008] and
[roadmap → v0.8](roadmap.md#v08--power-battery--sleep--).
[macroplan → v0.8](macroplan.md#v08--power-battery--sleep--).
- 10.3" e-ink upgrade path is preserved by keeping the renderer
resolution-agnostic. A _non_-e-ink swap (e.g. Sharp Memory LCD) would
invalidate [ADR-002]'s dirty-rect strategy and force a fresh medium ADR.
@@ -277,7 +277,7 @@ card alone is not enough.
[v0.1 product → provisioning](v0.1-mvp-product.md#provisioning-build-time-dev-only).
- PAT is never logged. Validated in code review.
- Rotation in v0.1 = wipe NVS and re-run setup. Proper rotation UI is v0.9
— see [roadmap → v0.9](roadmap.md#v09--robustness--).
— see [macroplan → v0.9](macroplan.md#v09--robustness--).
- Revisit if we ever want to support multiple remotes per device with
different credentials.
@@ -408,7 +408,7 @@ Sizing a battery before measuring is guessing.
scope is in [v0.1 product → out of scope](v0.1-mvp-product.md#out-of-scope-for-v01).
- We can decide cell capacity from real numbers in v0.8, not specs sheets.
- Lid-close detection / deep sleep slips to v0.8 with the battery — see
[roadmap → v0.8](roadmap.md#v08--power-battery--sleep--).
[macroplan → v0.8](macroplan.md#v08--power-battery--sleep--).
---

228
docs/macroplan.md Normal file
View File

@@ -0,0 +1,228 @@
# Macroplan — version details
Frequent releases. Each version is a usable artifact, not a checkpoint.
This file holds the `macroplan` source block (below), a cross-version status
roll-up, and a one-line summary of each release linking to its dedicated page.
The user-facing requirements and engineering targets each release feeds into are
tracked in [`qfd.md`](qfd.md).
## Macro-plan
Macroplan source — paste into the macroplan app to render the week-by-week
view. `original` dates are the June 2026 baseline and never move; slips get
appended as `reestimates`, per-item actuals live in the Status block below.
```macroplan
title = "Typoena — macro plan"
[[feature]]
name = "v0.1 it writes, it pushes"
start = 2026-06-01
original = 2026-06-29
delivered = 2026-07-11
learning = "Shipped 12 days late. The long pole was hardware bring-up risk, not the editor: SD on a shared SPI bus (resolved by moving it to its own SPI3, ADR-012) and on-device git (gix killed, pivoted to libgit2 as an esp-idf CMake component, ADR-004). Splash landed as a vector wordmark, not the planned 1-bit bitmap — the asset-embed/blit path is deferred to v1.0."
[[feature]]
name = "v0.2 navigation"
start = 2026-06-29
original = 2026-07-20
delivered = 2026-07-11
learning = "Delivered 9 days early. Motions/modes, Ctrl-d/u, the UTF-8 buffer, and the absolute line-number gutter all landed 2026-07-11; the last gate, Spike 13's on-panel gutter refresh check, confirmed a single-line edit repaints only rows at/below it with no extra full refresh. Relative line numbering was dropped as an e-ink ghosting cost with no proportionate gain."
[[feature]]
name = "v0.2.5 international input"
start = 2026-07-20
original = 2026-08-03
delivered = 2026-07-11
learning = "Delivered 23 days early — ahead of its own start window. Dead-key accent composer in the keymap crate (US-International, à é ê ë ñ ç), editor buffer made UTF-8-correct, typed on the bench with no panic. The side-panel pending-accent marker was dropped by decision: at typing speed it is stale before the ~630 ms panel repaint, so it conveyed nothing. Bonus: physical Esc (HID 0x29) remapped to backtick/tilde so code fences + grave/tilde accents work on a 60% board without a Fn layer."
[[feature]]
name = "v0.3 editing"
start = 2026-08-03
original = 2026-08-24
delivered = 2026-07-11
learning = "Core complete 44 days early, host-tested and partially smoke-tested on the panel. Register + yank/paste (yy/p/P), snapshot undo/redo (u/Ctrl-r, bounded 100 groups in PSRAM), and keystroke-recorded `.` repeat all landed 2026-07-11; the d/c operator grammar + text objects were already done ahead of schedule. Firmware bumped to 0.3.0. On device dd/yy/Ctrl-r confirmed; the one bug found was a multi-line paste leaving its later lines below the fold (adjust_scroll only tracked the caret) — fixed with a reveal() that scrolls the block end into view."
[[feature]]
name = "v0.4 visual + ex"
start = 2026-08-24
original = 2026-09-07
delivered = 2026-07-11
learning = "Core complete 58 days early, host-tested. Visual (v) and VisualLine (V) selection with y/d/c landed 2026-07-11 (charwise vim-inclusive of the char under the caret; linewise spans whole lines and pastes like yy/dd), plus the recorded v/V→Visual reassignment: the read-only View mode moved to `gr` (go-read). Selection is drawn as reverse-video cells on the 1-bit panel with the caret punched back to normal video so the active end stands out; 18 new editor tests (83 total). The `:` command mechanism and :fmt were already done; `:e <path>` was deliberately deferred to v0.5 where its multi-file/buffer-lifecycle machinery (Spikes 11/14) lives, rather than half-building file-open here. Firmware bumped to 0.4.0. On-device smoke-test of Visual still pending (pure editor-core, low risk)."
[[feature]]
name = "v0.5 palette + multi-file"
start = 2026-09-07
original = 2026-09-28
delivered = 2026-07-12
learning = "Delivered 2026-07-12, well ahead of the 2026-09-28 baseline, and fully on-device confirmed. Four slices: the drained Effect queue + parked-buffer LRU foundation; the Cmd-P fuzzy file palette (Spike 11 — no ghosting on the transient panel); :enew + file delete (Spike 14 caught that add_all alone doesn't stage a deletion on this libgit2 — fixed with update_all, i.e. git add -A); and the git-tracked .typoena.toml prefs with a stay-open palette `>` command mode + :settings. Both directions of the prefs loop are proven on hardware — boot-read (byte-exact parse) and on-device palette edit (a device publish flipped line_numbers on origin). Three decide-before-build calls: the idle auto-save is unformatted, and both the per-device auto_sync override and the `> auto sync` command are deferred to v0.7 where auto_sync gains behaviour. Amended 2026-07-12: a light/dark `theme` key and a set-ahead `> auto sync` preset command (2m/5m/10m/15m/30m) were added on top — the palette generalised so Enter rotates any pref to its next value (a bool is the two-option case); auto_sync is still read by nothing until v0.7. Descoped from v0.5 (not the four slices): explicit buffer close, the grey-Publish-in-Local panel cue, and the multi-file publish count."
[[feature]]
name = "v0.6 markdown"
start = 2026-09-28
original = 2026-10-12
delivered = 2026-07-12
learning = "Core complete 2026-07-12, ~92 days ahead of the 2026-10-12 baseline, host-tested (187 editor tests). The snippet feature was reshaped 2026-07-08→07-12 from a hard-coded table into a git-synced, Zed-compatible .typoena.snippets.json library: a forward-only tab-stop session ($1..$n/$0, ${n:label} stripped to $n) driven by two surfaces — inline Tab-expansion in Insert and a $ palette launcher — plus a quiet pause hint in the side panel. The Cmd-P palette generalised into a verb split: bare = files, > = a real command registry (toggles stay open, one-shots format/publish close, the parameterised `new file` two-step), $ = snippets — retiring :e. Firmware bumped 0.5.0→0.6.0; the boot-read of the library was confirmed to build for xtensa (serde_json, the one new dep — cargo check passes). `just init` now seeds a curated 17-snippet catalog (three opt-in groups). On-device smoke-test still pending (pure editor-core + a mirror of the proven prefs boot-read, low risk). Known caveat: two symbols the catalog inserts (arrow →, neq ≠) are outside ISO-8859-15, so they store/sync correctly but need a display-layer glyph overlay (in flight) to draw on the panel; the other 15 render on the stock font."
[[feature]]
name = "v0.7 search + git"
start = 2026-10-12
original = 2026-11-02
[[feature]]
name = "v0.8 battery + sleep"
start = 2026-11-02
original = 2026-11-30
[[feature]]
name = "v0.9 robustness"
start = 2026-11-30
original = 2026-12-28
[[feature]]
name = "v1.0 polish"
start = 2026-12-28
original = 2027-01-25
[[milestone]]
name = "MVP ships"
week = 2026-06-29
requires = ["v0.1 it writes, it pushes"]
```
## Status — synced 2026-07-12
The editor **core** has been built 23 versions ahead of the device
**releases**, and is now **extracted into a host-testable `editor` crate** (plus
a `display` crate for the panel framebuffer) so `cargo test` exercises it off the
xtensa target. **v0.1 shipped 2026-07-11** (late against the 2026-06-29
baseline): SD storage, save, and **git publish are all wired into the app binary
and hardware-verified** (`:sync` commits on the SD `/sd/repo` and pushes to a
test repo), and the **boot splash (Spike 9) is confirmed on the panel** — a
vector `typoena`-in-a-circle shown at startup while the SD mounts, then the
editor comes up. **Cold boot verified at 4258 ms** (power-on → cursor,
2026-07-11; 742 ms under the ≤ 5 s gate). It first measured ~5.5 s; the fix was
to bring the editor up with a full-area partial (~630 ms) instead of a second
full refresh (~1.9 s) — panel confirmed clean, no ghosting. The 1-hour soak is
attested from real use; the remaining post-ship acceptance checks are power-pull
recovery, 1000-word no-drop, and `Ctrl-G`'s not-yet-built pull-then-retry
(→ v0.9). **v0.2 navigation is COMPLETE 2026-07-11** — Spike 13's on-panel gutter
refresh check passed (single-line edit repaints only rows at/below it, no extra
full refresh), closing the last gate. **v0.2.5 international input** is
hardware-verified (2026-07-11), and **v0.3 editing is complete in core** the same
day (register + yank/paste, snapshot undo/redo, `.` repeat — host-tested, and
partially smoke-tested on the panel: `dd`/`yy`/`Ctrl-r` good, a multi-line-paste
scroll bug found + fixed). **v0.4 visual + ex is complete in core** the same day
too — charwise/linewise **Visual** selection (`v`/`V` with `y`/`d`/`c`), the
read-only View mode moved to `gr`, and the selection drawn as reverse-video on
the panel; `:e` was deferred to v0.5. Host-tested (83 editor tests); on-device
smoke-test pending. The firmware crate is bumped to **0.4.0**. Most of v0.6
Markdown also already runs. Version numbers track shippable device releases, not
raw core progress — the 0.4.0 bump reflects the v0.4 feature set being met.
**v0.5 palette + multi-file is DELIVERED 2026-07-12** (firmware **0.5.0**), fully
on-device confirmed: the Cmd-P fuzzy palette, `:e`/`:enew`/delete across the
`/sd/repo` + `/sd/local` scopes, and the git-tracked `.typoena.toml` prefs
(boot-read plus a stay-open palette `>` command mode + `:settings` that edits them
live and syncs the change). Descoped to later: explicit buffer close, the
grey-Publish-in-Local panel cue, and the multi-file publish count.
**v0.6 Markdown is COMPLETE in core 2026-07-12** (firmware **0.6.0**), host-tested
(187 editor tests), on-device smoke-test pending. The render affordances (heading
bold, list continuation, soft-wrap) were done early; the headline is the
**snippet engine** — a forward-only tab-stop session reached both inline (type a
prefix + Tab in Insert) and from a **`$` palette** launcher, fed by a git-synced,
Zed-compatible `.typoena.snippets.json` read at boot (serde_json, confirmed to
build for xtensa). The `Cmd-P` palette **generalised** into a verb split — bare =
files, `>` = a command registry (toggles stay open, `format`/`publish` one-shots
close, a two-step `new file`), `$` = snippets — which **retired `:e`**. `just init`
seeds a curated 17-snippet catalog (Symbols · Structure · Prose, opt-in). One
caveat: `→`/`≠` sit outside ISO-8859-15 and need a display-layer glyph overlay
(in flight) to render on the panel; the other 15 draw on the stock font.
Marks: `[x]` done in core · `[~]` partially done · `[ ]` not started. An
inline `(✓)` marks the done half of a split item.
Each version below links to its dedicated page, which carries the full scope
checklist and status.
---
## v0.1 — MVP: "it writes, it pushes" — [x]
The minimum thing that justifies the hardware existing — boot, type one file,
`:w` to save, `:sync` to publish to GitHub. **SHIPPED 2026-07-11** (late vs the
2026-06-29 baseline); cold boot verified at 4258 ms.
**Design:** [product](v0.1-mvp-product.md) · [technical](v0.1-mvp-technical.md).
## v0.2 — Vim navigation — [x]
Modal Normal/Insert/View, `h j k l`/`w b e`/`0 $`/`gg G` motions, `Ctrl-d/u`
half-page scroll, the UTF-8-correct buffer, and the absolute line-number gutter.
**COMPLETE 2026-07-11.** Detail: [v0.2-navigation.md](v0.2-navigation.md).
## v0.2.5 — International input — [x]
US-International dead-key accent composition (à é ê ë ñ ç) in the `keymap`
crate, plus the Esc→backtick/tilde remap for a 60% board.
**Hardware-verified 2026-07-11.**
Detail: [v0.2.5-international-input.md](v0.2.5-international-input.md).
## v0.3 — Vim editing — [x]
Register + yank/paste (`yy`/`p`/`P`), snapshot undo/redo (`u`/`Ctrl-r`), `.`
repeat, and the `d`/`c` operator grammar + text objects.
**COMPLETE in core 2026-07-11**, partially smoke-tested on the panel.
Detail: [v0.3-editing.md](v0.3-editing.md).
## v0.4 — Visual mode + ex commands — [x]
Charwise `v` / linewise `V` selection with `y`/`d`/`c`, the `:` command line
(`:w`/`:fmt`/`:sync`/`:gl`), and View mode moved to `gr`.
**COMPLETE in core 2026-07-11**, on-device smoke-test pending.
Detail: [v0.4-visual-and-ex.md](v0.4-visual-and-ex.md).
## v0.5 — File palette + multi-file — [x]
The `Cmd-P` fuzzy file palette, `:e`/`:enew`/delete across `/sd/repo` +
`/sd/local`, the parked-buffer LRU, and the git-tracked `.typoena.toml` prefs
with a palette `>` command mode + `:settings`.
**DELIVERED 2026-07-12** (firmware 0.5.0), fully on-device confirmed.
Detail: [v0.5-palette-and-multi-file.md](v0.5-palette-and-multi-file.md).
## v0.6 — Markdown affordances — [x]
Heading bolding, list continuation, and soft-wrap, plus the trigger-driven
snippet engine (net-new scope, added 2026-07-08): a tab-stop session reached
inline (prefix + Tab) and from the `$` palette, fed by a git-synced,
Zed-compatible `.typoena.snippets.json`; the `Cmd-P` palette generalised into a
`files`/`>` commands/`$` snippets split (retiring `:e`); and a `just init`
catalog. **COMPLETE in core 2026-07-12** (firmware 0.6.0), on-device smoke-test
pending. Detail: [v0.6-markdown.md](v0.6-markdown.md).
## v0.7 — Search + better git — [~]
`/` forward search (`n`/`N`) and `:gl` pull (fetch + fast-forward only). The
`:gl` editor command landed 2026-07-11; the on-device fetch and search are still
to build. Detail: [v0.7-search-and-git.md](v0.7-search-and-git.md).
## v0.8 — Power: battery + sleep — [ ]
Bench current-draw measurement, 18650 + charge board, light/deep sleep, and a
battery indicator. **Not started.**
Detail: [v0.8-battery-and-sleep.md](v0.8-battery-and-sleep.md).
## v0.9 — Robustness — [ ]
Crash-safe writes, interrupted-push recovery, SD removal handling, Wi-Fi
reconnect, and on-device provisioning (the first release usable by a non-author).
**Not started.** Detail: [v0.9-robustness.md](v0.9-robustness.md).
## v1.0 — Polish — [ ]
≤ 3 s boot, runtime-switchable fonts, enclosure files, and a user guide
(light/dark theme landed early, in v0.5). **Not started.** Detail:
[v1.0-polish.md](v1.0-polish.md).
## v1.x — Stretch / nice-to-have
Post-1.0 ideas, not committed to any release (10.3" panel, multiple remotes,
writing stats, BLE-HID fallback). Detail: [v1.x-stretch.md](v1.x-stretch.md).

View File

@@ -9,7 +9,7 @@
> Notes index: [`README.md`](README.md). Docs index:
> [`../README.md`](../README.md). Backs the boot-time acceptance criterion in
> [`../v0.1-mvp-product.md`](../v0.1-mvp-product.md#acceptance-criteria) and the
> v1.0 goal in [`../roadmap.md`](../roadmap.md). Refresh cost model:
> v1.0 goal in [`../macroplan.md`](../macroplan.md). Refresh cost model:
> [`../tradeoff-curves/epd-refresh-latency.md`](../tradeoff-curves/epd-refresh-latency.md).
## The waterfall

View File

@@ -75,10 +75,12 @@ The big rocks are physics or protocol, not slack:
- **TLS handshake ~2.4 s** and **push negotiate/upload ~4.4 s** are inherent to
libgit2-over-mbedTLS on this part; the payload is tiny, so there's little to
shave.
- **stage + commit ~3.1 s** is the one soft spot: staging `notes.md` directly
instead of `add_all(["*"])` would skip the SD/FAT tree walk (likely →
sub-second), at the cost of the file-agnostic design that a future multi-file
publish wants. Deferred, on purpose.
- **stage + commit ~3.1 s** is the one soft spot: staging over the editor's dirty
set (`add_path`) instead of `add_all(["*"])` would skip the SD/FAT tree walk
(likely → sub-second) *without* losing multi-file — the dirty set is the file
list. Whether the walk actually dominates the ~4 s commit is now being measured
by the `commit split —` log line; the cost model and the rule it decides live in
[`../tradeoff-curves/sync-commit-staging.md`](../tradeoff-curves/sync-commit-staging.md).
**Conclusion:** ~16 s cold / ~10 s warm is close to the floor for "commit to FAT +
one TLS push over Wi-Fi with a fresh clock." It reads as slow only if you wait on

View File

@@ -0,0 +1,164 @@
# Editor freeze — SPI-DMA out-of-memory during a background `:sync`
> Date: 2026-07-11 · Build at time of failure: `07-11 18:02Z @229c259-dirty`
> Status: **Safety net shipped** (paints are non-fatal, the appliance no longer
> bricks) — pending hardware re-test. **Root cause not yet eradicated**: a paint
> that lands during a sync still fails and drops the frame; the permanent fix (a
> persistent internal DMA scratch buffer) is specced below and tracked in the
> follow-ups.
>
> Context: editor loop [`../../firmware/src/main.rs`](../../firmware/src/main.rs),
> EPD driver [`../../firmware/src/epd.rs`](../../firmware/src/epd.rs), git
> transport [`../../firmware/src/git_sync.rs`](../../firmware/src/git_sync.rs).
> Display medium [ADR-003](../adr.md#adr-003-display-medium--e-ink-gdey0579t93-panel);
> concurrency model (dedicated git thread) [ADR-006](../adr.md).
## Summary
First-ever field freeze. After an hour of clean editing, the user ran `:sync`;
the commit and push **succeeded** (`push accepted by remote`, commit
`48a2c0a8`), but partway through the push the panel stopped updating. Keystrokes
kept being logged, so the device wasn't hung — but nothing repainted again.
The editor task had died. A screen refresh that happened to run **while Wi-Fi +
TLS were up for the push** failed to allocate an internal DMA buffer
(`ESP_ERR_NO_MEM`), and that error propagated through a `?` straight out of
`main()`. ESP-IDF's `main_task` returned from `app_main()`; the editor loop lived
on that task, so it stopped. The USB-keyboard and git threads are **separate
FreeRTOS tasks** (ADR-006) and kept running — hence keys still logged while the
panel was frozen. A zombie, not a crash.
## Symptom
```
I (305749) firmware::git_sync: verifying github.com TLS chain against embedded GitHub CA bundle
I (305979) firmware::usb_kbd: key: HalfPageUp ← scroll arrives → editor loop paints
E (306259) spi_master: setup_dma_priv_buffer(1206): Failed to allocate priv TX buffer
Error: ESP_ERR_NO_MEM
I (306259) main_task: Returned from app_main() ← editor task exits
...
I (310119) firmware::git_sync: push accepted by remote ← git thread unaffected; commit landed
I (311069) firmware::usb_kbd: key: Escape ← keys still logged, no refresh ever again
I (312729) firmware::usb_kbd: key: HalfPageDown
```
## Root cause
The EPD is on SPI2 configured `Dma::Auto(4096)` (`main.rs`), and the driver hands
frame data to `spi.write()` as ordinary Rust `Vec<u8>` buffers (`epd.rs`
`write_frame_bank` / `data`). With PSRAM added to the heap allocator, those
`Vec`s can be allocated **from PSRAM**, which is **not DMA-capable**. When a
transfer buffer isn't DMA-capable, esp-idf's `spi_master` bounces it through a
temporary internal buffer it mallocs on the fly (`setup_dma_priv_buffer`, using
`MALLOC_CAP_DMA`**internal RAM only**).
For the whole session that bounce allocation succeeded, because internal RAM was
plentiful. The instant `:sync` brought up **Wi-Fi + TLS**, they consumed the
small internal pool. The next paint's bounce allocation returned
`ESP_ERR_NO_MEM`. The real defect is not the low-memory moment itself — it's that
the paint's `?` made a **transient, retryable** I/O failure **fatal to the whole
appliance**.
Two things made this easy to misread:
- **"8.4 MB free heap" is a red herring.** That figure is dominated by PSRAM.
The starved pools are the tiny internal ones (~265 KB / 21 KB / 32 KB), and DMA
bounce buffers can only come from those.
- **It looked like a git/sync bug, but the push was fine.** The git thread
finished and the remote accepted the commit. Only the UI task died.
## Timeline
1. Hour of normal editing — hundreds of ~630 ms partial refreshes, no issue
(internal RAM never under pressure; Wi-Fi off — the radio is lazy, ADR-006).
2. `:sync` → save → git thread brings up Wi-Fi, SNTP, TLS, commits `48a2c0a8`,
starts the push. Internal RAM now under heavy Wi-Fi/TLS load.
3. User keeps scrolling (`HalfPageUp`) during the push. Each scroll triggers a
full-area partial refresh → SPI-DMA transfer → bounce-buffer malloc.
4. One such malloc fails (`ESP_ERR_NO_MEM`); the `?` in the refresh call
propagates out of `main()`; `main_task` returns from `app_main()`.
5. Push completes normally; Wi-Fi torn down; internal RAM freed — but the editor
task is already gone, so nothing repaints. Keys log into the void.
## What it was *not*
- **Not out of heap** — plenty of PSRAM free; it was internal *DMA-capable* RAM
specifically.
- **Not a git or TLS bug** — the sync succeeded end to end.
- **Not an editor-core bug** — `editor`/`keymap` never ran; the failure is in the
firmware paint path's error handling.
- **Not a panel/wiring fault** — the same paint path worked for an hour and works
again after Wi-Fi is down.
## Remediation shipped — paints are non-fatal
A screen refresh is idempotent and retryable: the editor buffer is the source of
truth, so a dropped frame costs nothing and the next paint recovers. This is the
exact contract already written into `save_note` ("errors are logged, never
propagated"). Every paint site in the editor loop (`main.rs`) now:
- logs the failure and **drops the frame** instead of `?`-propagating it;
- leaves `shown` untouched so the next paint repaints the same diff;
- sets a `force_full` flag so the next paint is a **full refresh**, which
rewrites both RAM banks and recovers from a partial that may have died
mid-transfer and left the `0x24`/`0x26` banks inconsistent.
Effect: a paint that lands during a sync is dropped (panel goes stale for the
~15 s of the push), then self-heals the moment the push finishes and internal RAM
frees. A permanent brick becomes a brief stale window. The boot-time first render
stays `?`-fatal on purpose — it runs before Wi-Fi, can't hit this, and a dead
panel at boot is a legitimate hard fault.
## Root-cause eradication (specced, not yet built)
The safety net stops the brick but not the underlying **contention**: paints
during a sync still fail and drop. To keep the editor fully live while a push
runs — the whole point of the async git thread (ADR-006) — remove the on-the-fly
DMA allocation entirely.
**Fix: a persistent, internal, DMA-capable scratch buffer owned by `Epd`.**
- Allocate it **once at `Epd` construction** (boot, before Wi-Fi is ever up, when
internal RAM is plentiful) via
`heap_caps_malloc(SPI_CHUNK, MALLOC_CAP_DMA | MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT)`.
- `data()` already chunks writes to `SPI_CHUNK` (4096 B), so a **4 KB** scratch
buffer is sufficient: copy each chunk into it and hand that DMA-capable slice to
`spi.write()`.
- Because the source is now DMA-capable, `spi_master` DMAs directly from it —
`setup_dma_priv_buffer` is never invoked, there is **no per-refresh
allocation**, and paints no longer compete with Wi-Fi/TLS for internal RAM.
Refreshes then succeed during a sync, and `force_full` recovery is only ever
exercised by genuine faults, not by normal syncing.
**Cost:** ~4 KB of internal RAM reserved for the life of the device; one small
`unsafe` block for the alloc + slice; a `memcpy` per 4 KB chunk (negligible
against the ~630 ms panel waveform).
**Alternatives considered and rejected:**
- *Trim Wi-Fi's internal buffer counts* to leave headroom — fragile tuning that
risks Wi-Fi stability/throughput and only widens the margin instead of removing
the failure mode.
- *Force the whole ~13.6 KB frame buffer internal* (custom allocator / full
`heap_caps` buffer) — larger reservation and more churn than the 4 KB
chunk-scratch, for no extra benefit since `data()` already chunks.
- *Serialize paints against sync* (skip painting while a push is in flight) —
defeats the async-git design and freezes the panel for the whole push; the
safety net already makes this unnecessary.
**Verification when built:** reproduce the exact failing scenario — edit,
`:sync`, and keep scrolling/paging through the *entire* push. Expect: every
refresh succeeds (no `refresh … FAILED` warnings, no dropped frames), min-ever
internal heap stays comfortably above zero throughout, and `force_full` is not
triggered by the sync.
## Follow-ups
- [x] Make all editor-loop paints non-fatal + `force_full` recovery (`main.rs`);
release build green.
- [ ] Reflash and hardware-verify the safety net against the repro (edit →
`:sync` → scroll through the push): panel must recover, not freeze.
- [ ] Implement the persistent internal DMA scratch buffer in `Epd` (eradication
above) if the stale-during-sync window proves annoying in real use.
- [ ] After eradication, confirm refreshes succeed *during* a push and drop the
stale window entirely.

View File

@@ -12,3 +12,4 @@
| ---------- | ------------------------------------------------------------------------ | ------ |
| 2026-07-05 | [Spike 3 (SD) — card rejects CMD59 (SPI-mode CRC)](2026-07-05-spike3-sd-cmd59.md) | Paused — awaiting a compliant microSD; wiring + firmware proven |
| 2026-07-05 | [Spike 7 (git push) — ADR-004 kill-switch fired: gix can't push over HTTPS](2026-07-05-spike7-gix-https-push.md) | Turned — pivoted to libgit2; git mechanics proven on desktop, device build next |
| 2026-07-11 | [Editor freeze — SPI-DMA OOM during a background `:sync`](2026-07-11-editor-freeze-spi-dma-oom-during-sync.md) | Safety net shipped (paints non-fatal); root-cause eradication specced, not yet built |

View File

@@ -10,7 +10,7 @@ Scope: v0.1 MVP — see
[`v0.1-mvp-product.md`](v0.1-mvp-product.md) for user-facing scope and
[`v0.1-mvp-technical.md`](v0.1-mvp-technical.md) for implementation —
with the v0.2v1.0 trajectory ([README](../README.md),
[roadmap](roadmap.md)) in mind so we don't paint into a corner. Terminology
[macroplan](macroplan.md)) in mind so we don't paint into a corner. Terminology
(e.g. **Tracked**, **Local**, **Save**, **Publish**) follows the project
glossary at [`../CONTEXT.md`](../CONTEXT.md).
@@ -592,17 +592,17 @@ What a user (= me) values about the device, with importance weights on a
| W1 | Sub-second visible response to typing | 10 | [product → Write](v0.1-mvp-product.md#user-stories), [README → UX](../README.md#ux-boundaries-set-by-the-medium) |
| W2 | **Publishing** is one deliberate action away | 9 | [product → Publish](v0.1-mvp-product.md#user-stories), [CONTEXT → Publish](../CONTEXT.md#user-facing-actions) |
| W3 | Pulling power never corrupts the file | 10 | [product → Recover](v0.1-mvp-product.md#user-stories), [acceptance](v0.1-mvp-product.md#acceptance-criteria) |
| W4 | Provisioning never interrupts a writing session | 7 | [product → Provisioning](v0.1-mvp-product.md#provisioning-build-time-dev-only), [roadmap → v0.9](roadmap.md#v09--robustness--) |
| W4 | Provisioning never interrupts a writing session | 7 | [product → Provisioning](v0.1-mvp-product.md#provisioning-build-time-dev-only), [macroplan → v0.9](macroplan.md#v09--robustness--) |
| W5 | Quick boot to a writing cursor | 6 | [product → acceptance](v0.1-mvp-product.md#acceptance-criteria) (≤ 5 s) |
| W6 | Long sessions without crash / lag / drift | 9 | [product → acceptance](v0.1-mvp-product.md#acceptance-criteria) (1 h soak) |
| W7 | Nothing on the device competes with prose | 8 | [README → vision](../README.md#vision) |
| W8 | The UI never moves except when I move it | 7 | [README → UX](../README.md#ux-boundaries-set-by-the-medium) |
| W9 | Codebase absorbs the planned roadmap without rewrite | 8 | [roadmap](roadmap.md) |
| W9 | Codebase absorbs the planned roadmap without rewrite | 8 | [macroplan](macroplan.md) |
| W10 | I can repair or fork it with hobbyist tools | 5 | [README → vision](../README.md#vision) |
| W11 | Multi-day battery life (v0.8 onward) | 4 | [roadmap → v0.8](roadmap.md#v08--power-battery--sleep--) |
| W12 | Local-only file scope coexists with git scope (v0.5+) | 5 | [README → scopes](../README.md#vision), [roadmap → v0.5](roadmap.md#v05--file-palette--multi-file--) |
| W13 | Typography sets a writing-tool tone — typewriter or developer editor, never gadget | 7 | [roadmap → v1.0](roadmap.md), [README → UX](../README.md#ux-boundaries-set-by-the-medium) |
| W14 | I can carry the device and write away from a desk | 8 | [roadmap → v0.8](roadmap.md#v08--power-battery--sleep--), [README → hardware](../README.md#hardware) |
| W11 | Multi-day battery life (v0.8 onward) | 4 | [macroplan → v0.8](macroplan.md#v08--power-battery--sleep--) |
| W12 | Local-only file scope coexists with git scope (v0.5+) | 5 | [README → scopes](../README.md#vision), [macroplan → v0.5](macroplan.md#v05--file-palette--multi-file--) |
| W13 | Typography sets a writing-tool tone — typewriter or developer editor, never gadget | 7 | [macroplan → v1.0](macroplan.md), [README → UX](../README.md#ux-boundaries-set-by-the-medium) |
| W14 | I can carry the device and write away from a desk | 8 | [macroplan → v0.8](macroplan.md#v08--power-battery--sleep--), [README → hardware](../README.md#hardware) |
---

View File

@@ -1,389 +0,0 @@
# Roadmap — version details
Frequent releases. Each version is a usable artifact, not a checkpoint.
This file holds the macro-plan (Macroplan block below) and the per-version
scope. The user-facing requirements and engineering targets each release feeds
into are tracked in [`qfd.md`](qfd.md).
## Status — synced 2026-07-11
The editor **core** has been built 23 versions ahead of the device
**releases**, and is now **extracted into a host-testable `editor` crate** (plus
a `display` crate for the panel framebuffer) so `cargo test` exercises it off the
xtensa target. **v0.1 shipped 2026-07-11** (late against the 2026-06-29
baseline): SD storage, save, and **git publish are all wired into the app binary
and hardware-verified** (`:sync` commits on the SD `/sd/repo` and pushes to a
test repo), and the **boot splash (Spike 9) is confirmed on the panel** — a
vector `typoena`-in-a-circle shown at startup while the SD mounts, then the
editor comes up. **Cold boot verified at 4258 ms** (power-on → cursor,
2026-07-11; 742 ms under the ≤ 5 s gate). It first measured ~5.5 s; the fix was
to bring the editor up with a full-area partial (~630 ms) instead of a second
full refresh (~1.9 s) — panel confirmed clean, no ghosting. The 1-hour soak is
attested from real use; the remaining post-ship acceptance checks are power-pull
recovery, 1000-word no-drop, and `Ctrl-G`'s not-yet-built pull-then-retry
(→ v0.9). Beyond v0.1, v0.2 navigation, **v0.2.5 international input (hardware-verified
2026-07-11)**, and most of v0.6 Markdown already run. Version numbers are
unchanged — they track shippable device releases, not core progress.
Marks: `[x]` done in core · `[~]` partially done · `[ ]` not started. An
inline `(✓)` marks the done half of a split item.
## Macro-plan
Macroplan source — paste into the macroplan app to render the week-by-week
view. `original` dates are the June 2026 baseline and never move; slips get
appended as `reestimates`, per-item actuals live in the Status block above.
```macroplan
title = "Typoena — macro plan"
[[feature]]
name = "v0.1 it writes, it pushes"
start = 2026-06-01
original = 2026-06-29
delivered = 2026-07-11
learning = "Shipped 12 days late. The long pole was hardware bring-up risk, not the editor: SD on a shared SPI bus (resolved by moving it to its own SPI3, ADR-012) and on-device git (gix killed, pivoted to libgit2 as an esp-idf CMake component, ADR-004). Splash landed as a vector wordmark, not the planned 1-bit bitmap — the asset-embed/blit path is deferred to v1.0."
[[feature]]
name = "v0.2 navigation"
start = 2026-06-29
original = 2026-07-20
status = "on-track"
note = "Core work landed early: motions and modes already run; gutter, Ctrl-d/u, UTF-8 buffer remain."
[[feature]]
name = "v0.2.5 international input"
start = 2026-07-20
original = 2026-08-03
status = "on-track"
note = "Done early + hardware-verified 2026-07-11: dead-key accent composer in the keymap crate, editor buffer made UTF-8-correct. Side-panel pending marker dropped by decision (stale before the ~630 ms panel repaint)."
[[feature]]
name = "v0.3 editing"
start = 2026-08-03
original = 2026-08-24
status = "on-track"
note = "Deletes, counts, and operator grammar done early in core; yank/paste, undo/redo remain."
[[feature]]
name = "v0.4 visual + ex"
start = 2026-08-24
original = 2026-09-07
status = "on-track"
note = ": command-line mechanism and :fmt done early; Visual mode not started."
[[feature]]
name = "v0.5 palette + multi-file"
start = 2026-09-07
original = 2026-09-28
note = "Also adds the git-tracked .typoena.toml preferences file (save_on_idle, auto_sync cadence) and the palette `>` command mode that edits it live."
[[feature]]
name = "v0.6 markdown"
start = 2026-09-28
original = 2026-10-12
status = "on-track"
note = "Render affordances done early; 80-col ruler + snippet engine (added 2026-07-08) remain."
[[feature]]
name = "v0.7 search + git"
start = 2026-10-12
original = 2026-11-02
[[feature]]
name = "v0.8 battery + sleep"
start = 2026-11-02
original = 2026-11-30
[[feature]]
name = "v0.9 robustness"
start = 2026-11-30
original = 2026-12-28
[[feature]]
name = "v1.0 polish"
start = 2026-12-28
original = 2027-01-25
[[milestone]]
name = "MVP ships"
week = 2026-06-29
requires = ["v0.1 it writes, it pushes"]
```
---
## v0.1 — MVP: "it writes, it pushes" — [x]
The minimum thing that justifies the hardware existing. Full design:
[product](v0.1-mvp-product.md) · [technical](v0.1-mvp-technical.md).
**Status:** SHIPPED 2026-07-11 (late vs the 2026-06-29 baseline). Core editing +
partial refresh run on device; **SD mount + save are wired into `main.rs`**
(Spike 3 resolved — a genuine ≤32 GB card mounts, verified on its own SPI3 host
per ADR-012); **git publish is wired** (`:sync` → commit + fast-forward push on
the SD `/sd/repo`, hardware-verified against a test repo); and the **boot splash
(Spike 9) is confirmed on the panel** — [`Frame::splash`](../display/src/lib.rs)
shows a vector `typoena`-in-a-circle at startup while the SD mounts, then the
editor comes up. Cold boot **verified at 4258 ms** (power-on → cursor, 2026-07-11; 742 ms under
the ≤ 5 s gate). It first measured ~5.5 s; the fix was to bring the editor up
with a full-area partial (~630 ms) instead of a second full refresh (~1.9 s) —
panel confirmed clean. The 1-hour soak is attested from real use; the remaining
post-ship acceptance checks are power-pull recovery, 1000-word no-drop, and
`Ctrl-G` pull-then-retry (→ v0.9) — see
[product → acceptance](v0.1-mvp-product.md#acceptance-criteria).
- [x] ESP32-S3 boots (✓); e-ink shows Typoena splash (✓ Spike 9, confirmed on
panel 2026-07-11); boot status surfaces via the panel snackbar (no serial on device)
- [x] USB host enumerates the Nuphy, key events reach the editor (Spike 4)
- [x] One hard-coded file (`/sd/repo/notes.md`) opens on boot — **wired in
`main.rs`** (`boot_storage` mounts the SD and loads the note; a missing
card / repo / unreadable note halts with a panel message). The card is
pre-seeded from a computer (`just init` copies a full clone to `/sd/repo` +
writes config), never cold-cloned on device — see
[note](notes/git-sync-images-and-repo-size.md).
- [x] Insert-only editing, backspace, enter, arrow keys — modal editor overshot this early (see v0.2)
- [x] Line wrap, no line numbers yet — soft-wrap done early (see v0.6)
- [x] Save to SD via `:w` (and `:sync`) — **wired in `main.rs`** through the
`persistence` module's atomic write (unlink-then-rename + `*.tmp`
boot-recovery)
- [~] Wi-Fi credentials + remote URL + PAT + author: today baked into the binary
via `env!()` (no NVS, no on-device provisioning UI in v0.1). Migrating to
`/sd/typoena.conf` on the card, provisioned by `just provision` (or
`just init` for a fresh card) from the same `firmware/.env` the build uses
(minimum input — rotate the PAT or switch networks without a reflash, no
card re-copy). Firmware to read it at boot instead of
`env!()` — the git-publish wiring landed with baked config (2026-07-11);
the `typoena.conf` migration itself is deferred to v0.9 (on-device
provisioning).
- [x] Publish on **`:sync`** (the editor's command; the roadmap originally
planned `Ctrl-G`): stage `notes.md` → commit with a timestamp message →
fast-forward `push`; on a rejected push, fetch + reconcile then retry once
(no-op short-circuit when the tree is unchanged). **Wired into the editor and
hardware-verified 2026-07-11** — `firmware::git_sync` opens the SD `/sd/repo`,
runs on a dedicated 96 KB git thread with lazy Wi-Fi, and pushes over mbedTLS
HTTPS+PAT; the panel snackbar shows `synced <oid>` / `up to date` /
`sync failed`. (Interrupted-push auto-retry deferred to v0.9.)
- [x] Split the display into a **writing column** (60 cols) + a **side panel**
(~30 cols at FONT_6X10) for metadata — the surface every later panel
feature writes to. **Built** in the `editor` crate (`draw_panel`): a
full-height divider at x=600, with the panel currently showing the word
count, the mode indicator, a NO-KBD flag, and a transient save/publish
**snackbar** (below). Later fields (filename, clock, Wi-Fi, battery) add to
the same surface. Defined in
[`CONTEXT.md` § Screen regions](../CONTEXT.md#screen-regions) and
[product § Screen layout](v0.1-mvp-product.md#screen-layout).
- [x] **Snackbar** — a transient side-panel notice for host events (added
2026-07-11). On-device there is no serial log, so boot posts `loaded
<name>` (the note's filename without suffix), `:w` posts `saved` /
`save FAILED - retry :w`, and `:sync` posts `syncing...` then the push
result (`synced <oid>` / `up to date` / `sync failed`). Set via
`Editor::set_notice`; cleared on the next keystroke
rather than a timer — a timed auto-dismiss would cost a ~630 ms full-area
e-ink flash purely to erase text, which the panel deliberately avoids (cf.
the dropped pending-accent marker in v0.2.5).
- [x] Partial refresh on edits (✓ Spike 5); save wired (full-area partial
repaint on `:w`)
Out of scope: Vim, palette, multiple files, branches, conflict handling.
## v0.2 — Vim navigation — [~]
**Status:** navigation done in core; the **UTF-8-correct buffer landed
2026-07-11** (hardware-verified). Remaining = `Ctrl-d/u` and the line-number
gutter. Shipped early beyond scope: a read-only **View** mode and the full
`d`/`c` operator + text-object grammar (see v0.3 / v0.4).
- [x] Mode state machine (Normal / Insert / View), mode indicator in the status strip
- [~] Movement: `h j k l`, `w b e`, `0 $`, `gg G` (✓); `Ctrl-d Ctrl-u` remain
- [x] `i a o O A` to enter Insert
- [x] `Esc` returns to Normal
- [ ] Line numbers in the left gutter: relative in Normal mode (current line
shown as its absolute number), absolute in Insert mode — Spike 13 first
- [x] Groundwork — UTF-8-correct buffer: caret motions and edits step by
character, not byte (dropped the ASCII == byte-offset assumption), so every
motion stays correct with accented input. **Done 2026-07-11** alongside
extracting the editor into a host-testable crate — char-step
motions/deletes, byte-vs-char split in `layout`/`caret_rc`, `word_end`/`de`
fixed; 15 host tests. Render font is ISO-8859-15 (Latin-9), so accented
glyphs display.
## v0.2.5 — International input — [x]
**Status:** DONE in core, **hardware-verified 2026-07-11** (typed ç é è ñ on the
bench, no crash). US-International dead-key accent composition lives in the
`keymap` crate — a `Composer` downstream of the decoder — wired into
`usb_kbd.rs` so the editor still receives a single `Key::Char`. Builds on the
v0.2 UTF-8-correct buffer and the ISO-8859-15 render font. Host-tested.
- [x] Dead keys — grave, acute, circumflex, diaeresis, tilde — compose with
the next letter: à é ê ë ñ, ç (via `'`+c), both cases
- [x] `'`+space emits a literal apostrophe (the everyday apostrophe path); a
dead key followed by a non-composing letter emits the accent then the
letter
- [x] A non-character event (Enter, Backspace, arrows) flushes any pending
accent as its literal first
- [ ] ~~Pending-accent indicator in the side-panel status strip~~ — **DROPPED
(2026-07-11 decision):** at typing speed it would be stale before the
~630 ms panel repaint, so it conveys nothing. Left unbuilt on purpose.
- [x] Bonus (2026-07-11): the physical **Esc key** (HID 0x29) now types
`` ` ``/`~` — Esc comes from the Caps tap — so grave/tilde accents and
Markdown code fences are reachable on a 60% board without a Fn layer.
## v0.3 — Vim editing — [~]
**Status:** deletes and counts done; **yank/paste, undo/redo, and `.` repeat
remain** — there is no register or recorded-op mechanism yet. The `d`/`c`
operator grammar and text objects landed here ahead of schedule (the roadmap
had scheduled only `dd`/`dw`/`d$`).
- [~] `x dd`, `dw dd d$` (✓); `yy p P` and repeat with `.` remain (need a register / recorded op)
- [ ] Undo / redo (`u`, `Ctrl-r`) — bounded history in PSRAM
- [x] Numeric prefixes (`3dd`, `5j`)
- [x] Ahead of schedule: `c` change operator + text objects
(`ciw`, `di(`, `ca"`, … — inner/around, nesting-aware)
## v0.4 — Visual mode + ex commands — [~]
**Status:** the `:` command-line mechanism is built (Command mode + status-strip
echo), but only `:fmt` exists — `:w :q :wq :e` remain. Visual mode is not
started.
**DECISION (2026-07-07):** `v`/`V` = **Visual** selection (vim-standard). The
read-only **View** (reading/scroll) mode currently bound to `v`/`V` moves off
those keys and gets its own trigger (exact key TBD when Visual lands). View mode
stays — it just frees `v`/`V` for Visual.
- [ ] Visual char (`v`) and line (`V`) modes, `y d c` on selections
- [~] `:` command line: `:w :q :wq :e <path>` (mechanism ✓; these commands remain)
- [x] Ahead of schedule / unscheduled: `:fmt` Markdown formatter
(table alignment, blank-line collapse, trailing-whitespace strip)
## v0.5 — File palette + multi-file — [ ]
**Status:** not started (Spikes 11 + 14 retire the panel-mechanism and
buffer-lifecycle risk first).
- [ ] `Ctrl-P` opens fuzzy file palette over **both** `/sd/repo/` and
`/sd/local/`, with a scope marker (e.g. `[git]` / `[local]`) per result
- [ ] Open, switch, close buffers (keep ≤ 3 in memory)
- [ ] `:e` and palette share the same recent-files list
- [ ] `:enew` creates a new file — prompts for scope (tracked vs local)
- [ ] Delete a file — removes it from the SD card; for a Tracked file the
removal reaches the next `Ctrl-G` Publish's staged set (`git rm` / `add -A`
semantics, not plain `git add .`); a Local file is just unlinked
- [ ] `Ctrl-G` is disabled / hidden when the current buffer is local-scope
- [ ] The side panel briefly shows file count on `Ctrl-G` when the publish bundles
more than one dirty Tracked file (e.g. `"publishing 3 files: abc1234"`),
so workspace-scoped behaviour stays visible to the user
- [ ] **Preferences file** `/sd/repo/.typoena.toml` — a git-tracked,
hand-editable TOML file for editor behaviour, deliberately **distinct from
the `/sd/typoena.conf` card secrets** (Wi-Fi / PAT / remote / author,
gitignored, never committed — see v0.1). Read at boot; a missing file or
key falls back to the defaults below. Keys:
- [ ] `save_on_idle` (bool, default `true`) — auto-save the current buffer on
the existing idle pause (the ≥ 1 s typing-pause the panel already uses
for its refresh), so `:w` becomes optional rather than required.
- [ ] `auto_sync` (duration string, default `"10m"`; `"0"` / omitted
disables; **min clamp ~`"2m"`** so a palette typo can't drain the
battery) — a *max-staleness cap*, not a wall-clock timer:
**opportunistic, rate-limited** Publish. Push when already awake + dirty
(coalesced into the idle-pause, ≤ once per `auto_sync`) and once on the
way into sleep if dirty; **never wake from deep sleep purely to sync**.
Wi-Fi energy is a `1/T` curve whose knee sits at 510 min, and
`save_on_idle` already owns local data safety — so 10 min halves the
sync energy of a 5-min default for no real risk. Full derivation:
[`tradeoff-curves/wifi-auto-sync.md`](tradeoff-curves/wifi-auto-sync.md).
The **schema + defaults live here in v0.5**; the periodic side rides the
better-git work (v0.7) and must interact with light / deep sleep (v0.8).
- [ ] Open question: because the file is committed, these prefs **sync to
every device** that clones the repo — a per-device sync cadence may
instead want a card-local override (in `typoena.conf`). Decide before
build.
- [ ] **Palette command mode** — typing `>` at the `Ctrl-P` palette switches it
from file search to a command list (VS Code-style). The v0.5 commands edit
the `.typoena.toml` prefs above — e.g. `> save on idle: on/off` and
`> auto sync: 10m` — writing the value back to the file and applying it
live. This command list is the discoverable surface that later actions
(`:fmt`, theme, font) also register into.
## v0.6 — Markdown affordances — [~]
**Status:** render affordances done early; the 80-col ruler and the snippet
engine remain (snippets are net-new scope, added 2026-07-08).
- [x] Heading lines bolded in render (faux-bold double-strike)
- [x] List continuation on Enter inside `- ` / `1. ` (with empty-item exit)
- [x] Soft-wrap at word boundaries
- [ ] Optional column ruler at 80
- [ ] **Snippets** — trigger-driven text expansion for Markdown authoring
(Zed-inspired, but no completion popup: e-ink's ~630 ms refresh rules out
a live filtering menu, and it fights the distraction-free premise). Shape,
mirroring the existing `list_marker` insert-transform:
- [ ] Tab in Insert mode triggers expansion: if the word immediately before
the caret matches a snippet prefix, expand it; otherwise insert spaces
as today (`expand_snippet(word) -> Option<(body, stops)>`, alongside
`list_marker`).
- [ ] A snippet body is literal text plus numbered empty tab stops `$1 … $n`
and a final `$0`. There is no placeholder text (`${1:label}`) — the
editor has no selection/overtype model, so a placeholder would just be
text to delete. There are no dynamic or computed values either (e.g. no
`date` — there's no RTC; the wall clock is valid only after Wi-Fi+SNTP,
so it'd stamp 1970 on a cold boot).
- [ ] After expansion the caret lands on `$1`; Tab advances to the next stop,
forward only (no Shift-Tab). Stored stop offsets shift with edits at the
caret (all pending stops are always after it). The session auto-aborts
on Esc, a mode change, or a motion that leaves the stops.
- [ ] On a typing pause (same throttle as the insert cursor / word-count
refresh — the panel never repaints per keystroke), if the word before
the caret is a snippet prefix, the side panel shows the hint (the target
expansion). Quiet while typing; the hint appears on pause.
- [ ] The snippet table is hard-coded in the binary to start; a git-syncable
file on SD (`/sd/repo/.snippets`) is a later option, deferred while SD
is still blocked.
- [ ] Starter set: link `[$1]($2)$0`, image `![$1]($2)$0`, fenced code block,
etc.
## v0.7 — Search + better git — [ ]
**Status:** not started.
- [ ] `/` forward search, `n N`
- [ ] `:Gpull` (fetch + fast-forward only; refuse on conflict and surface it)
## v0.8 — Power: battery + sleep — [ ]
- [ ] Measure idle / typing / push current draw on bench
- [ ] 18650 + IP5306 charge board, soft power switch
- [ ] Light sleep on idle > 30 s (keyboard interrupt wakes)
- [ ] Deep sleep on lid close (reed switch); restore cursor + buffer
- [ ] Battery indicator in the side panel
## v0.9 — Robustness — [ ]
- [ ] Crash-safe writes (write to `.tmp`, fsync, rename)
- [ ] Recover from interrupted push (re-attempt on next save)
- [ ] SD card removal / reinsert handling
- [ ] Wi-Fi reconnect with backoff
- [ ] On-device provisioning + settings screen: SSID, PAT rotation, default
remote, commit author (replaces the v0.1 dev-only NVS-flashing path —
first release usable by someone who is not the firmware author)
## v1.0 — Polish — [ ]
- [ ] Boot time ≤ 3 s to usable cursor — currently ~4.26 s; the ~1.9 s cold-boot
full refresh is a hard e-ink floor, so ≤ 3 s is marginal (see
[`notes/boot-time-budget.md`](notes/boot-time-budget.md))
- [ ] Font selection (at least one serif + one mono) with adjustable font
size, switchable at runtime and persisted across reboots
- [ ] Theme: light / dark (inverted e-ink), switchable at runtime and
persisted across reboots
- [ ] Enclosure design files in `hardware/`
- [ ] User guide
## v1.x — Stretch / nice-to-have
- 10.3" panel upgrade via IT8951
- Multiple remotes / repos
- Stats: words today, streak
- BLE-HID fallback for wireless keyboards

View File

@@ -8,7 +8,7 @@
>
> They ride on the `epd::Frame` `DrawTarget` and the Spike 5 partial-refresh
> path. Project overview: [`../README.md`](../README.md). Vocabulary:
> [`../CONTEXT.md`](../CONTEXT.md). Release sequence: [`roadmap.md`](roadmap.md).
> [`../CONTEXT.md`](../CONTEXT.md). Release sequence: [`macroplan.md`](macroplan.md).
These prove display/UX risks, not stack risks, so they sit outside the 17
"prove before integration" gate. **Run Spike 8 first:** it partitions the panel
@@ -114,23 +114,22 @@ risk early.
a design decision this spike hands data to.)
13. **Spike 13 — Line-number gutter.** Draw a fixed-width digit column left of
the text area. The v0.2 spec is the hard case: **relative** numbers in
Normal mode (current line as its absolute number), **absolute** in Insert.
The naive part — reserving columns and drawing digits — is trivial; the
e-ink risk is churn. Relative numbering renumbers the *entire visible gutter
on every `j`/`k`*, so a single cursor move becomes a partial refresh of a
tall digit column, straight into the Spike 5 windowed-Y path and the "20
partials → forced full refresh" ghosting counter
([`firmware/src/epd.rs`](../firmware/src/epd.rs), render module). Absolute
numbering is cheap by comparison — only the rows below an inserted/deleted
line renumber. Two more interactions: the gutter steals horizontal columns
from the render-time soft-wrap, and wrapped continuation rows have no number
(blank vs. tilde — a layout decision). Prove: measure gutter partial-refresh
cost for (a) a held-`j` scan that renumbers the whole relative gutter vs.
(b) a single-line absolute edit, and confirm neither blows the ghosting
budget or forces extra full refreshes. Decides whether relative numbering is
viable on this panel or must be gated (absolute-only, or a batched/coalesced
gutter repaint). (Feeds v0.2's line-number gutter; genuine new e-ink risk.)
the text area. **Absolute numbering only**relative numbering was dropped
(2026-07-11). It renumbered the *entire visible gutter on every `j`/`k`*, so
a single cursor move became a partial refresh of a tall digit column,
straight into the Spike 5 windowed-Y path and the "20 partials → forced full
refresh" ghosting counter ([`firmware/src/epd.rs`](../firmware/src/epd.rs),
render module) — real e-ink cost for no proportionate gain on a
distraction-first panel. Dropping it removes the genuine e-ink risk this
spike existed to prove: absolute numbering is cheap — only the rows below an
inserted/deleted line renumber. What remains is a **layout decision**, not a
latency one: the gutter steals horizontal columns from the render-time
soft-wrap, and wrapped continuation rows have no number (blank vs. tilde).
Prove: reserve the gutter width, then confirm a single-line edit repaints
only the rows at/below the change and forces no extra full refresh. (Feeds
v0.2's line-number gutter; now low-risk after the relative drop.)
**VERIFIED on the panel 2026-07-11** — a single-line edit repaints only the
rows at/below the change, no extra full refresh; closes the last v0.2 gate.
14. **Spike 14 — Multi-file navigation (open / switch / new / delete).** The
panel *mechanism* is already Spike 11 (which names this v0.5 file palette),
@@ -153,4 +152,4 @@ risk early.
artifact-free full-refresh swap on every buffer change (leaning on Spike
11). Latency is not the risk here; buffer-state and git-staging correctness
are. (Feeds v0.5 multi-file, delete included — now authorized in
[roadmap](roadmap.md#v05--file-palette--multi-file--) v0.5.)
[macroplan](macroplan.md#v05--file-palette--multi-file--) v0.5.)

View File

@@ -11,3 +11,4 @@
| --- | --- |
| [`wifi-auto-sync.md`](wifi-auto-sync.md) | `auto_sync` interval vs Wi-Fi energy (a `1/T` hyperbola) — why the default is 10 min and opportunistic, not a wall-clock timer. |
| [`epd-refresh-latency.md`](epd-refresh-latency.md) | E-ink refresh latency vs rows driven — the full / full-area-partial / windowed-Y cost model behind typing responsiveness and the boot splash→editor swap. |
| [`sync-commit-staging.md`](sync-commit-staging.md) | Commit-staging strategy vs working-tree size — RESOLVED: every index-based path is O(N_tree) and fails on the real repo (611 s / OOM); the O(depth) TreeBuilder splice is benched at ~22.8 s and ships. Holds the full measurement trail plus the firmware plumbing plan (merged from the retired handoff note). |

View File

@@ -0,0 +1,638 @@
# Commit-staging cost vs working-tree size
> **Decision (RESOLVED 2026-07-12, real-repo bench):** neither `add_all(["*"])`
> nor an index-free `read_tree`+`write_tree` is viable on the real
> `jcalixte/notes` clone — **both are O(N_tree)** and blow up on the 570 MB pack /
> 1179-file tree (611 s hash one end, 77 s tree-read + OOM the other; see
> [Real-repo run](#real-repo-run-2026-07-12-jcalixtenotes-570-mb-pack--the-index-is-the-wrong-primitive)
> below). The commit must be rebuilt with an **O(depth) TreeBuilder walk** —
> patch only the edited file's ancestor subtree chain onto HEAD's tree, never
> materialise all 1179 entries.
>
> **Final state (2026-07-13, after four localization rounds):** the splice walk
> is benched and the block is lifted. It first measured 6.5 s (each loose-object
> write cost ~1.5 s); the root cause was FatFS's lseek cluster-chain walk, and
> the fast-seek fix cut it to **2.8 s cold / ~2 s steady-state**. The `esp_map.c`
> window cache was **removed entirely** (0 hits across four instrumented runs —
> `mwindow` absorbs any true repetition above `p_mmap`). The sub-second bar
> failed, but ~22.8 s against 611 s/OOM for every alternative ships: a full
> cold real-repo `:sync` lands at ~910 s. **Decision: wire the splice in —
> done 2026-07-13** ([how it landed](#the-fix--wiring-the-odepth-splice-into-the-firmware),
> merged from the retired `notes/sync-commit-handoff.md`; on-device `:sync`
> against the real repo still pending). Shrinking the repo is **not** an option
> ([`../notes/git-sync-images-and-repo-size.md`](../notes/git-sync-images-and-repo-size.md):
> the images are load-bearing for another app), which is exactly why the O(depth)
> mechanism is the only lever left. This note records the cost model and the full
> measurement trail that got here.
>
> Tradeoff-curves index: [`README.md`](README.md). Docs index:
> [`../README.md`](../README.md). Where the whole sync goes:
> [`../notes/sync-latency.md`](../notes/sync-latency.md). Sibling curve on the
> radio cost of *how often* we sync: [`wifi-auto-sync.md`](wifi-auto-sync.md).
## The model
`:sync` commits the working tree on the SD/FAT card before it pushes. The commit
is two kinds of work against the card over SPI (10 MHz today, ADR-012):
```
stage write
─────────────────────────────── ─────────────────────────────
add_all(["*"]) + update_all(["*"]) index.write + write_tree + commit-obj
→ stat() every file in the tree, → serialise the index and three loose
hash the ones whose stat moved objects, each a FAT create+write+fsync
cost ∝ tree size (O(N_tree)) cost ∝ churn (O(N_changed)) + fixed
```
The two have different curves against **N = files in `/sd/repo`**:
- **Walk** rises with N. `add_all(["*"])` visits the whole working tree every
sync regardless of how little changed, and each visit is a FAT `stat` (and a
re-hash when the entry looks dirty) over SPI. This is the term explicit-path
staging removes: the editor already knows which buffers are dirty and which
were `:delete`d, so `index.add_path(p)` / `index.remove_path(p)` over that set
touches `N_changed` files (≈1 for a writing appliance), not `N`.
- **Write** is flat in N. A text commit is the index + a blob + a tree + a commit
object — a handful of small FAT writes whose cost is set by SPI clock and
`fsync`, not by tree size. Explicit-path staging cannot shave this; only a
faster card bus (SD 10 → 20 MHz on a clean PCB, `persistence.rs`) does.
```
Commit latency vs working-tree size two staging strategies
ms
| walk-all: add_all(["*"])
4000 | . * stat()s every file in the
| . * tree each sync → O(N)
| . *
3000 | . *
| . *
2000 | . *
| *
1000 |····································· explicit-path: add_path(dirty)
| FAT object-write floor → O(churn); flat in N. The gap
0 +----+----+----+----+----+----+----+---→ up to walk-all is the avoidable
10 50 100 200 400 800 1179 per-sync tree walk.
└── jcalixte/notes today (N files)
```
The gap between the lines at a given N is exactly what switching buys, and it
**grows without bound** as the notes tree fills.
### The real operating point (measured 2026-07-12, `jcalixte/notes`)
The device syncs into a clone of the actual notes repo, not a `notes.md` toy. Its
working tree is **not small**:
| | count | working-tree bytes |
| --- | ---: | ---: |
| Markdown (`.md`) | 875 | ~1.5 MB |
| Images (png/jpg/webp/bmp/gif) | ~260 | **~150 MB** |
| Other (json/ts/pdf/…) | ~44 | ~20 MB |
| **Total (N)** | **1179 files, 158 dirs** | **~170 MB** |
| `.git` history | | ~570 MB |
So `add_all(["*"])` walks **1179 files across 158 directories every sync** — and
~260 of them are images that a text edit never changes. That does two things the
toy-repo baseline hides:
1. **The walk term is large and paid on every sync** — 1179 `stat`s + 158 dir
reads over SPI, for a one-line note change. This is the O(N) cost the curve
above predicts, at N ≈ 1179 rather than N ≈ 2.
2. **Re-hash risk.** libgit2 decides a file is unchanged from `stat` metadata
(mtime/size). FAT's coarse mtime and lack of a stable inode can make entries
look racy, forcing a content re-hash. If even a slice of the ~150 MB of images
gets re-hashed over a 10 MHz SPI bus, the commit balloons far past 4 s. The
`walk` timer will show it; explicit-path staging sidesteps it entirely by never
visiting the images.
## Measurement (2026-07-12, toy `notes.md` tree, N ≈ 2)
Split from two back-to-back `:sync`es on the small test repo (commits `95ac56ef`
cold, `ab260bde` warm), via the `commit split —` log lines:
| Sub-phase | Kind | Cold (ms) | Warm (ms) |
| --- | --- | ---: | ---: |
| `walk(add_all+update_all)` | scan (O(N)) + likely 1 blob write | 1402 | 1456 |
| `index.write` | FAT write | 204 | 204 |
| `write_tree` | **1 tree object → FAT** | 710 | 715 |
| `parent-load` | FAT read | 102 | 105 |
| `commit-obj` | **1 commit object + ref → FAT** | 914 | 924 |
| **commit total** | | **3332** | **3404** |
### It is not the card — it's libgit2 (`sd_bench`, 2026-07-12)
My first read of the table was "a loose-object write to this SD card costs
~700900 ms." **That was wrong.** `sd_bench` (`firmware/src/bin/sd_bench.rs`) times
the raw FAT primitives on the same card at the same 10 MHz:
| Raw FAT op (200-byte payload) | p50 |
| --- | ---: |
| create + write + close | 21.7 ms |
| rename | 12.8 ms |
| stat (hit / miss) | ~5 ms |
| remove | 14.9 ms |
| **loose-object composite** (stat + create + write + rename) | **86 ms** |
The card does a *complete* loose-object write in **~86 ms**. Yet `write_tree`
(one tree object) took **710 ms** and `commit-obj` **914 ms** — an **~8× gap that
is pure libgit2 overhead, not FAT I/O.** So the earlier "object-write floor / SD
write amplification / better card / SPI-clock" framing is refuted: **the SD card is
not the bottleneck.** fsync is still confirmed off; the extra ~600 ms/op is CPU or
repeated `.git` I/O *inside* libgit2 (candidates: ODB refresh scanning
`objects/`, the treebuilder's per-entry `git_odb_exists`, ref-lock + reflog writes,
config/attributes re-reads). `git_bench` (`firmware/src/bin/git_bench.rs`) localizes
it — see below.
### It's the pack, read through an un-caching emulated mmap (`git_bench`, 2026-07-12)
`git_bench` times the git2 primitives in isolation on `/sd/repo` (git ops on the
same 96 KB thread the real service uses — the main-task stack overflows on
`index.write`, which is itself the reason the service has a dedicated thread):
| git2 op | p50 | note |
| --- | ---: | --- |
| `Repository::open` | 100 ms | one-time |
| `odb.write(blob)` (unique) | **45 ms** | writes a fresh object; touches no existing object |
| `repo.index()` open | ~0 ms | cached |
| `index.write()` | 376 ms | index + `index.lock` rename + tree-cache |
| `write_tree` [unchanged] | ~0 ms | tree exists → freshen-skips the write |
| **`write_tree` [changed]** | **1136 ms** | writes ONE 45 ms object |
| **`commit(None)` orphan obj** | **563 ms** | writes ONE 45 ms object, no ref/reflog |
Writing a fresh object is 45 ms; the ops that wrap one are 825×. The cause, from
the vendored source: `git_odb_write` calls `git_odb__freshen` (odb.c:1011), which
on a not-found object runs **`git_odb_refresh`** (re-reads the pack dir + reloads
pack indexes), and existence checks (`freshen(tree)` in `commit.c:169`, base-object
lookups in `write_tree`) hit the **pack**. Pack access goes through our
`p_mmap` (`esp_map.c`), which **`malloc`s and `read()`s the mapped range from the
card on every call — no cache** — with a 32 MB window on this 32-bit target. So
each write re-reads pack bytes from SD; `odb.write` of a fresh blob is 45 ms only
because it touches no packed object.
**This scales with pack size.** The toy repo's pack is tiny; the real
`jcalixte/notes` clone has a **570 MB pack**, and provisioning rsyncs a full clone
onto the card — so a real-repo commit has **never been benchmarked** and, on this
mechanism, will be far worse than the ~3.3 s toy number. That is the single biggest
open risk in sync.
### Real-repo run (2026-07-12, `jcalixte/notes`, 570 MB pack) — the index is the wrong primitive
`git_bench` was finally run against a full clone of the real repo (1179 files, 158
dirs, 570 MB pack). It settles the design: **any index-based commit is O(N_tree)
and does not fit this device.** Two independent walls:
| op | result | reading |
| --- | ---: | --- |
| `Repository::open` | 88 ms | fine |
| odb open (implicit) | ~6 s cold | maps the 1.7 MB pack `.idx` once (16 miss / 1790 KB) |
| `odb.write(blob)` | **142 ms** p50 | the mmap cache win **holds** (was 862 ms uncached) ✅ |
| `repo.index()` load (1179 entries) | 514 ms max | the on-disk index we were trying to avoid |
| `index.write()` | **min 360 ms / p50 12.8 s / max 611 s** | ⚠️ hangs — see root cause |
| **seed `read_tree(HEAD)` (cold, 1×)** | **~77 s** | ⚠️ reads all ~158 tree objects, 22.7 MB of pack windows |
| `Index::new + read_tree` (warm) | 447 ms p50 | windows still mapped → pure CPU |
| **index-free `stage→tree`** | **💥 crash** | `zlib (5)`: `deflateInit` failed, **508 KB heap left** |
**Wall 1 — `index.write()` hashes the whole working tree (up to 611 s).**
`git_index_write` unconditionally calls `truncate_racily_clean` (index.c:822),
which runs `git_diff_index_to_workdir` over **every** entry flagged "racy" and
re-hashes its file. On a fresh FAT clone the mtime granularity is 2 s and
`index.stamp.mtime <= entry.mtime` for ~all 1179 entries (index.h:117), so the
whole tree looks racy → it re-hashes ~170 MB (mostly the 150 MB of images) over
10 MHz SPI. The 611 s → 12.8 s → 360 ms decay across three iterations is the
signature: each write bumps the index mtime, shrinking the racy set. **Implication:
the shipping `stage_and_commit` calls `index.write()`, so `:sync` on the real repo
effectively bricks on the first commit** — the user sees a 10-minute freeze,
resets, the index mtime never advances, and it re-hashes forever. The real repo has
almost certainly never completed a sync on device (only the toy `typoena-test` has).
**Wall 2 — the index-free path is still O(N_tree), and the mmap cache OOMs.**
Skipping `index.write()` entirely (fresh `Index::new()`, stamp = 0, so
`truncate_racily_clean` can never fire) removes Wall 1. But to seed the in-memory
index, `read_tree(HEAD)` materialises all 1179 entries and reads every tree object
from the 570 MB pack — **77 s cold** (447 ms only once the windows are resident).
`write_tree_to` is O(changed), but you pay O(N_tree) to build the cache it needs, so
the index-free path only trades a 611 s hash for a 77 s tree-read. Worse, that
`read_tree` drove the `esp_map.c` cache to **7.4 MB resident** — past its own 4 MB
soft cap — which left 508 KB of heap and made `repo.blob()`'s zlib `deflateInit`
fail. **The one write we cannot skip crashed.** Root cause of the OOM: our cache
holds pack windows *after* libgit2 `p_munmap`s them (refcount 0, freed only lazily
on the next `p_mmap`), which **defeats `GIT_OPT_SET_MWINDOW_MAPPED_LIMIT`**
libgit2 thinks it released the memory; we didn't.
**Conclusion — use an O(depth) TreeBuilder walk.** "Replace K files in a 1179-file
tree" should touch `O(depth × K)` objects, not `O(N_tree)`. Walk HEAD's tree down
the edited path (`tree.get_name`/`get_path` → read ~depth subtree objects), then
rebuild bottom-up with `repo.treebuilder(Some(&subtree))``insert`/`remove`
`write()`, and `commit` the new root. That never materialises the 1179 entries,
never re-hashes anything, never visits the 150 MB of images, reads only a handful
of tree windows (so the cache stays small and zlib keeps its heap), and — crucially
— **carries the image entries forward untouched from HEAD's tree, so the device
does not even need the images in its working tree.** The `esp_map.c` cache still
needs an evict-on-`munmap` fix (drop the cap, free past a low-water mark) so it can
never again starve a downstream `git__malloc`, but with the TreeBuilder walk the
pressure it was under largely disappears.
### Splice bench (2026-07-12, second real-repo run) — the walk is right, the loose-object write is the new wall
The O(depth) splice op was added to `git_bench` (1 blob + 3 tree writes onto the
depth-3 path `.claude/commands/bsky.md`, run FIRST so its first iteration is
cold; the index ops moved last so their OOM can't cost the new data — it did
crash again, after everything was logged):
| op | result | reading |
| --- | ---: | --- |
| `splice stage→tree` (1 blob + 3 trees) | **6.5 s p50, warm ≈ cold** | O(depth) confirmed — cost is 4 loose writes × ~1.6 s |
| `commit(None)` orphan obj | 1.7 s p50 | one more loose write |
| `odb.write(blob)` | **1.5 s p50** | ⚠️ was 142 ms in the previous run |
| `repo.index()` load | 524 ms max | matches previous run |
| seed `read_tree(HEAD)` cold (now timed) | 81.6 s | reproduces the 77 s |
| `index-free stage→tree` | 💥 crash, 508 KB heap | reproduces the zlib OOM exactly |
Three readings:
1. **The splice mechanism is validated as a mechanism.** Pack reads stayed flat
(~40 KB per write; 6.4 MB heap free through splice + commit + odb.write), so
it really is O(depth) and it cannot OOM. The 6.5 s is not tree-walk cost.
2. **The wall moved to the loose-object write: ~1.5 s each, ×4 per splice.**
The isolated `odb.write(blob)` — one tiny orphan blob — took 1.5 s where the
raw FAT composite is 86 ms. Projected full commit (splice 6.5 s + commit-obj
1.7 s + ref/reflog update) ≈ **89 s**: enormously better than 611 s, still
far off the bar.
3. **The mmap cache scored 0 hits over the entire run** — the documented
862→142 ms `odb.write` win did **not reproduce** (same `esp_map.c`, same
card). Either the earlier run's conditions differed (orphan-object
population? FAT allocation state?) or the win was misattributed. Whatever the
1.5 s is, it is *not* SD data volume: each write moves ~40 KB read + ~1 KB
written.
### ROOT CAUSE FOUND (2026-07-12, `sd_bench` seek op): FatFS lseek walks the cluster chain
Two `sd_bench` re-runs on the ~740 MB-full card settled it:
1. **Free-cluster-scan hypothesis: refuted.** Raw FAT write ops are unchanged on
the full card — loose-object composite **77 ms p50** (was 86 ms), create
20 ms, rename 10 ms. The card is exonerated a second time.
2. **Long seeks are the cost.** A new op opens the repo's largest packfile
(263 MB — the "570 MB pack" was actually the whole `.git`) read-only and does
seek+read(4 KB): **@offset 0 = 5.8 ms; @end = 198.7 ms** — dead constant
across 20 iters. Without `CONFIG_FATFS_USE_FASTSEEK`, FatFS resolves lseek by
walking the file's FAT cluster chain over SPI: forward from the current
position, **from the chain head on any backward seek**. 263 MB ≈ 16.8k
clusters ≈ ~67 KB of FAT reads ≈ ~190 ms per long walk.
**Why FAT behaves this way:** FAT has no extent map or inode — a file is a
singly-linked list of clusters, and the only way to find "byte 260,000,000" is
to follow that list entry by entry through the allocation table. FatFS walks
forward from the current position when it can, but a backward seek restarts
from the chain head ([FatFS `f_lseek` docs, elm-chan.org](http://elm-chan.org/fsw/ff/doc/lseek.html)).
The fast-seek feature fixes exactly this: a pre-computed **cluster link map
table (CLMT)** per file object, "(fragments + 1) × 2" words, after which "no
FAT access is occured in subsequent f_read/f_write/f_lseek" (same page). On
esp-idf it's `CONFIG_FATFS_USE_FASTSEEK` — the official docs recommend it "for
read-heavy workloads with long backward seeks" and note it does not apply to
files opened in write mode
([ESP-IDF FatFS docs](https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/storage/fatfs.html)).
The budget closes: each loose write does ~810 small (~4 KB) `p_mmap`s (freshen
→ trailer/idx probes) interleaved with low-offset reads, so ~8 of them pay a
fresh ~190 ms walk → **~1.5 s per object**. It also explains everything the
cache couldn't: warm ≈ cold (the walk is paid inside `lseek` before any data
moves, and the maps are below the 64 KB cache floor), the 142 ms vs 1.5 s
run-1/run-2 discrepancy (run 1's `odb.write` bench ran first and hammered only
the trailer — the file position stayed there, so its seeks were forward/no-ops),
and a large slice of the 81.6 s `read_tree` (133 windows × backward seeks ≈ 25 s
of walking on top of the 25 MB of data).
**Fix (config, not code): `CONFIG_FATFS_USE_FASTSEEK=y` +
`CONFIG_FATFS_FAST_SEEK_BUFFER_SIZE=256`** (landed in `sdkconfig.defaults`
2026-07-12). Fast seek builds an in-memory cluster-link map per read-mode file —
exactly how the pack is opened — making lseek O(1); write-mode files fall back
to the walk transparently. 256 words = 1 KB per open read-only file, covering
~127 fragments (default 64 covers ~31; a fragmented pack would silently fall
back to slow seeks, so the headroom matters).
**A/B measured (same evening): a 2.3× partial win, not a full one.**
| op | fast-seek off | fast-seek on |
| --- | ---: | ---: |
| `splice stage→tree` | 6.5 s | **2.81 s** |
| `odb.write(blob)` | 1.5 s | **416 ms** |
| `commit(None)` | 1.7 s | **1.72 s — unchanged** |
`odb.write` dropped by almost exactly the ~6 chain walks the model predicted —
the seek theory holds — but two residuals remain: **~400 ms per loose write**
(vs the 77 ms raw-FAT floor) and **`commit(None)`'s ~1.3 s premium over a plain
write, which was never seek-bound at all**. Prime suspect for the commit
premium: strict object creation makes `git_commit_create` validate its parent +
tree OIDs with pack header resolves, and `git_treebuilder_insert` does the same
per inserted entry — `git_bench` grew `odb.read_header(packed)` /
`odb.exists(missing)` probes and strict-off re-benches to test it.
### Second localization round (2026-07-12, run 3b + sd_bench re-run)
**Fast-seek verified on the metal:** re-running the sd_bench seek op with
`CONFIG_FATFS_USE_FASTSEEK=y` dropped `pack seek+read 4KB @end` from
**198.7 ms → 20.4 ms** (the CLMT fits the pack in the 256-word buffer — the
pack is not too fragmented). A far seek is now ~15 ms, i.e. effectively fixed.
**The strict-creation theory is refuted; the probes found the real unit cost:**
| op | p50 | reading |
| --- | ---: | --- |
| `odb.read_header(packed)` | **470 ms** | ONE pack header resolve costs ~½ s |
| `odb.exists(missing)` | **968 ms** (±0.1 ms) | miss path (scan → refresh → rescan) ≈ 2× |
| `commit(None)` strict OFF | 1.80 s | vs 1.93 s strict on — validation is NOT the premium |
| `splice` strict OFF | 5.7 s | noise-worse; also not validation |
The ±0.1 ms constancy of `exists(missing)` = a fixed, deterministic SD-op
sequence. The map counters identify it: **~78 small (~4 KB) `p_mmap` reads per
op** — pack trailer probes, idx fanout reads and delta-base windows, repeated
at the *same offsets* on every freshen/refresh. Post-fast-seek those cost
~20 ms each (~150 ms/op); the rest of `read_header`'s 470 ms is CPU-side
delta-chain inflation on the 160 MHz core plus repeated re-reads. Two other
observations from run 3b: the loose-object orphan population from bench runs
is creeping costs upward (splice 2.81 s → 3.21 s between consecutive
fast-seek runs — a re-provision resets it), and the mmap cache STILL scored 0
hits — because its 64 KB map-length floor excluded exactly these hot small
maps.
**Fix built (esp_map.c v2):** cache admission re-keyed from
map length to **file size ≥ 1 MB** — the pack/idx's small repeated windows now
cache (RAM hits after first touch) while small mutable working-tree files stay
excluded — plus **evict-on-`p_munmap` down to a 2 MB low-water mark**, fixing
the 7.4 MB OOM from the first real-repo run (released windows are actually
returned to `git__malloc`, so `MWINDOW_MAPPED_LIMIT` stays honest). Expected:
`read_header` collapses toward CPU-only, `odb.write` toward ~150250 ms,
splice at or under the sub-second bar, and no end-of-run zlib OOM.
### Final bench (run 4, esp_map v2) — memory fix works, cache theory dead, bar failed
Run 4 (2026-07-12 evening, same card state as run 3b plus its orphans):
| op | run 3b (fast-seek) | run 4 (+ esp_map v2) |
| --- | ---: | ---: |
| `splice stage→tree` (cold, first op) | 2.81 s | **2.83 s — unchanged** |
| `splice` again (warm, strict-off phase) | 3.21 s | 1.95 s |
| `commit(None)` | 1.72 s | 713 ms |
| `odb.write(blob)` | 416 ms | 366 ms |
| `odb.read_header(packed)` | 470 ms | 412 ms |
| `odb.exists(missing)` | 968 ms | 852 ms |
| mmap-cache hits | 0 | **0** (313 misses) |
| cache resident / heap free | grew to 7.4 MB → zlib OOM | **1833 KB flat / 6.4 MB free all run** |
Three findings:
1. **The memory discipline is verified.** Resident sits at 1833 KB through
every phase (under the 2 MB low-water, so nothing is being churned) and
heap never drops below 6.2 MB. The one uncaptured datum is the index-free
`read_tree` tail (the section that OOM'd runs 13) — the monitor was cut
before it ran. Not blocking: the shipping splice path never calls
`read_tree`; the tail would only re-confirm eviction under burst.
2. **The repeated-small-window theory is REFUTED — theory #3 down** (after
strict-creation and free-cluster-scan). v2 demonstrably admits and retains
the small maps now — the 1833 KB resident *is* them, held below low-water so
nothing is evicted before reuse — and still scored 0 hits in 313 misses.
So the ~8 small reads per loose write hit **unique (offset, len) every
time**: `mwindow` was already absorbing any true repetition above `p_mmap`,
and what reaches the emulation layer is distinct data (different objects,
different delta bases). A window cache cannot help. The residual
~360 ms/loose-write ≈ 8 distinct small SD round-trips × ~45 ms each
(post-fast-seek) — I/O count, not I/O size or seek cost.
3. **Within-run drift cuts both ways, so cross-run tables are mushy.** In this
single run `commit(None)` degraded 713 ms → 1.79 s between the early and
late (strict-off) phases, while splice *improved* 2.83 → 1.95 s. Two
competing effects: first-touch warm-up fading (CLMT build, first pack
reads — helps later ops) and orphan loose objects accumulating in
`.git/objects/xx/` slowing every freshen existence check (FAT directory
lookups are linear scans — hurts later ops). Steady-state on a clean
objects dir ≈ **~2 s per splice+commit**.
**Run 5 (confirmation, cache removed entirely):** esp_map.c stripped back to
plain malloc-read/free-at-munmap (stats counters kept). Read pattern
**byte-identical** to run 4 at every checkpoint (118 maps / 2314 KB after
splice, 148/2434, 163/2494, 208/2674) — except the strict-off phase did **15
fewer reads (~60 KB)** than v2: the low-water eviction had been kicking out
buffers mwindow still wanted, forcing re-reads. The cache was marginally
worse than nothing. Warm splice identical (1953 vs 1949 ms); the cold-op
+1015 % drift (splice 2.83 → 3.26 s) is the known orphan-creep signature,
not the removal. Run 5 also reframes run 4's "resident": **1854 KB `live` is
mwindow's open-window working set** (pinned mappings under the bench's 4 MB
mapped limit), not retained cache buffers — the cache had been retaining
essentially nothing. Removal CONFIRMED free; simpler emulation ships.
**Verdict: the sub-second bar FAILED — wire the splice in anyway.** The bar
was aspirational; measured reality is ~22.8 s to commit on the real
263 MB-pack repo versus 611 s (or a hard OOM) for every alternative benched.
That puts a full real-repo `:sync` at roughly **910 s cold**, which ships.
The remaining ~2 s has survived four localization rounds; the next suspect —
FAT *directory-op* cost in the freshen/refresh path (open/stat/rename by path
walk FAT directories linearly; consistent with the orphan-creep signal) — is
one instrumentation pass for later (log each `p_mmap` miss + bench dir ops in
`sd_bench`), not a prerequisite for the plumbing.
### The walk is ~1.4 s even at N ≈ 2
Mostly fixed cost — the worktree-diff setup and the second (`update_all`) pass —
not per-file `stat` (one raw `stat` is ~5 ms, so N ≈ 2 can't be the 1.4 s). The
O(N) slope only bites on the real `jcalixte/notes` clone (N ≈ 1179), which this run
did **not** exercise. That slope is still unmeasured.
For orientation: `publish(commit+push)` was 9846 ms cold, so the **network half is
~6.5 s** — still the biggest single block of a warm sync (10.1 s total), a separate
floor ([`../notes/sync-latency.md`](../notes/sync-latency.md)).
## The verdict
The real-repo run (above) overturned the earlier ranking. Both index strategies
are O(N_tree) and fail on the 570 MB-pack clone, and the repo cannot be shrunk. The
work, ranked:
1. **Rewrite the commit as an O(depth) TreeBuilder walk (the fix — build this).**
Rebuild only the edited path's ancestor subtree chain onto HEAD's tree; never
materialise the 1179-entry index, never `index.write()`, never `read_tree` the
whole tree. This is the ONLY mechanism that fits: O(depth × dirty) reads/writes,
flat in repo size, small heap, images carried forward untouched. Replaces
`stage_and_commit`'s `add_all`/`update_all`/`index.write`/`write_tree`. Needs the
editor's dirty set (+ deleted set) plumbed to the git service — the editor
already knows both. **Benched to completion 2026-07-12: 6.5 s → 2.8 s cold /
~2 s steady-state after the fast-seek fix (run 4). The sub-second bar failed
but the block is lifted — wire it in** (see the final-bench section above:
the residual is unique small SD round-trips, not something a cache or seek
fix can remove).
2. **Fix the `esp_map.c` cache so it can't OOM — RESOLVED BY REMOVAL (run 5).**
The cache never scored a hit in four instrumented real-repo runs
(`mwindow` absorbs true repetition above `p_mmap`; only new ranges reach
the emulation), and the 7.4 MB OOM it was patched to avoid was caused by
the cache itself holding buffers past `p_munmap`. esp_map.c is now the
plain malloc-read/free-at-munmap emulation: honest with
`MWINDOW_MAPPED_LIMIT` by construction, ~120 lines lighter, and run 5
confirmed removal is I/O-neutral (even 15 reads *better* than v2, whose
low-water eviction fought mwindow). Stats counters kept to spot any future
workload that does repeat ranges.
3. **Retired: `add_all`/explicit-path *index* staging.** Explicit-path `add_path`
still goes through the index and `index.write``truncate_racily_clean`, so it
hits Wall 1 just the same. The TreeBuilder walk supersedes it entirely; the
"explicit-path staging" idea survives only as "the editor's dirty set feeds the
walk."
4. **Retired: SD clock / better card.** The card does a full object write in
~86 ms; raw I/O is not the bottleneck. Do not spend the PCB's 20 MHz budget
expecting a commit-latency win.
5. **Kept: the mmap cache + mwindow tuning** (`GIT_OPT_SET_MWINDOW_*`, 256 KB
window / 4 MB mapped limit). It fixed `odb.write` and the push read path; #2 just
makes it well-behaved under memory pressure.
**Recommendation:** build #1 (the O(depth) TreeBuilder walk) — #2 resolved
itself by removal. The concrete plumbing plan (exact call sites, dirty-set
threading, FD budget) is the
[next section](#the-fix--wiring-the-odepth-splice-into-the-firmware).
## The fix — wiring the O(depth) splice into the firmware
> Merged from `docs/notes/sync-commit-handoff.md` (written 2026-07-12, retired
> 2026-07-13 once the bench phase closed). The handoff's bench half is done —
> the splice op lives in `git_bench` and the numbers above are its output.
> **The firmware plumbing below SHIPPED 2026-07-13** (compile-verified both
> feature flavors; on-device `:sync` against the real repo still pending) —
> each item now records how it landed rather than what to do.
### The splice walk
Rebuild only the edited file's ancestor subtree chain onto HEAD's tree. Never
materialise the 1179-entry index; never `index.write()`; never `read_tree` the
whole tree. Cost is O(depth × dirty_files), flat in repo size, tiny heap, and it
carries every unchanged entry (all 260 images, the other 1176 files) forward
untouched — which means **the device doesn't even need the images in its
working tree.**
Shipped as `git_sync::splice` (git2 0.20: `Repository::treebuilder(Option<&Tree>)`
+ `TreeBuilder::{insert,remove,write}`). Signature:
```rust
fn splice(repo: &Repository, base: Option<&Tree>, path: &[&str], blob: Option<Oid>) -> Result<Oid>
```
`Some(blob)` inserts/replaces, `None` removes; `base: None` synthesizes a
missing intermediate directory on the way down, and a directory emptied by a
remove is pruned on the way up (the empty tree is never re-inserted).
`stage_and_commit` folds every dirty path through it (threading the running
root tree), then `commit(Some("HEAD"), …)` exactly as before — the
`tree unchanged → nothing to publish` check and the `commit split —` timing
log survive. The benched reference is `git_bench`'s `splice stage→tree` op.
### `firmware/src/git_sync.rs` — how it landed
- **mwindow options set at service start** (top of `run_git_service`, before
any `Repository::open`): `set_mwindow_size(256 KB)` +
`set_mwindow_mapped_limit(4 MB)`. Without them the 32-bit defaults (32 MB
window / 256 MB mapped limit, mwindow.c:16) would `git__malloc` a 32 MB
window on the first pack access and die on the 8 MB PSRAM heap. Run 5
sharpened the stakes: the ~1.85 MB bench "resident" **is** mwindow's live
open-window set, so these opts are the only thing bounding it.
- `stage_and_commit(repo, paths)` is the splice walk; `add_all`/`update_all`/
`index.write`/`write_tree` are gone. **The request carries one path set, not
`{changed, deleted}`:** the working tree is the source of truth, so at commit
time a recorded path that exists on the card is spliced in from disk and a
missing one is spliced out. An unchanged path is a no-op — over-reporting is
free, which is what makes the retry/journal semantics below simple.
- **Stranded-commit recovery (new):** when the splice yields the parent's tree
(nothing to commit), the service now compares HEAD against
`refs/remotes/origin/<branch>` and still pushes if origin lacks HEAD — a
previous cycle that committed and then failed mid-push used to strand that
commit forever (the old path reported "up to date" and never retried).
- **Radio-free up-to-date (new):** an empty dirty set + origin already at HEAD
short-circuits before Wi-Fi even comes up — `:sync` with nothing to do
answers in ~150 ms instead of a Wi-Fi/TLS round.
- `reconcile_onto_origin` now `ResetType::Soft` (ref move only) — there is no
index to reset, and a Mixed reset's index write is exactly the racy-clean
wall the splice avoids. Side win: a remote-only added file is now *carried*
by the replay (origin's tree is the splice base) where the old `add --all`
replay dropped it.
- The macOS-cruft filter (`skip_macos_cruft`) is gone with the walk — the
splice only touches paths the editor recorded, so `._*`/`.DS_Store` can't
sneak in the way they once did (07d87772, the Spike-14-era lesson).
- **Deliberate behavior change (now in effect):** only paths the editor
saved/deleted are ever committed. Files changed on the card outside the
editor (card mounted on a Mac) were swept in by `add_all` before; they will
now never be committed, and the working tree will show a permanent diff
against HEAD if inspected on a desktop. Correct for the appliance; recorded
here so it reads as intent, not accident.
### Dirty-set source — `firmware/src/persistence.rs` + `main.rs` (how it landed)
- `Storage` owns the record: `save_path`/`delete_path` note their repo-relative
path in a `RefCell<Dirty>` (paths outside `/sd/repo` are skipped), and the
set is **journaled to `/sd/.typoena-dirty`** — atomic write, rewritten only
when the set actually grows. Without the journal a power pull would strand
every file saved-but-not-published that session: nothing walks the tree
anymore, so an unrecorded change would never reach the remote. The journal
is loaded at mount and its paths ride the next `:sync`.
- Lifecycle: `take_dirty()` snapshots pending → in-flight for one publish
(journal keeps carrying the union); the outcome settles it —
`publish_succeeded()` forgets the snapshot and shrinks the journal,
`publish_failed()` returns it to pending for the next `:sync`. A save landing
*while* a publish runs re-enters pending and rides the next one. Recording
happens *before* the file write, so a crash between the two only
over-approximates (a no-op splice), never under-records.
- `Effect::Publish` in `main.rs` sends `PublishRequest { paths: take_dirty() }`;
the outcome handler in the idle branch calls the matching settle method.
- **FD budget:** a git build now mounts `Storage::mount_for_git()` (16 FDs) in
`boot_storage` — libgit2 keeps the pack + `.idx` descriptors open and opens
loose objects on top, which overruns the editor's 4-FD budget. The light
build keeps the editor's own budget.
### `esp_map.c` — nothing left to do
The handoff's third work item (an evict-on-`p_munmap` cache fix) is superseded:
run 5 removed the cache outright and `esp_map.c` is already the plain
malloc-read/free-at-munmap emulation (see verdict item 2). One surviving
operational note: editing `components/libgit2/*.c` won't rebuild via plain
`cargo build` — first
`rm -rf firmware/target/xtensa-esp32s3-espidf/release/.fingerprint/esp-idf-sys-*`
(the `esp-idf-component-rebuild` lesson).
### How to bench / flash
`git_bench.rs` runs git ops on the 96 KB `GIT_STACK` thread (the main task stack
overflows on these ops — that's why the real service has a dedicated thread). It's
Rust-only, so a plain rebuild picks it up (no fingerprint dance unless you also
touched `esp_map.c`).
```
just flash-gitbench
# = . ~/export-esp.sh && LIBGIT2_SRC=<repo>/firmware/components/libgit2/vendor \
# LIBGIT2_NO_VENDOR=1 PKG_CONFIG_ALLOW_CROSS=1 \
# PKG_CONFIG_LIBDIR=<repo>/firmware/pkgconfig \
# cargo run --release --bin git_bench --features git
```
Bench on the **real repo** clone (`/sd/repo` = full `jcalixte/notes`), not the
toy — the toy pack understates everything by ~2 orders of magnitude.
### Still open (none block the plumbing)
- The residual ~360 ms/loose-write ≈ 8 unique small SD round-trips; next
suspect is FAT **directory-op** cost in the freshen/refresh path. One
instrumentation pass for later (log each `p_mmap` miss + bench dir ops in
`sd_bench`).
- Ref/reflog-update cost on the real repo — the bench's `commit(None)` writes
no ref, so the shipping commit's last leg is unmeasured.
- The push's ~6.5 s network floor
([`../notes/sync-latency.md`](../notes/sync-latency.md)) — a separate curve.
## Adjacent lever: should the images be on the card at all?
Explicit-path staging makes the walk skip the images, but they still cost 150 MB
of SD space, inflate the 570 MB clone, and slow provisioning + the pull-before-push
paths. Whether the device should carry image blobs at all — vs. markdown-only, or
Git-LFS-style pointers — is a separate decision tracked in
[`../notes/git-sync-images-and-repo-size.md`](../notes/git-sync-images-and-repo-size.md).
That lever shrinks N and the clone; this one stops the walk from paying for N. They
compose.
## What this does *not* touch
The network half of `:sync` (TLS handshake + push round-trips, ~6.5 s of the warm
path) is a separate floor covered in [`sync-latency.md`](../notes/sync-latency.md);
this curve is only about the local commit. Radio *frequency* (how often we pay any
sync at all) is [`wifi-auto-sync.md`](wifi-auto-sync.md).

View File

@@ -3,7 +3,7 @@
> **Decision:** `auto_sync` defaults to **10 min**, and is an *opportunistic,
> rate-limited* push — not a wall-clock timer that wakes the device. See
> [Policy](#policy). Backs the `.typoena.toml` `auto_sync` key in
> [`../roadmap.md`](../roadmap.md) (v0.5), whose runtime timer lands in v0.7 and
> [`../macroplan.md`](../macroplan.md) (v0.5), whose runtime timer lands in v0.7 and
> must respect sleep (v0.8).
>
> Tradeoff-curves index: [`README.md`](README.md). Docs index:

147
docs/typoena-snippets.md Normal file
View File

@@ -0,0 +1,147 @@
# `.typoena.snippets.json` — snippet library
> The git-tracked file that holds your trigger-driven text expansions for
> Markdown authoring. Hand-editable (and Zed-compatible, so you can paste your
> existing snippets straight in), synced across devices like your notes. Landed
> in **v0.6** (see [`macroplan.md`](macroplan.md)). The editing surfaces — inline
> Tab-expansion and the `$` palette — are specified in
> [`v0.6-markdown.md`](v0.6-markdown.md).
>
> **Three files, three concerns, don't confuse them.** `.typoena.snippets.json`
> is *content* (your templates). [`.typoena.toml`](typoena-toml.md) is *behaviour*
> (auto-save, gutter). `/sd/typoena.conf` is *secrets* (Wi-Fi, PAT), gitignored
> and never committed. The first two live in the repo and sync; the third is
> per-device.
## Location
```
/sd/repo/.typoena.snippets.json
```
It sits in the Tracked repo beside [`.typoena.toml`](typoena-toml.md), so it is
**committed and pushed** like any note and **syncs to every device** that clones
the repo. Your snippet library follows you. It is read **once at boot**; a
**missing, empty, or malformed file is fine** — you simply have no snippets, and
the editor runs unchanged.
## Format
Deliberately **Zed's snippet JSON shape**, so the contents of a Zed
`snippets/markdown.json` paste in unmodified:
```json
{
"Markdown link": {
"prefix": "link",
"body": "[$1]($2)$0",
"description": "Inline link"
},
"Book notes": {
"prefix": "fiche",
"body": ["# $1", "", "## $2 — $3", "", "## What the book is about", ""],
"description": "Fiche de lecture"
}
}
```
- The top-level key is the **display name** (what the `$` palette shows).
- `prefix` — the word that triggers inline Tab-expansion.
- `body` — a **string**, or an **array of lines** joined with `\n` (Zed's form;
it sidesteps embedded-newline escaping and reads cleanly for multi-line
templates).
- `description` — optional but recommended: the `$` palette fuzzy-matches it and
shows it, so it is how you find a snippet you don't remember the prefix for.
### Tab stops
A body is literal text plus numbered stops:
- `$1 … $n` — empty stops the caret visits in order.
- `$0` — the final resting place (defaults to the end of the insertion if absent).
- `${n:label}`**accepted, but the label is stripped** to a bare `$n`. The
editor has no selection/overtype model, so a label would just be text to
delete; on a device with no completion popup it could never be shown as a
prompt anyway. The **headings and structure carry the template** — the labels
were only hints. This is what lets a Zed file with `${1:Titre}` load as-is.
- **No dynamic or computed values** (no `date`, no `clipboard`). There is no RTC
— the wall clock is valid only after Wi-Fi + SNTP, so a `date` snippet would
stamp 1970 on a cold boot. A stop is empty or it is literal; nothing else.
## The two surfaces
Every snippet works both ways — there is **no hidden two-tier rule** where some
are "inline only" and some are "palette only". Inline Tab is the fast path you
reach for once a prefix is in muscle memory; the `$` palette is discovery.
### Inline Tab-expansion (Insert mode)
Type a prefix, press **Tab**. If the word immediately before the caret matches a
snippet prefix, it expands; otherwise Tab inserts spaces as it does today. (Tab
arrives as an ordinary character, so this is a check inside the Insert-mode
handler, alongside the existing list-continuation transform.)
On a **typing pause** — the same throttle as the word-count / cursor refresh, so
never a per-keystroke e-ink flash — if the word before the caret is a prefix, the
right side panel shows a quiet hint (`» Book notes`), on the row above the mode
line. The panel is ~15 columns, so the hint is the **snippet name**, not the whole
body; the full preview is what the `$` palette is for. (The marker is `»`, not a
tab glyph: the panel font is ISO-8859-15, which has no `↹`.)
### `$` palette (browse + insert)
Open the palette (`Cmd-P`) and type **`$`** — the same sigil mechanism as `>` for
commands. The query after the `$` fuzzy-matches name, prefix, and description;
`Ctrl-N`/`Ctrl-P` move the selection; **Enter inserts the body at the caret** and
starts the tab-stop session (dropping you into Insert at `$1`). The empty-palette
placeholder legends the sigils: `Go to file · > settings · $ snippets`.
## The tab-stop session
Identical whether the snippet was expanded inline or inserted from the palette:
- After insertion the caret lands on **`$1`** (or the end, if the body has no
stops), in **Insert** mode.
- **Tab advances** to the next stop, **forward only** (no Shift-Tab). The last
Tab lands on `$0` / the end and ends the session.
- Pending stop offsets sit **after the caret** and shift with the edits you make
at each stop, so typing at `$1` keeps `$2 … $n` correctly placed.
- The session **auto-aborts** on Esc, a mode change, or a motion that leaves the
stop range — after which the buffer is just text and Tab inserts spaces again.
## Parsing
The parse lives in the host-testable `editor` crate (`Snippets::parse`), using
`serde_json` — JSON string escapes (`\n`, `\"`, `\uXXXX`) are a foot-gun to
hand-roll, and `serde_json` is battle-tested; the editor crate is `std`, so it
compiles for xtensa via esp-idf. This is the **one new dependency** the feature
adds. The firmware reads the file at boot and hands the parsed list to
`Editor::set_snippets`, mirroring how `.typoena.toml` is read and applied via
`set_prefs`. A parse error is **non-fatal**: log it and boot with no snippets,
rather than refusing to start over a stray comma.
## Editing it
- **On your computer (the normal path).** It's plain JSON in your notes repo —
edit it in your real editor, copy entries over from Zed, commit, and it reaches
the device on the next clone/sync. This is deliberately where the heavy editing
happens; the appliance is for writing, not for maintaining a JSON library.
- **First-time setup.** [`just init`](../firmware/README.md#provisioning-an-sd-card)
seeds this file from a curated catalog (`firmware/snippets-catalog/`) — you pick
which snippet groups you want and it `jq`-merges the selected subset into
`repo/.typoena.snippets.json` (committed on the device's first `:sync`). It
writes **only if the file is absent**, so re-running `init` on a card whose
clone already carries your library never overwrites it. See
[`v0.6-markdown.md`](v0.6-markdown.md) for the catalog.
- **On-device hand-edit — deferred.** The palette hides dotfiles, and `:e` was
dropped in v0.6, so there is no in-editor path to this file yet. When one is
wanted it returns as a discoverable `> edit snippets` command that opens the
file directly, rather than resurrecting a general `:e`.
## See also
- [`v0.6-markdown.md`](v0.6-markdown.md) — the editing surfaces, the `$`/`>`
palette model, and the setup-recipe snippet catalog.
- [`typoena-toml.md`](typoena-toml.md) — the sibling prefs file this is kept
separate from, and the `>` command palette snippets share the surface with.
- [`macroplan.md`](macroplan.md) — v0.6 scope.

176
docs/typoena-toml.md Normal file
View File

@@ -0,0 +1,176 @@
# `.typoena.toml` — editor preferences
> The git-tracked file that controls how the editor behaves — auto-save,
> format-on-save, the line-number gutter, and the panel theme. Hand-editable, or
> changed live from the `Cmd-P` palette (booleans flip; the theme and auto-sync
> interval rotate through preset options on **Enter**). Landed in **v0.5** (see
> [`macroplan.md`](macroplan.md)).
>
> **Not to be confused with `/sd/typoena.conf`** — that holds the device
> *secrets* (Wi-Fi, PAT, remote URL, commit author), is gitignored, and is never
> committed. `.typoena.toml` is *behaviour*, shared across devices; `typoena.conf`
> is *secrets*, per-device. See [v0.1 product](v0.1-mvp-product.md).
## Location
```
/sd/repo/.typoena.toml
```
It lives inside the Tracked repo (`/sd/repo`), so it is **committed and pushed**
like any note — which means the preferences **sync to every device** that clones
the repo. That is deliberate: your editor behaviour follows you. (A per-device
override for the one genuinely device-specific key, `auto_sync`, may layer on top
later via `typoena.conf` — worth it only once `auto_sync` actually does something
in v0.7. See the [auto_sync](#auto_sync) note.)
The file is read **once at boot**, before the first screen is drawn (so
`line_numbers` shapes the opening frame). A **missing, empty, or partial file is
fine** — every absent key falls back to its default below, so a fresh card just
works with no config present.
## Keys
| Key | Type | Default | Options | Effect |
| --- | --- | --- | --- | --- |
| `save_on_idle` | bool | `true` | `true` / `false` | Auto-save the current buffer on the idle typing-pause, so `:w` is optional. |
| `format_on_save` | bool | `true` | `true` / `false` | Run `:fmt` on the buffer before an explicit `:w`/`:sync`. |
| `line_numbers` | bool | `true` | `true` / `false` | Show the absolute line-number gutter. Off reclaims its columns for text. |
| `theme` | string | `"light"` | `light` / `dark` | Panel colour polarity. `dark` inverts the whole frame to white-on-black. |
| `auto_sync` | string | `"10m"` | `2m` / `5m` / `10m` / `15m` / `30m` | Max-staleness cap for opportunistic auto-publish. **Value only — no behaviour yet** (rides v0.7). |
The **Options** column is what the palette rotates through on **Enter**; a
boolean is just the two-option case. Hand-editing a string key can still set any
value — the palette only cycles the presets.
### Example
```toml
# Typoena editor preferences — hand-editable, git-tracked.
# Edit here, or change live from the Cmd-P palette (type `>`).
save_on_idle = true
format_on_save = true
line_numbers = true
theme = "light"
auto_sync = "10m"
```
### `save_on_idle`
When on, the firmware quietly persists a dirty, named buffer once typing has
paused (~1.5 s), so a power pull can't cost more than the last couple of seconds
of writing. It is a **safety net, not an action**:
- **Silent.** No snackbar, no forced screen refresh. A visible confirmation on
every pause would cost a ~630 ms e-ink flash purely to say "saved" — exactly
the gratuitous flashing the panel avoids elsewhere. `:w` remains the *loud*
save (it posts `saved`).
- **Unformatted.** The idle save never runs `:fmt` — see the
[format_on_save](#format_on_save) note for why.
- Fires **once per typing burst**; a failed save doesn't retry-storm (it's kept
in RAM and re-attempted on the next burst, or on `:w`).
### `format_on_save`
Runs `:fmt` — table alignment, blank-line collapse, trailing-whitespace strip —
on the buffer *before* it is persisted, so `:sync` is **fmt → save → commit →
push** and `:w` saves formatted.
**Formatting only happens on an explicit `:w`/`:sync`.** The `save_on_idle`
auto-save is deliberately left unformatted: if it reformatted on every idle
pause, tables would reflow and blank lines collapse *mid-session*, with the caret
jumping under you every time you paused to think. Formatting is a deliberate act;
the safety-net save is not.
### `line_numbers`
Shows the absolute line-number gutter (built always-on in v0.2). Turning it off
returns the gutter's columns to the text, so prose gets the full writing width.
Applied **live** — toggling it from the palette redraws immediately with (or
without) the gutter.
### `theme`
Panel colour polarity: `light` (the native black-ink-on-white-paper) or `dark`
(white-on-black). On the 1-bit e-paper panel this is not a palette swap but a
**whole-frame invert** applied at the very end of the render, so text, selection,
caret, side panel and command palette all flip together and each stays legible.
Any value other than `dark` reads as light. Applied **live** — cycling it from
the palette repaints inverted at once.
> **On e-paper, `dark` is not free.** Partial refreshes over a mostly-black field
> ghost more than over white, and the panel is tuned for white-background reading.
> It works, but expect a slightly muddier refresh than `light` — verify on-device.
### `auto_sync`
A duration string that will one day cap how stale the published copy is allowed
to get — an *opportunistic, rate-limited* push, not a wall-clock timer. The
palette rotates it through the presets `2m` / `5m` / `10m` / `15m` / `30m`
(hand-editing can still set any string, e.g. `"0"`/empty to disable). **The value
is only stored and displayed in v0.5 — nothing reads it yet:** the periodic push
rides the better-git work in v0.7 and must interact with sleep in v0.8, so
cycling the interval today changes what will be honoured *then*, not now.
Rationale for the `"10m"` default:
[`tradeoff-curves/wifi-auto-sync.md`](tradeoff-curves/wifi-auto-sync.md).
## Editing it
Two ways, both landing in the same file:
1. **By hand** — it's plain text on the card; edit it on your computer and reboot
to apply. (The palette hides dotfiles, but you can still open it in-editor with
`:e repo/.typoena.toml`.)
2. **Live, from the device** — open the settings list either way:
- **`:settings`** — drops you straight into it, or
- **`Cmd-P`** then type **`>`** — switches the file palette to the command
list (VS Code semantics).
Every pref appears carrying its current state:
```
> save on idle: on
format on save: on
line numbers: on
theme: light
auto sync: 10m
```
`Ctrl-N`/`Ctrl-P` move the selection; **Enter** advances the selected pref to
its next value, applies it at once, writes the change back to `.typoena.toml`,
and confirms the new state on the snackbar (e.g. `theme: dark - saved`). A
boolean flips; the theme and auto-sync interval **rotate through their preset
options and wrap** — same key, so the palette is uniformly "press Enter to
change". **The list stays open** so you can change several prefs in a row;
**Esc** (or `Cmd-P`) closes it. Each change rides the next `:sync` to your
other devices.
`auto_sync` is a value command now, but has no behaviour to drive until v0.7 —
cycling it sets the interval that the future periodic push will honour.
## Parsing
The reader is a deliberately tiny **line-based** parser, not a general TOML
library — the file is flat `key = value` pairs (a bool, or a quoted string) with
`#` comments, so a full TOML crate isn't worth pulling onto the firmware build.
It lives in the host-testable `editor` crate (`Prefs::parse` / `Prefs::to_toml`).
Rules:
- A `#` starts a comment to end of line (whole-line or trailing).
- Blank lines and lines without `=` are ignored.
- An **unrecognized key** is ignored; an **unparseable value** (e.g.
`save_on_idle = yes`) leaves *that key* at its default rather than reading as
`false`.
- Any key not present falls back to its default, so partial files are valid.
Because `Prefs::to_toml` round-trips with `Prefs::parse`, a palette edit rewrites
the whole file in canonical form (with the header comment) — hand-added comments
elsewhere in the file are not preserved across a palette toggle.
## See also
- [`macroplan.md`](macroplan.md) — v0.5 scope and the decisions behind these keys.
- [`v0.1-mvp-product.md`](v0.1-mvp-product.md) — the `typoena.conf` device secrets
this file is kept separate from.
- [`tradeoff-curves/wifi-auto-sync.md`](tradeoff-curves/wifi-auto-sync.md) — why
`auto_sync` defaults to 10 minutes.

View File

@@ -8,7 +8,7 @@
> [technical design](v0.1-mvp-technical.md) (how it's built) ·
> [ADR log](adr.md) (load-bearing decisions) ·
> [QFD](qfd.md) (requirements ↔ functions ↔ components) ·
> [roadmap](roadmap.md) (where v0.1 sits in the sequence).
> [macroplan](macroplan.md) (where v0.1 sits in the sequence).
## One-line summary

View File

@@ -6,7 +6,7 @@
> Decisions referenced inline point at [`adr.md`](adr.md). Tradeoff weights
> and the critical-performance budget live in [`qfd.md`](qfd.md). Project
> overview: [`../README.md`](../README.md). Release sequence:
> [`roadmap.md`](roadmap.md).
> [`macroplan.md`](macroplan.md).
## Architecture

39
docs/v0.2-navigation.md Normal file
View File

@@ -0,0 +1,39 @@
# v0.2 — Vim navigation
> Part of the [Typoena macro plan](macroplan.md). Requirements and targets:
> [qfd.md](qfd.md). Load-bearing decisions: [adr.md](adr.md).
**Status:** COMPLETE 2026-07-11. Navigation done in core; the **UTF-8-correct
buffer** and **`Ctrl-d/u` half-page scroll** landed and are hardware-verified,
and the **absolute line-number gutter** is built, host-tested, and **confirmed
on the panel (Spike 13) 2026-07-11** — a single-line edit repaints only the rows
at/below the change and forces no extra full refresh. Shipped early beyond scope:
a read-only **View** mode and the full `d`/`c` operator + text-object grammar
(see [v0.3](v0.3-editing.md) / [v0.4](v0.4-visual-and-ex.md)).
- [x] Mode state machine (Normal / Insert / View), mode indicator in the status strip
- [x] Movement: `h j k l`, `w b e`, `0 $`, `gg G`, `Ctrl-d Ctrl-u`. `Ctrl-d/u`
step **display** (soft-wrapped) rows, not logical lines — half a page is
half the visible window however prose wraps; decoded as `HalfPageDown/Up`
intents in the keymap, caret moves and the viewport follows.
- [x] `i a o O A` to enter Insert
- [x] `Esc` returns to Normal
- [x] Line numbers in the left gutter: **absolute**, built + host-tested
2026-07-11, **confirmed on the panel (Spike 13) 2026-07-11** — numbered on a
logical line's first display row, blank on wrapped continuation rows; the
gutter width tracks the buffer's line count (2 digits + separator, widening
past 99 lines) and steals its columns from the soft-wrap. **Always on** in
v0.2; the on/off toggle rides the [v0.5](v0.5-palette-and-multi-file.md)
`.typoena.toml` prefs.
Relative numbering was dropped (2026-07-11): renumbering the whole gutter on
every `j`/`k` burns the e-ink ghosting budget for no proportionate gain,
whereas absolute renumbers only the rows below an edit — the on-panel check
confirmed a single-line edit repaints only rows at/below it with no extra
full refresh.
- [x] Groundwork — UTF-8-correct buffer: caret motions and edits step by
character, not byte (dropped the ASCII == byte-offset assumption), so every
motion stays correct with accented input. **Done 2026-07-11** alongside
extracting the editor into a host-testable crate — char-step
motions/deletes, byte-vs-char split in `layout`/`caret_rc`, `word_end`/`de`
fixed; 15 host tests. Render font is ISO-8859-15 (Latin-9), so accented
glyphs display.

View File

@@ -0,0 +1,25 @@
# v0.2.5 — International input
> Part of the [Typoena macro plan](macroplan.md). Requirements and targets:
> [qfd.md](qfd.md). Load-bearing decisions: [adr.md](adr.md).
**Status:** DONE in core, **hardware-verified 2026-07-11** (typed ç é è ñ on the
bench, no crash). US-International dead-key accent composition lives in the
`keymap` crate — a `Composer` downstream of the decoder — wired into
`usb_kbd.rs` so the editor still receives a single `Key::Char`. Builds on the
[v0.2](v0.2-navigation.md) UTF-8-correct buffer and the ISO-8859-15 render font.
Host-tested.
- [x] Dead keys — grave, acute, circumflex, diaeresis, tilde — compose with
the next letter: à é ê ë ñ, ç (via `'`+c), both cases
- [x] `'`+space emits a literal apostrophe (the everyday apostrophe path); a
dead key followed by a non-composing letter emits the accent then the
letter
- [x] A non-character event (Enter, Backspace, arrows) flushes any pending
accent as its literal first
- [ ] ~~Pending-accent indicator in the side-panel status strip~~ — **DROPPED
(2026-07-11 decision):** at typing speed it would be stale before the
~630 ms panel repaint, so it conveys nothing. Left unbuilt on purpose.
- [x] Bonus (2026-07-11): the physical **Esc key** (HID 0x29) now types
`` ` ``/`~` — Esc comes from the Caps tap — so grave/tilde accents and
Markdown code fences are reachable on a 60% board without a Fn layer.

30
docs/v0.3-editing.md Normal file
View File

@@ -0,0 +1,30 @@
# v0.3 — Vim editing
> Part of the [Typoena macro plan](macroplan.md). Requirements and targets:
> [qfd.md](qfd.md). Load-bearing decisions: [adr.md](adr.md).
**Status:** COMPLETE in core 2026-07-11, host-tested (65 editor + 28 keymap
tests) and **partially smoke-tested on the panel 2026-07-11**. The three
remaining pieces landed together: a single unnamed **register** with
`y`/`yy`/`p`/`P` (and `x`/`d`/`c` filling it, so `dd``p` moves a line),
**undo/redo** (`u`/`Ctrl-r`, snapshot-based, bounded to 100 groups in PSRAM — a
whole Insert session undoes as one group), and **`.` repeat** (keystroke-recorded,
so it replays an insert session like `ciwfoo<Esc>`). The `d`/`c` operator grammar
and text objects had already landed ahead of schedule. On device, `dd`, `yy`, and
`Ctrl-r` confirmed good; the one issue found was that a **multi-line paste near
the bottom left its later lines below the fold** — `adjust_scroll` only kept the
caret's (first) pasted line visible. Fixed by a `reveal()` that scrolls the end of
the pasted block into view while the caret stays on its first line (reflash to
re-confirm on panel).
- [x] `x dd`, `dw dd d$` (✓); `yy p P` (✓) and `.` repeat (✓) — register + a
keystroke-recorded last-change both landed 2026-07-11
- [x] Undo / redo (`u`, `Ctrl-r`) — snapshot history bounded to 100 groups in
PSRAM; one Insert session = one undo group
- [x] Numeric prefixes (`3dd`, `5j`)
- [x] Ahead of schedule: `c` change operator + text objects
(`ciw`, `di(`, `ca"`, … — inner/around, nesting-aware)
Known limits (deferred): `.` drops a *leading* count (`3x` then `.` deletes one;
a count inside an operator like `d2w` is kept); no named registers; `.` after an
aborted operator (`d<Esc>`) is a no-op.

View File

@@ -0,0 +1,34 @@
# v0.4 — Visual mode + ex commands
> Part of the [Typoena macro plan](macroplan.md). Requirements and targets:
> [qfd.md](qfd.md). Load-bearing decisions: [adr.md](adr.md).
**Status:** COMPLETE in core 2026-07-11, host-tested (83 editor tests), on-device
smoke-test pending. Charwise **Visual** (`v`) and linewise **VisualLine** (`V`)
selection landed with `y`/`d`/`c` on the span: charwise is vim-inclusive of the
char under the further caret, linewise spans whole logical lines and fills the
register linewise (so `Vy``p` copies a line, `Vd` deletes it like `dd`). Motions
(`h j k l`, `w b e`, `0 $`, `gg G`, `Ctrl-d/u`) and counts extend the selection;
`v`/`V` toggle/switch submode, `Esc` cancels. The selection renders as
reverse-video cells (black fill, glyphs redrawn white) — the only selection
affordance on a 1-bit panel — with the caret cell punched back to *normal* video
so the active end stands out. The Normal-mode motions were factored into a shared
`move_by` helper so Normal and Visual can't drift.
**DECISION (2026-07-07, resolved 2026-07-11):** `v`/`V` = **Visual** selection
(vim-standard). The read-only **View** (reading/scroll) mode that used to sit on
`v`/`V` moved to **`gr`** (go-read) — a `g`-prefixed gesture reusing the existing
pending-`g` machinery, no vim clash. View mode stays; `v`/`V` are now Visual.
- [x] Visual char (`v`) and line (`V`) modes, `y d c` on selections — landed
2026-07-11 (18 new tests). Known limits (deferred): no `o` swap-ends, no
`x`/`s` operator aliases, no Visual `.` repeat, no `:'<,'>` range commands.
- [~] `:` command line (mechanism ✓; `:w`/`:wq`/`:x` save, `:fmt`/`:sync`/`:gl`
wired; `:q` deliberately dropped — nothing to quit to). Command-line
editing added 2026-07-11: Ctrl-W deletes the previous word, Cmd-Backspace
clears the line. **`:e <path>` deferred to [v0.5](v0.5-palette-and-multi-file.md)** — opening another file
needs host file-IO + buffer switching, which is v0.5's multi-file work
(gated behind Spikes 11/14); half-building it here ahead of its
dirty-buffer handling wasn't worth it.
- [x] Ahead of schedule / unscheduled: `:fmt` Markdown formatter
(table alignment, blank-line collapse, trailing-whitespace strip)

View File

@@ -0,0 +1,267 @@
# v0.5 — File palette + multi-file
> Part of the [Typoena macro plan](macroplan.md). Requirements and targets:
> [qfd.md](qfd.md). Load-bearing decisions: [adr.md](adr.md). Prefs reference:
> [typoena-toml.md](typoena-toml.md).
**Status:** buffer **foundation** landed in core 2026-07-11 (slice 1 of 4),
host-tested; the palette + transient panel (Spike 11) and delete → git-staging
(Spike 14) remain the on-device gates. The single-file `Effect` return became a
drained **effect queue** (`Save{path,contents}` / `Load{path}` / `Publish` /
`Pull`), so one action can ask the host for several steps in order — opening a
non-resident file queues a `Save` of the outgoing dirty buffer *then* a `Load` of
the target. The multi-buffer state deliberately avoids a rope-per-buffer rewrite:
the active buffer keeps its fields inline on `Editor`, inactive buffers park in a
small LRU `Vec<Buffer>` (≤ 3 resident = active + 2), and a switch marshals fields
in/out so the ~3k-line editing engine is untouched. A dirty parked buffer is
saved before it is evicted (nothing leaves RAM unsaved); `:e <path>` opens by
prefix (`/sd/repo` → Tracked, `/sd/local` → Local); `:sync` is refused in-core in
a Local buffer. Firmware drains the queue **to empty** each batch (a `Load` can
cascade an eviction `Save`), and `persistence::{load_path,save_path}` generalise
the atomic save off the hard-coded `notes.md`.
**Slice 2 of 4 landed in core 2026-07-11**, host-tested: the `Cmd-P` file
**palette** — a modal transient panel over the writing column with a bare
fuzzy-search input (no `>` prefix: `>` is reserved for the command palette,
slice 4 — VS Code semantics), the ranked list, and the selected row in reverse
video. A pure host-testable fuzzy matcher (`fuzzy_score`: subsequence match,
boundary + consecutive-run bonuses, no penalties) ranks results; an in-core MRU
floats recently-opened files to the top on an empty query and is **shared with
`:e`** (both flow through `open_path`). The host feeds the file list once at boot
(`set_file_list`, enumerating `/sd/repo` + `/sd/local`, dotfiles skipped);
`Ctrl-n`/`Ctrl-p` (fzf-style; `Ctrl-d`/`Ctrl-u` too) move the selection — the
60 % board has no arrow keys — Enter opens via the same park/evict path as `:e`,
Esc (or `Cmd-P` again) closes. Same slice: **`Ctrl-n`/`Ctrl-p` also work as
down/up line motions in Normal mode** (vim `CTRL-N``j`, `CTRL-P``k`,
count-aware), which is why the palette opener moved to `Cmd-P` alone. Scope
shows as the inline `repo/…` vs `local/…` label rather than the planned
`[git]`/`[local]` badge — it also disambiguates subpaths, not just scope. 111
editor tests + 28 keymap tests pass; the no-git firmware binary builds clean.
The transient-panel refresh (**Spike 11**) is **CONFIRMED ON DEVICE 2026-07-12 —
no ghosting** (user flashed it and eyeballed the full-area partial the palette
forces); Cmd-P opens it on-device too. Remaining v0.5 slice: 4 prefs +
palette command mode.
**Slice 3 (`:enew` + delete) COMPLETE + CONFIRMED ON DEVICE 2026-07-12**
(committed `c9c0716`). `:enew <name>` creates a new file: empty, active, marked **dirty**
so eviction/`:w` persists it, and added to the in-core file list so the palette
finds it without a disk re-enumeration — no card IO until it is saved. `:delete`
unlinks the **current** file (a new `Effect::Delete` the host services), then
switches to the most-recently-parked buffer or an empty scratch; the discarded
buffer is never saved even when dirty. **Scope for a new file is read from the
path, not a modal prompt** — `local/x` / `repo/x` (the palette label form) select
the scope, a bare name uses the current buffer's scope. Same change made the
**`/sd` prefix optional everywhere** in `resolve_path`: `/sd/repo/x`, `/repo/x`,
and `repo/x` all name one file and nothing resolves outside `/sd` (the writer
can't reach anything else). **Spike 14 (delete → git-staging) DID need a firmware
fix.** The first on-device test found `add_all(["*"])` alone does **not** stage a
deletion on this libgit2 build (the tree came back unchanged, so the second push
was a silent "up to date" no-op — the "delete didn't work" report). Fix:
`stage_and_commit` now runs `add_all` **then `update_all(["*"])`** (`git add -u`),
which removes index entries whose working-tree file is gone — together they are
`git add -A`. Also, `:delete` gave no clear feedback, so the snackbar now names
the scoped file and, for a Tracked file, that it is local until `:sync`
(`deleted repo/notes.md - :sync to publish`). Deferred to later: greying the
Publish affordance for a Local buffer, and the multi-file publish count. 123
editor tests + 28 keymap tests pass; the no-git firmware binary builds clean. The
`update_all` fix (behind `--features git`, unbuildable locally) was **verified on
device 2026-07-12** — `:enew test.txt``:sync``:delete``:sync` removed
test.txt from origin.
**Slice 4 (`.typoena.toml` prefs + palette `>` command mode) COMPLETE in core
2026-07-12, HOST-TESTED not yet on-device.** A `Prefs` type (host-testable
line-based TOML parse/serialize — flat `key = value` bools + one string with `#`
comments, no crate pulled onto xtensa) lives on `Editor`; the host reads
`/sd/repo/.typoena.toml` at boot and applies it before the first render, and a
missing/partial file falls back to per-key defaults. Keys: `save_on_idle`,
`format_on_save`, `line_numbers` (all bool, default on) and `auto_sync`
(string, default `"10m"`, **schema + default only** — nothing reads it yet).
`line_numbers` is live: `gutter_cols()` returns 0 when off, so the text reclaims
the gutter's columns (the `gutter - 1` field width made saturating to avoid the
underflow). The palette `>` command mode (VS Code semantics — a leading `>` in
the query switches file search to the command list) exposes the three booleans
as live toggles; Enter flips the pref, applies it at once, queues a new
`Effect::SavePrefs` (the editor serializes; the host does the atomic write to
`.typoena.toml`, which rides the next `:sync` to other devices), and confirms
the new state on the snackbar. **The list stays open after a toggle** so several
prefs flip in one visit (Esc/`Cmd-P` closes); **`:settings` opens the palette
straight into `>` mode** as a one-command shortcut (both requested by the user
2026-07-12, chosen over a separate settings modal — same surface, no duplicate
machinery). Committed `c535864`. **Three "decide before build" calls:** (1) the
idle auto-save is **unformatted**`:fmt` runs only on explicit `:w`/`:sync`, so
tables/blank-lines are never reflowed mid-session; (2) the per-device `auto_sync`
override (card-local `typoena.conf`) is **deferred** — auto_sync is inert in
v0.5, so there is nothing yet to override; (3) `> auto sync: <dur>` as a palette
command is **deferred to v0.7** — a control that changes a value nothing reads
would be a dead switch. `save_on_idle` is honoured host-side: a silent idle
auto-save (no snackbar, no forced e-ink flash — a safety net, not an action)
fires once per typing burst after a 1.5 s pause. 141 editor tests + 28 keymap
tests pass; the no-git firmware binary builds clean. Firmware bumped **0.4.0 →
0.5.0** (the v0.5 feature set is met). **Boot-read of the prefs file CONFIRMED ON
DEVICE 2026-07-12** — a `.typoena.toml` in `typoena-test` with non-default values
(`save_on_idle=false`, `line_numbers=false`, `auto_sync="5m"`) logged back
`prefs: Prefs { save_on_idle: false, format_on_save: true, line_numbers: false,
auto_sync: "5m" }` at boot, a byte-exact parse (comments skipped, bools + quoted
string read). **Full gate CLOSED 2026-07-12:** the palette `>` live-toggle
round-trip is confirmed — origin's `.typoena.toml` went `line_numbers` false →
true via a *device*-authored publish (`3c79f38`), proving toggle → `SavePrefs`
atomic write → `git add -A` → push — and the `save_on_idle` autosave works on
device too. **v0.5 slice 4 fully DONE + on-device confirmed.**
**Amendment 2026-07-12 — non-boolean prefs (`theme`, `auto_sync`).** Two of the
"decide before build" calls above are **superseded**: `auto_sync` is now a live
palette command, and a new `theme` (`light`/`dark`) key ships. The generalising
idea is that a boolean toggle is just the two-option case of *rotate through a
preset list on Enter* — so the palette gains one uniform gesture: **Enter
advances the selected pref to its next value and wraps** (a bool flips; a string
pref cycles its options). `theme` rotates `light``dark` and is applied by a
single whole-frame invert at the end of the render ([`Frame::invert`]), so text,
selection, caret, panel and palette all flip together; `auto_sync` rotates
`2m`/`5m`/`10m`/`15m`/`30m`. Decision (3) — "a value control that changes nothing
readable would be a dead switch" — is knowingly overridden: cycling `auto_sync`
persists and displays the interval, but **still drives no behaviour until v0.7**;
we accept a set-ahead control so the surface is ready and the value syncs now.
Decision (2) (per-device `typoena.conf` override) stays deferred. Kept simple: no
enum machinery — both string prefs share a `next_option(current, &OPTIONS)`
helper, and hand-editing the TOML can still set any value (the palette only
cycles presets; an off-list value snaps to the head on the next Enter). Editor
tests cover the rotate/wrap/snap, the live theme invert, and the round-trip.
**Trailing-newline handling — a saved note ends with a *visible* blank line
(commit `d14d9e7`, 2026-07-12; host-tested, on-device gate open).** Two
adjustments to how the buffer meets the file. First, format-on-save no longer
strips *every* trailing blank line — it collapses a run to **at most one** and
keeps it, so a writer who presses Enter to open the next line doesn't have that
line (and the caret) yanked away on save (the caret used to jump up to the last
non-empty line). Second, persistence treats the file's POSIX terminator as
content the editor *shows*: `load_path` reads the file **verbatim** and
`save_path` writes the buffer, appending a final newline **only if one is
missing** (guarded, not unconditional). Because the editor lays out
`rows = #\n + 1`, that terminating newline renders as a **visible trailing empty
line** the caret can land on — open a note and the blank line the newline stands
for is there. The two are an identity round-trip for any device-written file
(all end in `\n`); the file stays git-clean (exactly one terminator — vim and
GitHub show no phantom blank line); and a trailing blank line the writer leaves
is mirrored, never doubled. This replaced an interim model that stripped the
terminator on load and hid it (the file was correct, the newline just wasn't
shown). On-device check: reflash → open a note (trailing empty line visible) →
`:w` (caret stays on it) → reopen (still there). `Prefs::to_toml` ends in a
newline for the same reason; the guarded save leaves the prefs file with exactly
one.
**Amendment 2026-07-13 — recursive enumeration + a 2-char search threshold.**
Loading a real repo (`jcalixte/notes`) exposed that `enumerate_files` listed
only the **top-level** files of `/sd/repo` and `/sd/local` — a nested notes tree
showed a single file in the palette (subpaths always *opened* fine via
`:e repo/sub/x.md`; only the listing was flat). The enumeration is now a
recursive walk: dot entries are skipped at every level (so `.git` is never
descended into), each directory is read fully before recursing (one FatFS dir
handle open at a time — the `remove_dir_recursive` pattern, kind to the
FD-bounded mount), depth is capped at 8, and the boot-time walk logs its file
count and duration (`file walk: N files in Xms`) so the FAT dir-IO cost on a
big repo is measurable, not assumed. With the list now card-sized, the palette
gained a **search threshold**: below 2 typed chars the result list is the
**recents (MRU) only** — quick-switch (`Cmd-P`, `Enter`) stays one keystroke
away — and the full fuzzy-ranked list appears from 2 chars on
(`PALETTE_MIN_QUERY`). A fresh boot with no opens yet shows `(type to search)`.
`>` commands and `$` snippets are short curated lists; the threshold does not
apply to them.
- [x] `Cmd-P` opens fuzzy file palette over **both** `/sd/repo/` and
`/sd/local/`**landed and CONFIRMED ON DEVICE 2026-07-12** (Spike 11: no
ghosting on the transient panel); scope shows as the inline
`repo/…` / `local/…` label instead of a `[git]`/`[local]` badge.
- [~] Open, switch, close buffers (keep ≤ 3 in memory) — **open + switch + the
≤ 3 LRU-resident model with dirty-aware save-before-evict done in core**
(host-tested); `:e <path>` **and the palette** drive it today. Explicit
**close** still to come.
- [x] `:e` and palette share the same recent-files list — both open via
`open_path`, which pushes to the in-core MRU that orders the palette.
- [x] `:enew` creates a new file — **done in core (host-tested) 2026-07-12.**
Scope is read from the path (`local/x` / `repo/x` select it, the palette
label form; a bare name uses the current scope) rather than a modal
prompt — the resolved scope is echoed in the snackbar. The `/sd` prefix is
optional throughout (`/sd/repo/x` = `/repo/x` = `repo/x`).
- [x] Delete a file — **core done (host-tested) 2026-07-12;** `:delete` unlinks
the current file via `Effect::Delete`. For a Tracked file the removal reaches
the next `:sync` Publish's staged set. **Spike 14 (on-device) found the
staging incomplete:** `add_all(["*"])` alone did not stage the deletion, so
`stage_and_commit` now also runs `update_all(["*"])` (`git add -u`) — the
two together are `git add -A`. A Local file is just unlinked. The snackbar
now confirms the delete and flags that a Tracked file needs `:sync`.
**Verified on device 2026-07-12** — the `:enew``:sync``:delete``:sync`
cycle removed test.txt from origin.
- [~] `Ctrl-G` is disabled / hidden when the current buffer is local-scope —
**`:sync` / Publish is blocked in-core for a Local buffer** (posts "Publish
unavailable (Local)"); the side-panel affordance that hides/greys the
gesture is the remaining half.
- [ ] The side panel briefly shows file count on `Ctrl-G` when the publish bundles
more than one dirty Tracked file (e.g. `"publishing 3 files: abc1234"`),
so workspace-scoped behaviour stays visible to the user
- [x] **Preferences file** `/sd/repo/.typoena.toml` — a git-tracked,
hand-editable TOML file for editor behaviour, deliberately **distinct from
the `/sd/typoena.conf` card secrets** (Wi-Fi / PAT / remote / author,
gitignored, never committed — see [v0.1](v0.1-mvp-product.md)). Read at boot; a missing file or
key falls back to the defaults below. **Core done 2026-07-12** (a `Prefs`
type on `Editor`, host-testable parse/serialize, applied via
`Editor::set_prefs` before the first render); full reference:
[`typoena-toml.md`](typoena-toml.md). Keys:
- [x] `save_on_idle` (bool, default `true`) — auto-save the current buffer on
the idle typing-pause, so `:w` becomes optional rather than required.
**Honoured host-side** as a *silent* save (no snackbar, no forced e-ink
flash — a safety net, not an action), unformatted, once per typing burst
after a 1.5 s pause.
- [x] `format_on_save` (bool, default `true`) — run `:fmt` (table alignment,
blank-line collapse, trailing-whitespace strip) on the buffer before it
is persisted, so `:sync` is **fmt → save → commit → push** and `:w`
saves formatted. Implemented in-core 2026-07-11 (`Editor`), now **driven
by this key**. **Open question RESOLVED (2026-07-12):** fmt runs only on
an explicit `:w`/`:sync`; the `save_on_idle` auto-save is deliberately
**unformatted**, so tables/blank lines are never reflowed mid-session
(the caret would jump under you on every thinking pause).
- [x] `line_numbers` (bool, default `true`) — show the absolute line-number
gutter (built always-on in v0.2). Off reclaims the gutter's columns for
text (`gutter_cols()` → 0); the palette `> line numbers: on/off` command
toggles it live. **Done 2026-07-12.**
- [x] `theme` (string, default `"light"`; options `light`/`dark`) — panel
colour polarity. `dark` is a single whole-frame invert at the end of the
render (`Frame::invert`), so everything flips together and stays legible.
Palette `> theme` rotates `light``dark` live. **Added 2026-07-12
(amendment); host-tested, on-device check pending.** Caveat: `dark`
partial-refreshes ghost more on e-paper than `light` — verify on panel.
- [ ] `auto_sync` (duration string, default `"10m"`; `"0"` / omitted
disables; **min clamp ~`"2m"`** so a palette typo can't drain the
battery) — a *max-staleness cap*, not a wall-clock timer:
**opportunistic, rate-limited** Publish. Push when already awake + dirty
(coalesced into the idle-pause, ≤ once per `auto_sync`) and once on the
way into sleep if dirty; **never wake from deep sleep purely to sync**.
Wi-Fi energy is a `1/T` curve whose knee sits at 510 min, and
`save_on_idle` already owns local data safety — so 10 min halves the
sync energy of a 5-min default for no real risk. Full derivation:
[`tradeoff-curves/wifi-auto-sync.md`](tradeoff-curves/wifi-auto-sync.md).
The **schema + default (`"10m"`) live here in v0.5** and round-trip
through `Prefs`; **nothing reads the value yet** — the periodic side
rides the better-git work (v0.7) and must interact with light / deep
sleep (v0.8). Marked `[~]`: parsed and preserved, no behaviour.
**Amended 2026-07-12:** now a palette preset command — Enter rotates it
through `2m`/`5m`/`10m`/`15m`/`30m` (the `~2m` min is baked into the
preset list). Set-ahead only: still read by nothing until v0.7.
- [x] Open question RESOLVED (2026-07-12): the per-device sync cadence override
(a card-local `typoena.conf` layer over the committed prefs) is
**deferred**`auto_sync` is inert in v0.5, so there is nothing yet to
override; revisit when v0.7 makes the periodic push real.
- [x] **Palette command mode** — typing `>` at the `Cmd-P` palette switches it
from file search to a command list (VS Code-style). **Done in core
2026-07-12.** The v0.5 commands toggle the three boolean `.typoena.toml`
prefs — `> save on idle`, `> format on save`, `> line numbers` — each label
carrying its live state; Enter flips the pref, applies it at once, queues
`Effect::SavePrefs` (persist to the file), and confirms on the snackbar.
**The list stays open after a toggle** (flip several, Esc/`Cmd-P` closes),
and **`:settings` opens it directly** — both added 2026-07-12 as the "change
config from the device" surface (chosen over a separate settings modal).
This command list is the discoverable surface later actions (`:fmt`, font)
also register into. **Amended 2026-07-12:** the palette now also carries the
non-boolean prefs `> theme` (`light`/`dark`, live whole-frame invert) and
`> auto sync` (`2m`..`30m`), both cycled by the same Enter-rotates-to-next
gesture. `auto_sync` is exposed **set-ahead** (no behaviour until v0.7),
knowingly overriding the earlier "dead switch" call.

112
docs/v0.6-markdown.md Normal file
View File

@@ -0,0 +1,112 @@
# v0.6 — Markdown affordances
> Part of the [Typoena macro plan](macroplan.md). Requirements and targets:
> [qfd.md](qfd.md). Load-bearing decisions: [adr.md](adr.md).
**Status:** render affordances done early. The snippet engine (tab-stop core,
inline Tab-expansion, hint-on-pause, `$` palette) and the `>` command-palette
generalisation are **done in core, host-tested** (187 editor tests); the
remaining work is the **firmware boot-read + `just init` catalog** (slice 5, the
on-device gate). Snippets are net-new scope, added 2026-07-08; reshaped
2026-07-12 from a hard-coded table into a git-synced, Zed-compatible library with
a `$` palette launcher — see [`typoena-snippets.md`](typoena-snippets.md) for the
file format.
- [x] Heading lines bolded in render (faux-bold double-strike)
- [x] List continuation on Enter inside `- ` / `1. ` (with empty-item exit)
- [x] Soft-wrap at word boundaries
## Snippets
Trigger-driven text expansion for Markdown authoring (Zed-inspired, but **no
completion popup**: e-ink's ~630 ms refresh rules out a live filtering menu, and
it fights the distraction-free premise). The library is a git-synced,
Zed-compatible JSON file — full file-format reference in
[`typoena-snippets.md`](typoena-snippets.md). This section is the *editor*
behaviour.
- [x] **The tab-stop engine** — the shared core both surfaces drive. A body is
literal text plus numbered stops `$1 … $n` and a final `$0`; on insertion
the caret lands on `$1` (or the end), in Insert. Tab advances to the next
stop, **forward only** (no Shift-Tab); pending stops sit after the caret
and shift with edits there. The session auto-aborts on Esc, a mode change,
or a motion that leaves the stops. `${n:label}` parses to a bare `$n` (the
label is stripped — no selection model to fill); no dynamic values (no RTC,
so no `date`).
- [x] **Inline Tab-expansion (Insert mode).** If the word immediately before the
caret matches a snippet prefix, Tab expands it and starts the tab-stop
session; otherwise Tab inserts spaces as today. A check in the Insert
handler alongside the existing `list_marker` transform
(`expand_snippet(word) -> Option<Snippet>`). Tab already arrives as
`Key::Char('\t')`, so no new key event.
- [x] **Hint-on-pause.** On the typing pause (same throttle as the word-count /
cursor refresh — never a per-keystroke repaint), if the word before the
caret is a prefix, the right side panel shows a quiet hint (`» name`, on the
row above the mode line, sharing the slot with the `NO KBD` flag — the two
never co-occur since the hint means you're typing). The panel is ~15 cols,
so the hint is the snippet **name**, not the whole body — the full preview is
the `$` palette's job. Snapshotted in `refresh_stats` so it rides the pause,
and the firmware's Insert-pause repaint already carries it (no firmware
change). Latin-9 has no `↹` glyph, hence the `»` marker.
- [x] **`$` palette (browse + insert).** `Cmd-P` then `$` switches the palette to
the snippet list (the same sigil mechanism as `>`). Fuzzy-matches name /
prefix / description; Enter inserts the body at the caret and starts the
tab-stop session. Lists **all** snippets — the fuzzy filter handles clutter,
so there's no hidden "inline-only vs palette-only" split. Rows read
`Name [prefix]`, so browsing also teaches the inline trigger.
- [x] **Boot wiring.** The host reads `/sd/repo/.typoena.snippets.json` at boot
and calls `Editor::set_snippets` (mirroring `set_prefs`); a missing or
malformed file is non-fatal (no snippets, editor runs). Parse lives in the
host-testable `editor` crate via `serde_json` — the one new dependency,
confirmed to build for xtensa (`cargo check`, firmware 0.6.0). On-device
smoke-test still pending.
## The palette, generalised (`Cmd-P` · `>` · `$`) — done in core
v0.5 shipped `Cmd-P` = files and `>` = a five-entry settings list (`save_on_idle`,
`format_on_save`, `line_numbers` toggles + `theme`/`auto_sync` rotations). v0.6
makes the sigils a clean split by verb, and the empty-palette placeholder legends
them: `Go to file · > settings · $ snippets`.
- **bare `Cmd-P`** → *navigate*: go to file (unchanged).
- **`>`** → *act on the editor* — the command palette, a real action registry (the
`PALETTE_CMDS` list, actions first then settings). The pref toggles are just its
stateful entries, not a special section. Dispatch is by `PaletteCmd::kind`: a
**toggle** flips and the list **stays open**; a **one-shot** (`format`,
`publish` — the latter shares `run_publish` with `:sync`) runs and **closes**; a
**parameterised** command (`new file...`) morphs the palette into a second
**input step** (the box becomes a filename prompt → Enter creates it, scope read
from a `repo/`/`local/` prefix as `:enew` does; backspacing past the start steps
back to the `>` list). This **retired `:e`** — bare `Cmd-P` covers file-opening;
dotfiles can get a dedicated `> edit ...` command if/when wanted.
- **`$`** → *insert content* — the snippet launcher above.
Labels avoid `…`/`↹` — the palette/panel fonts are ISO-8859-15, which has neither
glyph — so `new file...` uses ASCII dots and the pause hint leads with `»`.
## First-time setup — snippet catalog (`just init`) — done
[`just init`](../firmware/README.md#provisioning-an-sd-card) seeds the two
git-tracked config files into `repo/` (so they commit + sync on the device's
first `:sync`), **writing only a file that is absent** so a re-`init` never
clobbers a synced library: a starter [`.typoena.toml`](typoena-toml.md) (the five
keys at their defaults) and a [`.typoena.snippets.json`](typoena-snippets.md)
assembled from a **curated catalog** (`firmware/snippets-catalog/`, `jq`-merged).
The catalog is grouped and opt-in — you pick the groups (each `[Y/n]`, all-yes on
a non-interactive run) rather than getting one writer's whole personal set. Not
every Zed snippet is worth proposing: the Slidev/blog-pipeline ones (`@[youtube]`,
`<v-clicks>`, frontmatter, mermaid) and hyper-specific personal ones are left out,
since a distraction-free prose appliance can't render them and you'd never miss
them. Prefixes are **English** (except `edanso`, the mnemonic for `œ`), bodies
translated from the source Zed snippets. **Three groups, 17 snippets:**
- [x] **Symbols** (inline, the keyboard can't type them): `arrow``→`,
`neq``≠`, `times``×`, `middot``·`, `deg``°`, `euro``€`, `edanso``œ`
(dead keys in v0.2.5 cover accents, *not* these). **Caveat:** `→` (U+2192)
and `≠` (U+2260) are outside ISO-8859-15, so they store correctly and sync
but render as a missing-glyph box on the device panel until the font is
extended; the other five are Latin-9 and draw fine. Kept deliberately.
- [x] **Structure**: `todo``- [ ] `, `link``[$1]($2)$0`, `img``![$1]($2)$0`,
`table` (2-col), `code` (fenced block). `img`/`code` are net-new.
- [x] **Prose / PKM templates** (`${n:label}` stripped to `$n`): `booknotes`,
`reference` (the English reference block — `refangl` folded in), `bias`,
`capture`, `standard`. (`5w1h` dropped.)

View File

@@ -0,0 +1,14 @@
# v0.7 — Search + better git
> Part of the [Typoena macro plan](macroplan.md). Requirements and targets:
> [qfd.md](qfd.md). Load-bearing decisions: [adr.md](adr.md).
**Status:** the **`:gl` pull command landed in the editor** (2026-07-11,
host-tested) — `Effect::Pull` + a firmware stub; the on-device fetch +
fast-forward is still to build. Search not started.
- [ ] `/` forward search, `n N`
- [~] `:gl` — pull: fetch + **fast-forward only**, refuse on divergence and
surface it (renamed from the planned `:Gpull`). Editor command +
`Effect::Pull` done 2026-07-11 (host-tested); the git-thread
fetch/fast-forward in `git_sync` remains (only push is wired today).

View File

@@ -0,0 +1,12 @@
# v0.8 — Power: battery + sleep
> Part of the [Typoena macro plan](macroplan.md). Requirements and targets:
> [qfd.md](qfd.md). Load-bearing decisions: [adr.md](adr.md).
**Status:** not started.
- [ ] Measure idle / typing / push current draw on bench
- [ ] 18650 + IP5306 charge board, soft power switch
- [ ] Light sleep on idle > 30 s (keyboard interrupt wakes)
- [ ] Deep sleep on lid close (reed switch); restore cursor + buffer
- [ ] Battery indicator in the side panel

14
docs/v0.9-robustness.md Normal file
View File

@@ -0,0 +1,14 @@
# v0.9 — Robustness
> Part of the [Typoena macro plan](macroplan.md). Requirements and targets:
> [qfd.md](qfd.md). Load-bearing decisions: [adr.md](adr.md).
**Status:** not started.
- [ ] Crash-safe writes (write to `.tmp`, fsync, rename)
- [ ] Recover from interrupted push (re-attempt on next save)
- [ ] SD card removal / reinsert handling
- [ ] Wi-Fi reconnect with backoff
- [ ] On-device provisioning + settings screen: SSID, PAT rotation, default
remote, commit author (replaces the v0.1 dev-only NVS-flashing path —
first release usable by someone who is not the firmware author)

16
docs/v1.0-polish.md Normal file
View File

@@ -0,0 +1,16 @@
# v1.0 — Polish
> Part of the [Typoena macro plan](macroplan.md). Requirements and targets:
> [qfd.md](qfd.md). Load-bearing decisions: [adr.md](adr.md).
**Status:** not started.
- [ ] Boot time ≤ 3 s to usable cursor — currently ~4.26 s; the ~1.9 s cold-boot
full refresh is a hard e-ink floor, so ≤ 3 s is marginal (see
[`notes/boot-time-budget.md`](notes/boot-time-budget.md))
- [ ] Font selection (at least one serif + one mono) with adjustable font
size, switchable at runtime and persisted across reboots
- [ ] Theme: light / dark (inverted e-ink), switchable at runtime and
persisted across reboots
- [ ] Enclosure design files in `hardware/`
- [ ] User guide

11
docs/v1.x-stretch.md Normal file
View File

@@ -0,0 +1,11 @@
# v1.x — Stretch / nice-to-have
> Part of the [Typoena macro plan](macroplan.md). Requirements and targets:
> [qfd.md](qfd.md). Load-bearing decisions: [adr.md](adr.md).
Post-1.0 ideas, not committed to any release:
- 10.3" panel upgrade via IT8951
- Multiple remotes / repos
- Stats: words today, streak
- BLE-HID fallback for wireless keyboards

View File

@@ -8,3 +8,8 @@ description = "Modal (vim-style) text-editor core: the buffer, motions, edits, a
embedded-graphics = "0.8"
display = { path = "../display" }
keymap = { path = "../keymap" }
# Snippet library parse (v0.6): the `.typoena.snippets.json` file uses Zed's JSON
# shape. serde_json handles the string-escape corner cases a hand-rolled reader
# would get wrong; the crate is `std` (esp-idf provides std on xtensa).
serde = { version = "1", features = ["derive"] }
serde_json = "1"

File diff suppressed because it is too large Load Diff

View File

@@ -3,7 +3,7 @@ target = "xtensa-esp32s3-espidf"
[target.'cfg(target_os = "espidf")']
linker = "ldproxy"
runner = "espflash flash --monitor"
runner = "espflash flash --monitor --baud 921600"
rustflags = [ "--cfg", "espidf_time64"]
[unstable]

View File

@@ -1,6 +1,6 @@
[package]
name = "firmware"
version = "0.1.0"
version = "0.6.0"
authors = ["Julien Calixte <juliencalixte@gmail.com>"]
edition = "2024"
resolver = "2"
@@ -36,6 +36,22 @@ name = "sd_fat"
path = "src/bin/sd_fat.rs"
harness = false
# SD/FAT primitive-op micro-benchmark — attributes the ~700 ms/loose-object write
# floor (see docs/tradeoff-curves/sync-commit-staging.md). Pure SD, no git feature.
# Flash with `just flash-bench`.
[[bin]]
name = "sd_bench"
path = "src/bin/sd_bench.rs"
harness = false
# git-level micro-benchmark — localizes the ~700 ms/object libgit2 overhead (see
# docs/tradeoff-curves/sync-commit-staging.md). Needs the `git` feature.
[[bin]]
name = "git_bench"
path = "src/bin/git_bench.rs"
harness = false
required-features = ["git"]
# Spike 9 — boot splash. Standalone bench program that paints the Typoena
# wordmark-in-a-circle on the EPD. No git, no SD, no Wi-Fi. Flash with
# `just flash-splash`.

View File

@@ -7,9 +7,19 @@
* Allocations go through git__malloc, so with PSRAM in the heap they land in
* the 8 MB external RAM rather than the ~340 KB internal DRAM.
*
* Limitation: writable/shared mappings are not written back (libgit2 does not
* mmap for writing in the paths we exercise). If that ever changes it will
* surface at runtime, not here.
* There is deliberately NO cache here. A window cache was built and removed
* on 2026-07-12: across four instrumented real-repo bench runs it scored
* exactly 0 hits, because libgit2's mwindow layer reuses its open windows and
* only genuinely new (offset, len) ranges ever reach p_mmap — and the one
* memory bug it "fixed" (7.4 MB resident starving zlib) was caused by the
* cache itself holding buffers past p_munmap. Free-at-munmap keeps
* GIT_OPT_SET_MWINDOW_MAPPED_LIMIT honest by construction: when libgit2
* releases a window, the memory really is back in git__malloc's pool. Full
* trail: docs/tradeoff-curves/sync-commit-staging.md (final-bench section).
* The stats counters below are kept so a future workload that *does* repeat
* ranges (push? reconcile?) can be spotted before anyone rebuilds a cache.
*
* Limitation: writable/shared mappings are not written back.
*/
#include "git2_util.h"
@@ -18,6 +28,7 @@
#include <unistd.h>
#include <string.h>
#include <errno.h>
#include <stdint.h>
int git__page_size(size_t *page_size)
{
@@ -31,10 +42,49 @@ int git__mmap_alignment(size_t *alignment)
return 0;
}
/* Diagnostics, read from the bench via esp_map_stats(). The signature predates
* the cache removal: `hits` is always 0, `misses` counts every mapping, and
* `cached_kb` now reports the LIVE mapped bytes (the mwindow working set). */
static uint32_t g_maps;
static uint64_t g_read_bytes;
static size_t g_live_bytes;
void esp_map_stats(uint32_t *hits, uint32_t *misses, uint32_t *read_kb, uint32_t *cached_kb)
{
if (hits) *hits = 0;
if (misses) *misses = g_maps;
if (read_kb) *read_kb = (uint32_t)(g_read_bytes / 1024);
if (cached_kb) *cached_kb = (uint32_t)(g_live_bytes / 1024);
}
static int read_range(int fd, off64_t offset, size_t len, unsigned char *data)
{
size_t got = 0;
if (lseek(fd, offset, SEEK_SET) < 0) {
git_error_set(GIT_ERROR_OS, "failed to seek for mmap emulation");
return -1;
}
while (got < len) {
ssize_t n = read(fd, data + got, len - got);
if (n < 0) {
if (errno == EINTR)
continue;
git_error_set(GIT_ERROR_OS, "failed to read for mmap emulation");
return -1;
}
if (n == 0)
break; /* short file: zero-fill the tail, like a real mapping */
got += (size_t)n;
}
if (got < len)
memset(data + got, 0, len - got);
return 0;
}
int p_mmap(git_map *out, size_t len, int prot, int flags, int fd, off64_t offset)
{
unsigned char *data;
size_t got = 0;
GIT_UNUSED(prot);
GIT_UNUSED(flags);
@@ -45,29 +95,13 @@ int p_mmap(git_map *out, size_t len, int prot, int flags, int fd, off64_t offset
data = git__malloc(len);
GIT_ERROR_CHECK_ALLOC(data);
if (lseek(fd, offset, SEEK_SET) < 0) {
git_error_set(GIT_ERROR_OS, "failed to seek for mmap emulation");
if (read_range(fd, offset, len, data) < 0) {
git__free(data);
return -1;
}
while (got < len) {
ssize_t n = read(fd, data + got, len - got);
if (n < 0) {
if (errno == EINTR)
continue;
git_error_set(GIT_ERROR_OS, "failed to read for mmap emulation");
git__free(data);
return -1;
}
if (n == 0)
break; /* short file: zero-fill the tail, like a real mapping */
got += (size_t)n;
}
if (got < len)
memset(data + got, 0, len - got);
g_maps++;
g_read_bytes += len;
g_live_bytes += len;
out->data = data;
out->len = len;
@@ -77,7 +111,12 @@ int p_mmap(git_map *out, size_t len, int prot, int flags, int fd, off64_t offset
int p_munmap(git_map *map)
{
GIT_ASSERT_ARG(map);
git__free(map->data);
if (g_live_bytes >= map->len)
g_live_bytes -= map->len;
else
g_live_bytes = 0;
map->data = NULL;
map->len = 0;
return 0;

View File

@@ -80,6 +80,32 @@ flash-sd:
monitor-sd:
espflash monitor --elf {{elf_sd}}
# SD primitive-op micro-benchmark — attributes the ~700 ms/loose-object write
# floor (docs/tradeoff-curves/sync-commit-staging.md). Writes only to /sd/sdbench.
build-bench:
{{esp_env}} cargo build --release --bin sd_bench
# build + flash + monitor the SD bench
flash-bench:
{{esp_env}} cargo run --release --bin sd_bench
# serial monitor for the SD bench, with decoded backtraces
monitor-bench:
espflash monitor --elf target/xtensa-esp32s3-espidf/release/sd_bench
# git-level micro-benchmark — localizes the ~700 ms/object libgit2 overhead
# (docs/tradeoff-curves/sync-commit-staging.md). Read-mostly on /sd/repo.
build-gitbench:
{{esp_env}} {{git_env}} cargo build --release --bin git_bench --features git
# build + flash + monitor the git-level bench
flash-gitbench:
{{esp_env}} {{git_env}} cargo run --release --bin git_bench --features git
# serial monitor for the git-level bench, with decoded backtraces
monitor-gitbench:
espflash monitor --elf target/xtensa-esp32s3-espidf/release/git_bench
# Spike 9 — build the boot splash spike (no .env needed)
build-splash:
{{esp_env}} cargo build --release --bin splash
@@ -112,7 +138,7 @@ build-git-push:
# (so the `storage` FAT partition exists) and sets 16 MB flash. Uses espflash
# directly, not `cargo run`, so the table is applied only to this binary.
flash-git-push: build-git-push
espflash flash --monitor --partition-table {{partition_table}} --flash-size 16mb {{elf_git_push}}
espflash flash --monitor --baud 921600 --partition-table {{partition_table}} --flash-size 16mb {{elf_git_push}}
# serial monitor for the git push, with decoded backtraces
monitor-git-push:
@@ -125,7 +151,7 @@ build-git-sync:
# Milestone #2A — flash + monitor. Applies the same custom partition table as
# git-push (the `storage` FAT partition holds the persistent clone).
flash-git-sync: build-git-sync
espflash flash --monitor --partition-table {{partition_table}} --flash-size 16mb {{elf_git_sync}}
espflash flash --monitor --baud 921600 --partition-table {{partition_table}} --flash-size 16mb {{elf_git_sync}}
# serial monitor for the git sync, with decoded backtraces
monitor-git-sync:
@@ -161,18 +187,21 @@ ports:
# The `_`-prefixed recipes are shared internals (hidden from `just --list`).
sd_repo_dir := "repo"
# Full prep of a fresh card: copy the notes repo + write config, then eject.
# Run this once per card. <repo-src> is a clone made on this computer.
# Full prep of a fresh card: copy the notes repo, seed the git-tracked config
# files (first-time only), write the runtime config, then eject. Run this once
# per card. <repo-src> is a clone made on this computer.
init repo_src sd_volume="":
#!/usr/bin/env bash
set -euo pipefail
vol="$(just _card "{{sd_volume}}" | tail -n1)"
just _load-repo "{{repo_src}}" "$vol"
just _seed-configs "$vol"
just _write-conf "$vol" "{{repo_src}}"
just _eject "$vol"
# (Re)copy just the notes repo to /sd/repo (full clone, gitignored paths
# excluded), then eject — e.g. to refresh the card after big upstream changes.
# (Re)copy just the notes repo to /sd/repo: fast-forward the source clone from
# its remote first, then rsync it across (full clone, gitignored paths excluded)
# and eject — e.g. to refresh the card after the device pushed new notes.
load repo_src sd_volume="":
#!/usr/bin/env bash
set -euo pipefail
@@ -241,7 +270,10 @@ _card sd_volume="":
# .gitignore ignores is excluded (node_modules is 3.9 GB, gitignored, never in
# .git) — the device needs .git + the checkout (~720 MB), not the JS deps or
# secrets like firmware/.env. Copying (not a fresh `git clone` of the local
# path) preserves origin → GitHub so on-device fetch/push still work.
# path) preserves origin → GitHub so on-device fetch/push still work — except
# the URL scheme: the card copy's origin is rewritten to HTTPS at the end,
# because the device's libgit2 has no SSH transport (HTTPS+PAT over mbedTLS
# only). The source clone is never touched.
_load-repo repo_src vol:
#!/usr/bin/env bash
set -euo pipefail
@@ -259,6 +291,29 @@ _load-repo repo_src vol:
echo "warning: origin ($origin) != TW_REMOTE_URL (${TW_REMOTE_URL})"
fi
# Refresh the source clone from its remote before mirroring, so the card
# carries the latest — e.g. notes the device pushed since the last load.
# Fast-forward-only and best-effort: a dirty tree, diverged history, detached
# HEAD, or being offline just warns and copies the current on-disk state
# rather than aborting the card prep. Set TW_NO_PULL=1 to skip the pull.
case "$origin" in
https://*|git@*|ssh://*|git://*)
if [ -n "${TW_NO_PULL:-}" ]; then
echo "TW_NO_PULL set — skipping pull, copying current on-disk state"
else
branch="$(git -C "$src" symbolic-ref --quiet --short HEAD || true)"
if [ -z "$branch" ]; then
echo "warning: '$src' is in detached HEAD — skipping pull, copying current state" >&2
elif git -C "$src" pull --ff-only origin "$branch"; then
:
else
echo "warning: 'git pull --ff-only origin $branch' failed (dirty, diverged, or offline) —" >&2
echo " copying the current on-disk state instead" >&2
fi
fi
;;
esac
dest="{{vol}}/{{sd_repo_dir}}"
echo "copying repo: $src -> $dest"
mkdir -p "$dest"
@@ -280,6 +335,101 @@ _load-repo repo_src vol:
# -P for progress on the ~700 MB copy. Scoped to repo/, never the card root.
rsync -rtP --delete --modify-window=1 --exclude-from="$ignore_list" "$src/" "$dest/"
# The device's libgit2 speaks HTTPS+PAT only (mbedTLS — no SSH transport),
# so an SSH origin copied verbatim fails every on-device push/fetch with
# "unsupported URL protocol" (found the hard way, 2026-07-13). Rewrite the
# CARD copy's origin to the HTTPS equivalent; the source clone keeps its
# own URL. Its embedded trust store carries GitHub's roots, hence the
# non-github warning.
case "$origin" in
git@*:*)
host="${origin#git@}"; host="${host%%:*}"
card_origin="https://$host/${origin#git@*:}"
;;
ssh://git@*)
rest="${origin#ssh://git@}"
card_origin="https://${rest%%[:/]*}/${rest#*/}"
;;
https://*) card_origin="$origin" ;;
*) card_origin="" ;;
esac
if [ -n "$card_origin" ]; then
if [ "$card_origin" != "$origin" ]; then
echo "rewriting card origin for the device: $origin -> $card_origin"
fi
git -C "$dest" remote set-url origin "$card_origin"
case "$card_origin" in
https://github.com/*) ;;
*) echo "warning: origin host isn't github.com — the device's embedded trust store" \
"only carries GitHub roots, so on-device TLS will fail without its CA" ;;
esac
elif [ -n "$origin" ]; then
echo "warning: origin '$origin' has no HTTPS equivalent I can derive —"
echo " on-device push/fetch will fail; set it by hand:"
echo " git -C '$dest' remote set-url origin https://github.com/<owner>/<repo>.git"
fi
# First-time setup: seed the two git-tracked config files into the card's repo/
# so they commit + sync on the device's first `:sync`. Writes ONLY a file that is
# absent — a card whose clone already carries `.typoena.toml` /
# `.typoena.snippets.json` keeps its own, so a re-`init` never clobbers a synced
# library. The starter `.typoena.toml` mirrors the editor defaults; the snippet
# library is assembled from the curated catalog (snippets-catalog/, opt-in per
# group) with `jq`. Non-interactive runs take every default (all groups).
_seed-configs vol:
#!/usr/bin/env bash
set -euo pipefail
repo="{{vol}}/{{sd_repo_dir}}"
catalog="{{justfile_directory()}}/snippets-catalog"
interactive=1; [ -t 0 ] || interactive=0
# ask_yn "label" → 0 (yes) / 1 (no); default yes, non-interactive → yes.
ask_yn() {
local ans
if [ "$interactive" = 0 ]; then return 0; fi
read -r -p " $1 [Y/n]: " ans || ans=""
[[ "${ans:-Y}" =~ ^[Yy] ]]
}
# ── starter prefs (.typoena.toml) ────────────────────────────────────────
prefs="$repo/.typoena.toml"
if [ -e "$prefs" ]; then
echo "keeping existing .typoena.toml"
elif ask_yn "seed a starter .typoena.toml (editor defaults)?"; then
# printf per line (matching _write-conf), so no heredoc indentation trap.
{
printf '# Typoena editor preferences — hand-editable, git-tracked.\n'
printf '# Edit here, or change live from the Cmd-P palette (type `>`).\n'
printf 'save_on_idle = true\n'
printf 'format_on_save = true\n'
printf 'line_numbers = true\n'
printf 'theme = "light"\n'
printf 'auto_sync = "10m"\n'
} > "$prefs"
echo "wrote .typoena.toml"
fi
# ── snippet library (.typoena.snippets.json) from the catalog ────────────
snips="$repo/.typoena.snippets.json"
if [ -e "$snips" ]; then
echo "keeping existing .typoena.snippets.json"
elif ! command -v jq >/dev/null 2>&1; then
echo "warning: jq not found — skipping the snippet catalog (install jq, or" >&2
echo " hand-write $snips later)" >&2
else
echo "snippet catalog — choose groups to include:" >&2
picked=()
ask_yn "Symbols (→ ≠ × · ° € œ)" && picked+=("$catalog/symbols.json")
ask_yn "Structure (todo, link, img, table, code)" && picked+=("$catalog/structure.json")
ask_yn "Prose (booknotes, reference, bias, capture, standard)" && picked+=("$catalog/prose.json")
if [ "${#picked[@]}" -gt 0 ]; then
jq -s 'add' "${picked[@]}" > "$snips"
echo "wrote .typoena.snippets.json ($(jq 'length' "$snips") snippets, ${#picked[@]} group(s))"
else
echo "no groups chosen — skipping .typoena.snippets.json"
fi
fi
# Resolve + write <vol>/typoena.conf (Wi-Fi + PAT + git identity). Each value
# runs a ladder: firmware/.env (loaded via `set dotenv-load`) → derived from
# tools already on the machine (git config, gh, the active Wi-Fi network + its

View File

@@ -44,6 +44,30 @@ CONFIG_SPIRAM=y
CONFIG_SPIRAM_MODE_OCT=y
CONFIG_SPIRAM_USE_MALLOC=y
# FatFS fast seek (sync-latency root cause, 2026-07-12). Without it, lseek
# resolves by walking the file's FAT cluster chain over SPI — forward from the
# current position, from the CHAIN HEAD on any backward seek. sd_bench measured
# ~190 ms per long seek into the 263 MB packfile (5.8 ms at offset 0), and
# libgit2 pays ~8 such seeks per loose-object write via p_mmap's lseek+read →
# the ~1.5 s/object commit cost (docs/tradeoff-curves/sync-commit-staging.md).
# Fast seek builds an in-memory cluster-link map (CLMT) per READ-mode file,
# making lseek O(1); write-mode files transparently fall back to the walk. The
# buffer is per-open-file, 4 B × size: 256 words = 1 KB, covering ~127 fragments
# (the default 64 covers ~31 — a freshly-provisioned pack is near-contiguous,
# but don't let card aging silently disable the fix; vfs_fat falls back to slow
# seeks when the map doesn't fit).
CONFIG_FATFS_USE_FASTSEEK=y
CONFIG_FATFS_FAST_SEEK_BUFFER_SIZE=256
# Silence the legacy-I2C boot warning. esp-idf-hal's i2c.rs is always compiled
# and binds the old `driver/i2c.h` API, so the legacy driver's TU (and its
# `__attribute__((constructor))` deprecation warning) gets linked into every
# image — even though Typoena uses no I2C at all (EPD/SD are SPI, keyboard is
# USB). This flag wraps that constructor out. Safe here: we link neither the
# new i2c_master driver nor call the old one, so the conflict-abort check it
# also removes can never fire.
CONFIG_I2C_SKIP_LEGACY_CONFLICT_CHECK=y
# TLS trust store (Spike 6 — Wi-Fi + TLS, the gate for Spike 7 git push).
# The certificate bundle backs esp_crt_bundle_attach so an HTTPS GET to
# api.github.com validates against real roots. FULL rather than the common

View File

@@ -0,0 +1,58 @@
{
"Book notes": {
"prefix": "booknotes",
"body": ["# $1", "", "## $2 - $3", "", "## What the book is about", ""],
"description": "Reading notes (title / author - year)"
},
"Reference block": {
"prefix": "reference",
"body": ["___", "", "## References", ""],
"description": "Reference block"
},
"Cognitive bias": {
"prefix": "bias",
"body": [
"# $1",
"",
"## Misconception",
"",
"$3",
"",
"## The reality",
"",
"$4",
"",
"___",
"",
"## References",
"",
"- You are not so smart - p. $2",
""
],
"description": "Cognitive-bias note"
},
"Capture": {
"prefix": "capture",
"body": ["# $1", "", "## Captured inspiration", "", "$2", ""],
"description": "Inspiration capture"
},
"Standard": {
"prefix": "standard",
"body": [
"# $1",
"",
"## Key points",
"",
"## Mistakes to avoid",
"",
"___",
"",
"## Examples",
"",
"___",
"",
"## References"
],
"description": "Standard / playbook"
}
}

View File

@@ -0,0 +1,27 @@
{
"Todo item": {
"prefix": "todo",
"body": "- [ ] ",
"description": "Task-list item"
},
"Link": {
"prefix": "link",
"body": "[$1]($2)$0",
"description": "Inline link"
},
"Image": {
"prefix": "img",
"body": "![$1]($2)$0",
"description": "Image"
},
"Table": {
"prefix": "table",
"body": ["| $1 | $2 |", "|-|-|", "| $3 | $4 |"],
"description": "Two-column table"
},
"Code block": {
"prefix": "code",
"body": ["```$1", "$2", "```"],
"description": "Fenced code block"
}
}

View File

@@ -0,0 +1,37 @@
{
"Arrow": {
"prefix": "arrow",
"body": "→",
"description": "Rightwards arrow (→)"
},
"Not equal": {
"prefix": "neq",
"body": "≠",
"description": "Not-equal sign (≠)"
},
"Times": {
"prefix": "times",
"body": "×",
"description": "Multiplication sign (×)"
},
"Middle dot": {
"prefix": "middot",
"body": "·",
"description": "Middle dot (·), inclusive writing"
},
"Degree": {
"prefix": "deg",
"body": "°",
"description": "Degree sign (°)"
},
"Euro": {
"prefix": "euro",
"body": "€",
"description": "Euro sign (€)"
},
"OE ligature": {
"prefix": "edanso",
"body": "œ",
"description": "Ligature œ (e dans l'o)"
}
}

View File

@@ -0,0 +1,371 @@
//! git-level micro-benchmark — localizes the ~700 ms/object libgit2 overhead the
//! `:sync` commit split showed (2026-07-12), now that `sd_bench` proved the raw
//! card does a *full* loose-object write (stat+create+write+rename) in ~86 ms.
//! The ~8× gap between that and `write_tree`'s 710 ms lives inside libgit2, not
//! FAT — this bench times the git2 ODB/index primitives in isolation to find it.
//!
//! HEADLINE OP (since the 2026-07-12 real-repo run): `splice stage→tree` — the
//! O(depth) TreeBuilder walk that replaces the index-based commit entirely
//! (docs/tradeoff-curves/sync-commit-staging.md). It runs FIRST so its first iteration is
//! the cold number; acceptance bar: **sub-second cold on the real 570 MB-pack
//! clone, heap staying healthy**. The index paths it supersedes run LAST, for
//! regression tracking — they previously OOM'd, and a late crash can't cost the
//! splice data.
//!
//! Read-mostly on `/sd/repo`: the only writes are unreferenced ("orphan") loose
//! blobs/trees/commits — never reachable from a ref, so never pushed, and
//! gc-able. Safe on the test card.
//!
//! Flash with `just flash-gitbench` (needs the `git` feature; env in the recipe).
use std::time::Instant;
use anyhow::{Context, Result};
use esp_idf_svc::hal::delay::FreeRtos;
use git2::{IndexEntry, IndexTime, ObjectType, Oid, Repository, Signature, Tree};
use firmware::git_sync::GIT_STACK;
use firmware::persistence::{Storage, REPO_DIR};
const BUILD_TAG: &str = concat!("build ", env!("BUILD_TIME"), " @", env!("BUILD_GIT"));
/// Iterations per op. Small — some ops write to the card, and the first vs rest
/// spread (min vs max) is itself the signal (e.g. cold vs warm, write vs
/// freshen-skip). Kept low (3) on the real 570 MB-pack clone so a slow op still
/// finishes in seconds.
const N: usize = 3;
fn main() -> Result<()> {
esp_idf_svc::sys::link_patches();
esp_idf_svc::log::EspLogger::initialize_default();
log::info!("Typoena — git-level bench, {BUILD_TAG}");
// libgit2 nests ~67 KB of GIT_PATH_MAX stack buffers (postmortem #3), so the
// git work must run on the same 96 KB stack the real git service uses. On the
// small main-task stack `index.write()` overflows → nested panic → boot loop.
let handle = std::thread::Builder::new()
.name("git_bench".into())
.stack_size(GIT_STACK)
.spawn(run)
.expect("spawn git_bench thread");
match handle.join() {
Ok(Ok(())) => log::info!("git_bench: done"),
Ok(Err(e)) => log::error!("git_bench failed: {e:?}"),
Err(_) => log::error!("git_bench thread panicked"),
}
loop {
FreeRtos::delay_ms(1000);
}
}
fn run() -> Result<()> {
// libgit2 holds the pack + idx (+ commit-graph) fds open and reads loose
// objects on top; the editor's default 4-FD budget can't cover read_tree.
let _sd = Storage::mount_for_git().context("mounting SD")?;
// A 32 MB default mwindow window (mwindow.c) would git__malloc > PSRAM on the
// real 570 MB pack; small windows keep each p_mmap read cheap, and the
// esp_map cache keeps them from being re-read on every freshen→refresh.
// SAFETY: process-global libgit2 options, set once before any repo work.
unsafe {
git2::opts::set_mwindow_size(256 * 1024).ok();
git2::opts::set_mwindow_mapped_limit(4 * 1024 * 1024).ok();
}
// Repository open — one-time, but shows the cost of scanning .git (config,
// refs, ODB backends/packs) which every later op may implicitly refresh.
let t = Instant::now();
let repo = Repository::open(REPO_DIR)
.with_context(|| format!("opening git repo at {REPO_DIR}"))?;
log::info!("Repository::open {:.1} ms", t.elapsed().as_micros() as f64 / 1000.0);
log_map_stats("open");
// 1) THE FIX — `splice stage→tree`, the O(depth) TreeBuilder walk
// (docs/tradeoff-curves/sync-commit-staging.md): patch the edited file's ancestor
// subtree chain onto HEAD's tree; never materialise the 1179-entry index,
// never index.write(), never read_tree the whole tree. Runs FIRST so
// iteration #1 is genuinely cold (only `open` has touched the pack).
let head_tree = repo
.head()?
.peel_to_commit()
.context("HEAD → commit")?
.tree()
.context("HEAD tree")?;
// A nested path that already exists in HEAD's tree, found by an O(depth)
// descent — NOT read_tree, which is itself the 77 s op — so the splice
// REPLACES a real file and rebuilds a real ancestor chain, not just the root.
let edit_path = find_edit_path(&repo, &head_tree)?;
log::info!(
"splice: editing {} (depth {})",
edit_path.join("/"),
edit_path.len()
);
// The blob write is inside the timing: the real commit pays blob + trees, and
// it keeps the number comparable to `index-free stage→tree` below. Not
// measured here: the ref/reflog update (commit(Some("HEAD"))) — flat FAT
// writes, ~350 ms on the toy repo.
bench("splice stage→tree", |i| {
let data = format!("typoena splice bench edit #{i}\n");
let oid = repo.blob(data.as_bytes()).context("write blob")?;
let parts: Vec<&str> = edit_path.iter().map(String::as_str).collect();
splice(&repo, Some(&head_tree), &parts, Some(oid)).map(|_| ())
})?;
log_map_stats("splice");
// 2) commit(None, …) — create a commit OBJECT without moving HEAD or writing a
// reflog (update_ref = None → an orphan commit, gc-able). Isolates commit-
// object creation from the ref-update + reflog cost; splice + this projects
// the full real-repo commit. Reuses the parent's tree (no new tree needed);
// unique message each iter forces a real write.
let parent = repo.head()?.peel_to_commit().context("HEAD → commit")?;
let sig = Signature::now("typoena-bench", "bench@typoena.local").context("sig")?;
bench("commit(None) orphan obj", |i| {
let msg = format!("typoena git_bench orphan commit #{i}");
repo.commit(None, &sig, &sig, &msg, &head_tree, &[&parent])
.map(|_| ())
.context("commit(None)")
})?;
log_map_stats("commit");
// 3) odb.write(blob) in isolation — unique content each iter forces a real
// write (no freshen-skip). If ~100 ms the ODB write path is fine and any
// slow op above is in the tree/ref layer; if ~1 s the cost is inside the
// ODB write itself (deflate/sha/freshen) and the mmap cache regressed.
let odb = repo.odb().context("opening odb")?;
bench("odb.write(blob)", |i| {
let data = format!("typoena git_bench orphan blob #{i} — unique so the write is real\n");
odb.write(ObjectType::Blob, data.as_bytes())
.map(|_| ())
.context("odb.write")
})?;
log_map_stats("odb.write");
// 3b) LOCALIZE the commit-vs-blob gap. The fast-seek A/B (2026-07-12) left
// `commit(None)` at 1.7 s while `odb.write` dropped to ~0.4 s — commit
// additionally VALIDATES its parent + tree OIDs against the odb (strict
// object creation → pack header resolves) and freshens the packed tree.
// Price the two suspects, then re-bench commit + splice with strict
// creation OFF. If commit collapses toward odb.write, validation was the
// gap — and git_sync can ship with strict off (every OID it inserts
// comes from HEAD's tree or a blob it just wrote).
let parent_id = parent.id();
let tree_id = head_tree.id();
bench("odb.read_header(packed)", |i| {
let id = if i % 2 == 0 { tree_id } else { parent_id };
odb.read_header(id).map(|_| ()).context("read_header")
})?;
bench("odb.exists(missing)", |i| {
let id = Oid::from_str(&format!("{:040x}", 0xdead_beef_u64 + i as u64))?;
let _ = odb.exists(id); // miss → freshen fails → git_odb_refresh path
Ok(())
})?;
log_map_stats("probes");
// Process-global libgit2 flag; this bench owns the process.
git2::opts::strict_object_creation(false);
bench("commit(None) [strict off]", |i| {
let msg = format!("typoena git_bench strict-off commit #{i}");
repo.commit(None, &sig, &sig, &msg, &head_tree, &[&parent])
.map(|_| ())
.context("commit strict-off")
})?;
bench("splice [strict off]", |i| {
let data = format!("typoena splice strict-off edit #{i}\n");
let oid = repo.blob(data.as_bytes()).context("write blob")?;
let parts: Vec<&str> = edit_path.iter().map(String::as_str).collect();
splice(&repo, Some(&head_tree), &parts, Some(oid)).map(|_| ())
})?;
git2::opts::strict_object_creation(true);
log_map_stats("strict-off");
// 4) on-disk index LOAD (no write). Times loading all ~1179 entries from the
// card and prints the count. We deliberately do NOT bench index.write():
// it calls truncate_racily_clean, which diffs the whole working tree
// against the index and — because a fresh FAT clone makes every entry look
// "racy" (2 s mtime granularity) — re-hashes ~170 MB over SPI, up to ~10 min
// on this repo (proven 2026-07-12, index.write max 611 s). The splice
// never touches the on-disk index, so that path never runs.
bench("repo.index() load", |_| {
repo.index().map(|_| ()).context("index open")
})?;
let n_entries = repo.index().map(|i| i.len()).unwrap_or(0);
log::info!("on-disk index has {n_entries} entries");
log_map_stats("index load");
// 5) REFUTED ALTERNATIVE — the index-free in-memory-index commit
// (read_tree(HEAD) + add + write_tree_to). It dodges truncate_racily_clean
// but is still O(N_tree): the 2026-07-12 real-repo run measured ~77 s for
// the cold read_tree and drove the mmap cache to 7.4 MB (zlib OOM). Kept
// for regression tracking, run LAST so a crash here can't cost the splice
// data above. The cold read_tree is now timed explicitly (the 77 s was
// previously visible only via log timestamps); the ops above warmed only
// ~depth of the ~158 tree windows, so this is still ~cold.
let t = Instant::now();
{
let mut idx = git2::Index::new().context("Index::new")?;
idx.read_tree(&head_tree).context("seed read_tree")?;
}
log::info!(
"seed read_tree(HEAD) cold {:.1} ms",
t.elapsed().as_micros() as f64 / 1000.0
);
log_map_stats("read_tree");
// Warm repeats: windows resident → pure CPU + cache lookups.
bench("Index::new + read_tree", |_| {
let mut idx = git2::Index::new().context("Index::new")?;
idx.read_tree(&head_tree).context("read_tree")?;
Ok(())
})?;
let edit_path_bytes = edit_path.join("/").into_bytes();
bench("index-free stage→tree", |i| {
let mut idx = git2::Index::new().context("Index::new")?;
idx.read_tree(&head_tree).context("read_tree")?;
let data = format!("typoena index-free bench edit #{i}\n");
let oid = repo.blob(data.as_bytes()).context("write blob")?;
idx.add(&blob_entry(&edit_path_bytes, oid)).context("index.add")?;
idx.write_tree_to(&repo).map(|_| ()).context("write_tree_to")
})?;
log_map_stats("index-free");
Ok(())
}
/// PROTOTYPE of the real fix (destined for `git_sync::stage_and_commit`): return
/// a new tree OID equal to `base` with `path` set to `new` — `Some(blob)` to
/// add/replace, `None` to delete. Reads ~depth subtree objects, writes ~depth
/// trees; every other entry (all 1179 files, the 150 MB of images) is carried
/// forward by OID without ever being read. `base = None` builds a fresh subtree
/// chain (new file in a new directory). The git_sync version must additionally
/// drop a directory entry when a delete empties its subtree; the bench only
/// exercises replace.
fn splice(repo: &Repository, base: Option<&Tree>, path: &[&str], new: Option<Oid>) -> Result<Oid> {
let (head, rest) = path.split_first().context("splice: empty path")?;
let mut tb = repo.treebuilder(base).context("treebuilder")?;
if rest.is_empty() {
match new {
Some(oid) => {
tb.insert(*head, oid, 0o100644).context("insert blob")?;
}
None => {
let _ = tb.remove(*head); // already absent ⇒ nothing to delete
}
}
} else {
let sub = match base.and_then(|b| b.get_name(head)) {
Some(e) if e.kind() == Some(ObjectType::Tree) => {
Some(repo.find_tree(e.id()).context("loading subtree")?)
}
_ => None, // no such dir yet (or a blob in the way): build from empty
};
let new_sub = splice(repo, sub.as_ref(), rest, new)?;
tb.insert(*head, new_sub, 0o040000).context("insert subtree")?;
}
tb.write().context("treebuilder write")
}
/// Find a real file to "edit": descend the first subtree at each level (capped),
/// then take the first blob of the deepest tree reached. Reads O(depth) tree
/// objects — never `read_tree`/materialise the whole tree (that's the 77 s op
/// this bench exists to retire).
fn find_edit_path(repo: &Repository, root: &Tree) -> Result<Vec<String>> {
let mut path = Vec::new();
let mut cur_id = root.id();
for _ in 0..6 {
let cur = repo.find_tree(cur_id).context("descending tree")?;
match cur.iter().find(|e| e.kind() == Some(ObjectType::Tree)) {
Some(sub) => {
path.push(sub.name().context("non-utf8 tree name")?.to_string());
cur_id = sub.id();
}
None => break,
}
}
let cur = repo.find_tree(cur_id).context("leaf tree")?;
let blob = cur
.iter()
.find(|e| e.kind() == Some(ObjectType::Blob))
.context("no blob along the first-subtree chain — pick an edit path manually")?;
path.push(blob.name().context("non-utf8 blob name")?.to_string());
Ok(path)
}
unsafe extern "C" {
/// Counters from the p_mmap emulation in `components/libgit2/esp_map.c`.
/// Post cache-removal: `hits` is always 0, `misses` counts every mapping,
/// `cached_kb` reports the LIVE mapped bytes (the mwindow working set).
fn esp_map_stats(hits: *mut u32, misses: *mut u32, read_kb: *mut u32, cached_kb: *mut u32);
}
/// Log the p_mmap counters — mappings performed, total KB read from the card,
/// and KB currently live-mapped (should track mwindow's open windows and stay
/// well under MWINDOW_MAPPED_LIMIT now that munmap frees immediately).
fn log_map_stats(label: &str) {
let (mut hits, mut misses, mut read_kb, mut cached_kb) = (0u32, 0u32, 0u32, 0u32);
unsafe { esp_map_stats(&mut hits, &mut misses, &mut read_kb, &mut cached_kb) };
let _ = hits; // always 0 since the cache removal; slot kept for ABI stability
// Free heap spans PSRAM here; a drop toward 0 during write_tree/commit on the
// real repo would point at mwindow/idx allocation pressure (or thrash) as the
// cause of an apparent hang, not CPU.
let free_kb = unsafe { esp_idf_svc::sys::esp_get_free_heap_size() } / 1024;
log::info!(
"mmap @ {label:<11} {misses} maps, {read_kb} KB read, {cached_kb} KB live, {free_kb} KB heap free"
);
}
/// Announce, time, and summarize an op. The `→ label …` line prints BEFORE the op
/// runs, so if an op hangs on the real 570 MB-pack repo we can see which one it
/// entered — a bare `summarize` prints only after all N iters, hiding the culprit.
fn bench<F: FnMut(usize) -> Result<()>>(label: &str, op: F) -> Result<()> {
log::info!("→ {label} …");
summarize(label, time_each(op)?);
Ok(())
}
/// A minimal index entry pointing at an already-written blob — for `index.add`,
/// which (unlike `add_frombuffer`) needs no repo owner, so it works on a bare
/// in-memory index. Only `id`, `path` and `mode` feed the tree write.
fn blob_entry(path: &[u8], oid: Oid) -> IndexEntry {
IndexEntry {
ctime: IndexTime::new(0, 0),
mtime: IndexTime::new(0, 0),
dev: 0,
ino: 0,
mode: 0o100644,
uid: 0,
gid: 0,
file_size: 0,
id: oid,
flags: 0,
flags_extended: 0,
path: path.to_vec(),
}
}
/// Run `op(i)` for `i in 0..N`, returning each call's wall time in microseconds.
fn time_each<F: FnMut(usize) -> Result<()>>(mut op: F) -> Result<Vec<u64>> {
let mut times = Vec::with_capacity(N);
for i in 0..N {
let t = Instant::now();
op(i)?;
times.push(t.elapsed().as_micros() as u64);
}
Ok(times)
}
/// Log min / p50 / mean / max in ms for a set of per-call microsecond timings.
fn summarize(label: &str, mut times: Vec<u64>) {
times.sort_unstable();
let n = times.len();
let mean = times.iter().sum::<u64>() / n as u64;
let ms = |us: u64| us as f64 / 1000.0;
log::info!(
"{label:<26} min {:>6.1} p50 {:>6.1} mean {:>6.1} max {:>6.1} ms",
ms(times[0]),
ms(times[n / 2]),
ms(mean),
ms(times[n - 1]),
);
}

View File

@@ -0,0 +1,212 @@
//! SD/FAT primitive-op micro-benchmark — investigating the ~700 ms-per-loose-
//! object write floor found in the `:sync` commit split (2026-07-12, see
//! `docs/tradeoff-curves/sync-commit-staging.md`).
//!
//! The split showed a single small git loose object (`write_tree` = one tree
//! object) takes ~710 ms to land on the card, and it is **not** fsync
//! (`GIT_OPT_ENABLE_FSYNC_GITDIR` is off). libgit2's loose-object write
//! (`odb_loose.c` `loose_backend__write` → `git_filebuf_commit_at`) is, per object:
//!
//! stat(final) — freshen probe, misses (our `utimes` stub → `stat`)
//! open+write+close — a temp file (`GIT_FILEBUF_TEMPORARY`)
//! [mkdir objects/xx once per fan-out]
//! p_rename — our stub: remove(final) [ENOENT] + rename(temp → final)
//!
//! i.e. **two directory-mutating writes** (temp create + rename) per object. This
//! bench times each FAT primitive in isolation, then a composite that mirrors the
//! sequence above, so we can attribute the ~700 ms to specific ops and get a
//! baseline to compare an A1/A2 card or a 20 MHz bus against. All writes go to
//! `/sd/sdbench` (cleaned up at the end); the pack-seek op additionally opens
//! `/sd/repo`'s packfile READ-ONLY — it never writes there.
//!
//! Flash with `just flash-bench`. Needs no `.env`, no `git` feature (pure SD).
use std::fs::{self, File};
use std::io::{Read, Seek, SeekFrom, Write};
use std::time::Instant;
use anyhow::{Context, Result};
use esp_idf_svc::hal::delay::FreeRtos;
use firmware::persistence::Storage;
/// Injected by build.rs so serial output identifies the exact build.
const BUILD_TAG: &str = concat!("build ", env!("BUILD_TIME"), " @", env!("BUILD_GIT"));
/// Scratch dir on the card ROOT — outside `/sd/repo`, so a later `:sync` never
/// stages it and the user's notes are never touched.
const BENCH_DIR: &str = "/sd/sdbench";
/// Iterations per op: enough to read min/p50/mean past controller jitter, few
/// enough that total write volume stays tiny.
const N: usize = 20;
/// ~ the size of a small deflated git loose object (blob/tree/commit).
const PAYLOAD: [u8; 200] = [b'x'; 200];
fn main() -> Result<()> {
// Required once before any esp-idf-svc call (see esp-idf-template#71).
esp_idf_svc::sys::link_patches();
esp_idf_svc::log::EspLogger::initialize_default();
log::info!("Typoena — SD primitive bench, {BUILD_TAG}");
match run() {
Ok(()) => log::info!("sd_bench: done"),
Err(e) => log::error!("sd_bench failed: {e:?}"),
}
loop {
FreeRtos::delay_ms(1000);
}
}
fn run() -> Result<()> {
let sd = Storage::mount().context("mounting SD")?;
let (max_khz, real_khz) = sd.negotiated_khz();
log::info!("bus: max {max_khz} kHz, negotiated {real_khz} kHz — {N} iters, {}-byte payload", PAYLOAD.len());
// Fresh scratch dir.
let _ = fs::remove_dir_all(BENCH_DIR);
fs::create_dir_all(BENCH_DIR).with_context(|| format!("creating {BENCH_DIR}"))?;
// Warm-up: the first write after mount pays one-time settling — don't measure it.
{
let mut f = File::create(format!("{BENCH_DIR}/warmup"))?;
f.write_all(&PAYLOAD)?;
}
// 1) create + write(200B) + close, a fresh unique file each time. The drop at
// the block's end is the close (FatFS f_close flushes dir entry + data).
summarize("create+write(200B)+close", time_each(|i| {
let mut f = File::create(format!("{BENCH_DIR}/c{i}"))?;
f.write_all(&PAYLOAD)?;
Ok(())
})?);
// 2) rename c{i} -> o{i}. Sources exist from step 1 (untimed setup).
summarize("rename", time_each(|i| {
fs::rename(format!("{BENCH_DIR}/c{i}"), format!("{BENCH_DIR}/o{i}"))
.map_err(Into::into)
})?);
// 3) stat, hit.
summarize("stat (hit)", time_each(|i| {
fs::metadata(format!("{BENCH_DIR}/o{i}")).map(|_| ()).map_err(Into::into)
})?);
// 4) stat, miss (ENOENT) — the freshen-probe analogue. A read, expected cheap.
summarize("stat (miss/ENOENT)", time_each(|i| {
let _ = fs::metadata(format!("{BENCH_DIR}/nope{i}"));
Ok(())
})?);
// 5) remove o{i}.
summarize("remove", time_each(|i| {
fs::remove_file(format!("{BENCH_DIR}/o{i}")).map_err(Into::into)
})?);
// 6) Composite: the exact loose-object write sequence libgit2 performs, with a
// git-length (38-hex) final name so LFN directory-entry cost is included.
// If the model is right this lands near the ~700 ms/object from the split.
summarize("loose-object composite", time_each(|i| {
let tmp = format!("{BENCH_DIR}/tmp_obj{i}");
let fin = format!("{BENCH_DIR}/{i:038x}");
let _ = fs::metadata(&fin); // freshen probe, misses
{
let mut f = File::create(&tmp)?; // temp create + write + close
f.write_all(&PAYLOAD)?;
}
let _ = fs::remove_file(&fin); // p_rename's remove(to) — ENOENT
fs::rename(&tmp, &fin)?; // temp -> final
Ok(())
})?);
// Clean up so the card is left as we found it.
fs::remove_dir_all(BENCH_DIR).with_context(|| format!("removing {BENCH_DIR}"))?;
// 7) THE ~1.5 s LOOSE-WRITE SUSPECT (git_bench, 2026-07-12 second real-repo
// run): lseek inside a huge file. Without CONFIG_FATFS_USE_FASTSEEK,
// FatFS resolves lseek by walking the file's FAT cluster chain — forward
// from the current position, from the CHAIN HEAD on any backward seek.
// The 570 MB pack is ~36k clusters ≈ ~146 KB of FAT reads over SPI per
// long walk. `p_mmap` (esp_map.c) does lseek+read per window, and
// libgit2's freshen path probes the pack TRAILER (near the end) while
// tree windows sit at low offsets — so each loose write pays ~one full
// walk. Prediction: "@start" stays ~ms; "@end" costs ~1.5 s per iter.
// If so, the fix is CONFIG_FATFS_USE_FASTSEEK=y (fast-seek applies to
// read-mode files only — exactly how the pack is opened).
match find_pack()? {
Some(pack) => {
let len = fs::metadata(&pack)?.len();
log::info!("pack seek bench: {pack} ({} MB)", len / (1024 * 1024));
if len < 1024 * 1024 {
log::info!("pack too small to show chain-walk cost — skipping (toy card?)");
} else {
let mut f = File::open(&pack).with_context(|| format!("opening {pack}"))?;
let mut buf = vec![0u8; 4096];
// Baseline: rewind + read at the chain head — no walk to resolve.
summarize("pack seek+read 4KB @start", time_each(|_| {
f.seek(SeekFrom::Start(0))?;
f.read_exact(&mut buf)?;
Ok(())
})?);
// Rewind (cheap, measured above), then seek near the end — pays
// one full cluster-chain walk per iteration if fast-seek is off.
let high = len - 4096;
summarize("pack seek+read 4KB @end", time_each(|_| {
f.seek(SeekFrom::Start(0))?;
f.read_exact(&mut buf)?;
f.seek(SeekFrom::Start(high))?;
f.read_exact(&mut buf)?;
Ok(())
})?);
}
}
None => log::info!("no packfile under /sd/repo/.git/objects/pack — skipping seek bench"),
}
Ok(())
}
/// Largest `*.pack` under the repo's pack dir, if the card carries a clone.
/// Skips macOS AppleDouble sidecars (`._pack-*.pack`, 4 KB of Finder metadata) —
/// the Spike-14 cruft in its latest disguise.
fn find_pack() -> Result<Option<String>> {
let Ok(entries) = fs::read_dir("/sd/repo/.git/objects/pack") else {
return Ok(None);
};
let mut best: Option<(u64, String)> = None;
for e in entries.flatten() {
let p = e.path();
let name = p.file_name().and_then(|n| n.to_str()).unwrap_or("");
if name.starts_with("._") || !name.ends_with(".pack") {
continue;
}
let len = fs::metadata(&p).map(|m| m.len()).unwrap_or(0);
if best.as_ref().is_none_or(|(l, _)| len > *l) {
best = Some((len, p.to_string_lossy().into_owned()));
}
}
Ok(best.map(|(_, p)| p))
}
/// Run `op(i)` for `i in 0..N`, returning each call's wall time in microseconds.
fn time_each<F: FnMut(usize) -> Result<()>>(mut op: F) -> Result<Vec<u64>> {
let mut times = Vec::with_capacity(N);
for i in 0..N {
let t = Instant::now();
op(i)?;
times.push(t.elapsed().as_micros() as u64);
}
Ok(times)
}
/// Log min / p50 / mean / max in ms for a set of per-call microsecond timings.
fn summarize(label: &str, mut times: Vec<u64>) {
times.sort_unstable();
let n = times.len();
let mean = times.iter().sum::<u64>() / n as u64;
let ms = |us: u64| us as f64 / 1000.0;
log::info!(
"{label:<26} min {:>6.1} p50 {:>6.1} mean {:>6.1} max {:>6.1} ms",
ms(times[0]),
ms(times[n / 2]),
ms(mean),
ms(times[n - 1]),
);
}

View File

@@ -94,6 +94,11 @@ fn write_test(storage: &Storage) -> Result<()> {
log::info!("{REPO_DIR} missing — creating it (bench setup) so the write test can run");
fs::create_dir_all(REPO_DIR).with_context(|| format!("create {REPO_DIR}"))?;
}
// End the payload with '\n', like a real editor buffer (whose visible trailing
// blank line is exactly that terminating newline). `save` inserts a final
// newline only when one is missing and `load` reads verbatim, so a payload that
// already ends in '\n' round-trips byte-for-byte — which this exact-equality
// check relies on.
let payload = format!("typoena spike 3\n{BUILD_TAG}\ndedicated SPI3: SCK14 MOSI15 MISO13 CS10\n");
storage.save(&payload).context("Storage::save")?;
let back = storage.load().context("Storage::load after save")?;

View File

@@ -17,15 +17,23 @@
//! notes. A `/sd/repo` that isn't a valid repo is a provisioning error
//! (`just init`), surfaced as such, not papered over.
//! 3. **No synthetic content.** The spike appended a marker line; here the
//! editor has already saved the user's `notes.md` before `:sync` signals us,
//! so we just stage + commit + push what's on disk.
//! editor has already saved the user's buffers before `:sync` signals us,
//! so we just commit + push what's on disk.
//! 4. **The commit is an O(depth) TreeBuilder splice, not an index pass.**
//! The request carries the repo-relative paths saved/deleted since the last
//! confirmed publish (`Storage`'s journaled dirty set); `stage_and_commit`
//! patches exactly those onto HEAD's tree. The index pipeline it replaced
//! (`add_all` → `index.write` → `write_tree`) is O(N_tree) and measured up
//! to 611 s on the real 1179-file / 570 MB-pack clone — see
//! docs/tradeoff-curves/sync-commit-staging.md for the whole trail.
//!
//! Runs on a dedicated 96 KB thread (libgit2's init→push chain nests ~67 KB of
//! `GIT_PATH_MAX` stack buffers — see git_push.rs / postmortem #3). Config is
//! baked at build time (`TW_*`, ADR-007: v0.1 device config is compiled in).
use std::cell::RefCell;
use std::path::Path;
use std::collections::BTreeSet;
use std::fs;
use std::rc::Rc;
use std::sync::mpsc::{Receiver, Sender};
use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH};
@@ -39,8 +47,8 @@ use esp_idf_svc::sntp::{EspSntp, SyncStatus};
use esp_idf_svc::sys;
use esp_idf_svc::wifi::{BlockingWifi, EspWifi};
use git2::{
CertificateCheckStatus, Commit, Cred, CredentialType, FetchOptions, IndexAddOption,
PushOptions, RemoteCallbacks, Repository, Signature,
CertificateCheckStatus, Commit, Cred, CredentialType, FetchOptions, ObjectType, Oid,
PushOptions, RemoteCallbacks, Repository, Signature, Tree,
};
use crate::net::connect_wifi;
@@ -73,10 +81,15 @@ const SNTP_TIMEOUT: Duration = Duration::from_secs(20);
/// now also runs here, but it's shallow next to libgit2's path-buffer nesting.
pub const GIT_STACK: usize = 96 * 1024;
/// A request to publish. The note is already saved to `/sd/repo/notes.md` by the
/// UI task before this is sent, so the request carries no payload (a future
/// multi-file publish can grow one).
pub struct PublishRequest;
/// A request to publish. The UI task has already saved every dirty buffer to
/// the card before sending this; `paths` is `Storage::take_dirty`'s snapshot —
/// the repo-relative paths saved or `:delete`d since the last confirmed
/// publish. The working tree stays the source of truth: at commit time a path
/// that exists on the card is spliced into the tree from disk, a missing one
/// is spliced out. An unchanged path is a no-op, so over-reporting is safe.
pub struct PublishRequest {
pub paths: BTreeSet<String>,
}
/// Result of a publish attempt, sent back to the UI task for the snackbar. The
/// detailed error always goes to the serial log; the panel gets a short line.
@@ -103,6 +116,22 @@ pub fn run_git_service(
rx: Receiver<PublishRequest>,
tx: Sender<PublishOutcome>,
) {
// Process-global libgit2 tuning, once, before any repo work. The 32-bit
// defaults (32 MB window / 256 MB mapped budget, mwindow.c) would
// git__malloc past PSRAM on the first pack access of the real 570 MB-pack
// clone; these are the bench-proven values (git_bench), and ~1.9 MB of
// windows stays live during git ops — the p_mmap emulation (esp_map.c)
// relies on this mapped limit being real.
// SAFETY: set on the git thread before any Repository is opened.
unsafe {
if let Err(e) = git2::opts::set_mwindow_size(256 * 1024) {
log::error!("set_mwindow_size failed ({e}); first pack access may OOM");
}
if let Err(e) = git2::opts::set_mwindow_mapped_limit(4 * 1024 * 1024) {
log::error!("set_mwindow_mapped_limit failed ({e}); first pack access may OOM");
}
}
// Lazily initialised on the first request, then reused across publishes.
let mut wifi: Option<BlockingWifi<EspWifi<'static>>> = None;
let mut modem = Some(modem);
@@ -110,7 +139,7 @@ pub fn run_git_service(
let mut clock_synced = false;
let mut tls_ready = false;
while rx.recv().is_ok() {
while let Ok(req) = rx.recv() {
let outcome = publish_cycle(
&sys_loop,
&mut wifi,
@@ -118,6 +147,7 @@ pub fn run_git_service(
&mut nvs,
&mut clock_synced,
&mut tls_ready,
&req.paths,
);
let msg = match outcome {
Ok(o) => o,
@@ -143,11 +173,22 @@ fn publish_cycle(
nvs: &mut Option<EspDefaultNvsPartition>,
clock_synced: &mut bool,
tls_ready: &mut bool,
paths: &BTreeSet<String>,
) -> Result<PublishOutcome> {
if REMOTE_URL.is_empty() || GH_USER.is_empty() || PAT.is_empty() || WIFI_SSID.is_empty() {
bail!("git config missing — set TW_WIFI_SSID / TW_REMOTE_URL / TW_GH_USER / TW_PAT in firmware/.env and rebuild");
}
// Nothing recorded dirty and origin's tracking ref already has HEAD: this
// `:sync` has nothing to do — say so without touching the radio (~150 ms
// instead of a Wi-Fi + TLS round). A stranded local commit (committed but
// never pushed, e.g. a push that failed mid-air) makes the check false and
// takes the full path below, where publish_once pushes it.
if paths.is_empty() && remote_current().unwrap_or(false) {
log::info!(":sync — no dirty paths and origin has HEAD; up to date, radio untouched");
return Ok(PublishOutcome::UpToDate);
}
// Phases are timed so a cold :sync reports where the seconds go. Wi-Fi, clock
// and TLS run only on the first sync of a session; a warm sync skips them, so
// they read 0 ms and the total collapses to just publish(fetch+commit+push).
@@ -186,7 +227,7 @@ fn publish_cycle(
}
let t_publish = Instant::now();
let outcome = publish_once()?;
let outcome = publish_once(paths)?;
log::info!(
":sync timing — wifi {wifi_ms}ms, clock {clock_ms}ms, tls {tls_ms}ms, publish(commit+push) {}ms, total {}ms",
t_publish.elapsed().as_millis(),
@@ -205,15 +246,16 @@ fn publish_cycle(
///
/// Never clones or wipes: a `/sd/repo` that isn't a valid repo is a provisioning
/// error, surfaced as such.
fn publish_once() -> Result<PublishOutcome> {
log::info!("publish started — free heap {}", free_heap());
fn publish_once(paths: &BTreeSet<String>) -> Result<PublishOutcome> {
log::info!(
"publish started — {} dirty path(s), free heap {}",
paths.len(),
free_heap()
);
let repo = Repository::open(REPO_DIR).with_context(|| {
format!("opening git repo at {REPO_DIR} — provision the card with a clone (just init) whose origin is your remote")
})?;
let Some(mut oid) = stage_and_commit(&repo)? else {
return Ok(PublishOutcome::UpToDate);
};
let branch = repo
.head()?
.shorthand()
@@ -221,17 +263,46 @@ fn publish_once() -> Result<PublishOutcome> {
.to_string();
let refspec = format!("refs/heads/{branch}:refs/heads/{branch}");
// Optimistic push. A non-fast-forward rejection means the remote moved under
// us: reconcile onto origin and replay the note on the new tip, then retry
// once. reconcile_onto_origin mixed-resets, so the just-saved note survives in
// the working tree and stage_and_commit lands it on top of origin.
if let Err(first) = try_push(&repo, &refspec) {
log::warn!("push rejected ({first}); reconciling onto origin and replaying the note");
let mut oid = match stage_and_commit(&repo, paths)? {
Some(oid) => oid,
None => {
// Nothing new to commit. Usually genuinely up to date — but a
// previous cycle may have committed and then failed to push,
// stranding a local-only commit (the old add_all path silently
// never retried those). Push whenever origin's tracking ref
// doesn't already have HEAD.
let head = repo.head()?.peel_to_commit()?.id();
if tracking_tip(&repo, &branch) == Some(head) {
return Ok(PublishOutcome::UpToDate);
}
log::info!(
"tree unchanged but origin/{branch} lacks HEAD {} — pushing the stranded commit",
short(head)
);
head
}
};
// Optimistic push. A non-fast-forward *rejection* means the remote moved
// under us: reconcile onto origin and replay the dirty paths on the new
// tip, then retry once (reconcile_onto_origin soft-resets — ref move only —
// so the notes stay on the card and stage_and_commit splices them on top of
// origin). A transport-level failure is surfaced as-is: its fetch would die
// the same way, and the commit is safe locally — the stranded-commit check
// above pushes it once the transport works again.
if let Err(failure) = try_push(&repo, &refspec) {
let rejection = match failure {
PushFailure::Rejected(msg) => msg,
PushFailure::Other(e) => return Err(e),
};
log::warn!("push rejected ({rejection}); reconciling onto origin and replaying the note");
reconcile_onto_origin(&repo, &branch).context("reconciling after a rejected push")?;
match stage_and_commit(&repo)? {
match stage_and_commit(&repo, paths)? {
Some(replayed) => {
oid = replayed;
try_push(&repo, &refspec).context("push after reconcile")?;
try_push(&repo, &refspec)
.map_err(PushFailure::into_error)
.context("push after reconcile")?;
}
// The note was already on origin (nothing to replay) — treat as done.
None => {
@@ -249,34 +320,59 @@ fn publish_once() -> Result<PublishOutcome> {
Ok(PublishOutcome::Pushed(short(oid)))
}
/// Stage the working tree and commit it on top of the current branch tip.
/// Returns the new commit id, or `None` when the tree already matches the parent
/// (nothing to publish). Called on the first attempt and again to replay the note
/// after a reconcile.
/// Build the commit for `paths` as an O(depth) TreeBuilder splice onto HEAD's
/// tree and return the new commit id or `None` when the result matches the
/// parent (nothing to publish). Called on the first attempt and again to
/// replay the dirty paths after a reconcile.
///
/// `add_all` runs a per-path filter that drops macOS AppleDouble sidecars
/// (`._name`) and `.DS_Store` that Finder/Spotlight sprinkle onto the FAT card
/// whenever it's mounted on a Mac — without it, a blind add --all sweeps them into
/// the commit (it did once: 07d87772 shipped `._.git`, `._README.md`,
/// `._notes.md`). Filtering here fixes it for *every* repo at the device level, so
/// no per-repo `.gitignore` is needed. (add --all also stages deletions, for a
/// future note-delete.)
fn stage_and_commit(repo: &Repository) -> Result<Option<git2::Oid>> {
let mut index = repo.index().context("opening index")?;
let mut skip_macos_cruft = |path: &Path, _matched: &[u8]| -> i32 {
match path.file_name().and_then(|n| n.to_str()) {
Some(name) if name.starts_with("._") || name == ".DS_Store" => 1, // skip
_ => 0, // add
}
};
index
.add_all(["*"], IndexAddOption::DEFAULT, Some(&mut skip_macos_cruft))
.context("staging (add --all)")?;
index.write().context("writing index")?;
let tree = repo.find_tree(index.write_tree().context("writing tree")?)?;
// Commit on top of the current branch tip (None on an empty/unborn remote).
/// This replaces the index pipeline (`add_all` → `index.write` → `write_tree`),
/// which is O(N_tree) and cannot run on the real 1179-file / 570 MB-pack clone:
/// `index.write`'s racy-clean pass re-hashes ~every entry on FAT's 2 s mtimes
/// (measured up to **611 s**), and even the index-free `read_tree` walk was
/// 77 s. The splice reads and writes only the dirty paths' ancestor chains —
/// O(depth × dirty), flat in repo size, **~22.8 s measured on the real
/// clone** — and carries every untouched entry (including the ~150 MB of
/// images) forward by OID without ever opening it. Trail + bench numbers:
/// docs/tradeoff-curves/sync-commit-staging.md.
///
/// The working tree is the source of truth: a recorded path that exists on the
/// card is spliced in from disk, a missing one is spliced out (a `:delete`).
/// Unrecorded paths are never visited — so Finder cruft (`._*`, `.DS_Store`)
/// on the FAT card can no longer ride into a commit the way it once did with
/// `add_all` (07d87772), and the old cruft filter is gone with the walk.
fn stage_and_commit(repo: &Repository, paths: &BTreeSet<String>) -> Result<Option<Oid>> {
// Commit on top of the current branch tip (None on an empty/unborn remote,
// where the splice starts from an empty base and makes a parentless commit).
let parent = repo.head().ok().and_then(|h| h.peel_to_commit().ok());
let base = match &parent {
Some(c) => Some(c.tree().context("loading HEAD tree")?),
None => None,
};
let t_splice = Instant::now();
let mut tree = base;
for path in paths {
let parts: Vec<&str> = path.split('/').filter(|p| !p.is_empty()).collect();
if parts.is_empty() {
continue;
}
let blob = match fs::read(format!("{REPO_DIR}/{path}")) {
Ok(bytes) => Some(
repo.blob(&bytes)
.with_context(|| format!("writing blob for {path}"))?,
),
Err(e) if e.kind() == std::io::ErrorKind::NotFound => None, // deleted → splice out
Err(e) => return Err(e).with_context(|| format!("reading {path}")),
};
let spliced = splice(repo, tree.as_ref(), &parts, blob)
.with_context(|| format!("splicing {path}"))?;
tree = Some(repo.find_tree(spliced).context("loading spliced tree")?);
}
let Some(tree) = tree else {
return Ok(None); // unborn branch and nothing dirty — nothing to commit
};
let splice_ms = t_splice.elapsed().as_millis();
if let Some(p) = &parent {
if p.tree_id() == tree.id() {
log::info!("nothing to publish — tree unchanged @ {}", short(p.id()));
@@ -287,18 +383,115 @@ fn stage_and_commit(repo: &Repository) -> Result<Option<git2::Oid>> {
let sig = Signature::now(AUTHOR_NAME, AUTHOR_EMAIL).context("building signature")?;
let message = format!("Typoena publish — unix {}", now_unix());
let parents: Vec<&Commit> = parent.iter().collect();
let t_commit = Instant::now();
let oid = repo
.commit(Some("HEAD"), &sig, &sig, &message, &tree, &parents)
.context("creating commit")?;
log::info!("committed {} — free heap {}", short(oid), free_heap());
log::info!(
"commit split — splice {splice_ms}ms ({} path(s)), commit-obj {}ms; committed {} — free heap {}",
paths.len(),
t_commit.elapsed().as_millis(),
short(oid),
free_heap()
);
Ok(Some(oid))
}
/// Return a new tree = `base` with `path` set to `blob` (`Some` inserts or
/// replaces, `None` removes). Recurses down the path's subtree chain: reads
/// ~depth tree objects and writes ~depth new ones, leaving every sibling entry
/// untouched (carried by OID — never opened). A missing intermediate directory
/// is synthesized on the way down; a directory emptied by a remove is pruned
/// on the way up rather than left behind as an empty tree entry.
fn splice(repo: &Repository, base: Option<&Tree>, path: &[&str], blob: Option<Oid>) -> Result<Oid> {
let (head, rest) = path.split_first().context("splice: empty path")?;
let mut tb = repo.treebuilder(base).context("treebuilder")?;
if rest.is_empty() {
match blob {
Some(oid) => {
tb.insert(*head, oid, 0o100644)
.context("inserting blob entry")?;
}
// Removing a never-committed path is a no-op, not an error (a note
// created and deleted between two syncs).
None => {
let _ = tb.remove(*head);
}
}
} else {
let sub = match base.and_then(|b| b.get_name(head)) {
Some(e) if e.kind() == Some(ObjectType::Tree) => {
Some(repo.find_tree(e.id()).context("loading subtree")?)
}
// Absent (a new directory) or a non-tree shadowing the name —
// build the subtree from scratch either way.
_ => None,
};
let new_sub = splice(repo, sub.as_ref(), rest, blob)?;
if repo.find_tree(new_sub)?.len() == 0 {
let _ = tb.remove(*head); // the remove emptied this directory — prune it
} else {
tb.insert(*head, new_sub, 0o040000)
.context("inserting subtree entry")?;
}
}
tb.write().context("writing spliced tree")
}
/// Origin's remote-tracking tip for `branch`, if the ref exists. libgit2
/// updates it after a successful push/fetch, so it is "the newest commit we
/// know origin has" — without touching the network.
fn tracking_tip(repo: &Repository, branch: &str) -> Option<Oid> {
repo.find_reference(&format!("refs/remotes/origin/{branch}"))
.ok()?
.peel_to_commit()
.ok()
.map(|c| c.id())
}
/// Whether origin is known to already have HEAD (local refs only, no network).
/// Errors read as "not current", so the caller falls through to the full
/// publish path where the real failure surfaces with context.
fn remote_current() -> Result<bool> {
let repo = Repository::open(REPO_DIR)?;
let head = repo.head()?.peel_to_commit()?.id();
let branch = repo
.head()?
.shorthand()
.context("HEAD has no branch shorthand")?
.to_string();
Ok(tracking_tip(&repo, &branch) == Some(head))
}
/// How a push attempt failed — this decides whether reconciling can help.
enum PushFailure {
/// The server processed the push but refused the ref update (arrives via
/// the `push_update_reference` callback — e.g. non-fast-forward): the
/// remote moved under us, and reconcile + replay is the right response.
Rejected(String),
/// Transport / TLS / auth / URL — the push never reached a ref decision,
/// so a reconcile (whose fetch needs the same transport) cannot help.
/// Surfaced directly; the 2026-07-13 on-device run burned a doomed
/// reconcile on an "unsupported URL protocol" because this wasn't split.
Other(anyhow::Error),
}
impl PushFailure {
fn into_error(self) -> anyhow::Error {
match self {
Self::Rejected(msg) => anyhow::anyhow!("remote rejected ref: {msg}"),
Self::Other(e) => e,
}
}
}
/// One push attempt over HTTPS. Binds the PAT credential + the cert-verify
/// callback, and surfaces a server-side ref rejection (e.g. non-fast-forward) as
/// an error (it arrives via `push_update_reference`, not as a `push()` error).
fn try_push(repo: &Repository, refspec: &str) -> Result<()> {
let mut remote = repo.find_remote("origin")?;
/// callback, and separates a server-side ref rejection (reconcilable) from a
/// transport-level failure (not).
fn try_push(repo: &Repository, refspec: &str) -> Result<(), PushFailure> {
let mut remote = repo
.find_remote("origin")
.map_err(|e| PushFailure::Other(anyhow::Error::new(e).context("finding remote origin")))?;
let rejection: Rc<RefCell<Option<String>>> = Rc::new(RefCell::new(None));
let mut cbs = auth_callbacks();
@@ -316,27 +509,30 @@ fn try_push(repo: &Repository, refspec: &str) -> Result<()> {
opts.remote_callbacks(cbs);
remote
.push(&[refspec], Some(&mut opts))
.context("push transport")?;
.map_err(|e| PushFailure::Other(anyhow::Error::new(e).context("push transport")))?;
if let Some(msg) = rejection.borrow().clone() {
bail!("remote rejected ref: {msg}");
return Err(PushFailure::Rejected(msg));
}
log::info!("push accepted by remote");
Ok(())
}
/// Fetch origin and mixed-reset the local branch onto it, so our just-made commit
/// can be replayed on the current tip. Only runs after a non-fast-forward push
/// Fetch origin and *soft*-reset the local branch onto it, so our changes can
/// be replayed on the current tip. Only runs after a non-fast-forward push
/// rejection — i.e. the remote moved under us.
///
/// **MIXED**, deliberately not a force checkout: the note we're publishing lives
/// in the working tree, and a force checkout would clobber it. Mixed moves the
/// branch ref + index onto origin but leaves the working tree, so the note
/// survives and `stage_and_commit` replays it on top. For a single-writer
/// appliance this resolves last-writer-wins — a concurrent remote *edit* to the
/// same note loses to ours, and a remote-only *added* file the card doesn't have
/// is dropped by the replay's add --all. Both need a real merge (increment B) and
/// don't arise from this device's own use.
/// **SOFT**, deliberately: it moves only the branch ref. The previous Mixed
/// reset also rewrote the index — pure waste now that the splice commit never
/// reads the index, and on the real repo an index write is exactly the
/// racy-clean wall the splice exists to avoid. Neither flavor touches the
/// working tree, so the notes being published survive on the card and the
/// replay splices them onto the new tip. For a single-writer appliance this
/// resolves last-writer-wins: a concurrent remote *edit* to a note we're
/// publishing loses to ours, while a remote-only added/changed file is simply
/// carried forward — origin's tree is now the splice base, so the replay
/// keeps it (an improvement over the old `add --all` replay, which dropped
/// files the card didn't have). A real merge stays increment-B work.
fn reconcile_onto_origin(repo: &Repository, branch: &str) -> Result<()> {
let mut remote = repo.find_remote("origin")?;
let mut fo = FetchOptions::new();
@@ -350,12 +546,12 @@ fn reconcile_onto_origin(repo: &Repository, branch: &str) -> Result<()> {
.context("no FETCH_HEAD after fetch")?;
let theirs = repo.reference_to_annotated_commit(&fetch_head)?;
log::info!(
"reconcile: resetting local {branch} onto origin @ {} (mixed, keeps the note)",
"reconcile: resetting local {branch} onto origin @ {} (soft — ref move only, notes stay on the card)",
short(theirs.id())
);
let their_obj = repo.find_object(theirs.id(), None)?;
repo.reset(&their_obj, git2::ResetType::Mixed, None)
.context("mixed reset onto origin")?;
repo.reset(&their_obj, git2::ResetType::Soft, None)
.context("soft reset onto origin")?;
Ok(())
}

View File

@@ -10,7 +10,10 @@ use esp_idf_svc::hal::spi::{Dma, SpiBusDriver, SpiDriver};
use esp_idf_svc::hal::units::FromValueType;
use display::Frame;
use editor::{Editor, Effect, Mode, CH};
use editor::{
Editor, Effect, Mode, Prefs, Scope, Snippets, CH, LOCAL_DIR, PREFS_PATH, REPO_DIR,
SNIPPETS_PATH,
};
use firmware::epd::{self, Epd};
use firmware::persistence::{Storage, NOTES};
@@ -26,6 +29,12 @@ const FULL_REFRESH_EVERY: u32 = 64;
/// reappears once you settle. Normal/View draw their own caret every action.
const CURSOR_DEBOUNCE_MS: u128 = 750;
/// How long input must pause before `save_on_idle` persists a dirty buffer.
/// Longer than the caret debounce so autosave settles after typing, not during
/// a mid-sentence pause. The save is silent (no snackbar, no forced e-ink
/// flash) — a safety net against power loss, not a user action.
const IDLE_SAVE_MS: u128 = 1500;
fn main() -> anyhow::Result<()> {
// Required once before any esp-idf-svc call; some runtime patches
// only link if this symbol is referenced. See esp-idf-template#71.
@@ -101,7 +110,9 @@ fn main() -> anyhow::Result<()> {
// Seed the editor from the saved note. Boots in Normal mode with the caret
// on the last character (the resume point) — press `i`/`a`/`o` to write.
let mut ed = Editor::with_text(saved);
// The boot note is Tracked (`/sd/repo/notes.md`); `:e` / the palette (v0.5)
// open others, Tracked or Local.
let mut ed = Editor::with_file(NOTES.to_string(), Scope::Tracked, saved);
// Confirm the boot-load on the panel (no serial console in normal use):
// "loaded <name>" using the note's filename without its suffix (notes.md ->
// notes). Cleared by the first keystroke, like any snackbar.
@@ -110,9 +121,44 @@ fn main() -> anyhow::Result<()> {
.and_then(|s| s.to_str())
.unwrap_or("notes");
ed.set_notice(format!("loaded {name}"));
// Feed the file palette (Ctrl-P). Enumerated once at boot — the v0.5 slices
// that create/delete files (`:enew`, delete) will re-feed it then.
ed.set_file_list(enumerate_files());
// Editor preferences (.typoena.toml, git-tracked). Read before the first
// render so `line_numbers` shapes the opening frame. A missing / unreadable /
// partial file falls back to defaults, so a fresh card just works.
let prefs = match storage.load_path(PREFS_PATH) {
Ok(src) => Prefs::parse(&src),
Err(_) => Prefs::default(),
};
log::info!("prefs: {prefs:?}");
ed.set_prefs(prefs);
// Snippet library (.typoena.snippets.json, git-tracked). Parsed with
// serde_json in the editor crate; a missing / unreadable / malformed file is
// non-fatal — the editor simply has no snippets and runs unchanged.
let snippets = match storage.load_path(SNIPPETS_PATH) {
Ok(src) => match Snippets::parse(&src) {
Ok(s) => s,
Err(e) => {
log::warn!("snippets parse FAILED ({e}); none loaded");
Snippets::default()
}
},
Err(_) => Snippets::default(),
};
log::info!("snippets: {} loaded", snippets.0.len());
ed.set_snippets(snippets);
let mut updates: u32 = 0;
let mut cursor_shown = true; // the initial render includes the caret
let mut last_activity = Instant::now();
// Whether `save_on_idle` already persisted the current idle window, so it
// fires once per typing burst (and doesn't retry-storm if a save fails).
// Reset on the next activity.
let mut idle_saved = false;
// Set when a paint fails (see the refresh block below): the next paint then
// does a full refresh to re-establish both RAM banks, since a partial that
// died mid-transfer may have left them inconsistent.
let mut force_full = false;
// Keyboard attach/detach state drives the panel's disconnect flag; seed it
// (and the word-count snapshot) before the first render.
@@ -145,35 +191,68 @@ fn main() -> anyhow::Result<()> {
// Drain all queued keystrokes (type-ahead absorbed during a refresh),
// apply them, then do a single refresh for the batch.
let mut keys = 0;
let mut effect = Effect::None;
while let Some(k) = usb_kbd::next_key() {
// A `:` command (only) yields an Effect; keep the last one in the batch.
match ed.handle(k) {
Effect::None => {}
e => effect = e,
}
ed.handle(k);
keys += 1;
}
// Carry out any host-side effect a `:` command asked for. Save is inline
// (fast). `:sync` persists, then hands off to the git thread the push
// is behind the `git` feature, so a light build carries no libgit2/git2.
match effect {
Effect::None => {}
Effect::Save => save_note(&storage, &mut ed),
Effect::Publish => {
// Publishing an unsaved buffer is meaningless, so persist first.
save_note(&storage, &mut ed);
// Then signal the git thread — non-blocking, so the ~10 s push
// never stalls the editor. The outcome returns on `git_rx` and
// updates the snackbar (see the idle branch below).
#[cfg(feature = "git")]
match git_tx.send(firmware::git_sync::PublishRequest) {
Ok(()) => ed.set_notice("syncing..."),
Err(_) => ed.set_notice("sync: git thread down"),
// Service the host-side effects the batch queued, in order. A file open
// queues a Save of the outgoing dirty buffer *then* a Load of the target;
// `:sync` queues a Save of the current buffer *then* Publish. Save/Load
// are inline (fast SD IO); Publish hands off to the git thread — behind
// the `git` feature, so a light build carries no libgit2/git2.
//
// Drain to empty rather than once: servicing a Load can itself queue an
// eviction Save (when the swap pushes a dirty parked buffer out of the
// ≤3 window), and that must be persisted now, not deferred to the next
// keystroke where a power-off could lose it. The queue strictly shrinks
// (a Save/Publish/Pull queues nothing; a Load queues at most one Save),
// so this terminates.
loop {
let effects = ed.take_effects();
if effects.is_empty() {
break;
}
for effect in effects {
match effect {
Effect::Save { path, contents, .. } => {
save_buffer(&storage, &mut ed, &path, &contents)
}
Effect::Load { path, scope } => open_buffer(&storage, &mut ed, path, scope),
Effect::Publish => {
// Non-blocking, so the ~10 s push never stalls the editor.
// The outcome returns on `git_rx` and updates the snackbar
// (see the idle branch below). The Save that preceded this
// in the batch already persisted the buffer, so this is a
// pure git publish of the recorded dirty paths — the
// outcome decides whether the snapshot is forgotten
// (publish_succeeded) or retried (publish_failed).
#[cfg(feature = "git")]
{
let paths = storage.take_dirty();
match git_tx.send(firmware::git_sync::PublishRequest { paths }) {
Ok(()) => ed.set_notice("syncing..."),
Err(_) => {
// Thread gone — nothing will report back, so
// return the snapshot to pending ourselves.
storage.publish_failed();
ed.set_notice("sync: git thread down");
}
}
}
#[cfg(not(feature = "git"))]
log::info!(":sync — saved; light build (no `git` feature) — push skipped");
}
Effect::Pull => {
// `:gl` — fetch + fast-forward from the remote. The
// on-device fetch/fast-forward on the git thread is v0.7
// work (git_sync only exposes push today), so acknowledge
// and no-op for now.
ed.set_notice("pull: not wired yet (v0.7)");
}
Effect::Delete { path, scope } => delete_buffer(&storage, &mut ed, path, scope),
Effect::SavePrefs { contents } => save_prefs(&storage, &mut ed, &contents),
}
#[cfg(not(feature = "git"))]
log::info!(":sync — saved; light build (no `git` feature) — push skipped");
}
}
@@ -190,13 +269,24 @@ fn main() -> anyhow::Result<()> {
#[cfg(feature = "git")]
if let Ok(outcome) = git_rx.try_recv() {
use firmware::git_sync::PublishOutcome::*;
// Settle the dirty snapshot this publish took: confirmed
// published (or up to date) → forget it; failed → back to
// pending so the next :sync retries the same paths.
match &outcome {
Pushed(_) | UpToDate => storage.publish_succeeded(),
Failed(_) => storage.publish_failed(),
}
ed.set_notice(match outcome {
Pushed(oid) => format!("synced {oid}"),
UpToDate => "up to date".to_string(),
Failed(reason) => reason,
});
let f = ed.draw(true);
epd.display_frame_partial_window(f.bytes(), 0, epd::HEIGHT)?;
if let Err(e) = epd.display_frame_partial_window(f.bytes(), 0, epd::HEIGHT) {
log::warn!("sync-notice repaint FAILED ({e}); full refresh next");
force_full = true;
continue;
}
shown = f;
cursor_shown = true;
continue;
@@ -205,12 +295,40 @@ fn main() -> anyhow::Result<()> {
// no keystroke will arrive to trigger it otherwise.
if kbd_changed {
let f = ed.draw(true);
epd.display_frame_partial_window(f.bytes(), 0, epd::HEIGHT)?;
if let Err(e) = epd.display_frame_partial_window(f.bytes(), 0, epd::HEIGHT) {
log::warn!("kbd-flag repaint FAILED ({e}); full refresh next");
force_full = true;
continue;
}
shown = f;
cursor_shown = true;
log::info!("keyboard {}", if kbd { "connected" } else { "disconnected" });
continue;
}
// save_on_idle: once input has paused, quietly persist a dirty named
// buffer so a power pull can't cost more than the last couple seconds.
// Silent — no snackbar and no forced e-ink flash (a safety net, not an
// action; `:w` is the loud save). Unformatted: fmt only runs on an
// explicit `:w`/`:sync`, never reflowing text mid-session. Fires once
// per idle window (`idle_saved`), so a failing save can't busy-loop.
if !idle_saved
&& ed.prefs().save_on_idle
&& ed.dirty()
&& !ed.path().is_empty()
&& last_activity.elapsed().as_millis() >= IDLE_SAVE_MS
{
idle_saved = true;
let path = ed.path().to_string();
match storage.save_path(&path, ed.text()) {
Ok(()) => {
log::info!("idle-save: {} bytes to {path}", ed.text().len());
ed.mark_saved(&path);
}
Err(e) => log::warn!("idle-save FAILED ({e:#}); buffer kept in RAM"),
}
// No repaint: `dirty` clearing has no visible effect, and a flash
// here would defeat the point. Fall through to the caret/idle path.
}
// Debounced caret, Insert mode only: once typing pauses, bring the
// bar caret back and refresh the panel word count with a silent
// full-area partial (no flash). Normal/View draw their caret on action.
@@ -220,10 +338,14 @@ fn main() -> anyhow::Result<()> {
{
ed.refresh_stats();
let f = ed.draw(true);
epd.display_frame_partial_window(f.bytes(), 0, epd::HEIGHT)?;
shown = f;
cursor_shown = true;
log::info!("caret shown");
if let Err(e) = epd.display_frame_partial_window(f.bytes(), 0, epd::HEIGHT) {
log::warn!("caret repaint FAILED ({e}); full refresh next");
force_full = true;
} else {
shown = f;
cursor_shown = true;
log::info!("caret shown");
}
} else {
FreeRtos::delay_ms(8);
}
@@ -231,6 +353,7 @@ fn main() -> anyhow::Result<()> {
}
last_activity = Instant::now();
idle_saved = false; // fresh activity reopens the save_on_idle window
// Non-Insert actions (Normal edits, mode switches) aren't rapid typing,
// so the panel word count can refresh immediately; in Insert the snapshot
// stays frozen until the typing-pause path above refreshes it.
@@ -267,17 +390,29 @@ fn main() -> anyhow::Result<()> {
&& only_adds_ink(shown.bytes(), frame.bytes(), y0, y1);
let t0 = Instant::now();
let refresh = if periodic {
epd.display_frame(frame.bytes())?;
"FULL"
// `force_full` promotes to a full refresh after a failed paint: it
// rewrites both RAM banks, recovering from a partial that may have died
// mid-transfer and desynced them.
let (result, refresh) = if periodic || force_full {
(epd.display_frame(frame.bytes()), "FULL")
} else if additive {
epd.display_frame_partial_window(frame.bytes(), y0, y1 - y0 + 1)?;
"windowed"
(epd.display_frame_partial_window(frame.bytes(), y0, y1 - y0 + 1), "windowed")
} else {
epd.display_frame_partial_window(frame.bytes(), 0, epd::HEIGHT)?;
"full-area"
(epd.display_frame_partial_window(frame.bytes(), 0, epd::HEIGHT), "full-area")
};
let ms = t0.elapsed().as_millis();
if let Err(e) = result {
// Never fatal — the buffer is the source of truth and safe in RAM,
// exactly like a failed `save_buffer`. Drop this frame, leave `shown`
// untouched so the next paint repaints the same diff, and force a
// clean full refresh then. Typical cause: internal DMA-capable RAM
// briefly starved by Wi-Fi/TLS during a background `:sync`; it frees
// the moment the push finishes.
log::warn!("{refresh} refresh #{updates} FAILED ({e}); frame dropped, full refresh next");
force_full = true;
continue;
}
force_full = false;
log::info!(
"{refresh} refresh #{updates} [{:?}]: {ms} ms (rows {y0}..={y1}, {keys} key(s))",
ed.mode()
@@ -292,7 +427,15 @@ fn main() -> anyhow::Result<()> {
/// `main`): the note is the whole point of the appliance, so we refuse to run
/// in a state where the next save could destroy it.
fn boot_storage(epd: &mut Epd) -> (Storage, String) {
let storage = match Storage::mount() {
// A git build shares this mount with the git thread, and libgit2 keeps the
// pack + idx descriptors open across a publish — that overruns the
// editor's tight 4-FD budget, so mount with the 16-FD one (persistence.rs,
// MAX_FILES_GIT). The light build keeps the editor's own budget.
#[cfg(feature = "git")]
let mounted = Storage::mount_for_git();
#[cfg(not(feature = "git"))]
let mounted = Storage::mount();
let storage = match mounted {
Ok(s) => s,
Err(e) => boot_halt(epd, "SD card not ready", &format!("{e:#}")),
};
@@ -331,23 +474,152 @@ fn show_message(epd: &mut Epd, msg: &str) -> anyhow::Result<()> {
Ok(())
}
/// Persist the buffer to SD. Errors are logged, never propagated: the in-RAM
/// buffer is the source of truth and must survive a failed write (e.g. a card
/// pulled mid-session) so the user can fix the card and retry `:w`.
fn save_note(storage: &Storage, ed: &mut Editor) {
let n = ed.text().len();
match storage.save(ed.text()) {
/// Persist a buffer to SD at `path`. Errors are logged, never propagated: the
/// in-RAM buffer is the source of truth and must survive a failed write (e.g. a
/// card pulled mid-session) so the user can fix the card and retry `:w`. On
/// success the editor's dirty flag for that path is cleared.
fn save_buffer(storage: &Storage, ed: &mut Editor, path: &str, contents: &str) {
match storage.save_path(path, contents) {
Ok(()) => {
log::info!(":w — saved {n} bytes to {NOTES}");
log::info!(":w — saved {} bytes to {path}", contents.len());
ed.mark_saved(path);
ed.set_notice("saved");
}
Err(e) => {
log::error!(":w — save FAILED ({e:#}); buffer kept in RAM, retry :w");
log::error!("save FAILED ({e:#}); buffer kept in RAM, retry :w");
ed.set_notice("save FAILED - retry :w");
}
}
}
/// Persist the preferences file after a palette `>` command changed a pref
/// (`Effect::SavePrefs`). The editor already applied the change live and
/// serialized it; this is a plain atomic write to the fixed `.typoena.toml`
/// path. Under `/sd/repo`, so it rides the next `:sync` to other devices.
fn save_prefs(storage: &Storage, ed: &mut Editor, contents: &str) {
match storage.save_path(PREFS_PATH, contents) {
Ok(()) => log::info!("prefs saved to {PREFS_PATH}"),
Err(e) => {
log::error!("prefs save FAILED ({e:#})");
ed.set_notice("prefs save FAILED");
}
}
}
/// Read `path` from SD and install it as the active buffer (the multi-file open
/// path, from `:e` / the palette). A read failure keeps the current buffer and
/// surfaces the reason on the snackbar rather than swapping to an empty screen.
fn open_buffer(storage: &Storage, ed: &mut Editor, path: String, scope: Scope) {
match storage.load_path(&path) {
Ok(text) => {
log::info!("opened {path} ({} bytes, {scope:?})", text.len());
let name = file_stem(&path);
ed.set_notice(format!("loaded {name}"));
ed.install_loaded(path, scope, text);
}
Err(e) => {
log::error!("open {path} FAILED ({e:#})");
ed.set_notice(format!("can't open {}", file_stem(&path)));
}
}
}
/// Unlink a file from the card (`:delete`). The editor has already dropped it
/// from its model and switched away, so this is pure IO plus the snackbar. For a
/// Tracked file the removal is left in the git working copy — the next `:sync`'s
/// `add --all` stages the deletion — so nothing git-specific happens here. A
/// failure keeps the file on disk and says so; the buffer has still switched, so
/// the file is recoverable by re-opening it.
fn delete_buffer(storage: &Storage, ed: &mut Editor, path: String, scope: Scope) {
// Scope-qualified label (`repo/notes.md`), so the snackbar names exactly which
// file left the card — and, for a Tracked file, that the removal is only local
// until the next `:sync` publishes it (deleting from the card alone never
// touches the remote — that mirrors how a Save is local until Publish).
let label = path.strip_prefix("/sd/").unwrap_or(&path);
match storage.delete_path(&path) {
Ok(()) => {
log::info!("deleted {path} ({scope:?})");
ed.set_notice(match scope {
Scope::Tracked => format!("deleted {label} - :sync to publish"),
Scope::Local => format!("deleted {label}"),
});
}
Err(e) => {
log::error!("delete {path} FAILED ({e:#})");
ed.set_notice(format!("delete FAILED: {label}"));
}
}
}
/// Enumerate the palette's openable files: the regular files under `/sd/repo`
/// and `/sd/local`, recursively, as absolute paths. Skips dot entries at every
/// level (so `.git` and its thousands of object files, `.typoena.toml`, and the
/// like never show or get walked). Best-effort: an unreadable directory (e.g.
/// no `/sd/local` yet) contributes nothing rather than failing. The editor
/// sorts and dedupes. Runs once at boot, so the walk time is logged — on a big
/// repo the FAT directory IO is the cost to watch.
fn enumerate_files() -> Vec<String> {
let start = std::time::Instant::now();
let mut out = Vec::new();
for dir in [REPO_DIR, LOCAL_DIR] {
walk_files(std::path::Path::new(dir), 0, &mut out);
}
log::info!("file walk: {} files in {}ms", out.len(), start.elapsed().as_millis());
out
}
/// Depth bound for [`walk_files`] — belt-and-braces against pathological
/// nesting on a hand-edited card; notes trees are a couple of levels deep.
const WALK_MAX_DEPTH: usize = 8;
/// Recursive helper for [`enumerate_files`]: push `dir`'s files onto `out`,
/// then descend into its subdirectories. Reads each directory fully before
/// recursing (the `remove_dir_recursive` pattern in `git_sync`), so only one
/// FatFS directory handle is open at a time regardless of depth — relevant on
/// the FD-bounded SD mount.
fn walk_files(dir: &std::path::Path, depth: usize, out: &mut Vec<String>) {
if depth > WALK_MAX_DEPTH {
log::warn!("file walk: {} exceeds depth {WALK_MAX_DEPTH}, skipped", dir.display());
return;
}
let Ok(entries) = std::fs::read_dir(dir) else {
return;
};
// Keep the dirent's own file type: esp-idf's FAT VFS always fills d_type
// (DT_DIR/DT_REG, straight from the FILINFO readdir already holds), so
// `file_type()` is free. A per-entry `metadata()` stat instead re-walks
// the directory by path every time — measured at ~32ms/file on the SD
// card, it turned a 1098-file walk into 35s.
let children: Vec<_> = entries
.flatten()
.filter_map(|e| e.file_type().ok().map(|t| (e.path(), t)))
.collect();
for (path, ftype) in children {
let Some(name) = path.file_name().and_then(|n| n.to_str()) else {
continue;
};
if name.starts_with('.') {
continue;
}
if ftype.is_file() {
if let Some(p) = path.to_str() {
out.push(p.to_string());
}
} else if ftype.is_dir() {
walk_files(&path, depth + 1, out);
}
}
}
/// A file's display name — its basename without extension (`/sd/repo/notes.md`
/// → `notes`), for the snackbar. Falls back to the raw path if it has no stem.
fn file_stem(path: &str) -> &str {
std::path::Path::new(path)
.file_stem()
.and_then(|s| s.to_str())
.unwrap_or(path)
}
/// First and last (inclusive) framebuffer rows that differ between two frames,
/// or `None` if identical. Lets the partial refresh target just the band a
/// keystroke touched instead of all 272 rows.

View File

@@ -34,6 +34,8 @@
//! sits in the tmp. [`Storage::recover`] closes the loop at boot — see its docs
//! for the exact case analysis, which is subtler than "promote the tmp."
use std::cell::RefCell;
use std::collections::BTreeSet;
use std::fs;
use std::io::Write as _;
use std::mem::MaybeUninit;
@@ -79,6 +81,23 @@ pub const MAX_FILE_BYTES: u64 = 256 * 1024;
/// The C mount point (`/sd\0`) for the esp-idf FFI calls.
const MOUNT_C: &std::ffi::CStr = c"/sd";
/// Dirty-path journal — one repo-relative path per line, mirroring the in-RAM
/// dirty set (see [`Storage::take_dirty`]). At the card root, *outside*
/// `/sd/repo`, so it can never itself be committed. Without it a power pull
/// would strand every file saved-but-not-yet-published in that session: the
/// splice commit only visits recorded paths (nothing walks the tree anymore),
/// so an unrecorded change would never reach the remote.
const DIRTY_JOURNAL: &str = "/sd/.typoena-dirty";
/// VFS open-file budget for the editor path: it opens only a note and its
/// `*.tmp`, so a tight budget keeps FatFS's per-file buffers off the heap.
const MAX_FILES_EDITOR: i32 = 4;
/// VFS open-file budget for the git tooling. libgit2 keeps the pack + `.idx`
/// (and commit-graph) descriptors open for the repo's lifetime and opens loose
/// objects on top, so a `read_tree` walk overruns [`MAX_FILES_EDITOR`] with a
/// "no free file descriptors" error. Matches the flash-FAT git binaries' 16.
const MAX_FILES_GIT: i32 = 16;
/// A mounted SD card. Holds the live card handle for its lifetime; v0.1 never
/// unmounts (the card stays up for the whole power session). Not `Send` — the
/// handle lives on the task that mounted it (the ui/main task). The git thread
@@ -86,6 +105,23 @@ const MOUNT_C: &std::ffi::CStr = c"/sd";
/// lock serialises the two, so no extra mutex is needed here.
pub struct Storage {
card: *mut sys::sdmmc_card_t,
/// Repo-relative paths saved or `:delete`d since the last confirmed
/// publish — the editor-side half of the O(depth) splice commit
/// (`git_sync::stage_and_commit` visits exactly these paths and nothing
/// else). Mirrored to [`DIRTY_JOURNAL`] whenever it changes, so the record
/// survives a power pull. `RefCell` because recording happens inside
/// `&self` save/delete calls; `Storage` already lives on one task only.
dirty: RefCell<Dirty>,
}
/// The two halves of the dirty record: `pending` accumulates between syncs;
/// `take_dirty` moves it to `in_flight` for the duration of a publish so a
/// failure can put it back (and a save landing *during* the publish re-enters
/// `pending`, riding the next one). The journal always carries the union.
#[derive(Default)]
struct Dirty {
pending: BTreeSet<String>,
in_flight: BTreeSet<String>,
}
/// What [`Storage::recover`] did with a leftover `*.tmp` at boot.
@@ -113,6 +149,17 @@ impl Storage {
/// (The Spike 3 bench binary sets it true for convenience on blank cards;
/// this path must not.)
pub fn mount() -> Result<Self> {
Self::mount_with_max_files(MAX_FILES_EDITOR)
}
/// Like [`Storage::mount`], but with the larger [`MAX_FILES_GIT`] open-file
/// budget the git tooling (bench / sync) needs — libgit2 holds several
/// descriptors open at once, which the editor's default budget can't cover.
pub fn mount_for_git() -> Result<Self> {
Self::mount_with_max_files(MAX_FILES_GIT)
}
fn mount_with_max_files(max_files: i32) -> Result<Self> {
// 1) SPI3 with the SD's four lines. Dedicated bus (ADR-012) — no EPD
// deselect needed: the panel is on SPI2 and can't contend here.
// SAFETY: a zeroed spi_bus_config_t is valid (all pins default 0); we
@@ -177,7 +224,7 @@ impl Storage {
// 4) Mount config. format_if_mount_failed = FALSE — see method docs.
let mount = sys::esp_vfs_fat_mount_config_t {
format_if_mount_failed: false,
max_files: 4,
max_files,
allocation_unit_size: 16 * 1024,
disk_status_check_enable: false,
use_one_fat: false,
@@ -203,7 +250,10 @@ impl Storage {
}
esp!(rc).context("esp_vfs_fat_sdspi_mount (card present? inserted? FAT-formatted?)")?;
let storage = Storage { card };
let storage = Storage {
card,
dirty: RefCell::new(Dirty::default()),
};
let (max_khz, real_khz) = storage.negotiated_khz();
log::info!("SD mounted at {MOUNT} — max {max_khz} kHz, negotiated {real_khz} kHz");
@@ -218,6 +268,13 @@ impl Storage {
newest complete copy)"
),
}
let carried = storage.load_dirty_journal();
if carried > 0 {
log::info!(
"dirty journal: {carried} unpublished path(s) carried over from a previous \
session — the next :sync will commit them"
);
}
Ok(storage)
}
@@ -250,45 +307,189 @@ impl Storage {
Path::new(REPO_DIR).is_dir()
}
/// Read `notes.md` into a `String`. Returns an empty string if the file
/// doesn't exist yet (fresh, but provisioned, repo). Refuses a file larger
/// than [`MAX_FILE_BYTES`] rather than loading it.
/// Read `notes.md` into a `String` — the boot default note. Thin wrapper over
/// [`Storage::load_path`].
pub fn load(&self) -> Result<String> {
match fs::metadata(NOTES) {
self.load_path(NOTES)
}
/// Read an arbitrary file under `/sd` into a `String`. Returns an empty string
/// if the file doesn't exist yet (a `:e` of a not-yet-created name, or a fresh
/// repo). Refuses a file larger than [`MAX_FILE_BYTES`] rather than loading it.
///
/// The multi-file (v0.5) load path: the editor names the file, the host reads
/// it here and hands the text back through `Editor::install_loaded`.
pub fn load_path(&self, path: &str) -> Result<String> {
match fs::metadata(path) {
Ok(m) if m.len() > MAX_FILE_BYTES => bail!(
"{NOTES} is {} KiB — over the {} KiB v0.1 limit; open it on a computer to split it",
"{path} is {} KiB — over the {} KiB limit; open it on a computer to split it",
m.len() / 1024,
MAX_FILE_BYTES / 1024
),
Ok(_) => fs::read_to_string(NOTES).with_context(|| format!("reading {NOTES}")),
// Read the file verbatim. The editor's `rows = #\n + 1` model renders a
// trailing '\n' as an empty last line, and we *want* that: a note ends
// with a visible blank line that reflects its POSIX terminator. Since
// `save_path` guarantees that terminator, this load and that save form an
// identity round-trip for any device-written file (which always ends in
// '\n') — no strip needed, and none wanted.
Ok(_) => fs::read_to_string(path).with_context(|| format!("reading {path}")),
Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(String::new()),
Err(e) => Err(e).with_context(|| format!("stat {NOTES}")),
Err(e) => Err(e).with_context(|| format!("stat {path}")),
}
}
/// Atomically persist `contents` to `notes.md`: write the tmp, fsync,
/// unlink the target, rename over it. See the module docs for why the unlink
/// is mandatory on FAT and [`Storage::recover`] for the crash window it opens.
/// Atomically persist `contents` to `notes.md`. Thin wrapper over
/// [`Storage::save_path`].
pub fn save(&self, contents: &str) -> Result<()> {
self.save_path(NOTES, contents)
}
/// Atomically persist `contents` to an arbitrary file under `/sd`: write the
/// tmp, fsync, unlink the target, rename over it. See the module docs for why
/// the unlink is mandatory on FAT. Boot recovery ([`Storage::recover`]) still
/// only covers the default `notes.md`; per-file recovery for the other v0.5
/// buffers is deferred to the v0.9 crash-safety work — the atomic swap here
/// already protects each individual save.
pub fn save_path(&self, path: &str, contents: &str) -> Result<()> {
// Record BEFORE writing: a crash in between leaves an over-approximate
// journal (the splice of an unchanged path is a no-op), whereas the
// reverse order could leave a changed file no record ever points at.
self.record_dirty(path);
Self::atomic_write(path, contents)
}
/// The atomic write primitive behind [`Storage::save_path`] and the dirty
/// journal: write `{path}.tmp`, fsync, unlink the target, rename over it.
fn atomic_write(path: &str, contents: &str) -> Result<()> {
let tmp = format!("{path}.tmp");
{
let mut f = fs::File::create(NOTES_TMP)
.with_context(|| format!("create {NOTES_TMP} (is {REPO_DIR} present?)"))?;
let mut f = fs::File::create(&tmp)
.with_context(|| format!("create {tmp} (does its directory exist?)"))?;
f.write_all(contents.as_bytes())
.with_context(|| format!("write {NOTES_TMP}"))?;
.with_context(|| format!("write {tmp}"))?;
// Insert a final newline only if the buffer lacks one (POSIX text
// convention; keeps git from flagging "No newline at end of file").
// `load_path` reads verbatim, so this is the sole place the terminator is
// guaranteed — and because it's guarded, the file mirrors the buffer's
// trailing newlines exactly: one visible trailing blank line stays one,
// never doubled. A buffer that already ends in '\n' passes through as-is.
if !contents.ends_with('\n') {
f.write_all(b"\n")
.with_context(|| format!("write final newline to {tmp}"))?;
}
// FatFS f_sync — flush the tmp fully before it can replace the target.
f.sync_all().with_context(|| format!("fsync {NOTES_TMP}"))?;
f.sync_all().with_context(|| format!("fsync {tmp}"))?;
}
// FatFS f_rename won't overwrite, so unlink the target first (tolerate a
// missing target: the first-ever save has nothing to remove).
match fs::remove_file(NOTES) {
match fs::remove_file(path) {
Ok(()) => {}
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {}
Err(e) => return Err(e).with_context(|| format!("unlink {NOTES} before rename")),
Err(e) => return Err(e).with_context(|| format!("unlink {path} before rename")),
}
fs::rename(NOTES_TMP, NOTES).with_context(|| format!("rename {NOTES_TMP} -> {NOTES}"))?;
fs::rename(&tmp, path).with_context(|| format!("rename {tmp} -> {path}"))?;
Ok(())
}
/// Unlink a file under `/sd` (`:delete`). Tolerates a missing target — an
/// already-gone file is a success, so the call is idempotent. Also clears a
/// stray `{path}.tmp` best-effort, so a crash-interrupted save can't leave the
/// file half-present after a delete. For a Tracked file this leaves the
/// working copy short one file; the next publish's `add --all` stages it.
pub fn delete_path(&self, path: &str) -> Result<()> {
// Same record-first rule as `save_path`: the splice treats a recorded
// path with no file behind it as "remove from the tree".
self.record_dirty(path);
let _ = fs::remove_file(format!("{path}.tmp"));
match fs::remove_file(path) {
Ok(()) => Ok(()),
Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(()),
Err(e) => Err(e).with_context(|| format!("unlink {path}")),
}
}
/// Note a working-copy file as (possibly) differing from HEAD. Paths
/// outside `/sd/repo` (`/sd/local`, `/sd/ca.pem`, the journal itself) are
/// not git's business and are skipped. The journal is rewritten only when
/// the set actually grows, so re-saving the same note between syncs costs
/// no extra card I/O.
fn record_dirty(&self, abs_path: &str) {
let Some(rel) = abs_path
.strip_prefix(REPO_DIR)
.and_then(|r| r.strip_prefix('/'))
else {
return;
};
if rel.is_empty() {
return;
}
let grew = self.dirty.borrow_mut().pending.insert(rel.to_string());
if grew {
self.persist_dirty();
}
}
/// Snapshot the dirty paths for a publish (repo-relative). The snapshot
/// moves to `in_flight` — the journal keeps carrying it — until the UI
/// task reports the outcome: [`Storage::publish_succeeded`] forgets it,
/// [`Storage::publish_failed`] returns it to pending for the next `:sync`.
pub fn take_dirty(&self) -> BTreeSet<String> {
let mut d = self.dirty.borrow_mut();
let taken = std::mem::take(&mut d.pending);
d.in_flight.extend(taken.iter().cloned());
taken
}
/// The publish that took the last snapshot committed (or confirmed
/// up-to-date): drop its paths and shrink the journal. Anything saved
/// while it ran is still in `pending` and rides the next sync.
pub fn publish_succeeded(&self) {
self.dirty.borrow_mut().in_flight.clear();
self.persist_dirty();
}
/// The publish failed: return its snapshot to pending so the next `:sync`
/// retries it (the splice is idempotent, so a retry of an already-clean
/// path is free). The journal already carries these paths — no rewrite.
pub fn publish_failed(&self) {
let mut d = self.dirty.borrow_mut();
let inflight = std::mem::take(&mut d.in_flight);
d.pending.extend(inflight);
}
/// Mirror `pending in_flight` to [`DIRTY_JOURNAL`], atomically.
/// Best-effort: a failed journal write must not fail the save that
/// triggered it — the set stays correct in RAM and the journal heals on
/// the next change.
fn persist_dirty(&self) {
let contents = {
let d = self.dirty.borrow();
let mut out = String::new();
for p in d.pending.union(&d.in_flight) {
out.push_str(p);
out.push('\n');
}
out
};
if let Err(e) = Self::atomic_write(DIRTY_JOURNAL, &contents) {
log::warn!("dirty journal write FAILED ({e:#}); set kept in RAM only");
}
}
/// Seed the dirty set from the journal at mount — the paths a previous
/// session saved but never got confirmed as published (power pull, failed
/// sync, or simply no `:sync` before shutdown). Returns how many.
fn load_dirty_journal(&self) -> usize {
let Ok(text) = fs::read_to_string(DIRTY_JOURNAL) else {
return 0; // no journal yet — nothing carried over
};
let mut d = self.dirty.borrow_mut();
for line in text.lines().map(str::trim).filter(|l| !l.is_empty()) {
d.pending.insert(line.to_string());
}
d.pending.len()
}
/// Reconcile a leftover `notes.md.tmp` at boot. The save sequence is
/// write-tmp → fsync → unlink-target → rename, so a lingering tmp means the
/// last save was interrupted. Which way to recover depends on whether the

View File

@@ -30,6 +30,23 @@ pub enum Key {
DeleteWord,
/// Cmd/GUI+Backspace — delete back to the start of the current line.
DeleteLine,
/// Ctrl+D — scroll down half a screen (vim `Ctrl-d`).
HalfPageDown,
/// Ctrl+U — scroll up half a screen (vim `Ctrl-u`).
HalfPageUp,
/// Ctrl+R — redo (vim `Ctrl-r`); the inverse of `u`. Meaningful in Normal;
/// ignored elsewhere.
Redo,
/// Cmd+P — open the file palette (fuzzy open, v0.5), VS Code "Go to File"
/// style. A Normal-mode gesture; **inside** the palette the same chord closes
/// it (toggle). Esc also closes. Ignored in Insert.
Palette,
/// Ctrl+N — move down: one line in Normal/View (vim `CTRL-N` ≡ `j`), or one
/// row in the file palette. Ignored in Insert.
Down,
/// Ctrl+P — move up: one line in Normal/View (vim `CTRL-P` ≡ `k`), or one row
/// in the file palette. Ignored in Insert.
Up,
/// Caps Lock tapped on its own. A no-op for now; groundwork for a future
/// vim-style normal mode.
Escape,
@@ -140,6 +157,12 @@ fn translate(usage: u8, shift: bool, ctrl: bool, cmd: bool) -> Option<Key> {
});
}
0x1a if ctrl => return Some(Key::DeleteWord), // Ctrl+W, readline-style
0x07 if ctrl => return Some(Key::HalfPageDown), // Ctrl+D, half-page down
0x18 if ctrl => return Some(Key::HalfPageUp), // Ctrl+U, half-page up
0x15 if ctrl => return Some(Key::Redo), // Ctrl+R, redo
0x13 if ctrl => return Some(Key::Up), // Ctrl+P, move up (vim CTRL-P)
0x13 if cmd => return Some(Key::Palette), // Cmd+P, file palette
0x11 if ctrl => return Some(Key::Down), // Ctrl+N, move down (vim CTRL-N)
_ => {}
}
@@ -379,6 +402,21 @@ mod tests {
assert_eq!(translate(0x29, true, false, false), Some(Key::Char('~')));
}
#[test]
fn translate_ctrl_navigation_and_redo_chords() {
assert_eq!(translate(0x07, false, true, false), Some(Key::HalfPageDown)); // Ctrl+D
assert_eq!(translate(0x18, false, true, false), Some(Key::HalfPageUp)); // Ctrl+U
assert_eq!(translate(0x15, false, true, false), Some(Key::Redo)); // Ctrl+R
assert_eq!(translate(0x13, false, true, false), Some(Key::Up)); // Ctrl+P, up
assert_eq!(translate(0x13, false, false, true), Some(Key::Palette)); // Cmd+P, palette
assert_eq!(translate(0x11, false, true, false), Some(Key::Down)); // Ctrl+N, down
assert_eq!(translate(0x11, false, false, true), None); // Cmd+N reserved (:enew, v0.5)
// Without a modifier these are ordinary letters, not intents.
assert_eq!(translate(0x15, false, false, false), Some(Key::Char('r')));
assert_eq!(translate(0x13, false, false, false), Some(Key::Char('p')));
assert_eq!(translate(0x11, false, false, false), Some(Key::Char('n')));
}
#[test]
fn translate_ctrl_or_cmd_swallows_plain_chars() {
assert_eq!(translate(0x04, false, true, false), None); // Ctrl+a