typewriter/docs/v0.5-palette-and-multi-file.md

v0.5 — File palette + multi-file

Part of the Typoena macro plan. Requirements and targets: qfd.md. Load-bearing decisions: adr.md. Prefs reference: typoena-toml.md.

Status: buffer foundation landed in core 2026-07-11 (slice 1 of 4), host-tested; the palette + transient panel (Spike 11) and delete → git-staging (Spike 14) remain the on-device gates. The single-file Effect return became a drained effect queue (Save{path,contents} / Load{path} / Publish / Pull), so one action can ask the host for several steps in order — opening a non-resident file queues a Save of the outgoing dirty buffer then a Load of the target. The multi-buffer state deliberately avoids a rope-per-buffer rewrite: the active buffer keeps its fields inline on Editor, inactive buffers park in a small LRU Vec<Buffer> (≤ 3 resident = active + 2), and a switch marshals fields in/out so the ~3k-line editing engine is untouched. A dirty parked buffer is saved before it is evicted (nothing leaves RAM unsaved); :e <path> opens by prefix (/sd/repo → Tracked, /sd/local → Local); :sync is refused in-core in a Local buffer. Firmware drains the queue to empty each batch (a Load can cascade an eviction Save), and persistence::{load_path,save_path} generalise the atomic save off the hard-coded notes.md.

Slice 2 of 4 landed in core 2026-07-11, host-tested: the Cmd-P file palette — a modal transient panel over the writing column with a bare fuzzy-search input (no > prefix: > is reserved for the command palette, slice 4 — VS Code semantics), the ranked list, and the selected row in reverse video. A pure host-testable fuzzy matcher (fuzzy_score: subsequence match, boundary + consecutive-run bonuses, no penalties) ranks results; an in-core MRU floats recently-opened files to the top on an empty query and is shared with :e (both flow through open_path). The host feeds the file list once at boot (set_file_list, enumerating /sd/repo + /sd/local, dotfiles skipped); Ctrl-n/Ctrl-p (fzf-style; Ctrl-d/Ctrl-u too) move the selection — the 60 % board has no arrow keys — Enter opens via the same park/evict path as :e, Esc (or Cmd-P again) closes. Same slice: Ctrl-n/Ctrl-p also work as down/up line motions in Normal mode (vim CTRL-N≡j, CTRL-P≡k, count-aware), which is why the palette opener moved to Cmd-P alone. Scope shows as the inline repo/… vs local/… label rather than the planned [git]/[local] badge — it also disambiguates subpaths, not just scope. 111 editor tests + 28 keymap tests pass; the no-git firmware binary builds clean. The transient-panel refresh (Spike 11) is CONFIRMED ON DEVICE 2026-07-12 — no ghosting (user flashed it and eyeballed the full-area partial the palette forces); Cmd-P opens it on-device too. Remaining v0.5 slice: 4 prefs + palette command mode.

Slice 3 (:enew + delete) COMPLETE + CONFIRMED ON DEVICE 2026-07-12 (committed c9c0716). :enew <name> creates a new file: empty, active, marked dirty so eviction/:w persists it, and added to the in-core file list so the palette finds it without a disk re-enumeration — no card IO until it is saved. :delete unlinks the current file (a new Effect::Delete the host services), then switches to the most-recently-parked buffer or an empty scratch; the discarded buffer is never saved even when dirty. Scope for a new file is read from the path, not a modal prompt — local/x / repo/x (the palette label form) select the scope, a bare name uses the current buffer's scope. Same change made the /sd prefix optional everywhere in resolve_path: /sd/repo/x, /repo/x, and repo/x all name one file and nothing resolves outside /sd (the writer can't reach anything else). Spike 14 (delete → git-staging) DID need a firmware fix. The first on-device test found add_all(["*"]) alone does not stage a deletion on this libgit2 build (the tree came back unchanged, so the second push was a silent "up to date" no-op — the "delete didn't work" report). Fix: stage_and_commit now runs add_all then update_all(["*"]) (git add -u), which removes index entries whose working-tree file is gone — together they are git add -A. Also, :delete gave no clear feedback, so the snackbar now names the scoped file and, for a Tracked file, that it is local until :sync (deleted repo/notes.md - :sync to publish). Deferred to later: greying the Publish affordance for a Local buffer, and the multi-file publish count. 123 editor tests + 28 keymap tests pass; the no-git firmware binary builds clean. The update_all fix (behind --features git, unbuildable locally) was verified on device 2026-07-12 — :enew test.txt → :sync → :delete → :sync removed test.txt from origin.

Slice 4 (.typoena.toml prefs + palette > command mode) COMPLETE in core 2026-07-12, HOST-TESTED not yet on-device. A Prefs type (host-testable line-based TOML parse/serialize — flat key = value bools + one string with # comments, no crate pulled onto xtensa) lives on Editor; the host reads /sd/repo/.typoena.toml at boot and applies it before the first render, and a missing/partial file falls back to per-key defaults. Keys: save_on_idle, format_on_save, line_numbers (all bool, default on) and auto_sync (string, default "10m", schema + default only — nothing reads it yet). line_numbers is live: gutter_cols() returns 0 when off, so the text reclaims the gutter's columns (the gutter - 1 field width made saturating to avoid the underflow). The palette > command mode (VS Code semantics — a leading > in the query switches file search to the command list) exposes the three booleans as live toggles; Enter flips the pref, applies it at once, queues a new Effect::SavePrefs (the editor serializes; the host does the atomic write to .typoena.toml, which rides the next :sync to other devices), and confirms the new state on the snackbar. The list stays open after a toggle so several prefs flip in one visit (Esc/Cmd-P closes); :settings opens the palette straight into > mode as a one-command shortcut (both requested by the user 2026-07-12, chosen over a separate settings modal — same surface, no duplicate machinery). Committed c535864. Three "decide before build" calls: (1) the idle auto-save is unformatted — :fmt runs only on explicit :w/:sync, so tables/blank-lines are never reflowed mid-session; (2) the per-device auto_sync override (card-local typoena.conf) is deferred — auto_sync is inert in v0.5, so there is nothing yet to override; (3) > auto sync: <dur> as a palette command is deferred to v0.7 — a control that changes a value nothing reads would be a dead switch. save_on_idle is honoured host-side: a silent idle auto-save (no snackbar, no forced e-ink flash — a safety net, not an action) fires once per typing burst after a 1.5 s pause. 141 editor tests + 28 keymap tests pass; the no-git firmware binary builds clean. Firmware bumped 0.4.0 → 0.5.0 (the v0.5 feature set is met). Boot-read of the prefs file CONFIRMED ON DEVICE 2026-07-12 — a .typoena.toml in typoena-test with non-default values (save_on_idle=false, line_numbers=false, auto_sync="5m") logged back prefs: Prefs { save_on_idle: false, format_on_save: true, line_numbers: false, auto_sync: "5m" } at boot, a byte-exact parse (comments skipped, bools + quoted string read). Full gate CLOSED 2026-07-12: the palette > live-toggle round-trip is confirmed — origin's .typoena.toml went line_numbers false → true via a device-authored publish (3c79f38), proving toggle → SavePrefs → atomic write → git add -A → push — and the save_on_idle autosave works on device too. v0.5 slice 4 fully DONE + on-device confirmed.

Amendment 2026-07-12 — non-boolean prefs (theme, auto_sync). Two of the "decide before build" calls above are superseded: auto_sync is now a live palette command, and a new theme (light/dark) key ships. The generalising idea is that a boolean toggle is just the two-option case of rotate through a preset list on Enter — so the palette gains one uniform gesture: Enter advances the selected pref to its next value and wraps (a bool flips; a string pref cycles its options). theme rotates light↔dark and is applied by a single whole-frame invert at the end of the render ([Frame::invert]), so text, selection, caret, panel and palette all flip together; auto_sync rotates 2m/5m/10m/15m/30m. Decision (3) — "a value control that changes nothing readable would be a dead switch" — is knowingly overridden: cycling auto_sync persists and displays the interval, but still drives no behaviour until v0.7; we accept a set-ahead control so the surface is ready and the value syncs now. Decision (2) (per-device typoena.conf override) stays deferred. Kept simple: no enum machinery — both string prefs share a next_option(current, &OPTIONS) helper, and hand-editing the TOML can still set any value (the palette only cycles presets; an off-list value snaps to the head on the next Enter). Editor tests cover the rotate/wrap/snap, the live theme invert, and the round-trip.

Trailing-newline handling — a saved note ends with a visible blank line (commit d14d9e7, 2026-07-12; host-tested, on-device gate open). Two adjustments to how the buffer meets the file. First, format-on-save no longer strips every trailing blank line — it collapses a run to at most one and keeps it, so a writer who presses Enter to open the next line doesn't have that line (and the caret) yanked away on save (the caret used to jump up to the last non-empty line). Second, persistence treats the file's POSIX terminator as content the editor shows: load_path reads the file verbatim and save_path writes the buffer, appending a final newline only if one is missing (guarded, not unconditional). Because the editor lays out rows = #\n + 1, that terminating newline renders as a visible trailing empty line the caret can land on — open a note and the blank line the newline stands for is there. The two are an identity round-trip for any device-written file (all end in \n); the file stays git-clean (exactly one terminator — vim and GitHub show no phantom blank line); and a trailing blank line the writer leaves is mirrored, never doubled. This replaced an interim model that stripped the terminator on load and hid it (the file was correct, the newline just wasn't shown). On-device check: reflash → open a note (trailing empty line visible) → :w (caret stays on it) → reopen (still there). Prefs::to_toml ends in a newline for the same reason; the guarded save leaves the prefs file with exactly one.

Amendment 2026-07-13 — recursive enumeration + a 2-char search threshold. Loading a real repo (jcalixte/notes) exposed that enumerate_files listed only the top-level files of /sd/repo and /sd/local — a nested notes tree showed a single file in the palette (subpaths always opened fine via :e repo/sub/x.md; only the listing was flat). The enumeration is now a recursive walk: dot entries are skipped at every level (so .git is never descended into), each directory is read fully before recursing (one FatFS dir handle open at a time — the remove_dir_recursive pattern, kind to the FD-bounded mount), depth is capped at 8, and the boot-time walk logs its file count and duration (file walk: N files in Xms) so the FAT dir-IO cost on a big repo is measurable, not assumed. With the list now card-sized, the palette gained a search threshold: below 2 typed chars the result list is the recents (MRU) only — quick-switch (Cmd-P, Enter) stays one keystroke away — and the full fuzzy-ranked list appears from 2 chars on (PALETTE_MIN_QUERY). A fresh boot with no opens yet shows (type to search). > commands and $ snippets are short curated lists; the threshold does not apply to them.

TODO (on-device, next time the device is on the bench) — two measurements from the same boot log, both already instrumented:

• Re-measure the walk time after the d_type fix (2660a3e — dirent file_type() instead of a per-entry metadata() stat, which cost ~32 ms/file and made run 1 take 35 s for 1098 files). Read the file walk: N files in Xms line. Only if it's still slow does the async-walk idea come back on the table.

• Read the file-list DRAM cost from the new file list: internal heap <before> -> <after> (<N> KB consumed) line (the build is bracketed with MALLOC_CAP_INTERNAL readings in main.rs). The 1098 path Strings are each below the 16 KB SPIRAM malloc threshold, so they all land in internal DRAM — estimated 60–70 KB, competing with Wi-Fi/TLS. Decision rule: ~60–70 KB confirms interning the paths into one shared buffer (a single >16 KB alloc goes to PSRAM; only a ~9 KB offset index stays in DRAM); well under that (≤~30 KB) kills the idea — the next DRAM suspect is then parked-buffer text.

• Cmd-P opens fuzzy file palette over both /sd/repo/ and /sd/local/ — landed and CONFIRMED ON DEVICE 2026-07-12 (Spike 11: no ghosting on the transient panel); scope shows as the inline repo/… / local/… label instead of a [git]/[local] badge.

• [~] Open, switch, close buffers (keep ≤ 3 in memory) — open + switch + the ≤ 3 LRU-resident model with dirty-aware save-before-evict done in core (host-tested); :e <path> and the palette drive it today. Explicit close still to come.

• :e and palette share the same recent-files list — both open via open_path, which pushes to the in-core MRU that orders the palette.

• :enew creates a new file — done in core (host-tested) 2026-07-12. Scope is read from the path (local/x / repo/x select it, the palette label form; a bare name uses the current scope) rather than a modal prompt — the resolved scope is echoed in the snackbar. The /sd prefix is optional throughout (/sd/repo/x = /repo/x = repo/x).

• Delete a file — core done (host-tested) 2026-07-12; :delete unlinks the current file via Effect::Delete. For a Tracked file the removal reaches the next :sync Publish's staged set. Spike 14 (on-device) found the staging incomplete: add_all(["*"]) alone did not stage the deletion, so stage_and_commit now also runs update_all(["*"]) (git add -u) — the two together are git add -A. A Local file is just unlinked. The snackbar now confirms the delete and flags that a Tracked file needs :sync. Verified on device 2026-07-12 — the :enew→:sync→:delete→:sync cycle removed test.txt from origin.

• [~] Ctrl-G is disabled / hidden when the current buffer is local-scope — :sync / Publish is blocked in-core for a Local buffer (posts "Publish unavailable (Local)"); the side-panel affordance that hides/greys the gesture is the remaining half.

• The side panel briefly shows file count on Ctrl-G when the publish bundles more than one dirty Tracked file (e.g. "publishing 3 files: abc1234"), so workspace-scoped behaviour stays visible to the user

• Preferences file /sd/repo/.typoena.toml — a git-tracked, hand-editable TOML file for editor behaviour, deliberately distinct from the /sd/typoena.conf card secrets (Wi-Fi / PAT / remote / author, gitignored, never committed — see v0.1). Read at boot; a missing file or key falls back to the defaults below. Core done 2026-07-12 (a Prefs type on Editor, host-testable parse/serialize, applied via Editor::set_prefs before the first render); full reference: typoena-toml.md. Keys:

  • save_on_idle (bool, default true) — auto-save the current buffer on the idle typing-pause, so :w becomes optional rather than required. Honoured host-side as a silent save (no snackbar, no forced e-ink flash — a safety net, not an action), unformatted, once per typing burst after a 1.5 s pause.
  • format_on_save (bool, default true) — run :fmt (table alignment, blank-line collapse, trailing-whitespace strip) on the buffer before it is persisted, so :sync is fmt → save → commit → push and :w saves formatted. Implemented in-core 2026-07-11 (Editor), now driven by this key. Open question RESOLVED (2026-07-12): fmt runs only on an explicit :w/:sync; the save_on_idle auto-save is deliberately unformatted, so tables/blank lines are never reflowed mid-session (the caret would jump under you on every thinking pause).
  • line_numbers (bool, default true) — show the absolute line-number gutter (built always-on in v0.2). Off reclaims the gutter's columns for text (gutter_cols() → 0); the palette > line numbers: on/off command toggles it live. Done 2026-07-12.
  • theme (string, default "light"; options light/dark) — panel colour polarity. dark is a single whole-frame invert at the end of the render (Frame::invert), so everything flips together and stays legible. Palette > theme rotates light↔dark live. Added 2026-07-12 (amendment); host-tested, on-device check pending. Caveat: dark partial-refreshes ghost more on e-paper than light — verify on panel.
  • auto_sync (duration string, default "10m"; "0" / omitted disables; min clamp ~"2m" so a palette typo can't drain the battery) — a max-staleness cap, not a wall-clock timer: opportunistic, rate-limited Publish. Push when already awake + dirty (coalesced into the idle-pause, ≤ once per auto_sync) and once on the way into sleep if dirty; never wake from deep sleep purely to sync. Wi-Fi energy is a 1/T curve whose knee sits at 5–10 min, and save_on_idle already owns local data safety — so 10 min halves the sync energy of a 5-min default for no real risk. Full derivation: tradeoff-curves/wifi-auto-sync.md. The schema + default ("10m") live here in v0.5 and round-trip through Prefs; nothing reads the value yet — the periodic side rides the better-git work (v0.7) and must interact with light / deep sleep (v0.8). Marked [~]: parsed and preserved, no behaviour. Amended 2026-07-12: now a palette preset command — Enter rotates it through 2m/5m/10m/15m/30m (the ~2m min is baked into the preset list). Set-ahead only: still read by nothing until v0.7.
  • Open question RESOLVED (2026-07-12): the per-device sync cadence override (a card-local typoena.conf layer over the committed prefs) is deferred — auto_sync is inert in v0.5, so there is nothing yet to override; revisit when v0.7 makes the periodic push real.

• Palette command mode — typing > at the Cmd-P palette switches it from file search to a command list (VS Code-style). Done in core 2026-07-12. The v0.5 commands toggle the three boolean .typoena.toml prefs — > save on idle, > format on save, > line numbers — each label carrying its live state; Enter flips the pref, applies it at once, queues Effect::SavePrefs (persist to the file), and confirms on the snackbar. The list stays open after a toggle (flip several, Esc/Cmd-P closes), and :settings opens it directly — both added 2026-07-12 as the "change config from the device" surface (chosen over a separate settings modal). This command list is the discoverable surface later actions (:fmt, font) also register into. Amended 2026-07-12: the palette now also carries the non-boolean prefs > theme (light/dark, live whole-frame invert) and > auto sync (2m..30m), both cycled by the same Enter-rotates-to-next gesture. auto_sync is exposed set-ahead (no behaviour until v0.7), knowingly overriding the earlier "dead switch" call.