p_mmap emulated a mapping by malloc+read()ing the range from SD on every
call. libgit2 re-hits the pack idx/windows on every object write (via
git_odb__freshen -> git_odb_refresh), so a commit re-read pack bytes over
SPI repeatedly -- ~500ms-1.1s/op on a small repo, far worse on the real
570 MB pack.
Cache read-only mappings >= 64 KB (pack idx/windows, commit-graph, midx,
packed-refs -- all immutable on this device) in a refcounted, PSRAM-backed
slot table keyed by (dev, ino, size, mtime, offset, len), LRU-evicted under
a soft cap. Small mutable working-tree maps (diff_file.c) fall below the
floor and are never served stale.
Break the stage+commit window into sub-phases (FAT working-tree walk vs
object writes) via `commit split —` log lines, and add two micro-benchmarks
(sd_bench for SD/FAT primitive ops, git_bench for libgit2 object overhead)
with justfile recipes. Documents the walk-vs-writes cost model in
tradeoff-curves/sync-commit-staging.md to decide whether explicit-path
staging over the editor's dirty set is worth replacing add_all(["*"]).
esp-idf-hal always compiles its i2c.rs, which binds the old driver/i2c.h
API, so ESP-IDF links the legacy driver's constructor into every image and
prints its deprecation warning at boot. Typoena uses no I2C (EPD/SD are SPI,
keyboard is USB), so the warning is pure noise; this flag compiles the
constructor (and its conflict-abort check) out.
`just load` / `just init` now fast-forward the source repo from its
origin before rsyncing to /sd/repo, so the card picks up notes the
device already pushed. Fast-forward-only and best-effort: dirty tree,
diverged history, detached HEAD, or offline warns and copies the
current on-disk state rather than aborting. TW_NO_PULL=1 skips it.
Default espflash baud (115200) made a full flash take ~2-3 min of
serial transfer. Bump the cargo runner and the two direct-espflash
recipes (flash-git-push, flash-git-sync) to 921600. Monitor recipes
keep 115200 to match the device UART log rate.
Add a curated snippet catalog (firmware/snippets-catalog/, three opt-in
groups: Symbols, Structure, Prose — 17 snippets, English prefixes,
translated from the source Zed set) and a _seed-configs step in `just
init` that jq-merges the chosen groups into repo/.typoena.snippets.json
and writes a starter repo/.typoena.toml. Seeds only files that are
absent, so re-running init never clobbers a synced library.
Mirror the prefs boot-read: load the git-tracked snippet library from the
SD repo, parse it with Snippets::parse, and hand it to the editor before
the first render. Missing/unreadable/malformed is non-fatal (no snippets,
editor runs). Verified serde_json builds for xtensa (cargo check, 0.6.0).
Read notes verbatim and insert the final newline only when it is missing,
instead of stripping the terminator on load and appending it unconditionally
on save. The editor's `rows = #\n + 1` model then renders a file's POSIX
terminator as a visible trailing blank line — what a writer expects: open a
note and see (and land the caret on) the empty line the newline stands for.
Supersedes the strip-on-load / unconditional-append handling that shipped
with the prefs work (c535864), which kept the buffer newline-free and hid the
terminator. Load + save are now an identity round-trip for any device-written
file (all end in '\n'); files stay git-clean (exactly one terminator); and a
trailing blank line the writer leaves is mirrored, never doubled.
- load_path: read verbatim (drop the strip)
- save_path: guarded final-newline (drop the unconditional append)
- Prefs::to_toml: ends in a newline again — the guarded save leaves exactly
one, so the prefs file is byte-identical to before and its device-verified
round-trip still holds
- sd_fat spike: payload ends in '\n' so its exact-equality round-trip holds
Git-tracked editor preferences read at boot and toggled live on-device:
- Prefs type (line-based TOML parse/serialize, no crate on xtensa) held on
Editor; firmware reads /sd/repo/.typoena.toml before the first render and
falls back to per-key defaults. Keys: save_on_idle, format_on_save,
line_numbers (bool) + auto_sync (string, schema/default only until v0.7).
- line_numbers applied live (gutter_cols -> 0 when off).
- Palette > command mode toggles the three bools; the list stays open so
several flip in one visit, and :settings opens it directly. Each toggle
applies live and queues Effect::SavePrefs (host atomic-writes the file,
which rides the next :sync).
- save_on_idle honoured host-side as a silent, unformatted idle auto-save.
- to_toml is newline-free; save_path now appends exactly one terminator
unconditionally so buffers round-trip byte-for-byte (trailing blanks kept).
- Firmware 0.4.0 -> 0.5.0; new docs/typoena-toml.md reference. Also refreshes
the slice-3 macroplan status (delete fix was confirmed on device).
Add the POSIX line terminator on save and strip it on load, so files
written by the editor no longer trip git's "No newline at end of file".
Done at the persistence choke point, not in :fmt: the editor buffer is
newline-free by design (rows = #\n + 1), so a trailing '\n' in the buffer
would render a phantom blank last line. save_path adds exactly one '\n'
(guarded against doubling); load_path strips one back off so the buffer
stays newline-free and round-trips byte-stable.
Update the sd_fat spike payload to be newline-free so its byte-identity
round-trip assertion holds under the new normalizing contract.
:enew <name> creates a new file (empty, dirty, added to the palette list);
:delete unlinks the current file via a new Effect::Delete and switches to a
parked buffer or scratch. Scope is read from the path (local/x, repo/x) rather
than a modal prompt, and the /sd prefix is now optional in resolve_path.
On-device testing showed deletions never reached the remote: add_all(["*"])
alone does not stage a removal on this libgit2, so the tree came back unchanged
and the push was a silent no-op. stage_and_commit now runs add_all then
update_all(["*"]) (git add -u) — together git add -A. The :delete snackbar now
confirms the scoped file and flags that a Tracked file is local until :sync.
Cmd-P opens a modal transient palette over the writing column: a bare
fuzzy-search input, the ranked file list, and the selected row in reverse
video. A pure host-testable fuzzy_score (subsequence + word-boundary and
consecutive-run bonuses) ranks results; an in-core MRU floats recently
opened files first and is shared with :e (both route through open_path).
The host feeds the file list once at boot (enumerate_files over /sd/repo
and /sd/local). Ctrl-n/Ctrl-p navigate the list; Enter opens via the same
park/evict path as :e; Esc closes.
Ctrl-n/Ctrl-p also become down/up line motions in Normal and View (vim
CTRL-N/CTRL-P, count-aware), which is why the palette opener is Cmd-P
alone. No `>` prefix on the file input — `>` is reserved for the command
palette (slice 4).
112 editor + 28 keymap tests; the no-git firmware binary builds clean.
Rework the single Effect return into a drained effect queue
(Save{path,scope,contents} / Load / Publish / Pull) so one action can
ask the host for several ordered steps: opening a non-resident file
queues a Save of the outgoing dirty buffer then a Load of the target.
Keep the active buffer's fields inline on Editor and park inactive
buffers in a small LRU (<=3 resident = active + 2); switching back to a
resident buffer restores its caret/scroll/undo without touching the SD.
A dirty parked buffer is saved before eviction, so nothing leaves RAM
unsaved. `:e <path>` opens by prefix (/sd/repo -> Tracked, /sd/local ->
Local); `:sync` is refused in-core for a Local buffer.
Firmware drains the queue to empty each batch (a Load can cascade an
eviction Save) and persistence generalises the atomic save off the
hard-coded notes.md into load_path/save_path.
Also bump the side panel to FONT_9X15 and the `:` command line to
FONT_10X20 for legibility, word-wrapping the snackbar so a long notice
keeps its actionable tail.
Charwise `v` and linewise `V` selection with yank/delete/change on the
span; motions and counts extend it. Read-only View moves off v/V to
`gr` (go-read). Selection renders reverse-video on the 1-bit panel.
Normal motions factored into a shared move_by. Firmware -> 0.4.0.
A screen refresh that ran while :sync had Wi-Fi + TLS up could fail to
allocate an internal DMA bounce buffer (ESP_ERR_NO_MEM); the error
propagated through ? out of main(), main_task returned from app_main(),
and the editor loop died while the USB and git threads kept running —
keys logged, panel frozen forever.
A paint is idempotent and retryable (the buffer is the source of truth),
so every editor-loop paint now logs and drops the frame instead of
propagating, leaves shown untouched so the next paint repaints the diff,
and forces a full refresh next to re-sync both RAM banks. Same contract
as save_note. The boot-time first render stays fatal on purpose.
Add Effect::Pull and the `:gl` command (fetch + fast-forward only, refuse on
divergence). The editor side is host-tested; the firmware arm is a stub posting
"pull: not wired yet (v0.7)" — the on-device fetch/fast-forward in git_sync is
v0.7 work (only push is wired today).
The pre-commit fetch cost ~6s on the measured cold sync (it did real work
absorbing a foreign push; ~3s + a full TLS handshake even when the remote is
unchanged). Drop it: push onto the current tip first, and only when the remote
rejects the push non-fast-forward do we fetch, mixed-reset onto origin, replay
the note, and retry. The happy path is now a single handshake.
- stage_and_commit extracted (used on the first attempt and the replay).
- reconcile_onto_origin replaces fast_forward_before_commit + fetch_and_integrate.
Mixed reset keeps the just-saved note; the replay lands it on origin's tip.
- Single-writer semantics: a foreign push resolves last-writer-wins instead of
bailing on divergence, so the device never gets stuck. A remote-only added file
it doesn't have would be dropped by the replay — needs the real merge path
(increment B), doesn't arise from this device's own use.
publish_cycle now times wifi assoc, SNTP, TLS setup and publish separately
and logs a one-line total. Cold sync shows all phases; a warm sync reads 0ms
for the first three. Makes the fetch added to publish measurable.
Two robustness fixes surfaced by commit 07d87772 (the first editor-driven
:sync), which shipped three macOS AppleDouble files to typoena-test:
- Staging filter: add_all now runs a per-path callback that skips ._* and
.DS_Store, so Finder/Spotlight cruft on the FAT card never lands in a
commit. Device-level, so it protects every repo without a per-repo
.gitignore.
- Pre-commit fast-forward: publish_once fetches origin and, if the local
branch is behind, fast-forwards to it via a MIXED reset before committing.
This lets the device absorb a foreign push (e.g. the sidecar cleanup) and
fast-forward cleanly instead of stacking a commit on a stale base and
diverging at push time. The mixed reset moves the ref+index but leaves the
working tree, so the just-saved unsynced note isn't lost.
Land the v0.1 editor integration: the git_sync module (libgit2 on the SD
/sd/repo, dedicated 96KB git thread, lazy Wi-Fi, :sync push with
synced/up-to-date/failed snackbars), the boot splash (Spike 9) plus its bin
and justfile recipes, and a power-on→cursor boot-timing log. Also re-syncs
the roadmap/spikes/v0.1-product status and adds the SD hardware reference
photo.
Boot posts "loaded <name>" (the note's filename without suffix); :w and
:sync post "saved" / "save FAILED - retry :w" via Editor::set_notice, so
save state is visible on-device, where there is no serial console.
Make `just build`/`just flash` the nominal product build (--features git
+ LIBGIT2_SRC), and move the fast editor-only build to `build-light`/
`flash-light`. Verified the git default links libgit2 into the firmware
ELF (1.2 MB) cleanly.
Route :sync through publish(), whose git-push path is #[cfg(feature =
"git")]. Git code in the firmware binary is now only ever compiled with
--features git, so the default build stays a light editor build with no
git2 crate and (since the justfile only sets LIBGIT2_SRC for git recipes)
no libgit2 component. Feature is off by default; :sync saves locally.
Seed the editor from Storage::load at boot and persist on :w/:sync
through Storage::save. A missing card/repo/unreadable note halts boot
with the reason on the panel rather than starting empty and clobbering
the note on the next save. Save errors are logged, buffer kept in RAM.
The git-push half of :sync stays a TODO — it needs git_sync graduated
into a module on the dedicated git thread; :sync saves for now.
Record that persistence is now implemented and hardware-verified, and correct
the crash-recovery rule everywhere it appeared as "just promote the tmp": a
crash during the tmp write leaves a partial tmp, so recovery keeps the
committed target when both files are present and only promotes the tmp when
the target was already unlinked. Also fix the stale README heading that still
said the SD was on shared SPI2.
sd_fat is now a thin on-device harness over firmware::persistence instead of
duplicating the mount FFI. It mounts, reports FAT usage + negotiated clock,
loads notes.md (non-destructive), and only runs the write round-trip when
notes.md is empty so a provisioned card is never clobbered. Verified on
hardware 2026-07-11: mount at 10 MHz, 91-byte round-trip byte-identical.
Graduate the proven Spike 3 storage stack into firmware::persistence so the
editor and the spike share one implementation. Storage::mount brings up the
dedicated SPI3 bus (ADR-012) and mounts FAT with format_if_mount_failed=false
(never reformat the user's card). save() does the FAT-safe atomic write
(tmp -> fsync -> unlink -> rename); recover() reconciles a leftover *.tmp at
boot, keeping the committed file when the crash point is ambiguous and only
promoting the tmp when the target was already unlinked. load() caps reads at
256 KiB.
Re-ran the spike on the dedicated SPI3 wiring (CLK 14 / MOSI 15) with
the same clean mount + atomic round-trip result. Flip the docs from
"pending a re-run" to verified.
The boot banner, success line, and round-trip payload still said
"shared SPI2" / "shared bus" after the move to SPI3. Cosmetic only —
serial output now matches the dedicated-bus wiring.
Document the shared-bus arbitration decision: rather than rework the
proven EPD SPI layer and add a cross-thread mutex on the save path, the
SD moves to a dedicated SPI3. Update the boot sequence, risk table,
Spike 3 postmortem follow-up, hardware overview, and firmware README.
The EPD's SpiBusDriver holds an exclusive SPI2 lock for its lifetime, so
an SD on SPI2 is locked out while the panel driver is alive. Give the SD
its own SPI3 (SCK 14, MOSI 15; MISO 13 / CS 10 unchanged) and drop the
now-pointless EPD-CS deselect. See ADR-012.
Add a "Provisioning an SD card" section covering the just init/load/provision
entry points, the config-resolution ladder (env → derive → prompt), the macOS
Keychain Wi-Fi lookup, and the plaintext-secret threat model for the card.
typoena.conf no longer requires a hand-written firmware/.env. _write-conf now
resolves each value through a ladder: .env if set, else derived from tools
already on the machine (git config, gh, the active Wi-Fi network + its System
keychain password), else an interactive prompt with the derived value as the
default. The PAT is never derived (a broad gh token on a plaintext card would
defeat the scoped-token model) and is always typed by hand. Missing required
values now abort before touching the card instead of ejecting a blank config.
Mark Spike 3 resolved (genuine 32 GB SDHC mounts, round-trips clean on the
shared SPI2 bus; the 133 GB SDXC's CMD59 rejection was the sole fault).
Capture the FatFS f_rename caveat and the ≤32 GB card-compatibility note in
ADR-007, the resolution in the postmortem, and the verified writeup in the
firmware README.
Flip format_if_mount_failed to true so a fresh exFAT/unformatted bench card is
reformatted to FAT on the device instead of failing — no Mac-side prep for the
spike. Fires only on a filesystem mount failure, not the CMD59 protocol
rejection. The real persistence module must keep this false.
FatFS's f_rename returns FR_EXIST on an existing destination — unlike POSIX
rename(2) it won't replace it, so re-running the round-trip failed at rename.
Unlink the target first (tolerating a missing one for the first save). This
opens a crash window the real persistence module must close with *.tmp boot
recovery (ADR-007).
The per-command sdmmc/sdspi DEBUG logs only existed to diagnose the CMD59
init failure, now resolved (verified 2026-07-11). Remove the build-time log
ceiling and the runtime esp_log_level_set block.
Replace the single seed-sd recipe with three entry points sharing
private helpers: init (repo + config), load (repo copy only), provision
(config only). Each excludes gitignored paths via git's own resolution
(so node_modules and firmware/.env never reach the card), writes
/sd/typoena.conf from firmware/.env (Wi-Fi + PAT + git identity, PAT
never printed), and ejects the card so it goes straight into Typoena.
The device never cold-clones the 566 MB notes repo over Wi-Fi + mbedTLS;
a laptop copies the clone onto the SD card via a reader so the device
only ever takes the open + fast-forward path. Excludes are driven off
git's own ignore resolution (collapsing node_modules, .env, etc.), and
the target card is auto-detected but refused on 0 or >1 match so rsync
--delete can't wipe the wrong disk.
The pure, no-IO editor core can't persist or push, so `handle()` now
returns an `Effect` (None/Save/Publish) that the firmware actions. `:w`
signals Save, `:sync` signals Publish (save then git push); `:wq`/`:x`
alias Save with vim's "quit" half dropped, and the `:q` family stays
absent — an always-on appliance has nothing to quit to. `:fmt` remains
in-core and yields no effect.
The main loop keeps the batch's last effect and matches on it. Actual
SD-write and git-push are the v0.1 gate and still stubbed as log lines.
Route decoded keys through keymap::Composer, so on device `'`+e yields é
(grave/acute/circumflex/diaeresis/tilde, plus ç), and reset it on
keyboard detach. Enabling it is now safe because the editor buffer is
UTF-8-correct. Not verified on the xtensa build in this environment.
Move the editor core out of the firmware crate (pinned to the xtensa
target, so it can't run `cargo test`) into a standalone `editor` crate,
with the panel framebuffer + geometry split into a `display` crate. Both
depend only on embedded-graphics + keymap, so the editor is now
host-buildable and unit-tested (8 characterization tests). Firmware links
them and re-exports the geometry from epd.rs; behaviour is unchanged.
The xtensa firmware build was not verified in this environment.
The single-shot connect_wifi aborted boot on any transient association or
DHCP miss — common right after a reset, when the AP still holds the stale
pre-reset association. Add a bounded retry (5 attempts, 500ms→4s backoff,
disconnect between tries) so those transient failures recover instead of
failing the boot.
Extract the logic into firmware::net (new lib target) and rewire the
wifi_tls / git_push / git_sync spikes to it, removing three duplicated
copies. main.rs reuses this when Wi-Fi lands there instead of adding a
fourth copy.
Adopt the keymap crate (drops the PREV_KEYS/CAPS_USED statics for one
owned Decoder) and harden device teardown:
- #1 UAF: track REPORT_INFLIGHT (set on submit, cleared first thing in
report_cb). Teardown moves to close_device, which pumps client events
until the transfer quiesces before freeing it, and leaks rather than
frees if it never does — a leak is recoverable, a use-after-free is
not. usb_host_transfer_free's return is now checked.
- #3 leaks: a new attach while a device is still open tears the old one
down first; control_request and start_report_polling free the transfer
they allocated on a submit error.
Decode correctness is covered by the keymap tests; the teardown paths
are FFI and still need an on-device hot-plug run to confirm.
Move the mode indicator (and the NO KBD flag above it) down so the
mode sits flush at the bottom-left with a 2 px margin, reclaiming the
dead space that was left below it.
Move the mode + pending-command echo from the panel's top to its
bottom-left (vim `-- INSERT --` style); word count takes the top slot
and the NO KBD flag sits just above the mode. Also drop the
words-this-session field (not wanted) and its session_base_words state.
