julien/typewriter
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
235c7e4d24ef89e09775ba5c4dbae442d317c0b0
typewriter/docs/v0.1-mvp-technical.md

20 KiB
Raw Blame History

v0.1 MVP — Technical design

Scope: ships the product surface in v0.1-mvp-product.md. No more, no less.

Decisions referenced inline point at adr.md. Tradeoff weights and the critical-performance budget live in qfd.md. Project overview: ../README.md. Release sequence: roadmap.md.

Architecture

Single Rust binary on esp-idf-rs (std) — ADR-001. Two cores, several threads, one shared editor state behind a mutex (ADR-006). Wi-Fi and git work happens off the input thread so the typing path is never blocked by I/O.

                ┌──────────────────────────────────────────┐
                │           Core 0 (PRO_CPU)               │
                │                                          │
   USB HID ───► │  usb_task     ─►  KeyEvent channel ──┐   │
                │                                       │   │
                │  wifi_task    ◄─►  esp-wifi internals │   │
                │                                       │   │
                └───────────────────────────────────────│───┘
                                                        │
                ┌───────────────────────────────────────▼───┐
                │           Core 1 (APP_CPU)                │
                │                                           │
                │  ui_task  ──► editor state ──► render_task│
                │     ▲              │               │      │
                │     │              ▼               ▼      │
                │     └── git_task ──┘          SPI ──► EPD │
                │              ▲                            │
                │              └── SD I/O                   │
                │                                           │
                └───────────────────────────────────────────┘

Threads / tasks

Task Core Stack Responsibility
usb_task 0 8 KB TinyUSB host loop, decode HID reports, post KeyEvents
wifi_task 0 8 KB On-demand station: off by default, brought up by Ctrl-G
ui_task 1 16 KB Consume KeyEvents, mutate editor state, enqueue render
render_task 1 12 KB Drain render queue, do partial/full refresh on EPD
git_task 1 96 KB Triggered by Ctrl-G; runs libgit2 commit + push

All inter-task communication is via crossbeam-channel or std::sync::mpsc bounded queues. The editor state is Arc<Mutex<EditorState>>; the lock is held only for the duration of a single mutation (key application) or a single read snapshot (render diff).

Boot sequence

1. ROM → bootloader → app_main
2. Init PSRAM allocator, set as default for large alloc
3. Config is compiled into the binary (`build.rs` reads env vars) — no
   filesystem read needed for it. LittleFS is unused in v0.1.
4. Mount FAT on SD → verify /sd/repo and /sd/repo/notes.md exist
   ├─ missing → fatal: "missing /sd/repo — re-mount SD and reboot"
   └─ present → continue
5. Init two SPI buses: EPD on SPI2, SD on SPI3 (separate hosts — ADR-012)
6. Init EPD, full refresh: splash + boot log
7. Start tasks: usb, wifi (spawned in `Off` state — no radio bring-up), ui, render
8. ui_task opens /sd/repo/notes.md, places cursor, enqueues full render
9. STEADY STATE

Target boot time: ≤ 5 s to cursor (v0.1). The 3 s target is v1.0.

Hardware bring-up order

We do not try to build the whole stack and turn it on. Each spike below is a small program that proves one risk before we commit to the next layer. Spike 7 is the kill-switch for ADR-004 (gitoxide); spike 4 is the gate for ADR-009 (USB host).

  1. Spike 1 — Blink. Confirm toolchain, flash, and basic GPIO.
  2. Spike 2 — EPD. Drive the GDEY0579T93 (via DESPI-c579 breakout) over SPI; full refresh "hello world." Validates SPI wiring, panel timings, and — critically — whether epd-waveshare already supports the panel's controller (SSD1683-class) or whether we write a thin custom driver against embedded-hal. Either way, ~300 LoC; this spike answers which.
  3. Spike 3 — SD. Mount FAT, read/write a file. Verified 2026-07-11; the shared-bus question resolved to a separate SPI3 for the SD (ADR-012).
  4. Spike 4 — USB host. Enumerate the Nuphy as a boot-protocol HID keyboard, log keycodes over UART.
  5. Spike 5 — Partial refresh. Type a string letter-by-letter, partial refresh per character, measure end-to-end latency.
  6. Spike 6 — Wi-Fi + TLS. Connect to home Wi-Fi, do an HTTPS GET to api.github.com, validate cert chain.
  7. Spike 7 — git push. Smoke test: from desktop-Rust first, then on device, push a single commit to a test repo over HTTPS+PAT. Done — the gitoxide fall-back to libgit2 (git2) fired here; push is proven on hardware (postmortem).

Only after spike 7 do we start integration. Any spike that fails forces a stack decision (e.g. fall back to libgit2; switch to a separate SD SPI bus).

Rendering spikes (post-bring-up)

The display/UX spikes (8–14) — panel layout, boot splash, theme, transient panel, scroll indicator, line-number gutter, multi-file navigation — are not part of the v0.1 integration gate and span several releases, so they live in their own log: spikes.md. Spikes 8 (layout) and 9 (splash) feed v0.1; the rest feed later releases. Run Spike 8 first — it partitions the panel into the writing column and side panel the others draw within.

Module breakdown

editor — text buffer + cursor

  • Buffer = Ropey::Rope backed by PSRAM allocator.
  • Cursor { line: usize, col: usize, byte_offset: usize } — single cursor, Insert mode only.
  • apply(KeyEvent) -> EditOp — pure function from key to operation; returns the dirty range for render.
  • Soft wrap is computed at render time from the rope, not stored.

keymap — v0.1 keymap table

Key Action
printable insert char
Backspace delete char before cursor
Enter insert \n
← → ↑ ↓ move cursor (visual lines for ↑↓)
Home End line start / end
Ctrl-S save
Ctrl-G publish (save → stage → commit → push, with pull-and-retry on remote divergence)

Anything else is ignored in v0.1.

render — dirty-rect to EPD

Custom widget layer on embedded-graphics — ADR-002. Owns the top-ranked engineering functions (latency, refresh-region area) in qfd.md §3.

  • A render request is a RenderOp { Lines(range) | Panel(field, text) | FullRefresh }. The side panel is multi-field (Spike 8), so the old single Status(text) op becomes a per-field write; the exact field enum is what Spike 8 decides.
  • The render thread maintains a shadow of the current screen contents per region (writing-column lines / side-panel fields).
  • For Lines(range): clear the region's bounding box, draw glyphs, partial refresh that rect.
  • Glyph cache: rasterised mono font kept in PSRAM; one entry per (codepoint).
  • After 20 partial refreshes, the next render is upgraded to a full refresh to clear ghosting. Counter persists across saves.
  • A FullRefresh is forced on save, on focus return, and on screen-clear.

persistence — SD I/O

Storage split rationale: ADR-007.

Implemented in firmware::persistence (Storage::{mount, load, save, recover}), graduated from Spike 3 and hardware-verified 2026-07-11.

  • Atomic save: write to notes.md.tmp, fsync, unlink the target (FatFS f_rename won't overwrite — ADR-007), rename. On power loss the user gets the previous version, the documented behavior. Boot recovery reconciles a leftover *.tmp: kept-target-if-both-present, promote-if-target-gone (see ADR-007 for why "just promote" is unsafe).
  • The file is read fully into the rope at boot. v0.1 caps file size at 256 KB; larger files refuse to open with a clear message. Saving is not capped — the buffer is always persistable.
  • Wired into main.rs (2026-07-11): the editor boot-loads the note via Editor::with_text(Storage::load()) — a missing card / repo / unreadable note halts boot with the reason painted on the panel — and :w/:sync persist through Storage::save, inline on the UI loop (a small-file write is tens of ms). Save errors are logged and the RAM buffer kept, so a card pulled mid-session doesn't lose work. The git-push half of :sync is not wired yet: it awaits git_sync being graduated into a module and run on the dedicated git thread (see git below), so :sync currently just saves.

wifi — on-demand station

  • Off by default. The radio is powered down (esp_wifi_stop) except during a Publish and a short grace window after. This is the load-bearing battery decision: always-on station mode burns ~410 mAh/day just for the radio; on-demand is closer to ~25 mAh/day at 10 Publishes/day. See qfd.md H13.
  • Bring-up on Ctrl-G. wifi_task transitions Off → Associating → Connected. Association timeout: 10 s. If association fails, the local commit is preserved and the side panel's Wi-Fi field shows ✗; no background retry.
  • Grace window of ~90 s after a Publish completes. A Ctrl-G inside the window reuses the connection (no second association cost). If no Publish fires before the window expires, the radio tears down.
  • No reconnect-backoff state machine. There is nothing to reconnect to when the user isn't trying to Publish. Single-shot try-on-demand replaces the exponential-backoff approach.
  • SNTP runs on the first successful bring-up of a power session. The result is cached for up to 24 h; subsequent Publishes use the cached time for the commit message. Falls back to device uptime if SNTP has never succeeded — logged but non-fatal.
  • Status exposed via a WifiState atomic enum (Off, Associating, Connected, Failed).

On-device provisioning (SSID, PAT rotation, etc.) is deliberately deferred to v0.9. v0.1 reads all config from the binary at build time — see "Provisioning — build-time only" below.

git — commit + push

Library + kill-switch: ADR-004 — the gitoxide bet was retired in spike 7 (gix has no HTTP(S) push at all), so the module is built on libgit2 via the git2 crate. Auth + transport are settled: HTTPS + PAT over esp-idf's mbedTLS (ADR-005) — no custom transport and no reqwest/rustls layer are needed; libgit2's own smart-HTTP client speaks TLS through the mbedTLS stream it is compiled against. The full init → commit → push path is proven on hardware — see the spike 7 postmortem. PSRAM heap during push is a top-3 watched metric — see qfd.md §6.

  • libgit2 1.9.4 is vendored as an esp-idf component (firmware/components/libgit2) built with GIT_HTTPS + GIT_MBEDTLS; git2 binds it in system mode. A short esp_stubs.c supplies the POSIX calls picolibc/FATFS lack (utimes existence-gate, p_rename replace, uid/symlink).
  • The server cert chain is verified against an embedded GitHub root bundle (GIT_OPT_SET_SSL_CERT_LOCATIONS); the certificate_check callback returns PASSTHROUGH so libgit2's own verdict stands → fail-closed on an untrusted / MITM cert. A product should refresh those roots (or reuse esp-idf's bundle) at provisioning.
  • Runs on a dedicated ~96 KB thread, not the shared UI/main stack: libgit2's init→push chain nests ~10 GIT_PATH_MAX (4 KB) buffers (~67 KB measured).
  • Operations needed in v0.1 (the gct shell function is the reference):
    • open a persistent clone at /sd/repo (spike 7 proved the flow on a fresh per-boot init pushing a throwaway device/<unix> branch, to isolate the transport from history reconciliation; the product keeps one clone and fast-forwards — storage is ADR-007, still SD-vs-flash-FAT while SD is blocked)
    • stage everything under the working copy (git add --all, incl. deletions)
    • short-circuit return if nothing is staged — status: "nothing to publish"
    • commit with author from config, message "<ISO-8601 timestamp>" (no wip prefix; the timestamp is the message)
    • push HEAD to origin/<current branch> (normally a clean fast-forward)
    • on push failure: git pull --no-edit (merge), then retry the push once. Only surface failure to the side panel's publish-state field if the pull conflicts or the second push also fails.
  • The PAT is handed to libgit2's credential callback (USER_PASS_PLAINTEXT); never logged, never persisted to the working copy.
  • The whole sequence is atomic from the user's view — see CONTEXT.md → Publish.

Provisioning — build-time only (no module on device)

v0.1 has no provisioning module, no NVS config, no LittleFS mount. All config (Wi-Fi creds, remote URL, GitHub user, PAT, commit author) is supplied as environment variables at build time:

export TW_WIFI_SSID=... TW_WIFI_PASS=...
export TW_REMOTE_URL=... TW_GH_USER=... TW_PAT=...
export TW_AUTHOR_NAME=... TW_AUTHOR_EMAIL=...
cargo espflash --release

build.rs reads these (or fails the build) and emits constants the runtime references directly. The .env file used to source these is gitignored.

The git working copy is set up out-of-band: the dev clones the remote onto the mounted SD card from their laptop. There is no "first clone on device" in v0.1.

Net savings vs. an on-device wizard: ~300–500 LoC of firmware (HTTP server, captive AP, form parsing, validation state machine, NVS read/write, TOML parser). On-device provisioning + NVS-backed config land in v0.9 when non-dev users enter the picture.

Memory plan

ESP32-S3-N16R8: 512 KB SRAM + 8 MB PSRAM. We budget conservatively.

Region Approx Use
Internal SRAM ~120 KB task stacks, DMA buffers, hot code paths
Internal SRAM ~80 KB mbedtls runtime (TLS handshake working set)
PSRAM ~128 KB EPD framebuffer (792×272×1 ≈ 27 KB) + shadow + glyph cache
PSRAM ~512 KB rope buffer + edit history headroom
PSRAM ~1.5 MB libgit2 working set during push (pack delta, etc.)
PSRAM rest heap headroom

PSRAM is the default for Box::new via a custom allocator wrapper; DMA-able allocations must explicitly request internal SRAM (epd-waveshare needs this for SPI buffers).

Concurrency model

Rationale and rejected alternatives: ADR-006.

  • Editor state behind a single Mutex. Holders: ui_task (writer), render_task (reader, snapshot then unlock), git_task (reader for save).
  • No await / no async runtime — std threads only. Simpler debugging on embedded; the workload doesn't justify async overhead.
  • git_task is spawned on demand by Ctrl-G. Only one push in flight; a second Ctrl-G while one is running is ignored with a status message.

Error handling

  • anyhow::Result at task boundaries.
  • Errors that reach a task's top level are formatted and posted to the Status channel.
  • A task never panics in v0.1 production builds (debug builds keep panics). Anything that would panic gets surfaced as a status message and the task restarts itself.

File layout

SD card (FAT):
  /sd/repo/              ← git working copy
    .git/
    notes.md             ← the only file v0.1 opens
  /sd/local/             ← reserved, unused in v0.1

Internal LittleFS:
  /nvs/config.toml       ← Wi-Fi creds, PAT (encrypted), remote URL,
                           author name/email, last-known branch

PAT encryption uses an HMAC key derived from the chip's eFuse block, so a stolen SD card alone isn't enough; an attacker would need the device. v0.1 does not implement key rotation.

Test plan

  • Host-side unit tests: rope ops, keymap dispatch, dirty-rect math, config parsing. Pure-Rust, run in CI.
  • Hardware-in-the-loop: a checklist run before declaring v0.1 done, matching the product doc's acceptance criteria. No automated rig yet.
  • Long-run soak: 1 hour continuous typing via a USB keystroke generator (any laptop) on the bench. Watch heap, dropped keys, ghosting.

Risks and how we'll know they bit us

Mirrored as live conflicts in qfd.md §7 "Conflicts left explicitly unresolved by v0.1".

Risk Symptom we'd see Fallback
gix smart-HTTP push doesn't work on esp-idf-rs mbedtls fired 2026-07-05: gix has no HTTP(S) push at all (only file:///ssh://) spike 7 (desktop) switched to libgit2 (git2) — mechanics proven on desktop; on-device libgit2/mbedtls cross-compile is the new gate. See postmortem.
TinyUSB host drops HID reports under load dropped keystrokes during fast typing enable larger USB rx buffer; if still bad, fall back to BLE-HID for v0.1
EPD partial refresh slower than 200 ms typing feels laggy reduce font size to shrink dirty area; or render multi-char bursts
TLS heap pressure on PSRAM OOM during push tune mbedtls to smaller cipher suites; force GC of glyph cache before push
SD + EPD on same SPI bus collide corruption on save during render ADOPTED (ADR-012): SD on its own SPI3 host

Every one of these is detected by a spike before integration starts — we are not finding them at the end.

Reference in New Issue View Git Blame Copy Permalink