libgit2 creates loose objects and packfiles with mode 0444 (read-only). FATFS
turns that into the AM_RDO attribute and then refuses to f_unlink them (EACCES),
and esp-idf's FATFS VFS chmod cannot clear AM_RDO — so a written object can
never be deleted, blocking re-clone recovery and (later) fetch/repack.
Shim p_open/p_creat in esp_stubs.c to force owner-write into every create mode
(compile posix.c's originals under throwaway names, same mechanism as p_rename),
so nothing is ever created read-only. Immutability is only a safety hint on an
appliance where nothing but libgit2 touches these files.
Non-regressive on hardware: git_push, which links the same component, still
commits + pushes cleanly with the shim. The delete path itself is validated via
git_sync's RECLONE_EACH_BOOT knob (pending a dedicated flash).
Increment A (persistent-clone publish cycle) is hardware-verified: clone +
persistent open + fast-forward push. Mark the follow-up checklist: git module
fold is in progress (A done; B = divergence + AM_RDO unlink shim; C = editor
wiring), tech-doc revision done, PAT-in-flash tracked as open ADR-011. Note
the FAT read-only-delete limitation and the real-notes-repo sizing caveat.
859c478 aborted if /spiflash/repo already existed — libgit2 refuses to
clone into a non-empty dir (code=Exists). Remove a writable stale/partial
clone and re-clone.
Milestone #2A now hardware-verified end to end: clone (boot 1) + persistent
open + fast-forward push (boot 2) against origin/main, on the 96 KB git
thread, min heap 6.8 MB — clone/checkout fit the stack, no bump needed.
Known limit: the remove only works on a *writable* leftover. libgit2 marks
objects/packs read-only and FATFS won't f_unlink a read-only file (EACCES);
POSIX chmod does not clear AM_RDO on esp-idf's FATFS VFS (verified). So
re-cloning over a run that wrote objects needs an AM_RDO-clearing unlink
shim in esp_stubs.c (increment B, also needed for fetch/repack); until then
the fallback is a one-time `espflash erase-region 0x310000 0x400000`.
Milestone #2 increment A: the product's real publish flow, distinct from
git_push's fresh-init-per-boot throwaway branch. open-or-clone a persistent
/spiflash/repo, append to a tracked notes.md, commit on top of the branch,
and fast-forward push over mbedTLS HTTPS+PAT. Reconcile handles up-to-date
and fast-forward; a true divergence (merge commit on FATFS) is deferred to
increment B. Runs on the dedicated 96 KB git thread; gated behind `git`.
Compiles clean; on-device clone/open+fast-forward not yet hardware-verified
(clone/checkout is deeper than the proven init path — the flash may need a
GIT_STACK bump).
The PAT-in-flash shortcut (ADR-005 v0.1) is safe for the dev's bench unit
but not for a shipped one: it's plaintext in flash, the same token on every
unit, and rotation needs a reflash. ADR-005 decided the auth model but left
provisioning mechanics open.
Capture that as ADR-011 (Proposed/Open): the options (build-time bake →
on-device paste → eFuse-encrypted NVS, per-device fine-grained PAT, GitHub
App) and the likely shape, deferred beyond v0.1. It gates the first non-dev
distribution. Point the Spike 7 postmortem's shortcut entry at it.
The v0.1 technical doc still described the gitoxide/gix plan with an
undecided transport and a 32 KB git_task stack. Bring it in line with
what spike 7 proved:
- gix → libgit2 via the git2 crate (ADR-004 kill-switch fired: no gix push)
- transport settled: HTTPS + PAT over esp-idf mbedTLS, no custom transport
or reqwest/rustls layer needed
- fail-closed cert verification (embedded GitHub roots + PASSTHROUGH)
- git_task stack 32 KB → 96 KB (measured ~67 KB depth); dedicated thread
- persistent-clone + fast-forward is the product flow (spike used a fresh
per-boot throwaway branch); add --all stages deletions
- PAT via libgit2 credential callback, never logged/persisted
Also mark the Spike 7 entry done in the spikes list.
Git now runs on its own 96 KB std::thread and the main-task stack is back
to 12 KB — hardware-verified off-main. Update the follow-up checklist and
the "three bugs" caveat to reflect the completed move.
libgit2's init→push chain is deeply stack-hungry (~67 KB measured for a
trivial config write; the push is deeper). Spike 7 sized the shared main
task stack at 96 KB for it, which forced the editor build to over-reserve
~80 KB.
Move git_publish onto its own std::thread (GIT_STACK = 96 KB via
Builder::stack_size) and join it from main. This lets
CONFIG_ESP_MAIN_TASK_STACK_SIZE drop back to 12288 — the Spike-6 value
proven to run the editor plus a TLS-on-main handshake — so the editor no
longer pays for git's stack.
Hardware-verified: the push ran off-main (mbedTLS + FATFS) with no panic,
no stack overflow, and no ENOMEM on the 96 KB spawn. This retires the
earlier "time() only works on the main task" misdiagnosis for good — it
was always the default 4 KB pthread stack overflowing, never thread-vs-main.
The certificate_check bypass is retired (real trust-store, hardware-verified
2519ed8) and the product sync transport is decided (HTTPS+PAT, not SSH, since
on-device libgit2 is HTTPS-only). Remaining shortcut is PAT-in-flash.
Retire the cert-check bypass. Embed GitHub's root CAs (github_roots.pem:
USERTrust ECC/RSA + DigiCert G2/Global Root CA), write them to /spiflash/ca.pem
at boot, and point libgit2's mbedTLS stream at them via
GIT_OPT_SET_SSL_CERT_LOCATIONS (CONFIG_MBEDTLS_FS_IO is on). The push callback
now returns CertificatePassthrough instead of CertificateOk, so the http
transport maps it to `is_valid ? 0 : -1` (httpclient.c:805) -- an untrusted or
MITM cert fails the push (fail-closed), no blanket-accept.
Hardware-verified 2026-07-06: `TLS trust store installed`, chain validated
against the embedded USERTrust ECC root, push accepted (branch
device/1783372683 on jcalixte/typoena-test). Roots must be refreshed if GitHub
rotates CAs; a product would prefer esp-idf's bundle via a custom subtransport.
The full init -> commit -> push over mbedTLS HTTPS now runs on hardware
(verified device-side + via git ls-remote). Documents the three fixes that
got there -- 96 KB main stack, remove-then-rename p_rename, and the utimes
existence-gate that had silently dropped every ODB write -- plus the
shortcuts still standing (cert bypass, PAT-in-flash, SSH-vs-HTTPS remote).
Wi-Fi + SNTP + flash-FAT + libgit2 in one bench binary: init a fresh working
copy, commit, and push a per-boot device/<unix> branch over mbedTLS HTTPS with
PAT auth. Gated behind the `git` feature; built/flashed via `just flash-git-push`.
Status: local git init verified on hardware; the ODB-write fix and the full
push are not yet confirmed end-to-end on device. Cert verification is bypassed
in the push callback (spike shortcut) -- real trust-store wiring must land
before this leaves the bench.
build.rs embeds TW_REMOTE_URL / TW_GH_USER / TW_PAT / TW_AUTHOR_* via env!()
so only the git_push binary carries them (the editor references none), and
.env.example documents them. NOTE: TW_PAT lands in the git_push flash image
-- a bench-only shortcut (ADR-005); a product must not embed the PAT.
The loose ODB silently dropped every write on device: utimes() returned 0
unconditionally, so libgit2's freshen probe (git_futils_touch -> p_utimes)
always reported "object exists" and git_odb_write skipped the write entirely
-- blobs/trees/commits never hit disk and write_tree failed with "invalid
object specified". Gate utimes on file existence (present -> 0, absent ->
ENOENT). Also add a remove-then-rename p_rename (FATFS f_rename can't replace
an existing target, no hardlinks), with posix.c's original scoped out in the
component CMakeLists, plus symlink/gai_strerror link stubs git_push pulls in.
p_rename was verified on hardware (cleared the rename error); the utimes fix
is diagnosed from the failure + libgit2 source and build-verified, on-device
confirmation still pending.
libgit2's repository_init -> config-write -> FATFS -> wear-leveling chain
nests ~10 GIT_PATH_MAX (4 KB) stack buffers deep; a trivial config write
measured ~67 KB on hardware and overflowed the previous 48 KB, corrupting a
newlib lock handle (LoadProhibited in xQueueGenericSend). Shared with the
editor build, so this is temporary -- git should move to a dedicated
large-stack task and this can drop back to ~16 KB.
16 MB layout adding a `storage` FAT data partition for the on-device git
working copy. Applied only by `just flash-git-push` (espflash
--partition-table); the editor flash keeps its default single-app layout.
Adds the extra_components metadata, git2 as an optional dep behind the
`git` feature (default-features off -> no openssl-sys/libssh2-sys), and a
git_smoke bin gated on it. libgit2-sys/libz-sys run in system mode against
fake pkg-config files (empty Libs) so they emit no link flags -- symbols
come from the esp-idf component. Proven on device: git2 version + a blob
SHA1 through the mbedTLS backend.
Builds vendored libgit2 as an esp-idf component so it inherits the
lwip/mbedtls/pthread/vfs/newlib include+link graph -- the include cascade
that sank CFLAGS injection never appears. Configured for mbedTLS via a
hand-written git2_features.h. Small port surface, libgit2 sources
untouched: poll.h shim, lstat==stat (esp_port.h), p_mmap via malloc+read
(esp_map.c), and identity/symlink stubs (esp_stubs.c). Registers empty
when LIBGIT2_SRC is unset so the editor build is unaffected.
Pins the exact version libgit2-sys 0.18.5 expects, so git2's safe API can
bind it in system mode. Sits under the esp-idf component that builds it;
clone with --recurse-submodules to fetch the source.
CFLAGS include-injection cascades through the whole esp-idf component
graph (arpa/inet.h -> sys/ioctl.h -> ...), so path 1 is out. Decide
path 2: libgit2 as an esp-idf CMake component with USE_HTTPS=mbedTLS,
which fixes the includes and the TLS backend together.
libgit2 C cross-compiles for xtensa; blocker is missing esp-idf lwIP
headers (arpa/inet.h) because libgit2-sys builds standalone. PSRAM
enabled and hardware-verified.
Adds the 8 MB octal PSRAM to the heap allocator (verified on the N16R8:
detected, memory-tested, 8192K pooled). OCT mode is required or init
fails; speed left at 40 MHz for a safe first enable. Prerequisite for
the git working set and the rope buffer.
Postmortem + ADR-004 outcome note + risk-table update: gitoxide has no
HTTP(S) push, so v0.1 falls back to libgit2; the on-device libgit2 <->
mbedtls cross-compile is the next gate.
Proves add -> commit -> push over HTTPS+PAT via libgit2 (git2), the
ADR-004 fallback after gitoxide was found to lack HTTP(S) push. Host
crate on stable, kept out of the xtensa firmware tree.
New docs/postmortems/ folder. The bench card (133 GB SDXC) rejects CMD59
(SPI-mode CRC) as illegal; CMD0/CMD8 succeed, so wiring and firmware are
proven and the fix is a compliant card. Captures the root cause, the
keep-CRC-required decision, and the reusable findings (EPD bus lock, LFN
requirement, R1-byte diagnostic) before the work pauses for hardware.
Standalone bench program (src/bin/sd_fat.rs, `just flash-sd`) that mounts
FAT over the EPD's shared SPI2 bus and proves the persistence module's
atomic save: write .tmp, fsync, rename, read back and compare.
Runs SD-only: the EPD's SpiBusDriver holds an exclusive bus lock for its
lifetime, so an arbitrated SD device can't be live alongside it yet. Keeps
CRC required and maps a card that rejects CMD59 to a clear "use a genuine
card" message rather than running the user's notes over an unchecked bus.
sdkconfig gains CONFIG_FATFS_LFN_HEAP (the atomic-save .tmp name isn't valid
8.3) and, temporarily, CONFIG_LOG_MAXIMUM_LEVEL_DEBUG to read the drivers'
per-command R1 bytes during bring-up.
US-International dead-key accents, slotted after navigation without
renumbering later versions so ADR/qfd/README anchors stay valid. Also
adds a UTF-8-correct buffer groundwork bullet to v0.2.
Swap the mono font from the ascii subset to iso_8859_15 (Latin-9) so
à é ê ç … œ € have glyphs. Cell size is identical, so ASCII rendering is
byte-for-byte unchanged; groundwork for v0.2.5 international input.
Spike 14 relies on file deletion, which the roadmap did not list. Add it
explicitly, flagging that a Tracked-file removal must reach the next
Publish's staged set (git rm / add -A, not plain git add .).
Spikes 8-14 are a cross-release display/UX bench batch, not part of the
v0.1 integration gate — only 8 and 9 feed v0.1. Move them out of the v0.1
technical doc into their own log (with a feeds-table and dependency notes),
leaving a pointer behind; link it from the README layout map.
Replace the header/status-bar layout with a full-height writing column
plus an always-visible side panel holding all metadata. Canonicalize the
new region vocabulary in CONTEXT.md (Writing column, Side panel) and retire
header / status line / edit area as terms-to-avoid.
Consequent sweeps: rewrite the Error UX section in side-panel terms and
give keyboard-disconnect a home (⌨ ✗, disconnect-only); drop the "line N"
readout; reconcile the render module ops and region shadow; sweep the
status-line/edit-area wording across the technical, roadmap, qfd, notes,
README, and ADR-003 docs (incl. the ~11→~13 line-count figure).
The product doc opened notes.md from /sd/local, the permanently-private
scope where Publish is unavailable — contradicting the headline Ctrl-G
flow and every other doc. Point it at the Tracked working copy.
Standalone `wifi_tls` binary: station assoc, SNTP, then a validated
HTTPS GET to api.github.com against the esp-idf cert bundle. Gates
Spike 7 (gitoxide push). Creds come from firmware/.env via build.rs.
Generalise the normal-mode command state machine into d/c operators that
compose with motions (dw, d$, …), doubled forms (dd, cc), and text
objects: iw/aw plus nesting-aware bracket pairs (i(, a{, …) and quotes
(i", i'). ciw/daw/di( and friends now work; pending op/object shows in
the status strip.
Per keystroke, diff the new frame against the last shown one to find the
changed row band and partial-refresh only those rows instead of all 272.
E-paper update time scales with the gate rows driven, so a single edited
text line refreshes far faster. Windowed in Y only (full width, both
controllers), so the seam/mirroring logic is unchanged. Also drop the
periodic full refresh to every 64 updates — the panel stays visually
clean, so it's now mainly for longevity.
Spike 5 — first spike where keyboard input and panel output run
together. usb_kbd changes from a blocking run() into start(): it brings
the USB host stack up on background threads and pushes edge-detected,
US-layout-translated key-downs onto a queue drained via next_key().
main.rs owns the panel, maintains a wrapped/scrolling text buffer, and
partial-refreshes per keystroke batch with a periodic full refresh to
clear ghosting. Logs per-refresh latency.
display_frame_partial writes the new image to bank 0x24, runs GxEPD2's
partial waveform (_Update_Part: 0x3C/0x80, 0x21/0x00,0x10, 0x22/0xFF,
0x20), then syncs bank 0x26 so the next partial has a correct baseline.
Like GxEPD2 for this dual-controller panel, the update covers the full
panel with the partial waveform — no windowing, since the waveform time
dominates. Much faster than a full refresh and without the flashing.
Spike 4. Drive the ESP-IDF USB Host Library directly through the raw
esp-idf-sys bindings (the convenience HID class driver is a managed
component not vendored in mainline, and a boot keyboard doesn't need
it). On attach, src/usb_kbd.rs enumerates the device and dumps its
descriptors, claims the boot-keyboard interface, sends SET_PROTOCOL(boot)
and SET_IDLE(0), then polls the interrupt-IN endpoint and decodes each
8-byte report into modifiers + keycodes logged over UART.
main.rs becomes the Spike 4 harness; the epd module is kept compiled
(allow(dead_code)) so it doesn't bit-rot. Verified on hardware with a
19f5:3255 keyboard: letters, digits, modifiers, and rollover all decode.
Add a thin dual-SSD1683 driver (src/epd.rs, ported from GxEPD2's
GxEPD2_579_GDEY0579T93) for the 792x272 e-paper panel: reset/init,
RAM addressing, the split-and-mirror full-frame blit across the
master/slave seam at x=396, and full-refresh waveform. Frame
implements embedded-graphics' DrawTarget.
Replace the Spike 1 blink harness in main.rs with the Spike 2 test:
SPI at 4 MHz over SCK 12 / DIN 11 / CS 7 / DC 6 / RST 5 / BUSY 4,
alternating normal/inverted frames with a seam-straddling circle,
"Typoena", and the build tag. Verified on hardware 2026-07-04.
Emit BUILD_TIME and BUILD_GIT (git describe --always --dirty) as
rustc env vars so the running firmware can identify itself on serial
and on-panel. A rerun-if-changed on a nonexistent file forces the
script every build, keeping the timestamp fresh. Bring-up lesson:
know which build you're diagnosing.
The bench board is a DevKitC-1 v1.0 (RGB LED on GPIO 48), not the v1.1
layout (GPIO 38). Vendor the official v1.0 pinout diagram under docs/ and
reference it from the README so the pinout is pinned offline, and fold in
the 2026-07-04 on-hardware blink verification note.
