Compare commits

..

3 Commits

Author SHA1 Message Date
Julien Calixte
7a1f25f5d1 docs(persistence): document module and refine FAT recovery to two cases
Record that persistence is now implemented and hardware-verified, and correct
the crash-recovery rule everywhere it appeared as "just promote the tmp": a
crash during the tmp write leaves a partial tmp, so recovery keeps the
committed target when both files are present and only promotes the tmp when
the target was already unlinked. Also fix the stale README heading that still
said the SD was on shared SPI2.
2026-07-11 12:02:30 +02:00
Julien Calixte
1e3220a95f refactor(sd): drive Spike 3 harness through persistence module
sd_fat is now a thin on-device harness over firmware::persistence instead of
duplicating the mount FFI. It mounts, reports FAT usage + negotiated clock,
loads notes.md (non-destructive), and only runs the write round-trip when
notes.md is empty so a provisioned card is never clobbered. Verified on
hardware 2026-07-11: mount at 10 MHz, 91-byte round-trip byte-identical.
2026-07-11 12:02:22 +02:00
Julien Calixte
9c338571f0 feat(persistence): add SD mount + atomic save/load module
Graduate the proven Spike 3 storage stack into firmware::persistence so the
editor and the spike share one implementation. Storage::mount brings up the
dedicated SPI3 bus (ADR-012) and mounts FAT with format_if_mount_failed=false
(never reformat the user's card). save() does the FAT-safe atomic write
(tmp -> fsync -> unlink -> rename); recover() reconciles a leftover *.tmp at
boot, keeping the committed file when the crash point is ambiguous and only
promoting the tmp when the target was already unlinked. load() caps reads at
256 KiB.
2026-07-11 12:02:15 +02:00
7 changed files with 433 additions and 260 deletions

View File

@@ -358,9 +358,16 @@ derived key.
- **FatFS caveat (Spike 3, verified 2026-07-11):** FatFS's `f_rename` returns
`FR_EXIST` on an existing destination — it does **not** replace like POSIX
`rename(2)`. So the atomic save must `f_unlink` the target before renaming the
`*.tmp` over it, and pair that with **boot recovery**: a lingering `*.tmp` at
startup means the last save didn't finish → promote it (it is the newest
complete, fsync'd copy). See the
`*.tmp` over it, and pair that with **boot recovery** of a lingering `*.tmp`.
Recovery is *not* simply "promote the tmp": a crash *during* the tmp write
leaves a partial tmp, so the choice depends on whether the target survived —
- **tmp + target both present** → the crash could have been mid-write, so the
tmp is untrustworthy. Keep the committed target, discard the tmp (this is the
documented "you get the previous version" behaviour).
- **tmp only, target absent** → the target was already unlinked, so the tmp is
the newest complete, fsync'd copy and the only one left. Promote it.
Implemented in `firmware::persistence::Storage::{save,recover}`. See the
[Spike 3 postmortem](postmortems/2026-07-05-spike3-sd-cmd59.md#resolution-2026-07-11).
- **SD-card compatibility:** use a genuine card, ideally **≤32 GB (SDHC/FAT32)**.
Large or counterfeit SDXC cards may reject `CMD59` (SPI-mode CRC) and fail to

View File

@@ -39,10 +39,18 @@ as-is on FAT; the destination must be unlinked first.
That unlink opens a crash window: between `f_unlink(target)` and `f_rename`, the
target is momentarily gone while `*.tmp` holds the complete, fsync'd new content.
The consequence for the real persistence module is a **boot-recovery** rule: a
lingering `*.tmp` at startup means the last save didn't finish → promote it
(it's the newest complete copy). The spike now does unlink-then-rename and
tolerates a missing target, so re-runs are idempotent; recorded in
The consequence for the real persistence module is a **boot-recovery** rule for a
lingering `*.tmp`. Implementing it (`firmware::persistence::Storage::recover`)
surfaced that "promote the tmp" is too blunt: a crash *during* the tmp write (a
second window, before the fsync) leaves a **partial** tmp, so recovery must look
at whether the target survived —
- **tmp + target both present** → crash point ambiguous, tmp may be partial →
keep the committed target, discard the tmp (the "previous version" guarantee);
- **tmp only, target absent** → target was already unlinked → tmp is the newest
complete copy → promote it.
The spike now does unlink-then-rename and tolerates a missing target, so re-runs
are idempotent; recorded in
[ADR-007](../adr.md#adr-007-storage-split--fat-on-sd-for-working-copy-littlefs-on-flash-for-config).
## Summary

View File

@@ -163,11 +163,18 @@ in [qfd.md §3](qfd.md#3-house-of-quality--whats--hows).
Storage split rationale:
[ADR-007](adr.md#adr-007-storage-split--fat-on-sd-for-working-copy-littlefs-on-flash-for-config).
- Atomic save: write to `notes.md.tmp`, fsync, rename. We accept FAT's
weakness here; on power loss between rename and dir flush, the user gets
the previous version, which is the documented behavior.
Implemented in [`firmware::persistence`](../firmware/src/persistence.rs)
(`Storage::{mount, load, save, recover}`), graduated from Spike 3 and
hardware-verified 2026-07-11.
- Atomic save: write to `notes.md.tmp`, fsync, unlink the target (FatFS
`f_rename` won't overwrite — ADR-007), rename. On power loss the user gets
the previous version, the documented behavior. Boot recovery reconciles a
leftover `*.tmp`: kept-target-if-both-present, promote-if-target-gone (see
ADR-007 for why "just promote" is unsafe).
- The file is read fully into the rope at boot. v0.1 caps file size at
256 KB; larger files refuse to open with a clear message.
256 KB; larger files refuse to open with a clear message. Saving is not
capped — the buffer is always persistable.
### `wifi` — on-demand station

View File

@@ -66,13 +66,15 @@ of it. `sdkconfig.defaults` gains the full certificate bundle and a bigger main
task stack for the mbedtls handshake — a one-time esp-idf reconfigure on the
next build.
**Spike 3 — SD card (FAT) on shared SPI2: verified 2026-07-11.** A separate
**Spike 3 — SD card (FAT) on dedicated SPI3: verified 2026-07-11.** A separate
binary — [`src/bin/sd_fat.rs`](src/bin/sd_fat.rs), flashed with `just flash-sd`
brings up the SD card, mounts FAT at `/sd`, and exercises the persistence
module's atomic save (write `*.tmp` → fsync → rename → read-back). Per ADR-012
the SD runs on its **own SPI3 host****SCK 14 · MOSI 15 · MISO 13 · SD CS 10**
— leaving the EPD alone on SPI2. Verified on the dedicated SPI3 bus 2026-07-11
(same mount + round-trip result as the initial shared-SPI2 bring-up).
is now a thin on-device harness over the real
[`firmware::persistence`](src/persistence.rs) module: it mounts the card, reports
FAT usage, and round-trips an atomic save/load (write `*.tmp` → fsync → unlink →
rename → read-back). Per ADR-012 the SD runs on its **own SPI3 host**
**SCK 14 · MOSI 15 · MISO 13 · SD CS 10** — leaving the EPD alone on SPI2.
Verified on the dedicated SPI3 bus 2026-07-11 (same mount + round-trip result as
the initial shared-SPI2 bring-up).
Bench result (genuine 32 GB SDHC card): mounts at 10 MHz, `29806 MiB total`,
atomic round-trip byte-identical. Two findings baked into the code:
@@ -82,8 +84,11 @@ atomic round-trip byte-identical. Two findings baked into the code:
with a swap-the-card message rather than run over an unchecked bus. See the
[Spike 3 postmortem](../docs/postmortems/2026-07-05-spike3-sd-cmd59.md).
- **FatFS rename ≠ POSIX rename.** `f_rename` won't overwrite an existing
target (returns `FR_EXIST`), so the atomic save unlinks the destination first;
the real persistence module must add `*.tmp` boot-recovery. Long filenames
target (returns `FR_EXIST`), so the atomic save unlinks the destination first.
`firmware::persistence` pairs this with `*.tmp` boot-recovery
(`Storage::recover`): if a `*.tmp` is found *alongside* the target the crash
may have been mid-write, so it keeps the committed file and discards the tmp;
it only promotes the tmp when the target was already unlinked. Long filenames
(`CONFIG_FATFS_LFN_HEAP`) are required for the two-dot `*.md.tmp` name.
**Arbitration resolved (ADR-012):** the EPD driver holds an exclusive SPI2 lock

View File

@@ -1,88 +1,44 @@
//! Spike 3 — SD card (FAT) on its own SPI3 host.
//! Spike 3 — SD card (FAT) on its own SPI3 host, now a thin on-device harness
//! over the real [`firmware::persistence`] module.
//!
//! A small standalone bench program (separate binary from the editor firmware)
//! that proves the storage stack the persistence module will sit on:
//! The raw storage stack (SPI3 bring-up, hand-rolled SDSPI descriptors, the
//! atomic write/fsync/unlink/rename dance) was proven here first and has since
//! graduated into `firmware::persistence` so the editor and this spike share one
//! implementation. This binary now just drives that module on hardware:
//!
//! 1. Bring up SPI3 with the SD's four lines — SCK 14, MOSI 15, MISO 13, and
//! its own chip-select (10). This is a *dedicated* bus: the EPD keeps SPI2
//! (SCK 12, MOSI 11, CS 7). See ADR-012.
//! 2. Mount a FAT filesystem on the card at `/sd` via `esp_vfs_fat_sdspi_mount`.
//! 3. Exercise the exact atomic-save pattern the persistence module specifies
//! (ADR-007): write `*.tmp`, fsync, rename over the target, then read back
//! and byte-compare. Report the card's negotiated clock and FAT usage.
//! 1. [`Storage::mount`] — SPI3 (SCK 14, MOSI 15, MISO 13, CS 10; ADR-012) +
//! FAT mount at `/sd` + boot crash-recovery. `format_if_mount_failed` is
//! false in the module, so the card must already be FAT-formatted (it is —
//! Spike 3 formatted it 2026-07-11).
//! 2. Report the card's negotiated clock and FAT usage.
//! 3. Load `/sd/repo/notes.md` (non-destructive) and report its size.
//! 4. Only if there is no notes.md yet (a blank bench card, nothing to lose)
//! exercise the real [`Storage::save`] → [`Storage::load`] round-trip and
//! byte-compare. On a provisioned card the write test is skipped so the
//! user's writing is never clobbered by the bench tool.
//!
//! Why a dedicated SPI3 (ADR-012, decided 2026-07-11): the EPD driver uses
//! esp-idf-hal's `SpiBusDriver`, whose constructor calls
//! `spi_device_acquire_bus(BLOCK)` and holds that *exclusive* bus lock for the
//! driver's whole lifetime (it needs CS held across a cmd→data sequence while DC
//! toggles). While that lock is held, no other device on the same host can
//! transact — so an SD on SPI2 would be locked out for as long as the panel
//! driver is alive, and persistence runs on its own thread (Spike 7) concurrently
//! with EPD refreshes. Rather than rewrite the proven EPD SPI layer and add a
//! cross-thread mutex on the save path, we take the risk-table fallback: the SD
//! gets SPI3 to itself. This spike still drives SD-only, but now because it *is*
//! a separate bus, not to dodge contention.
//!
//! Two esp-idf notes baked in below:
//! - The `SDSPI_HOST_DEFAULT()` / `SDSPI_DEVICE_CONFIG_DEFAULT()` C macros are
//! dropped by bindgen, so the descriptors are filled by hand. The
//! `SDMMC_HOST_FLAG_*` values are `BIT(n)` macros bindgen can't fold either,
//! so they're inlined with a reference to sd_protocol_types.h.
//! - The `.tmp` rename target (`notes.md.tmp`) is not a valid 8.3 name, and
//! FatFS defaults to 8.3-only. `CONFIG_FATFS_LFN_HEAP=y` (sdkconfig.defaults)
//! turns on long filenames — required here and by the real persistence path.
//!
//! Flash with `just flash-sd`. Needs no `.env` (unlike the Wi-Fi spike).
//! Flash with `just flash-sd`. Needs no `.env`.
use std::ffi::CStr;
use std::fs;
use std::io::{Read, Write};
use std::mem::MaybeUninit;
use std::ptr;
use anyhow::{bail, Context, Result};
use esp_idf_svc::hal::delay::FreeRtos;
use esp_idf_svc::sys::{self, esp};
use firmware::persistence::{Storage, MAX_FILE_BYTES, NOTES, REPO_DIR};
/// Injected by build.rs so serial output identifies the exact build.
const BUILD_TAG: &str = concat!("build ", env!("BUILD_TIME"), " @", env!("BUILD_GIT"));
// SD wiring on its own SPI3 host (ADR-012). MISO 13 and CS 10 are unchanged from
// the original shared-bus spike; only SCK/MOSI move off the EPD-shared 12/11 onto
// dedicated pins so the two buses are fully independent.
const PIN_SCK: i32 = 14;
const PIN_MOSI: i32 = 15;
const PIN_MISO: i32 = 13;
const PIN_CS: i32 = 10;
/// SD clock. Deliberately conservative for bench jumper wires: SDSPI's 20 MHz
/// default is prone to CRC errors on long unterminated jumpers, which would look
/// like a stack failure when it's really signal integrity. 10 MHz keeps margin
/// while staying a real speed; raise toward 20 MHz once on a clean PCB.
const SD_FREQ_KHZ: i32 = 10_000;
/// Host flags from sd_protocol_types.h — `BIT(3)` / `BIT(5)`. Inlined because
/// bindgen doesn't fold the nested `BIT()` macro into a constant.
const SDMMC_HOST_FLAG_SPI: u32 = 1 << 3;
const SDMMC_HOST_FLAG_DEINIT_ARG: u32 = 1 << 5;
/// VFS mount point. `MOUNT` is the C string handed to esp-idf; `MOUNT_STR` is
/// the same path for std::fs.
const MOUNT: &CStr = c"/sd";
const MOUNT_STR: &str = "/sd";
fn main() -> Result<()> {
// Required once before any esp-idf-svc call; some runtime patches only link
// if this symbol is referenced. See esp-idf-template#71.
esp_idf_svc::sys::link_patches();
esp_idf_svc::log::EspLogger::initialize_default();
log::info!("Typoena — Spike 3 (SD/FAT on dedicated SPI3), {BUILD_TAG}");
log::info!("Typoena — Spike 3 (SD/FAT via firmware::persistence), {BUILD_TAG}");
match run() {
Ok(()) => {
log::info!("✅ Spike 3 complete — mount + atomic write/fsync/rename/read-back on dedicated SPI3")
}
Ok(()) => log::info!("✅ Spike 3 complete — persistence::Storage mounts and round-trips"),
Err(e) => log::error!("❌ Spike 3 failed: {e:?}"),
}
@@ -93,165 +49,54 @@ fn main() -> Result<()> {
}
fn run() -> Result<()> {
let card = mount_sd().context("mounting SD over SPI3")?;
let storage = Storage::mount().context("mounting SD via persistence module")?;
// SAFETY: `card` is a live handle returned by a successful mount.
let (max_khz, real_khz) = unsafe { ((*card).max_freq_khz, (*card).real_freq_khz) };
log::info!("card mounted at /sd — max {max_khz} kHz, negotiated {real_khz} kHz");
let (max_khz, real_khz) = storage.negotiated_khz();
log::info!("card clock — max {max_khz} kHz, negotiated {real_khz} kHz");
let (total, free) = fs_info().context("reading FAT usage")?;
let (total, free) = storage.usage().context("reading FAT usage")?;
log::info!(
"FAT usage — {} MiB total, {} MiB free",
total / (1024 * 1024),
free / (1024 * 1024)
);
file_roundtrip().context("atomic write/fsync/rename/read-back")?;
list_root(); // best-effort, informational
if storage.repo_present() {
log::info!("{REPO_DIR} present (card is provisioned)");
} else {
log::warn!("{REPO_DIR} missing — card not provisioned; run `just init` on the host");
}
// Read-back is always safe. An empty string means "no notes.md yet".
let existing = storage.load().context("loading notes.md")?;
log::info!("notes.md load OK — {} bytes", existing.len());
if existing.is_empty() {
write_test(&storage).context("save/load round-trip")?;
} else {
log::info!(
"notes.md already has content — skipping the destructive write test to protect it \
(v0.1 caps notes at {} KiB)",
MAX_FILE_BYTES / 1024
);
}
Ok(())
}
/// Init the dedicated SPI3 bus and mount the card. Returns the card handle (kept
/// alive for the program's lifetime; the spike never unmounts).
fn mount_sd() -> Result<*mut sys::sdmmc_card_t> {
// 1) Initialize SPI3 with the SD's four lines. Dedicated bus (ADR-012) — no
// EPD deselect needed: the panel is on SPI2 and can't contend here.
// SAFETY: zeroed spi_bus_config_t is valid (all pins default 0); we then set
// the used pins and mark the quad lines unused (-1).
let mut bus: sys::spi_bus_config_t = unsafe { MaybeUninit::zeroed().assume_init() };
bus.__bindgen_anon_1.mosi_io_num = PIN_MOSI;
bus.__bindgen_anon_2.miso_io_num = PIN_MISO;
bus.sclk_io_num = PIN_SCK;
bus.__bindgen_anon_3.quadwp_io_num = -1;
bus.__bindgen_anon_4.quadhd_io_num = -1;
bus.max_transfer_sz = 4096;
esp!(unsafe {
sys::spi_bus_initialize(
sys::spi_host_device_t_SPI3_HOST,
&bus,
sys::spi_common_dma_t_SPI_DMA_CH_AUTO as _,
)
})
.context("spi_bus_initialize(SPI3)")?;
// 1b) Enable internal pull-ups on the SD lines. The SD spec wants ~10 kΩ
// pull-ups on the data lines; the bench jumpers have none, so MISO
// floats between response bytes and a stray bit reads back as a spurious
// R1 "illegal command" (ESP_ERR_NOT_SUPPORTED) that fails init. The
// ESP32's internal ~45 kΩ pull-ups are usually enough on short wires;
// an external 10 kΩ MISO→3V3 is the proper fix on a real board. Set
// after bus init so the SPI pin config doesn't clobber it (CS gets
// reconfigured by the mount below — harmless; MISO is the one that
// matters).
for pin in [PIN_SCK, PIN_MOSI, PIN_MISO, PIN_CS] {
esp!(unsafe { sys::gpio_set_pull_mode(pin, sys::gpio_pull_mode_t_GPIO_PULLUP_ONLY) })
.with_context(|| format!("pull-up on GPIO {pin}"))?;
/// Exercise the module's real atomic save + load, then confirm the bytes match.
/// Only called when notes.md is empty, so nothing of the user's is at risk.
fn write_test(storage: &Storage) -> Result<()> {
// The module deliberately never creates the repo dir (a missing one means an
// unprovisioned card, which the editor treats as fatal). On a blank bench
// card there's nothing to protect, so create it here as explicit bench setup
// to give `Storage::save` a directory to write into.
if !storage.repo_present() {
log::info!("{REPO_DIR} missing — creating it (bench setup) so the write test can run");
fs::create_dir_all(REPO_DIR).with_context(|| format!("create {REPO_DIR}"))?;
}
// 2) SDSPI host descriptor — hand-rolled SDSPI_HOST_DEFAULT(). The function
// pointers are esp-idf's sdspi_host_* ops; the driver calls them to drive
// the card. `slot` picks the SPI host the device attaches to.
// SAFETY: zeroed is a valid starting point (all fn-pointer Options = None);
// we fill exactly the fields the C macro sets.
let mut host: sys::sdmmc_host_t = unsafe { MaybeUninit::zeroed().assume_init() };
host.flags = SDMMC_HOST_FLAG_SPI | SDMMC_HOST_FLAG_DEINIT_ARG;
host.slot = sys::spi_host_device_t_SPI3_HOST as i32;
host.max_freq_khz = SD_FREQ_KHZ;
host.io_voltage = 3.3;
host.driver_strength = sys::sdmmc_driver_strength_t_SDMMC_DRIVER_STRENGTH_B;
host.current_limit = sys::sdmmc_current_limit_t_SDMMC_CURRENT_LIMIT_200MA;
host.init = Some(sys::sdspi_host_init);
host.set_card_clk = Some(sys::sdspi_host_set_card_clk);
host.do_transaction = Some(sys::sdspi_host_do_transaction);
host.__bindgen_anon_1.deinit_p = Some(sys::sdspi_host_remove_device);
host.io_int_enable = Some(sys::sdspi_host_io_int_enable);
host.io_int_wait = Some(sys::sdspi_host_io_int_wait);
host.get_real_freq = Some(sys::sdspi_host_get_real_freq);
host.input_delay_phase = sys::sdmmc_delay_phase_t_SDMMC_DELAY_PHASE_0;
host.check_buffer_alignment = Some(sys::sdspi_host_check_buffer_alignment);
// 3) Device (slot) config — CS 10, no card-detect / write-protect / SDIO int.
// SAFETY: zeroed is valid; we set the host, CS, and mark the rest unused.
let mut slot: sys::sdspi_device_config_t = unsafe { MaybeUninit::zeroed().assume_init() };
slot.host_id = sys::spi_host_device_t_SPI3_HOST;
slot.gpio_cs = PIN_CS;
slot.gpio_cd = -1;
slot.gpio_wp = -1;
slot.gpio_int = -1;
// 4) Mount config. format_if_mount_failed = true here (spike only): a fresh
// bench card that's exFAT or unformatted gets reformatted to FAT on the
// device instead of failing, so no Mac-side prep is needed. This fires
// only on a *filesystem* mount failure, not on the earlier CMD59 protocol
// rejection (that still bails with the actionable message below).
// The real persistence module MUST keep this false — it must never wipe
// the user's card on a transient mount hiccup. allocation_unit_size is
// used when formatting, so the 16 KiB below now applies.
let mount = sys::esp_vfs_fat_mount_config_t {
format_if_mount_failed: true,
max_files: 4,
allocation_unit_size: 16 * 1024,
disk_status_check_enable: false,
use_one_fat: false,
};
let mut card: *mut sys::sdmmc_card_t = ptr::null_mut();
let rc = unsafe {
sys::esp_vfs_fat_sdspi_mount(MOUNT.as_ptr(), &host, &slot, &mount, &mut card)
};
// Turn the driver's opaque error into something actionable. The one we hit
// in practice is a card that rejects CMD59 (SPI-mode CRC on/off): init gets
// through CMD0/CMD8 cleanly, then the CRC-enable step returns NOT_SUPPORTED.
// That's a card-firmware limitation (common on large/counterfeit SDXC), not
// a wiring fault — and we deliberately keep CRC required rather than run the
// user's notes over an unchecked bus, so we reject the card with guidance.
if rc == sys::ESP_ERR_NOT_SUPPORTED {
bail!(
"SD card rejected CMD59 (SPI-mode CRC). CMD0/CMD8 succeeded, so wiring is \
fine — this card's firmware just doesn't support CRC in SPI mode (common on \
large/counterfeit SDXC). Use a genuine card, ideally ≤32 GB. We keep CRC \
required on purpose: a writing device shouldn't run over an unchecked bus."
);
}
esp!(rc).context("esp_vfs_fat_sdspi_mount (card present? inserted? FAT-formatted?)")?;
Ok(card)
}
/// The persistence module's atomic save (ADR-007), proven end to end: write to
/// a temp file, fsync, rename over the target, then reopen and byte-compare.
fn file_roundtrip() -> Result<()> {
let path = format!("{MOUNT_STR}/spike3.md");
let tmp = format!("{path}.tmp"); // two dots → exercises long-filename support
let payload =
format!("typoena spike 3\n{BUILD_TAG}\ndedicated SPI3: SCK14 MOSI15 MISO13, SD CS10\n");
{
let mut f = fs::File::create(&tmp).context("create tmp")?;
f.write_all(payload.as_bytes()).context("write tmp")?;
f.sync_all().context("fsync tmp")?; // FatFS f_sync — flush before rename
}
// FatFS's f_rename — unlike POSIX rename(2) — refuses to overwrite an
// existing destination and returns FR_EXIST (EEXIST). So the classic
// write-tmp → rename-over-target idiom needs an explicit unlink of the
// target first on FAT. That opens a crash window: the target is briefly
// gone while `tmp` holds the complete, fsync'd new content. The real
// persistence module must pair this with boot recovery — a lingering
// `*.tmp` means the last save didn't finish and should be promoted. See
// ADR-007. (Tolerate a missing target so the first save works too.)
match fs::remove_file(&path) {
Ok(()) => {}
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {}
Err(e) => return Err(e).context("remove existing target before rename"),
}
fs::rename(&tmp, &path).context("rename tmp -> final")?;
let mut back = String::new();
fs::File::open(&path)
.context("reopen final")?
.read_to_string(&mut back)
.context("read back")?;
let payload = format!("typoena spike 3\n{BUILD_TAG}\ndedicated SPI3: SCK14 MOSI15 MISO13 CS10\n");
storage.save(&payload).context("Storage::save")?;
let back = storage.load().context("Storage::load after save")?;
if back != payload {
bail!(
"read-back mismatch: wrote {} bytes, read {} bytes",
@@ -260,32 +105,8 @@ fn file_roundtrip() -> Result<()> {
);
}
log::info!(
"round-trip OK — {} bytes: create {tmp} → fsync → rename {path} → read back identical",
"round-trip OK — {} bytes: save {NOTES} (tmp→fsync→unlink→rename) → load identical",
payload.len()
);
Ok(())
}
/// FAT total/free bytes for the mount.
fn fs_info() -> Result<(u64, u64)> {
let mut total: u64 = 0;
let mut free: u64 = 0;
esp!(unsafe { sys::esp_vfs_fat_info(MOUNT.as_ptr(), &mut total, &mut free) })
.context("esp_vfs_fat_info")?;
Ok((total, free))
}
/// Log the root directory (informational — shows the card's existing content
/// and confirms our file landed).
fn list_root() {
match fs::read_dir(MOUNT_STR) {
Ok(entries) => {
log::info!("/sd contents:");
for entry in entries.flatten() {
let len = entry.metadata().map(|m| m.len()).unwrap_or(0);
log::info!(" {} ({len} B)", entry.file_name().to_string_lossy());
}
}
Err(e) => log::warn!("could not list /sd: {e}"),
}
}

View File

@@ -2,9 +2,12 @@
//!
//! The editor binary (`src/main.rs`) and the spike binaries under `src/bin/`
//! are each separate crate roots; anything they need to share lives here and is
//! reached as `firmware::…`. Currently that is just the resilient Wi-Fi
//! bring-up, extracted from three duplicated `connect_wifi` copies so the retry
//! logic lives in exactly one place and the eventual editor integration reuses
//! it instead of growing a fourth copy.
//! reached as `firmware::…`:
//!
//! - [`net`] — resilient Wi-Fi bring-up, extracted from three duplicated
//! `connect_wifi` copies so the retry logic lives in exactly one place.
//! - [`persistence`] — SD mount + atomic save/load, graduated from the Spike 3
//! bench binary so the editor and the spike share one implementation.
pub mod net;
pub mod persistence;

322
firmware/src/persistence.rs Normal file
View File

@@ -0,0 +1,322 @@
//! SD-card persistence — mount, atomic save, crash recovery.
//!
//! The editor's notes live at `/sd/repo/notes.md` on a FAT filesystem on the
//! microSD card. This module owns bringing that card up and reading/writing the
//! buffer safely across power loss. It is the graduation of the Spike 3 bench
//! binary (`src/bin/sd_fat.rs`): that spike proved the raw stack on hardware
//! (verified 2026-07-11); the proven bits now live here so the editor and the
//! spike share one implementation instead of the spike being a dead-end proof.
//!
//! ## Storage split (ADR-007)
//!
//! FAT-on-SD holds the git working copy (`/sd/repo/`) and local scratch
//! (`/sd/local/`); device config is compiled into the binary in v0.1. The repo
//! is provisioned host-side (`just init` / `just load` copy a clone onto the
//! card) and opened — not cloned — on device, so this module never creates the
//! repo directory: a missing `/sd/repo` means the card wasn't provisioned, which
//! the boot path surfaces as a fatal "re-run `just init`" rather than silently
//! papering over.
//!
//! ## Dedicated SPI3 bus (ADR-012)
//!
//! The card sits on its own SPI3 host (SCK 14, MOSI 15, MISO 13, CS 10). The EPD
//! keeps SPI2. The EPD driver holds an exclusive `spi_device_acquire_bus` lock
//! for its whole lifetime, so a shared bus would lock the SD out; giving the SD
//! its own host sidesteps that for ~2 GPIOs. See the `mount` docs and ADR-012.
//!
//! ## Atomic save + crash recovery (the load-bearing part)
//!
//! FAT gives weak power-loss guarantees, so a save is: write `notes.md.tmp`,
//! `fsync`, unlink the target, rename the tmp over it. On FAT that unlink is
//! mandatory — FatFS's `f_rename` returns `FR_EXIST` on an existing destination
//! (it does *not* replace like POSIX `rename(2)`; Spike 3 finding). That unlink
//! opens a small window where the target is gone while the complete new content
//! sits in the tmp. [`Storage::recover`] closes the loop at boot — see its docs
//! for the exact case analysis, which is subtler than "promote the tmp."
use std::fs;
use std::io::Write as _;
use std::mem::MaybeUninit;
use std::path::Path;
use std::ptr;
use anyhow::{bail, Context, Result};
use esp_idf_svc::sys::{self, esp};
/// SD wiring on its own SPI3 host (ADR-012).
const PIN_SCK: i32 = 14;
const PIN_MOSI: i32 = 15;
const PIN_MISO: i32 = 13;
const PIN_CS: i32 = 10;
/// SD clock. Conservative for bench jumper wires: SDSPI's 20 MHz default is
/// prone to CRC errors on long unterminated jumpers, which look like a stack
/// failure when they're really signal integrity. 10 MHz keeps margin; raise
/// toward 20 MHz on a clean PCB. Init always runs at 400 kHz regardless.
const SD_FREQ_KHZ: i32 = 10_000;
/// Host flags from `sd_protocol_types.h` — `BIT(3)` / `BIT(5)`. Inlined because
/// bindgen doesn't fold the nested `BIT()` macro into a constant.
const SDMMC_HOST_FLAG_SPI: u32 = 1 << 3;
const SDMMC_HOST_FLAG_DEINIT_ARG: u32 = 1 << 5;
/// FAT mount point.
pub const MOUNT: &str = "/sd";
/// Git working copy — provisioned host-side, opened on device.
pub const REPO_DIR: &str = "/sd/repo";
/// The one file v0.1 opens.
pub const NOTES: &str = "/sd/repo/notes.md";
/// Staging name for the atomic save. Two dots → needs long-filename support
/// (`CONFIG_FATFS_LFN_HEAP=y`, set in sdkconfig.defaults).
const NOTES_TMP: &str = "/sd/repo/notes.md.tmp";
/// Largest file [`Storage::load`] will read into the buffer. v0.1 caps notes at
/// 256 KiB; a larger file refuses to open with a clear message rather than
/// exhausting the rope. Saving is *not* capped — never refuse to persist the
/// user's work once it's in the buffer.
pub const MAX_FILE_BYTES: u64 = 256 * 1024;
/// The C mount point (`/sd\0`) for the esp-idf FFI calls.
const MOUNT_C: &std::ffi::CStr = c"/sd";
/// A mounted SD card. Holds the live card handle for its lifetime; v0.1 never
/// unmounts (the card stays up for the whole power session). Not `Send` — the
/// handle lives on the task that mounted it (the ui/main task). The git thread
/// reaches `/sd/repo` through plain `std::fs`; FatFS's per-volume reentrancy
/// lock serialises the two, so no extra mutex is needed here.
pub struct Storage {
card: *mut sys::sdmmc_card_t,
}
/// What [`Storage::recover`] did with a leftover `*.tmp` at boot.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Recovery {
/// No `*.tmp` present — clean shutdown last time.
Clean,
/// `*.tmp` and the target both present: the crash could have landed
/// mid-write, so the tmp is untrustworthy. Kept the committed target,
/// discarded the tmp. The in-flight (unsaved) edit is lost — the documented
/// "you get the previous version" behaviour.
DiscardedTmp,
/// Only `*.tmp` present: the target had already been unlinked, so the tmp is
/// the newest complete, fsync'd copy. Promoted it to the target.
PromotedTmp,
}
impl Storage {
/// Bring up SPI3 and mount the FAT filesystem at `/sd`, then run crash
/// recovery ([`Storage::recover`]) so storage is in a consistent state
/// before the caller reads anything.
///
/// `format_if_mount_failed` is **false**: this is the user's card with their
/// writing on it, so a transient mount hiccup must never trigger a reformat.
/// (The Spike 3 bench binary sets it true for convenience on blank cards;
/// this path must not.)
pub fn mount() -> Result<Self> {
// 1) SPI3 with the SD's four lines. Dedicated bus (ADR-012) — no EPD
// deselect needed: the panel is on SPI2 and can't contend here.
// SAFETY: a zeroed spi_bus_config_t is valid (all pins default 0); we
// set the used pins and mark the quad lines unused (-1).
let mut bus: sys::spi_bus_config_t = unsafe { MaybeUninit::zeroed().assume_init() };
bus.__bindgen_anon_1.mosi_io_num = PIN_MOSI;
bus.__bindgen_anon_2.miso_io_num = PIN_MISO;
bus.sclk_io_num = PIN_SCK;
bus.__bindgen_anon_3.quadwp_io_num = -1;
bus.__bindgen_anon_4.quadhd_io_num = -1;
bus.max_transfer_sz = 4096;
esp!(unsafe {
sys::spi_bus_initialize(
sys::spi_host_device_t_SPI3_HOST,
&bus,
sys::spi_common_dma_t_SPI_DMA_CH_AUTO as _,
)
})
.context("spi_bus_initialize(SPI3)")?;
// 1b) Internal pull-ups on the SD lines. The SD spec wants ~10 kΩ
// pull-ups; bench jumpers have none, so MISO floats between response
// bytes and a stray bit reads back as a spurious R1 "illegal
// command" that fails init. The ESP32's internal ~45 kΩ pull-ups are
// usually enough on short wires; an external 10 kΩ MISO→3V3 is the
// proper fix on a real board.
for pin in [PIN_SCK, PIN_MOSI, PIN_MISO, PIN_CS] {
esp!(unsafe { sys::gpio_set_pull_mode(pin, sys::gpio_pull_mode_t_GPIO_PULLUP_ONLY) })
.with_context(|| format!("pull-up on GPIO {pin}"))?;
}
// 2) SDSPI host descriptor — hand-rolled SDSPI_HOST_DEFAULT() (bindgen
// drops the macro). The fn pointers are esp-idf's sdspi_host_* ops.
// SAFETY: zeroed is a valid start (all fn-pointer Options = None); we
// fill exactly the fields the C macro sets.
let mut host: sys::sdmmc_host_t = unsafe { MaybeUninit::zeroed().assume_init() };
host.flags = SDMMC_HOST_FLAG_SPI | SDMMC_HOST_FLAG_DEINIT_ARG;
host.slot = sys::spi_host_device_t_SPI3_HOST as i32;
host.max_freq_khz = SD_FREQ_KHZ;
host.io_voltage = 3.3;
host.driver_strength = sys::sdmmc_driver_strength_t_SDMMC_DRIVER_STRENGTH_B;
host.current_limit = sys::sdmmc_current_limit_t_SDMMC_CURRENT_LIMIT_200MA;
host.init = Some(sys::sdspi_host_init);
host.set_card_clk = Some(sys::sdspi_host_set_card_clk);
host.do_transaction = Some(sys::sdspi_host_do_transaction);
host.__bindgen_anon_1.deinit_p = Some(sys::sdspi_host_remove_device);
host.io_int_enable = Some(sys::sdspi_host_io_int_enable);
host.io_int_wait = Some(sys::sdspi_host_io_int_wait);
host.get_real_freq = Some(sys::sdspi_host_get_real_freq);
host.input_delay_phase = sys::sdmmc_delay_phase_t_SDMMC_DELAY_PHASE_0;
host.check_buffer_alignment = Some(sys::sdspi_host_check_buffer_alignment);
// 3) Device (slot) config — CS 10, no card-detect / write-protect / int.
// SAFETY: zeroed is valid; we set the host, CS, and mark the rest unused.
let mut slot: sys::sdspi_device_config_t = unsafe { MaybeUninit::zeroed().assume_init() };
slot.host_id = sys::spi_host_device_t_SPI3_HOST;
slot.gpio_cs = PIN_CS;
slot.gpio_cd = -1;
slot.gpio_wp = -1;
slot.gpio_int = -1;
// 4) Mount config. format_if_mount_failed = FALSE — see method docs.
let mount = sys::esp_vfs_fat_mount_config_t {
format_if_mount_failed: false,
max_files: 4,
allocation_unit_size: 16 * 1024,
disk_status_check_enable: false,
use_one_fat: false,
};
let mut card: *mut sys::sdmmc_card_t = ptr::null_mut();
let rc = unsafe {
sys::esp_vfs_fat_sdspi_mount(MOUNT_C.as_ptr(), &host, &slot, &mount, &mut card)
};
// Turn the driver's opaque error into something actionable. The one we
// hit in practice: a card that rejects CMD59 (SPI-mode CRC on/off) after
// CMD0/CMD8 succeed. That's a card-firmware limitation (common on
// large/counterfeit SDXC), not a wiring fault — and we keep CRC required
// rather than run the user's notes over an unchecked bus.
if rc == sys::ESP_ERR_NOT_SUPPORTED {
bail!(
"SD card rejected CMD59 (SPI-mode CRC). CMD0/CMD8 succeeded, so wiring is \
fine — this card's firmware just doesn't support CRC in SPI mode (common on \
large/counterfeit SDXC). Use a genuine card, ideally ≤32 GB. We keep CRC \
required on purpose: a writing device shouldn't run over an unchecked bus."
);
}
esp!(rc).context("esp_vfs_fat_sdspi_mount (card present? inserted? FAT-formatted?)")?;
let storage = Storage { card };
let (max_khz, real_khz) = storage.negotiated_khz();
log::info!("SD mounted at {MOUNT} — max {max_khz} kHz, negotiated {real_khz} kHz");
match storage.recover().context("boot crash recovery")? {
Recovery::Clean => {}
Recovery::DiscardedTmp => log::warn!(
"recovery: found {NOTES_TMP} alongside {NOTES} — last save didn't finish; \
kept the committed file, discarded the incomplete tmp"
),
Recovery::PromotedTmp => log::warn!(
"recovery: found {NOTES_TMP} with no {NOTES} — promoted the tmp (it is the \
newest complete copy)"
),
}
Ok(storage)
}
/// The card's ceiling and negotiated SPI clock, in kHz (`(max, real)`).
/// `real` is what SDSPI settled on after init and is the speed reads/writes
/// actually run at — worth logging on the bench where wiring caps it.
pub fn negotiated_khz(&self) -> (i32, i32) {
// SAFETY: `card` is a live handle for the lifetime of `self` (the mount
// is never torn down while a `Storage` exists).
unsafe {
(
(*self.card).max_freq_khz as i32,
(*self.card).real_freq_khz as i32,
)
}
}
/// Total / free bytes on the FAT volume.
pub fn usage(&self) -> Result<(u64, u64)> {
let mut total: u64 = 0;
let mut free: u64 = 0;
esp!(unsafe { sys::esp_vfs_fat_info(MOUNT_C.as_ptr(), &mut total, &mut free) })
.context("esp_vfs_fat_info")?;
Ok((total, free))
}
/// Whether the working copy exists. A missing `/sd/repo` means the card
/// wasn't provisioned (`just init`); the boot path treats that as fatal.
pub fn repo_present(&self) -> bool {
Path::new(REPO_DIR).is_dir()
}
/// Read `notes.md` into a `String`. Returns an empty string if the file
/// doesn't exist yet (fresh, but provisioned, repo). Refuses a file larger
/// than [`MAX_FILE_BYTES`] rather than loading it.
pub fn load(&self) -> Result<String> {
match fs::metadata(NOTES) {
Ok(m) if m.len() > MAX_FILE_BYTES => bail!(
"{NOTES} is {} KiB — over the {} KiB v0.1 limit; open it on a computer to split it",
m.len() / 1024,
MAX_FILE_BYTES / 1024
),
Ok(_) => fs::read_to_string(NOTES).with_context(|| format!("reading {NOTES}")),
Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(String::new()),
Err(e) => Err(e).with_context(|| format!("stat {NOTES}")),
}
}
/// Atomically persist `contents` to `notes.md`: write the tmp, fsync,
/// unlink the target, rename over it. See the module docs for why the unlink
/// is mandatory on FAT and [`Storage::recover`] for the crash window it opens.
pub fn save(&self, contents: &str) -> Result<()> {
{
let mut f = fs::File::create(NOTES_TMP)
.with_context(|| format!("create {NOTES_TMP} (is {REPO_DIR} present?)"))?;
f.write_all(contents.as_bytes())
.with_context(|| format!("write {NOTES_TMP}"))?;
// FatFS f_sync — flush the tmp fully before it can replace the target.
f.sync_all().with_context(|| format!("fsync {NOTES_TMP}"))?;
}
// FatFS f_rename won't overwrite, so unlink the target first (tolerate a
// missing target: the first-ever save has nothing to remove).
match fs::remove_file(NOTES) {
Ok(()) => {}
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {}
Err(e) => return Err(e).with_context(|| format!("unlink {NOTES} before rename")),
}
fs::rename(NOTES_TMP, NOTES).with_context(|| format!("rename {NOTES_TMP} -> {NOTES}"))?;
Ok(())
}
/// Reconcile a leftover `notes.md.tmp` at boot. The save sequence is
/// write-tmp → fsync → unlink-target → rename, so a lingering tmp means the
/// last save was interrupted. Which way to recover depends on whether the
/// target survived:
///
/// - **tmp + target both present** — the crash could have been *during* the
/// tmp write (before fsync completed), so the tmp may be partial. The
/// target is the last fully-committed version. Keep it, delete the tmp.
/// Promoting a possibly-partial tmp over good data would be data loss.
/// - **tmp only, target absent** — the target was already unlinked, so we
/// crashed between unlink and rename. The tmp is the newest complete,
/// fsync'd copy and the only one left. Promote it (rename over the target).
/// - **neither / target only** — nothing to do.
///
/// Idempotent and safe to call on every mount; a no-op when `/sd/repo`
/// doesn't exist (no tmp can be there).
fn recover(&self) -> Result<Recovery> {
if fs::metadata(NOTES_TMP).is_err() {
return Ok(Recovery::Clean);
}
if fs::metadata(NOTES).is_ok() {
fs::remove_file(NOTES_TMP)
.with_context(|| format!("discard stale {NOTES_TMP}"))?;
Ok(Recovery::DiscardedTmp)
} else {
fs::rename(NOTES_TMP, NOTES)
.with_context(|| format!("promote {NOTES_TMP} -> {NOTES}"))?;
Ok(Recovery::PromotedTmp)
}
}
}