Compare commits
60 Commits
f950abdc4a
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
5f50cf1b45 | ||
|
|
5c97e019ec | ||
|
|
7712de6153 | ||
|
|
1df25f75d6 | ||
|
|
acbafa3d6b | ||
|
|
e801d2d480 | ||
|
|
a771c3f6de | ||
|
|
4c92b0dddc | ||
|
|
af5b41a104 | ||
|
|
2660a3e9dd | ||
|
|
79fad4689c | ||
|
|
98fc817b3f | ||
|
|
d306caacf7 | ||
|
|
2166b932b6 | ||
|
|
02b7ed88b6 | ||
|
|
a5edaed810 | ||
|
|
e86a3b8254 | ||
|
|
9309f3f239 | ||
|
|
1a5e1736f5 | ||
|
|
ce3204a350 | ||
|
|
dd3d70cbec | ||
|
|
2c1c590c9e | ||
|
|
2c24ece3a5 | ||
|
|
da8ca7d8d2 | ||
|
|
dade5e6bb3 | ||
|
|
a6f52df8b6 | ||
|
|
6c5e666f4b | ||
|
|
c5e119f6be | ||
|
|
456c4c43e7 | ||
|
|
beb11eda5e | ||
|
|
88bb2b99cf | ||
|
|
69843086f9 | ||
|
|
45664fc956 | ||
|
|
56f5d18bd8 | ||
|
|
767742ba12 | ||
|
|
85cbeceea9 | ||
|
|
8527f75bf8 | ||
|
|
5a076e3226 | ||
|
|
c969d3e051 | ||
|
|
baf9715570 | ||
|
|
4aaf89d977 | ||
|
|
221135cd9b | ||
|
|
93f3a43b34 | ||
|
|
f37da21462 | ||
|
|
fdd59093db | ||
|
|
55a8d21e0f | ||
|
|
d14d9e77a5 | ||
|
|
d46cdb7e7f | ||
|
|
c535864ee7 | ||
|
|
82f305cea6 | ||
|
|
c9c07165e0 | ||
|
|
e967773bd6 | ||
|
|
2215da939d | ||
|
|
fa0ea56e1a | ||
|
|
470a9d25d0 | ||
|
|
937868dd85 | ||
|
|
884c8f2e48 | ||
|
|
32b9bc6a15 | ||
|
|
f8ef9c821c | ||
| 657aba3cf3 |
22
CONTEXT.md
22
CONTEXT.md
@@ -34,6 +34,25 @@ stay Local for their lifetime. Lives under `/sd/local/`.
|
||||
_Avoid_: draft, private, untracked, scratch (these all imply impermanence or
|
||||
promotability, which is not the model).
|
||||
|
||||
### Editing model
|
||||
|
||||
**Buffer**:
|
||||
A **File** loaded into memory for editing, with its own caret, scroll position,
|
||||
and undo history. Opening a file makes it the **active buffer** — the one the
|
||||
**Writing column** shows. Up to three buffers stay **resident** at once (the
|
||||
active one plus two parked in the background); switching back to a resident
|
||||
buffer restores its caret and undo without re-reading the card. A fourth open
|
||||
**evicts** the least-recently-used resident buffer — saved first if it has
|
||||
unsaved edits, so nothing is lost.
|
||||
_Avoid_: tab, window, document (a buffer is not a UI chrome element); "the file"
|
||||
when you mean the in-memory copy rather than the bytes on the card.
|
||||
|
||||
**Open**:
|
||||
Bringing a **File** into the **active buffer**, via `Cmd-P` (the file palette)
|
||||
or `:e`. Scope is read from where the file lives (`/sd/repo` → **Tracked**,
|
||||
`/sd/local` → **Local**), never chosen at open time.
|
||||
_Avoid_: load (implementation talk for the disk read behind an Open).
|
||||
|
||||
### User-facing actions
|
||||
|
||||
**Save**:
|
||||
@@ -68,7 +87,8 @@ _Avoid_: edit area, text area, main pane (superseded — they named the old
|
||||
full-width text region before the side panel carved out its right edge).
|
||||
|
||||
**Side panel**:
|
||||
The right region (~160 px / ~25 cols, full height) holding all metadata:
|
||||
The right region (~160 px / ~17 cols at its FONT_9X15 metadata font, full
|
||||
height) holding all metadata:
|
||||
filename + dirty dot, word count, elapsed time, clock, Wi-Fi,
|
||||
keyboard-disconnect flag, publish state, and the mode indicator at its
|
||||
bottom-left. Sits entirely in the master half
|
||||
|
||||
@@ -69,7 +69,7 @@ surface (mostly `usb_kbd.rs`) is in [`MEMORY_AUDIT.md`](MEMORY_AUDIT.md).
|
||||
| HAL / runtime | `esp-idf-svc`, `esp-idf-hal` | std build: heap, threads, VFS, mbedtls, Wi-Fi stack. |
|
||||
| Display | Custom SSD1683 driver (`src/epd.rs`) + `embedded-graphics` | Dual-controller 792×272 panel; dirty-rect partial refresh (~630 ms measured). |
|
||||
| UI layer | Custom thin widget layer | Ratatui's API _shape_ without its char-grid terminal model ([ADR-002](docs/adr.md#adr-002-ui-strategy--custom-widgets-on-embedded-graphics-not-ratatui)). |
|
||||
| Editor core | Custom, in-tree (`src/editor.rs`) | Modal (Normal / Insert / View / Command), motions, operators + text objects. Plain-ASCII buffer until the v0.2 UTF-8 work. |
|
||||
| Editor core | Custom, in-tree (`src/editor.rs`) | Modal (Normal / Insert / Visual / VisualLine / View / Command), motions, operators + text objects. Plain-ASCII buffer until the v0.2 UTF-8 work. |
|
||||
| USB host | `esp-idf` TinyUSB bindings | Boot-protocol HID; verified on hardware (Spike 4). |
|
||||
| Git | **libgit2 via `git2`**, built as an esp-idf component with mbedTLS (`firmware/components/libgit2/`) | `gix` was the original pick but can't push over HTTPS — the [ADR-004](docs/adr.md#adr-004-git-implementation--gitoxide-gix) kill-switch fired ([postmortem](docs/postmortems/2026-07-05-spike7-gix-https-push.md)). On-device add → commit → push verified; ~16 s cold-`:sync` [latency breakdown](docs/notes/sync-latency.md). |
|
||||
| TLS | `mbedtls` via `esp-idf` | GitHub HTTPS with the chain checked against embedded roots; ≈35 KB heap measured during handshake (Spike 6). |
|
||||
@@ -106,7 +106,7 @@ source live in [`docs/macroplan.md`](docs/macroplan.md).
|
||||
| [v0.2](docs/macroplan.md#v02--vim-navigation--) | Vim nav | Normal/Insert, motions, line numbers. |
|
||||
| [v0.2.5](docs/macroplan.md#v025--international-input--) | Intl input | US-Intl dead keys: à é ê ç, `'`+space = `'`. |
|
||||
| [v0.3](docs/macroplan.md#v03--vim-editing--) | Vim edit | `dd yy p`, undo/redo, counts. |
|
||||
| [v0.4](docs/macroplan.md#v04--visual-mode--ex-commands--) | Visual + ex | `v V`, `:w :q :e` command line. |
|
||||
| [v0.4](docs/macroplan.md#v04--visual-mode--ex-commands--) | Visual + ex | `v`/`V` select + `y d c`, `gr` read, `:` ex. |
|
||||
| [v0.5](docs/macroplan.md#v05--file-palette--multi-file--) | Files | `Ctrl-P` over `/repo` + `/local`, buffers. |
|
||||
| [v0.6](docs/macroplan.md#v06--markdown-affordances--) | Markdown | Headings, list continuation, soft-wrap. |
|
||||
| [v0.7](docs/macroplan.md#v07--search--better-git--) | Search + git | `/` search, `:Gpull`. |
|
||||
|
||||
341
display/src/glyphs.rs
Normal file
341
display/src/glyphs.rs
Normal file
@@ -0,0 +1,341 @@
|
||||
//! Extra glyphs the ISO-8859-15 render font lacks.
|
||||
//!
|
||||
//! The buffer is UTF-8, so it can hold codepoints outside Latin-9. The
|
||||
//! `embedded-graphics` `iso_8859_15` font draws those as a fallback box. This
|
||||
//! module carries hand-authored 10×20 bitmaps for a curated set of common
|
||||
//! single-width characters — typographic punctuation that arrives in imported
|
||||
//! prose (curly quotes, en/em dashes, ellipsis, bullet) and the math symbols the
|
||||
//! user's Markdown snippets insert (→ ≠ Σ). [`Editor::draw`] overlays these over
|
||||
//! the fallback box after the font has laid out the line.
|
||||
//!
|
||||
//! Every glyph is exactly one 10×20 cell (matching the body `FONT_10X20`, i.e.
|
||||
//! `editor::CW`×`editor::CH`), so the editor's 1-char = 1-cell layout invariant
|
||||
//! is untouched. A row is a 10-bit pattern: the leftmost column (x = 0) is the
|
||||
//! high bit, so a binary literal reads left-to-right as the pixels of that row.
|
||||
//! Rows are top (y = 0) to bottom (y = 19); 1 = ink.
|
||||
|
||||
use crate::Frame;
|
||||
use embedded_graphics::pixelcolor::BinaryColor;
|
||||
use embedded_graphics::prelude::*;
|
||||
|
||||
const GLYPH_W: usize = 10;
|
||||
const GLYPH_H: usize = 20;
|
||||
|
||||
/// A 10×20 1-bit glyph: one `u16` per row, low `GLYPH_W` bits used, high bit =
|
||||
/// leftmost pixel.
|
||||
pub type Glyph = [u16; GLYPH_H];
|
||||
|
||||
// → U+2192 RIGHTWARDS ARROW: a horizontal shaft with a `>` head at the right.
|
||||
#[rustfmt::skip]
|
||||
const ARROW_RIGHT: Glyph = [
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000010000,
|
||||
0b0000001000,
|
||||
0b0000000100,
|
||||
0b0000000010,
|
||||
0b0111111111,
|
||||
0b0111111111,
|
||||
0b0000000010,
|
||||
0b0000000100,
|
||||
0b0000001000,
|
||||
0b0000010000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
];
|
||||
|
||||
// ≠ U+2260 NOT EQUAL TO: two equals bars with a slash through them.
|
||||
#[rustfmt::skip]
|
||||
const NOT_EQUAL: Glyph = [
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000010,
|
||||
0b0000000100,
|
||||
0b0000000100,
|
||||
0b0000001000,
|
||||
0b0111111110,
|
||||
0b0000010000,
|
||||
0b0000010000,
|
||||
0b0000100000,
|
||||
0b0111111110,
|
||||
0b0001000000,
|
||||
0b0001000000,
|
||||
0b0010000000,
|
||||
0b0010000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
];
|
||||
|
||||
// Σ U+03A3 GREEK CAPITAL LETTER SIGMA: top & bottom bars with the diagonals
|
||||
// meeting at a vertex on the centre-right.
|
||||
#[rustfmt::skip]
|
||||
const SIGMA: Glyph = [
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0111111110,
|
||||
0b0100000000,
|
||||
0b0010000000,
|
||||
0b0001000000,
|
||||
0b0000100000,
|
||||
0b0000010000,
|
||||
0b0000100000,
|
||||
0b0001000000,
|
||||
0b0010000000,
|
||||
0b0100000000,
|
||||
0b0100000000,
|
||||
0b0111111110,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
];
|
||||
|
||||
// • U+2022 BULLET: a filled dot at mid-height.
|
||||
#[rustfmt::skip]
|
||||
const BULLET: Glyph = [
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0001111000,
|
||||
0b0011111100,
|
||||
0b0011111100,
|
||||
0b0011111100,
|
||||
0b0001111000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
];
|
||||
|
||||
// … U+2026 HORIZONTAL ELLIPSIS: three dots on the baseline.
|
||||
#[rustfmt::skip]
|
||||
const ELLIPSIS: Glyph = [
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0110110110,
|
||||
0b0110110110,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
];
|
||||
|
||||
// – U+2013 EN DASH: a short mid-height bar.
|
||||
#[rustfmt::skip]
|
||||
const EN_DASH: Glyph = [
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0011111100,
|
||||
0b0011111100,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
];
|
||||
|
||||
// — U+2014 EM DASH: a full-width mid-height bar.
|
||||
#[rustfmt::skip]
|
||||
const EM_DASH: Glyph = [
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b1111111111,
|
||||
0b1111111111,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
];
|
||||
|
||||
// ' U+2018 LEFT SINGLE QUOTATION MARK: raised, tail up (opening).
|
||||
#[rustfmt::skip]
|
||||
const LEFT_SINGLE_QUOTE: Glyph = [
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000100000,
|
||||
0b0001100000,
|
||||
0b0001100000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
];
|
||||
|
||||
// ' U+2019 RIGHT SINGLE QUOTATION MARK / apostrophe: raised, tail down (closing).
|
||||
#[rustfmt::skip]
|
||||
const RIGHT_SINGLE_QUOTE: Glyph = [
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0001100000,
|
||||
0b0001100000,
|
||||
0b0000100000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
];
|
||||
|
||||
// " U+201C LEFT DOUBLE QUOTATION MARK: two opening quotes.
|
||||
#[rustfmt::skip]
|
||||
const LEFT_DOUBLE_QUOTE: Glyph = [
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0010010000,
|
||||
0b0011011000,
|
||||
0b0011011000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
];
|
||||
|
||||
// " U+201D RIGHT DOUBLE QUOTATION MARK: two closing quotes.
|
||||
#[rustfmt::skip]
|
||||
const RIGHT_DOUBLE_QUOTE: Glyph = [
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0011011000,
|
||||
0b0011011000,
|
||||
0b0010010000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
0b0000000000,
|
||||
];
|
||||
|
||||
/// The 10×20 bitmap for a character the ISO-8859-15 font can't draw, or `None`
|
||||
/// if the base font already covers it (all ASCII and Latin-9, including `œ €`).
|
||||
pub fn extra_glyph(c: char) -> Option<&'static Glyph> {
|
||||
Some(match c {
|
||||
'\u{2192}' => &ARROW_RIGHT, // →
|
||||
'\u{2260}' => &NOT_EQUAL, // ≠
|
||||
'\u{03A3}' => &SIGMA, // Σ
|
||||
'\u{2022}' => &BULLET, // •
|
||||
'\u{2026}' => &ELLIPSIS, // …
|
||||
'\u{2013}' => &EN_DASH, // –
|
||||
'\u{2014}' => &EM_DASH, // —
|
||||
'\u{2018}' => &LEFT_SINGLE_QUOTE, // '
|
||||
'\u{2019}' => &RIGHT_SINGLE_QUOTE, // '
|
||||
'\u{201C}' => &LEFT_DOUBLE_QUOTE, // "
|
||||
'\u{201D}' => &RIGHT_DOUBLE_QUOTE, // "
|
||||
_ => return None,
|
||||
})
|
||||
}
|
||||
|
||||
/// Paint a glyph over one 10×20 cell at `(x, y)`. Fills the whole cell — set
|
||||
/// bits in `ink`, unset bits in `ink.invert()` — so it fully overwrites whatever
|
||||
/// the base font drew there (e.g. the fallback box). Pass `ink = On` for a
|
||||
/// normal black-on-white cell, `ink = Off` for a reverse-video cell (a white
|
||||
/// glyph over the black selection/caret fill).
|
||||
pub fn blit_glyph(f: &mut Frame, x: i32, y: i32, g: &Glyph, ink: BinaryColor) {
|
||||
let bg = ink.invert();
|
||||
let pixels = g.iter().enumerate().flat_map(move |(row, &bits)| {
|
||||
(0..GLYPH_W).map(move |c| {
|
||||
let on = (bits >> (GLYPH_W - 1 - c)) & 1 == 1;
|
||||
Pixel(
|
||||
Point::new(x + c as i32, y + row as i32),
|
||||
if on { ink } else { bg },
|
||||
)
|
||||
})
|
||||
});
|
||||
// Frame's DrawTarget error is Infallible.
|
||||
let _ = f.draw_iter(pixels);
|
||||
}
|
||||
@@ -13,6 +13,9 @@ use embedded_graphics::prelude::*;
|
||||
use embedded_graphics::primitives::{Circle, PrimitiveStyleBuilder};
|
||||
use embedded_graphics::text::{Alignment, Baseline, Text, TextStyleBuilder};
|
||||
|
||||
mod glyphs;
|
||||
pub use glyphs::{blit_glyph, extra_glyph, Glyph};
|
||||
|
||||
pub const WIDTH: u16 = 792;
|
||||
pub const HEIGHT: u16 = 272;
|
||||
|
||||
@@ -76,6 +79,16 @@ impl Frame {
|
||||
pub fn bytes(&self) -> &[u8] {
|
||||
&self.buf
|
||||
}
|
||||
|
||||
/// Flip every pixel black↔white across the whole framebuffer. The editor
|
||||
/// draws its native black-ink-on-white-paper frame, then calls this once at
|
||||
/// the end for the dark theme — so text, selection, caret, panel and palette
|
||||
/// all invert together and each stays legible against the flipped ground.
|
||||
pub fn invert(&mut self) {
|
||||
for b in &mut self.buf {
|
||||
*b = !*b;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl OriginDimensions for Frame {
|
||||
|
||||
@@ -15,6 +15,7 @@
|
||||
| [`v0.1-mvp-product.md`](v0.1-mvp-product.md) | v0.1 product design — boot, type one file, `Ctrl-S` to save, `Ctrl-G` to publish. |
|
||||
| [`v0.1-mvp-technical.md`](v0.1-mvp-technical.md) | v0.1 technical design — single Rust binary on `esp-idf-rs`, modules, threads, bring-up order. |
|
||||
| [`macroplan.md`](macroplan.md) | Version-by-version plan; each release is a usable artifact, not a checkpoint. |
|
||||
| [`typoena-toml.md`](typoena-toml.md) | `.typoena.toml` reference — the git-tracked editor preferences (auto-save, format-on-save, line numbers, auto-sync). |
|
||||
| [`hardware.md`](hardware.md) | Part choices for the bench build and the rationale behind them. |
|
||||
|
||||
## Quality method
|
||||
@@ -32,3 +33,4 @@
|
||||
| [`postmortems/`](postmortems/README.md) | Bring-up debugging write-ups: what broke, the root cause, and the decisions that came out of it. |
|
||||
| [`notes/`](notes/README.md) | Longer-form essays on the thinking behind specific choices — e.g. where the ~16 s cold [`:sync`](notes/sync-latency.md) goes. |
|
||||
| [`tradeoff-curves/`](tradeoff-curves/README.md) | Cost-vs-knob curves behind chosen defaults — energy, latency, memory. |
|
||||
| [`kaizen/real-repo-sync.md`](kaizen/real-repo-sync.md) | Six-step kaizen write-ups — the problem→analysis→fix story behind an improvement, e.g. the real-repo `:sync` brick. |
|
||||
|
||||
280
docs/kaizen/real-repo-sync.md
Normal file
280
docs/kaizen/real-repo-sync.md
Normal file
@@ -0,0 +1,280 @@
|
||||
# Kaizen — `:sync` never completes on the real notes repo
|
||||
|
||||
The writer presses `:sync` on the Typoena appliance and the device freezes for
|
||||
ten minutes; a power-cycle re-enters the same freeze forever. The real
|
||||
`jcalixte/notes` clone (1179 files, 263 MB pack) has never been synced from the
|
||||
device — only the toy test repo has. Full measurement trail:
|
||||
[`../tradeoff-curves/sync-commit-staging.md`](../tradeoff-curves/sync-commit-staging.md).
|
||||
|
||||
## 1) Improvement potential
|
||||
|
||||
**Value model for the writer (Julien on the appliance)**
|
||||
|
||||
| Want more of | Do less of |
|
||||
| ----------------------------------------------- | ------------------------------------------------------- |
|
||||
| + words safely on the remote with one keystroke | – waiting on a frozen device |
|
||||
| + trust that `:sync` completes, every time | – power-cycling mid-sync and losing the session's trust |
|
||||
| + keep writing while it publishes | – babysitting git from a desktop to rescue the card |
|
||||
|
||||
**Measurement**: seconds from `:sync` to the snackbar, on the real
|
||||
`jcalixte/notes` clone on the device.
|
||||
|
||||
```mermaid
|
||||
---
|
||||
config:
|
||||
xyChart:
|
||||
width: 900
|
||||
height: 500
|
||||
---
|
||||
xychart-beta
|
||||
title "Improvement potential — :sync → snackbar"
|
||||
x-axis ["Before kaizen (~611 s, never completes)", "After kaizen (target ≤ ~10 s)"]
|
||||
y-axis "seconds" 0 --> 650
|
||||
bar [611]
|
||||
line [10, 10]
|
||||
```
|
||||
|
||||
The before bar understates the reality: a reset re-enters the freeze, so the
|
||||
true value is ∞ — 0 successful syncs ever. The line is the ≤ ~10 s target;
|
||||
|
||||
## 2) Current method analysis
|
||||
|
||||
```
|
||||
:sync (current method, real repo)
|
||||
│
|
||||
▼
|
||||
┌─ local commit ─────────────────────────────────────┐ ┌─ network push ──────┐
|
||||
│ stage: add_all(["*"]) + update_all(["*"]) │ │ TLS + push │
|
||||
│ → stat()s 1179 files / 158 dirs over 10 MHz SPI │──▶│ ~6.5 s floor │
|
||||
│ index.write ⚡ ~611 s │ │ (separate curve) │
|
||||
│ → FAT's 2 s mtime marks ~every entry "racy" │ └─────────────────────┘
|
||||
│ → re-hashes ~170 MB (150 MB of images) │
|
||||
│ write_tree + commit-obj ⚡ seconds each │
|
||||
│ → loose writes trigger pack reads through │
|
||||
│ emulated mmap; each far lseek walks the │
|
||||
│ FAT cluster chain (~190 ms) │
|
||||
└────────────────────────────────────────────────────┘
|
||||
⚡ = weak points, measured on device
|
||||
```
|
||||
|
||||
Every factor was benched on the device (`sd_bench` for raw FAT, `git_bench` for
|
||||
git2 primitives on the real clone) — details and full tables in
|
||||
[`../tradeoff-curves/sync-commit-staging.md`](../tradeoff-curves/sync-commit-staging.md):
|
||||
|
||||
| Factor # | Factor | Hypothesis | Test method | Test result | True or false? |
|
||||
| -------- | --------------------------- | ------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- | -------------------- |
|
||||
| 1 | SD card raw I/O | The card itself is slow: a loose-object write costs ~700–900 ms | `sd_bench`: raw FAT create/write/rename composite, same card, same 10 MHz | complete loose-object composite **86 ms** — 8× under libgit2's number | **FALSE** |
|
||||
| 2 | `index.write()` racy-clean | FAT's 2 s mtime granularity makes ~all 1179 entries look racy → full ~170 MB re-hash over SPI | `git_bench` on the real clone, 3 consecutive `index.write()` | **611 s → 12.8 s → 360 ms** — the decay is the racy set shrinking as the index mtime advances | **TRUE** |
|
||||
| 3 | Index-free alternative | Skipping `index.write` (fresh in-memory index) escapes the wall | `git_bench`: `Index::new` + `read_tree(HEAD)` + `write_tree_to` | seed `read_tree` **77 s cold** + mmap grows to 7.4 MB → **zlib OOM crash** — still O(N_tree) | **FALSE** (as a fix) |
|
||||
| 4 | FatFS `lseek` | Long/backward seeks walk the FAT cluster chain over SPI (~67 KB of FAT reads for a 263 MB file) | `sd_bench`: seek+read 4 KB @offset 0 vs @end of the pack; A/B with `CONFIG_FATFS_USE_FASTSEEK` | @0 = 5.8 ms, @end = **198.7 ms**; fast-seek → **20.4 ms** | **TRUE** |
|
||||
| 5 | Free-cluster scan | A ~740 MB-full card slows FAT allocation | `sd_bench` re-run on the full card | composite 77 ms — unchanged | **FALSE** |
|
||||
| 6 | Strict object creation | OID validation on commit/treebuilder causes the ~1.3 s commit premium | strict-off A/B in `git_bench` | 1.80 s vs 1.93 s — no premium removed | **FALSE** |
|
||||
| 7 | Repeated small mmap windows | A window cache in `esp_map.c` would absorb the ~8 small pack reads per loose write | instrumented cache (v2), 4 real-repo runs | **0 hits / 313 misses** — every read hits a unique (offset, len); `mwindow` already absorbs true repetition | **FALSE** |
|
||||
| 8 | Cache memory discipline | The OOM in factor 3 is our own cache holding buffers past `p_munmap`, defeating `MWINDOW_MAPPED_LIMIT` | run-4 memory monitor after evict-on-munmap fix, then full removal (run 5) | resident flat at 1833 KB, heap ≥ 6.2 MB, no OOM; removal I/O-neutral | **TRUE** |
|
||||
|
||||
### Details on hypothesis #2 (the freeze itself)
|
||||
|
||||
`git_index_write` unconditionally runs `truncate_racily_clean`, which re-hashes
|
||||
every entry whose mtime is not strictly older than the index stamp. On a fresh
|
||||
FAT clone the 2 s mtime granularity makes the whole tree racy, so a one-line
|
||||
note edit re-hashes ~170 MB — mostly the ~260 images a text edit never touches —
|
||||
over the 10 MHz SPI bus. This is why the device freezes ~10 minutes and why a
|
||||
reset never helps: the index mtime doesn't advance, so it re-hashes forever.
|
||||
|
||||
**Selected weak point**: the local commit stage — its staging strategy is
|
||||
O(N_tree) in files and bytes, on a device where N_tree ≈ 1179 and the bus is
|
||||
10 MHz SPI. (The ~6.5 s network half is real but is a separate curve,
|
||||
[`../notes/sync-latency.md`](../notes/sync-latency.md), and it does not brick
|
||||
the device.)
|
||||
|
||||
## 3) Ideas
|
||||
|
||||
### Bibliography
|
||||
|
||||
- [FatFS `f_lseek` docs (elm-chan.org)](http://elm-chan.org/fsw/ff/doc/lseek.html) — the cluster-chain walk and the CLMT fast-seek mechanism.
|
||||
- [ESP-IDF FatFS docs](https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/storage/fatfs.html) — `CONFIG_FATFS_USE_FASTSEEK`, recommended for read-heavy workloads with long backward seeks.
|
||||
- Vendored libgit2 v1.9.4 source — `index.c` `truncate_racily_clean` (the re-hash wall), `odb.c` `git_odb__freshen`/`git_odb_refresh` (the per-write pack probes), `mwindow.c` (32-bit defaults: 32 MB window / 256 MB mapped limit).
|
||||
|
||||
### New ideas
|
||||
|
||||
| Name | Estimated gain | Estimated lead time | Cause addressed | Comments |
|
||||
| ---------------------------------------------------------------------------------------------- | ---------------------------------------------------------- | --------------------------------- | ------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **O(depth) TreeBuilder splice** — patch only the edited path's ancestor chain onto HEAD's tree | commit 611 s → ~2–3 s; **flat in repo size forever** | ~2 days (bench op, then plumbing) | #2 + #3 — never opens the index, never materialises the tree | **Chosen.** Bonus: carries the 150 MB of images forward by OID, so the device doesn't even need them in its working tree |
|
||||
| `CONFIG_FATFS_USE_FASTSEEK` + 256-word CLMT buffer | 2.3× on the splice (6.5 → 2.8 s); far seek 198.7 → 20.4 ms | hours — config, not code | #4 | Shipped as the companion fix; write-mode files fall back transparently |
|
||||
| Explicit-path index staging (`add_path` over the editor's dirty set) | removes the O(N) walk term only | ~1 day | #2 partially | Retired — still calls `index.write()`, so it hits the same racy-clean wall |
|
||||
| Index-free `write_tree_to` on a fresh in-memory index | — | ~1 day | #2 | Refuted by factor 3: seeding `read_tree(HEAD)` is 77 s + OOM — trades one O(N) for another |
|
||||
| `esp_map.c` window cache (v2: size-keyed admission, evict-on-munmap) | hoped ~150–250 ms per loose write | ~1 day | #7 | Built and instrumented → **0 hits in 4 runs** → removed entirely. The one build made on an unverified cause; see learnings |
|
||||
| Faster card / SD at 20 MHz | none for the commit | PCB respin budget | #1 | Rejected — factor 1 refuted; the card was exonerated twice (86 ms then 77 ms composites) |
|
||||
| Shrink the repo (images off-card / LFS-style pointers) | shrinks N and the 570 MB clone | unknown | N itself | Rejected for now — the images are load-bearing for another app ([`../notes/git-sync-images-and-repo-size.md`](../notes/git-sync-images-and-repo-size.md)). Composes with the splice later |
|
||||
|
||||
**Chosen idea**: the O(depth) TreeBuilder splice, with fast-seek as its config
|
||||
companion — it is the only candidate whose cost is independent of repo size, so
|
||||
the problem cannot come back as the notes tree grows. Everything index-shaped
|
||||
stays O(N_tree) and merely moves the wall.
|
||||
|
||||
## 4) Test plan
|
||||
|
||||
**What could go wrong?**
|
||||
|
||||
| Lens | Anticipated consequence | Mitigation |
|
||||
| ------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Stable | libgit2's 32-bit mwindow defaults `malloc` a 32 MB window on first pack access → instant OOM on the 8 MB PSRAM heap | `GIT_OPT_SET_MWINDOW_SIZE` 256 KB / `MAPPED_LIMIT` 4 MB set at service start, before any `Repository::open` |
|
||||
| Stable | Power pull between a save and the next `:sync` loses the dirty set — nothing walks the tree anymore, so an unrecorded file would **never** sync | Dirty set journaled to `/sd/.typoena-dirty` (atomic write), recorded _before_ the file write; over-reporting is a free no-op splice, so the semantics stay simple |
|
||||
| Stable | Commit lands, push fails → the commit strands forever behind "up to date" | Stranded-commit recovery: compare HEAD against `refs/remotes/origin/<branch>` and push even when there is nothing new to commit |
|
||||
| Method | Reconcile's `Mixed` reset writes the index → re-enters the racy-clean wall through the back door | `ResetType::Soft` — ref move only; there is no index to reset anymore |
|
||||
| Method | **Deliberate behavior change:** files edited on the card from a desktop are never committed anymore (`add_all` used to sweep them in) | Accepted and documented as intent — the appliance's editor is the only writer that counts |
|
||||
| Machine | libgit2 holds the pack + `.idx` descriptors open and opens loose objects on top → blows the editor's 4-FD mount | git builds mount with 16 FDs (`Storage::mount_for_git()`) |
|
||||
|
||||
The mechanism was stress-tested as a prototype first, never in production: the
|
||||
splice ran as a `git_bench` op against a full real-repo clone through four
|
||||
localization rounds before any firmware plumbing.
|
||||
|
||||
**Who must we convince?**
|
||||
|
||||
- Upstream libgit2 maintainers — the tlsf double-free in the mbedTLS stream
|
||||
error path (found during rollout; `wrap`'s error path frees the caller's
|
||||
socket stream, `new()` frees it again). Needs an upstream report; we ship a
|
||||
whole-file override meanwhile.
|
||||
- Desktop-me (and any future contributor) — the card's working tree now shows a
|
||||
permanent diff against HEAD when inspected on a Mac. That is intent, not a
|
||||
bug.
|
||||
|
||||
**Rollback**: the `git` feature flag gates the whole sync stack (the light
|
||||
build never links it), and the plumbing is a small commit range to revert. The
|
||||
journal file is additive — an old build simply ignores it.
|
||||
|
||||
**Measurement protocol for step 6**: full `:sync` on the device against the
|
||||
real clone — `commit split —` log lines plus snackbar-to-snackbar wall time,
|
||||
cold and steady-state.
|
||||
|
||||
## 5) Implementation
|
||||
|
||||
**Before** — `stage_and_commit` walked and hashed the world through the index
|
||||
(condensed; `firmware/src/git_sync.rs` at `a5edaed~1`):
|
||||
|
||||
```rust
|
||||
fn stage_and_commit(repo: &Repository) -> Result<Option<Oid>> {
|
||||
let mut index = repo.index()?;
|
||||
index.add_all(["*"], IndexAddOption::DEFAULT, Some(&mut skip_macos_cruft))?;
|
||||
index.update_all(["*"], Some(&mut skip_macos_cruft))?; // stat 1179 files
|
||||
index.write()?; // ⚡ racy-clean re-hash: ~611 s on FAT
|
||||
let tree = repo.find_tree(index.write_tree()?)?;
|
||||
// … commit(Some("HEAD"), …, &tree, &parents)
|
||||
}
|
||||
```
|
||||
|
||||
**After** — the editor's journaled dirty paths are spliced onto HEAD's tree;
|
||||
the index never exists (condensed; `firmware/src/git_sync.rs:345`):
|
||||
|
||||
```rust
|
||||
fn stage_and_commit(repo: &Repository, paths: &BTreeSet<String>) -> Result<Option<Oid>> {
|
||||
let parent = repo.head().ok().and_then(|h| h.peel_to_commit().ok());
|
||||
let mut tree = parent.as_ref().map(|c| c.tree()).transpose()?;
|
||||
for path in paths {
|
||||
let blob = match fs::read(format!("{REPO_DIR}/{path}")) {
|
||||
Ok(bytes) => Some(repo.blob(&bytes)?), // present → splice in
|
||||
Err(e) if e.kind() == NotFound => None, // deleted → splice out
|
||||
Err(e) => return Err(e.into()),
|
||||
};
|
||||
let spliced = splice(repo, tree.as_ref(), &parts(path), blob)?;
|
||||
tree = Some(repo.find_tree(spliced)?);
|
||||
}
|
||||
// … same "tree unchanged → nothing to publish" check, same commit call
|
||||
}
|
||||
|
||||
/// Reads ~depth tree objects, writes ~depth new ones; every sibling entry is
|
||||
/// carried by OID — never opened.
|
||||
fn splice(repo: &Repository, base: Option<&Tree>, path: &[&str], blob: Option<Oid>) -> Result<Oid>
|
||||
```
|
||||
|
||||
The same pipeline, redrawn with the change applied:
|
||||
|
||||
```
|
||||
:sync (new method, real repo)
|
||||
│
|
||||
▼
|
||||
┌─ local commit ───────────────────────────────┐ ┌─ network push ──────┐
|
||||
│ splice: for each journaled dirty path (≈1) │ │ TLS + push │
|
||||
│ read file → blob → rebuild its ancestor │──▶│ ~6.5 s floor │
|
||||
│ subtree chain (~depth tree objects) │ │ (untouched — next │
|
||||
│ commit-obj │ │ curve to attack) │
|
||||
│ no index · no walk · no re-hash · images │ └─────────────────────┘
|
||||
│ carried by OID · lseek O(1) via fast-seek │
|
||||
└──────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
Around the mechanism, the plumbing that makes it safe day-to-day: the
|
||||
`/sd/.typoena-dirty` journal with its pending → in-flight → settled lifecycle,
|
||||
stranded-commit recovery, a radio-free "up to date" answer (~150 ms instead of
|
||||
a Wi-Fi/TLS round), soft-reset reconcile, the 16-FD git mount, and the mwindow
|
||||
options at service start. Details:
|
||||
[the fix — wiring](../tradeoff-curves/sync-commit-staging.md#the-fix--wiring-the-odepth-splice-into-the-firmware).
|
||||
|
||||
## 6) Evaluation
|
||||
|
||||
**Measurement redone** (seconds, `:sync` → snackbar on the real repo):
|
||||
∞ (never completed) → **PENDING end-to-end** (target was ≤ ~10 s).
|
||||
|
||||
What is measured so far, on device against the real clone (2026-07-13):
|
||||
|
||||
- The commit half works for the first time ever: splice **4.2 s** + commit-obj
|
||||
**3.2 s** with the UI running concurrently (commits `a73bca0e`, then
|
||||
`8939168f` carrying a 2-path journal across a power cycle).
|
||||
- Projected full cold `:sync` ≈ **9–10 s** (commit + the ~6.5 s network half).
|
||||
- The push half is not yet verified: the first attempt hit the card's
|
||||
SSH-shaped origin (now rewritten to HTTPS at load), the second a tlsf
|
||||
double-free in libgit2's mbedTLS stream error path (fixed via the
|
||||
`esp_mbedtls_stream.c` override + `CONFIG_MBEDTLS_EXTERNAL_MEM_ALLOC=y`).
|
||||
The step-1 number lands when the next on-device `:sync` completes
|
||||
snackbar-to-snackbar — protocol in step 4.
|
||||
|
||||
**Learnings**:
|
||||
|
||||
- **Bench the real thing.** The toy repo understated everything by ~2 orders
|
||||
of magnitude; "works on the toy" hid a total brick. The real clone is now
|
||||
the only valid bench target.
|
||||
- **Refute before building.** Four theories died to measurement (card speed,
|
||||
free-cluster scan, strict creation, repeated windows). The one build made on
|
||||
an unverified cause — the `esp_map.c` window cache — was the one wasted
|
||||
build: 0 hits in 4 instrumented runs, removed entirely.
|
||||
- **FAT breaks POSIX assumptions.** 2 s mtime granularity, no inode,
|
||||
cluster-chain seeks: any POSIX-shaped library (libgit2's racy-clean check,
|
||||
the mmap emulation) needs its cost model audited on FAT before trusting it.
|
||||
- **Prefer mechanisms flat in N.** The splice cannot regress as the notes tree
|
||||
grows — the fix that prevents the problem from ever coming back, which is
|
||||
the kind kaizen prefers.
|
||||
|
||||
**Standard to update**:
|
||||
|
||||
- Bench and verify against the **real repo clone**, never the toy (the toy is
|
||||
~2 orders of magnitude too kind) — recorded in the tradeoff doc's
|
||||
[how to bench](../tradeoff-curves/sync-commit-staging.md#how-to-bench--flash).
|
||||
- Any new libgit2 entry point **must set the mwindow options before its first
|
||||
`Repository::open`** — the 32-bit defaults OOM the 8 MB heap (done in
|
||||
`git_sync` and `git_bench`; the rule is the standard).
|
||||
- `CONFIG_FATFS_USE_FASTSEEK=y` + 256-word CLMT buffer stay pinned in
|
||||
`sdkconfig.defaults`.
|
||||
|
||||
**Share with**:
|
||||
|
||||
- Upstream libgit2 — file the tlsf double-free report (mbedTLS stream error
|
||||
path: `wrap`'s error path frees the caller's socket stream, `new()` frees it
|
||||
again; v1.9.4).
|
||||
- A write-up for the notes/blog — the four-round localization story (611 s →
|
||||
~2.8 s commit on a microcontroller, four theories refuted on the way) stands
|
||||
on its own.
|
||||
|
||||
**Next steps**
|
||||
|
||||
- Run the end-to-end on-device `:sync` verify and close the step-1
|
||||
measurement (the stranded `8939168f` should push first).
|
||||
- File the libgit2 upstream bug report.
|
||||
- Instrument the residual ~360 ms/loose-write (suspect: FAT directory-op cost
|
||||
in the freshen/refresh path) — one `sd_bench` + `p_mmap`-miss logging pass.
|
||||
- Measure the ref/reflog leg of the shipping commit (the bench's
|
||||
`commit(None)` writes no ref).
|
||||
- Attack the next curve: the ~6.5 s network floor
|
||||
([`../notes/sync-latency.md`](../notes/sync-latency.md)); the
|
||||
images-off-card lever
|
||||
([`../notes/git-sync-images-and-repo-size.md`](../notes/git-sync-images-and-repo-size.md))
|
||||
composes with the splice.
|
||||
@@ -1,7 +1,8 @@
|
||||
# Macroplan — version details
|
||||
|
||||
Frequent releases. Each version is a usable artifact, not a checkpoint.
|
||||
This file holds the `macroplan` source block (below) and the per-version scope.
|
||||
This file holds the `macroplan` source block (below), a cross-version status
|
||||
roll-up, and a one-line summary of each release linking to its dedicated page.
|
||||
The user-facing requirements and engineering targets each release feeds into are
|
||||
tracked in [`qfd.md`](qfd.md).
|
||||
|
||||
@@ -40,27 +41,28 @@ name = "v0.3 editing"
|
||||
start = 2026-08-03
|
||||
original = 2026-08-24
|
||||
delivered = 2026-07-11
|
||||
learning = "Core complete 44 days early, host-tested. Register + yank/paste (yy/p/P), snapshot undo/redo (u/Ctrl-r, bounded 100 groups in PSRAM), and keystroke-recorded `.` repeat all landed 2026-07-11; the d/c operator grammar + text objects were already done ahead of schedule. Firmware version bumped to 0.3.0. On-device smoke-test still pending — pure editor-core, no new render surface."
|
||||
learning = "Core complete 44 days early, host-tested and partially smoke-tested on the panel. Register + yank/paste (yy/p/P), snapshot undo/redo (u/Ctrl-r, bounded 100 groups in PSRAM), and keystroke-recorded `.` repeat all landed 2026-07-11; the d/c operator grammar + text objects were already done ahead of schedule. Firmware bumped to 0.3.0. On device dd/yy/Ctrl-r confirmed; the one bug found was a multi-line paste leaving its later lines below the fold (adjust_scroll only tracked the caret) — fixed with a reveal() that scrolls the block end into view."
|
||||
|
||||
[[feature]]
|
||||
name = "v0.4 visual + ex"
|
||||
start = 2026-08-24
|
||||
original = 2026-09-07
|
||||
status = "on-track"
|
||||
note = ": command-line mechanism and :fmt done early; Visual mode not started."
|
||||
delivered = 2026-07-11
|
||||
learning = "Core complete 58 days early, host-tested. Visual (v) and VisualLine (V) selection with y/d/c landed 2026-07-11 (charwise vim-inclusive of the char under the caret; linewise spans whole lines and pastes like yy/dd), plus the recorded v/V→Visual reassignment: the read-only View mode moved to `gr` (go-read). Selection is drawn as reverse-video cells on the 1-bit panel with the caret punched back to normal video so the active end stands out; 18 new editor tests (83 total). The `:` command mechanism and :fmt were already done; `:e <path>` was deliberately deferred to v0.5 where its multi-file/buffer-lifecycle machinery (Spikes 11/14) lives, rather than half-building file-open here. Firmware bumped to 0.4.0. On-device smoke-test of Visual still pending (pure editor-core, low risk)."
|
||||
|
||||
[[feature]]
|
||||
name = "v0.5 palette + multi-file"
|
||||
start = 2026-09-07
|
||||
original = 2026-09-28
|
||||
note = "Also adds the git-tracked .typoena.toml preferences file (save_on_idle, format_on_save, auto_sync cadence, line_numbers) and the palette `>` command mode that edits it live."
|
||||
delivered = 2026-07-12
|
||||
learning = "Delivered 2026-07-12, well ahead of the 2026-09-28 baseline, and fully on-device confirmed. Four slices: the drained Effect queue + parked-buffer LRU foundation; the Cmd-P fuzzy file palette (Spike 11 — no ghosting on the transient panel); :enew + file delete (Spike 14 caught that add_all alone doesn't stage a deletion on this libgit2 — fixed with update_all, i.e. git add -A); and the git-tracked .typoena.toml prefs with a stay-open palette `>` command mode + :settings. Both directions of the prefs loop are proven on hardware — boot-read (byte-exact parse) and on-device palette edit (a device publish flipped line_numbers on origin). Three decide-before-build calls: the idle auto-save is unformatted, and both the per-device auto_sync override and the `> auto sync` command are deferred to v0.7 where auto_sync gains behaviour. Amended 2026-07-12: a light/dark `theme` key and a set-ahead `> auto sync` preset command (2m/5m/10m/15m/30m) were added on top — the palette generalised so Enter rotates any pref to its next value (a bool is the two-option case); auto_sync is still read by nothing until v0.7. Descoped from v0.5 (not the four slices): explicit buffer close, the grey-Publish-in-Local panel cue, and the multi-file publish count."
|
||||
|
||||
[[feature]]
|
||||
name = "v0.6 markdown"
|
||||
start = 2026-09-28
|
||||
original = 2026-10-12
|
||||
status = "on-track"
|
||||
note = "Render affordances done early; 80-col ruler + snippet engine (added 2026-07-08) remain."
|
||||
delivered = 2026-07-12
|
||||
learning = "Core complete 2026-07-12, ~92 days ahead of the 2026-10-12 baseline, host-tested (187 editor tests). The snippet feature was reshaped 2026-07-08→07-12 from a hard-coded table into a git-synced, Zed-compatible .typoena.snippets.json library: a forward-only tab-stop session ($1..$n/$0, ${n:label} stripped to $n) driven by two surfaces — inline Tab-expansion in Insert and a $ palette launcher — plus a quiet pause hint in the side panel. The Cmd-P palette generalised into a verb split: bare = files, > = a real command registry (toggles stay open, one-shots format/publish close, the parameterised `new file` two-step), $ = snippets — retiring :e. Firmware bumped 0.5.0→0.6.0; the boot-read of the library was confirmed to build for xtensa (serde_json, the one new dep — cargo check passes). `just init` now seeds a curated 17-snippet catalog (three opt-in groups). On-device smoke-test still pending (pure editor-core + a mirror of the proven prefs boot-read, low risk). Known caveat: two symbols the catalog inserts (arrow →, neq ≠) are outside ISO-8859-15, so they store/sync correctly but need a display-layer glyph overlay (in flight) to draw on the panel; the other 15 render on the stock font."
|
||||
|
||||
[[feature]]
|
||||
name = "v0.7 search + git"
|
||||
@@ -88,7 +90,7 @@ week = 2026-06-29
|
||||
requires = ["v0.1 it writes, it pushes"]
|
||||
```
|
||||
|
||||
## Status — synced 2026-07-11
|
||||
## Status — synced 2026-07-12
|
||||
|
||||
The editor **core** has been built 2–3 versions ahead of the device
|
||||
**releases**, and is now **extracted into a host-testable `editor` crate** (plus
|
||||
@@ -108,333 +110,119 @@ recovery, 1000-word no-drop, and `Ctrl-G`'s not-yet-built pull-then-retry
|
||||
refresh check passed (single-line edit repaints only rows at/below it, no extra
|
||||
full refresh), closing the last gate. **v0.2.5 international input** is
|
||||
hardware-verified (2026-07-11), and **v0.3 editing is complete in core** the same
|
||||
day (register + yank/paste, snapshot undo/redo, `.` repeat — host-tested,
|
||||
on-device smoke-test pending); the firmware crate is bumped to **0.3.0**. Most of
|
||||
v0.6 Markdown also already runs. Version numbers track shippable device releases,
|
||||
not raw core progress — the 0.3.0 bump reflects the v0.3 feature set being met.
|
||||
day (register + yank/paste, snapshot undo/redo, `.` repeat — host-tested, and
|
||||
partially smoke-tested on the panel: `dd`/`yy`/`Ctrl-r` good, a multi-line-paste
|
||||
scroll bug found + fixed). **v0.4 visual + ex is complete in core** the same day
|
||||
too — charwise/linewise **Visual** selection (`v`/`V` with `y`/`d`/`c`), the
|
||||
read-only View mode moved to `gr`, and the selection drawn as reverse-video on
|
||||
the panel; `:e` was deferred to v0.5. Host-tested (83 editor tests); on-device
|
||||
smoke-test pending. The firmware crate is bumped to **0.4.0**. Most of v0.6
|
||||
Markdown also already runs. Version numbers track shippable device releases, not
|
||||
raw core progress — the 0.4.0 bump reflects the v0.4 feature set being met.
|
||||
**v0.5 palette + multi-file is DELIVERED 2026-07-12** (firmware **0.5.0**), fully
|
||||
on-device confirmed: the Cmd-P fuzzy palette, `:e`/`:enew`/delete across the
|
||||
`/sd/repo` + `/sd/local` scopes, and the git-tracked `.typoena.toml` prefs
|
||||
(boot-read plus a stay-open palette `>` command mode + `:settings` that edits them
|
||||
live and syncs the change). Descoped to later: explicit buffer close, the
|
||||
grey-Publish-in-Local panel cue, and the multi-file publish count.
|
||||
**v0.6 Markdown is COMPLETE in core 2026-07-12** (firmware **0.6.0**), host-tested
|
||||
(187 editor tests), on-device smoke-test pending. The render affordances (heading
|
||||
bold, list continuation, soft-wrap) were done early; the headline is the
|
||||
**snippet engine** — a forward-only tab-stop session reached both inline (type a
|
||||
prefix + Tab in Insert) and from a **`$` palette** launcher, fed by a git-synced,
|
||||
Zed-compatible `.typoena.snippets.json` read at boot (serde_json, confirmed to
|
||||
build for xtensa). The `Cmd-P` palette **generalised** into a verb split — bare =
|
||||
files, `>` = a command registry (toggles stay open, `format`/`publish` one-shots
|
||||
close, a two-step `new file`), `$` = snippets — which **retired `:e`**. `just init`
|
||||
seeds a curated 17-snippet catalog (Symbols · Structure · Prose, opt-in). One
|
||||
caveat: `→`/`≠` sit outside ISO-8859-15 and need a display-layer glyph overlay
|
||||
(in flight) to render on the panel; the other 15 draw on the stock font.
|
||||
|
||||
Marks: `[x]` done in core · `[~]` partially done · `[ ]` not started. An
|
||||
inline `(✓)` marks the done half of a split item.
|
||||
|
||||
Each version below links to its dedicated page, which carries the full scope
|
||||
checklist and status.
|
||||
|
||||
---
|
||||
|
||||
## v0.1 — MVP: "it writes, it pushes" — [x]
|
||||
|
||||
The minimum thing that justifies the hardware existing. Full design:
|
||||
[product](v0.1-mvp-product.md) · [technical](v0.1-mvp-technical.md).
|
||||
|
||||
**Status:** SHIPPED 2026-07-11 (late vs the 2026-06-29 baseline). Core editing +
|
||||
partial refresh run on device; **SD mount + save are wired into `main.rs`**
|
||||
(Spike 3 resolved — a genuine ≤32 GB card mounts, verified on its own SPI3 host
|
||||
per ADR-012); **git publish is wired** (`:sync` → commit + fast-forward push on
|
||||
the SD `/sd/repo`, hardware-verified against a test repo); and the **boot splash
|
||||
(Spike 9) is confirmed on the panel** — [`Frame::splash`](../display/src/lib.rs)
|
||||
shows a vector `typoena`-in-a-circle at startup while the SD mounts, then the
|
||||
editor comes up. Cold boot **verified at 4258 ms** (power-on → cursor, 2026-07-11; 742 ms under
|
||||
the ≤ 5 s gate). It first measured ~5.5 s; the fix was to bring the editor up
|
||||
with a full-area partial (~630 ms) instead of a second full refresh (~1.9 s) —
|
||||
panel confirmed clean. The 1-hour soak is attested from real use; the remaining
|
||||
post-ship acceptance checks are power-pull recovery, 1000-word no-drop, and
|
||||
`Ctrl-G` pull-then-retry (→ v0.9) — see
|
||||
[product → acceptance](v0.1-mvp-product.md#acceptance-criteria).
|
||||
|
||||
- [x] ESP32-S3 boots (✓); e-ink shows Typoena splash (✓ Spike 9, confirmed on
|
||||
panel 2026-07-11); boot status surfaces via the panel snackbar (no serial on device)
|
||||
- [x] USB host enumerates the Nuphy, key events reach the editor (Spike 4)
|
||||
- [x] One hard-coded file (`/sd/repo/notes.md`) opens on boot — **wired in
|
||||
`main.rs`** (`boot_storage` mounts the SD and loads the note; a missing
|
||||
card / repo / unreadable note halts with a panel message). The card is
|
||||
pre-seeded from a computer (`just init` copies a full clone to `/sd/repo` +
|
||||
writes config), never cold-cloned on device — see
|
||||
[note](notes/git-sync-images-and-repo-size.md).
|
||||
- [x] Insert-only editing, backspace, enter, arrow keys — modal editor overshot this early (see v0.2)
|
||||
- [x] Line wrap, no line numbers yet — soft-wrap done early (see v0.6)
|
||||
- [x] Save to SD via `:w` (and `:sync`) — **wired in `main.rs`** through the
|
||||
`persistence` module's atomic write (unlink-then-rename + `*.tmp`
|
||||
boot-recovery)
|
||||
- [~] Wi-Fi credentials + remote URL + PAT + author: today baked into the binary
|
||||
via `env!()` (no NVS, no on-device provisioning UI in v0.1). Migrating to
|
||||
`/sd/typoena.conf` on the card, provisioned by `just provision` (or
|
||||
`just init` for a fresh card) from the same `firmware/.env` the build uses
|
||||
(minimum input — rotate the PAT or switch networks without a reflash, no
|
||||
card re-copy). Firmware to read it at boot instead of
|
||||
`env!()` — the git-publish wiring landed with baked config (2026-07-11);
|
||||
the `typoena.conf` migration itself is deferred to v0.9 (on-device
|
||||
provisioning).
|
||||
- [x] Publish on **`:sync`** (the editor's command; originally planned as
|
||||
`Ctrl-G`): format (`:fmt`, when `format_on_save`) → save → stage `notes.md`
|
||||
→ commit with a timestamp message →
|
||||
fast-forward `push`; on a rejected push, fetch + reconcile then retry once
|
||||
(no-op short-circuit when the tree is unchanged). **Wired into the editor and
|
||||
hardware-verified 2026-07-11** — `firmware::git_sync` opens the SD `/sd/repo`,
|
||||
runs on a dedicated 96 KB git thread with lazy Wi-Fi, and pushes over mbedTLS
|
||||
HTTPS+PAT; the panel snackbar shows `synced <oid>` / `up to date` /
|
||||
`sync failed`. (Interrupted-push auto-retry deferred to v0.9.)
|
||||
- [x] Split the display into a **writing column** (60 cols) + a **side panel**
|
||||
(~30 cols at FONT_6X10) for metadata — the surface every later panel
|
||||
feature writes to. **Built** in the `editor` crate (`draw_panel`): a
|
||||
full-height divider at x=600, with the panel currently showing the word
|
||||
count, the mode indicator, a NO-KBD flag, and a transient save/publish
|
||||
**snackbar** (below). Later fields (filename, clock, Wi-Fi, battery) add to
|
||||
the same surface. Defined in
|
||||
[`CONTEXT.md` § Screen regions](../CONTEXT.md#screen-regions) and
|
||||
[product § Screen layout](v0.1-mvp-product.md#screen-layout).
|
||||
- [x] **Snackbar** — a transient side-panel notice for host events (added
|
||||
2026-07-11). On-device there is no serial log, so boot posts `loaded
|
||||
<name>` (the note's filename without suffix), `:w` posts `saved` /
|
||||
`save FAILED - retry :w`, and `:sync` posts `syncing...` then the push
|
||||
result (`synced <oid>` / `up to date` / `sync failed`). Set via
|
||||
`Editor::set_notice`; cleared on the next keystroke
|
||||
rather than a timer — a timed auto-dismiss would cost a ~630 ms full-area
|
||||
e-ink flash purely to erase text, which the panel deliberately avoids (cf.
|
||||
the dropped pending-accent marker in v0.2.5).
|
||||
- [x] Partial refresh on edits (✓ Spike 5); save wired (full-area partial
|
||||
repaint on `:w`)
|
||||
|
||||
Out of scope: Vim, palette, multiple files, branches, conflict handling.
|
||||
The minimum thing that justifies the hardware existing — boot, type one file,
|
||||
`:w` to save, `:sync` to publish to GitHub. **SHIPPED 2026-07-11** (late vs the
|
||||
2026-06-29 baseline); cold boot verified at 4258 ms.
|
||||
**Design:** [product](v0.1-mvp-product.md) · [technical](v0.1-mvp-technical.md).
|
||||
|
||||
## v0.2 — Vim navigation — [x]
|
||||
|
||||
**Status:** COMPLETE 2026-07-11. Navigation done in core; the **UTF-8-correct
|
||||
buffer** and **`Ctrl-d/u` half-page scroll** landed and are hardware-verified,
|
||||
and the **absolute line-number gutter** is built, host-tested, and **confirmed
|
||||
on the panel (Spike 13) 2026-07-11** — a single-line edit repaints only the rows
|
||||
at/below the change and forces no extra full refresh. Shipped early beyond scope:
|
||||
a read-only **View** mode and the full `d`/`c` operator + text-object grammar
|
||||
(see v0.3 / v0.4).
|
||||
|
||||
- [x] Mode state machine (Normal / Insert / View), mode indicator in the status strip
|
||||
- [x] Movement: `h j k l`, `w b e`, `0 $`, `gg G`, `Ctrl-d Ctrl-u`. `Ctrl-d/u`
|
||||
step **display** (soft-wrapped) rows, not logical lines — half a page is
|
||||
half the visible window however prose wraps; decoded as `HalfPageDown/Up`
|
||||
intents in the keymap, caret moves and the viewport follows.
|
||||
- [x] `i a o O A` to enter Insert
|
||||
- [x] `Esc` returns to Normal
|
||||
- [x] Line numbers in the left gutter: **absolute**, built + host-tested
|
||||
2026-07-11, **confirmed on the panel (Spike 13) 2026-07-11** — numbered on a
|
||||
logical line's first display row, blank on wrapped continuation rows; the
|
||||
gutter width tracks the buffer's line count (2 digits + separator, widening
|
||||
past 99 lines) and steals its columns from the soft-wrap. **Always on** in
|
||||
v0.2; the on/off toggle rides the v0.5 `.typoena.toml` prefs (below).
|
||||
Relative numbering was dropped (2026-07-11): renumbering the whole gutter on
|
||||
every `j`/`k` burns the e-ink ghosting budget for no proportionate gain,
|
||||
whereas absolute renumbers only the rows below an edit — the on-panel check
|
||||
confirmed a single-line edit repaints only rows at/below it with no extra
|
||||
full refresh.
|
||||
- [x] Groundwork — UTF-8-correct buffer: caret motions and edits step by
|
||||
character, not byte (dropped the ASCII == byte-offset assumption), so every
|
||||
motion stays correct with accented input. **Done 2026-07-11** alongside
|
||||
extracting the editor into a host-testable crate — char-step
|
||||
motions/deletes, byte-vs-char split in `layout`/`caret_rc`, `word_end`/`de`
|
||||
fixed; 15 host tests. Render font is ISO-8859-15 (Latin-9), so accented
|
||||
glyphs display.
|
||||
Modal Normal/Insert/View, `h j k l`/`w b e`/`0 $`/`gg G` motions, `Ctrl-d/u`
|
||||
half-page scroll, the UTF-8-correct buffer, and the absolute line-number gutter.
|
||||
**COMPLETE 2026-07-11.** Detail: [v0.2-navigation.md](v0.2-navigation.md).
|
||||
|
||||
## v0.2.5 — International input — [x]
|
||||
|
||||
**Status:** DONE in core, **hardware-verified 2026-07-11** (typed ç é è ñ on the
|
||||
bench, no crash). US-International dead-key accent composition lives in the
|
||||
`keymap` crate — a `Composer` downstream of the decoder — wired into
|
||||
`usb_kbd.rs` so the editor still receives a single `Key::Char`. Builds on the
|
||||
v0.2 UTF-8-correct buffer and the ISO-8859-15 render font. Host-tested.
|
||||
|
||||
- [x] Dead keys — grave, acute, circumflex, diaeresis, tilde — compose with
|
||||
the next letter: à é ê ë ñ, ç (via `'`+c), both cases
|
||||
- [x] `'`+space emits a literal apostrophe (the everyday apostrophe path); a
|
||||
dead key followed by a non-composing letter emits the accent then the
|
||||
letter
|
||||
- [x] A non-character event (Enter, Backspace, arrows) flushes any pending
|
||||
accent as its literal first
|
||||
- [ ] ~~Pending-accent indicator in the side-panel status strip~~ — **DROPPED
|
||||
(2026-07-11 decision):** at typing speed it would be stale before the
|
||||
~630 ms panel repaint, so it conveys nothing. Left unbuilt on purpose.
|
||||
- [x] Bonus (2026-07-11): the physical **Esc key** (HID 0x29) now types
|
||||
`` ` ``/`~` — Esc comes from the Caps tap — so grave/tilde accents and
|
||||
Markdown code fences are reachable on a 60% board without a Fn layer.
|
||||
US-International dead-key accent composition (à é ê ë ñ ç) in the `keymap`
|
||||
crate, plus the Esc→backtick/tilde remap for a 60% board.
|
||||
**Hardware-verified 2026-07-11.**
|
||||
Detail: [v0.2.5-international-input.md](v0.2.5-international-input.md).
|
||||
|
||||
## v0.3 — Vim editing — [x]
|
||||
|
||||
**Status:** COMPLETE in core 2026-07-11, host-tested (64 editor + 28 keymap
|
||||
tests). The three remaining pieces landed together: a single unnamed **register**
|
||||
with `y`/`yy`/`p`/`P` (and `x`/`d`/`c` filling it, so `dd`…`p` moves a line),
|
||||
**undo/redo** (`u`/`Ctrl-r`, snapshot-based, bounded to 100 groups in PSRAM — a
|
||||
whole Insert session undoes as one group), and **`.` repeat** (keystroke-recorded,
|
||||
so it replays an insert session like `ciwfoo<Esc>`). The `d`/`c` operator grammar
|
||||
and text objects had already landed ahead of schedule. Pure editor-core work,
|
||||
riding the already-verified draw/refresh path; **on-device smoke-test pending**
|
||||
(nothing new to render, so low risk).
|
||||
Register + yank/paste (`yy`/`p`/`P`), snapshot undo/redo (`u`/`Ctrl-r`), `.`
|
||||
repeat, and the `d`/`c` operator grammar + text objects.
|
||||
**COMPLETE in core 2026-07-11**, partially smoke-tested on the panel.
|
||||
Detail: [v0.3-editing.md](v0.3-editing.md).
|
||||
|
||||
- [x] `x dd`, `dw dd d$` (✓); `yy p P` (✓) and `.` repeat (✓) — register + a
|
||||
keystroke-recorded last-change both landed 2026-07-11
|
||||
- [x] Undo / redo (`u`, `Ctrl-r`) — snapshot history bounded to 100 groups in
|
||||
PSRAM; one Insert session = one undo group
|
||||
- [x] Numeric prefixes (`3dd`, `5j`)
|
||||
- [x] Ahead of schedule: `c` change operator + text objects
|
||||
(`ciw`, `di(`, `ca"`, … — inner/around, nesting-aware)
|
||||
## v0.4 — Visual mode + ex commands — [x]
|
||||
|
||||
Known limits (deferred): `.` drops a *leading* count (`3x` then `.` deletes one;
|
||||
a count inside an operator like `d2w` is kept); no named registers; `.` after an
|
||||
aborted operator (`d<Esc>`) is a no-op.
|
||||
Charwise `v` / linewise `V` selection with `y`/`d`/`c`, the `:` command line
|
||||
(`:w`/`:fmt`/`:sync`/`:gl`), and View mode moved to `gr`.
|
||||
**COMPLETE in core 2026-07-11**, on-device smoke-test pending.
|
||||
Detail: [v0.4-visual-and-ex.md](v0.4-visual-and-ex.md).
|
||||
|
||||
## v0.4 — Visual mode + ex commands — [~]
|
||||
## v0.5 — File palette + multi-file — [x]
|
||||
|
||||
**Status:** the `:` command-line mechanism is built (Command mode + status-strip
|
||||
echo), but only `:fmt` exists — `:w :q :wq :e` remain. Visual mode is not
|
||||
started.
|
||||
The `Cmd-P` fuzzy file palette, `:e`/`:enew`/delete across `/sd/repo` +
|
||||
`/sd/local`, the parked-buffer LRU, and the git-tracked `.typoena.toml` prefs
|
||||
with a palette `>` command mode + `:settings`.
|
||||
**DELIVERED 2026-07-12** (firmware 0.5.0), fully on-device confirmed.
|
||||
Detail: [v0.5-palette-and-multi-file.md](v0.5-palette-and-multi-file.md).
|
||||
|
||||
**DECISION (2026-07-07):** `v`/`V` = **Visual** selection (vim-standard). The
|
||||
read-only **View** (reading/scroll) mode currently bound to `v`/`V` moves off
|
||||
those keys and gets its own trigger (exact key TBD when Visual lands). View mode
|
||||
stays — it just frees `v`/`V` for Visual.
|
||||
## v0.6 — Markdown affordances — [x]
|
||||
|
||||
- [ ] Visual char (`v`) and line (`V`) modes, `y d c` on selections
|
||||
- [~] `:` command line (mechanism ✓; `:w`/`:wq`/`:x` save, `:fmt`/`:sync`/`:gl`
|
||||
wired; `:e <path>` remains, `:q` deliberately dropped — nothing to quit
|
||||
to). Command-line editing added 2026-07-11: Ctrl-W deletes the previous
|
||||
word, Cmd-Backspace clears the line.
|
||||
- [x] Ahead of schedule / unscheduled: `:fmt` Markdown formatter
|
||||
(table alignment, blank-line collapse, trailing-whitespace strip)
|
||||
|
||||
## v0.5 — File palette + multi-file — [ ]
|
||||
|
||||
**Status:** not started (Spikes 11 + 14 retire the panel-mechanism and
|
||||
buffer-lifecycle risk first).
|
||||
|
||||
- [ ] `Ctrl-P` opens fuzzy file palette over **both** `/sd/repo/` and
|
||||
`/sd/local/`, with a scope marker (e.g. `[git]` / `[local]`) per result
|
||||
- [ ] Open, switch, close buffers (keep ≤ 3 in memory)
|
||||
- [ ] `:e` and palette share the same recent-files list
|
||||
- [ ] `:enew` creates a new file — prompts for scope (tracked vs local)
|
||||
- [ ] Delete a file — removes it from the SD card; for a Tracked file the
|
||||
removal reaches the next `Ctrl-G` Publish's staged set (`git rm` / `add -A`
|
||||
semantics, not plain `git add .`); a Local file is just unlinked
|
||||
- [ ] `Ctrl-G` is disabled / hidden when the current buffer is local-scope
|
||||
- [ ] The side panel briefly shows file count on `Ctrl-G` when the publish bundles
|
||||
more than one dirty Tracked file (e.g. `"publishing 3 files: abc1234"`),
|
||||
so workspace-scoped behaviour stays visible to the user
|
||||
- [ ] **Preferences file** `/sd/repo/.typoena.toml` — a git-tracked,
|
||||
hand-editable TOML file for editor behaviour, deliberately **distinct from
|
||||
the `/sd/typoena.conf` card secrets** (Wi-Fi / PAT / remote / author,
|
||||
gitignored, never committed — see v0.1). Read at boot; a missing file or
|
||||
key falls back to the defaults below. Keys:
|
||||
- [ ] `save_on_idle` (bool, default `true`) — auto-save the current buffer on
|
||||
the existing idle pause (the ≥ 1 s typing-pause the panel already uses
|
||||
for its refresh), so `:w` becomes optional rather than required.
|
||||
- [ ] `format_on_save` (bool, default `true`) — run `:fmt` (table alignment,
|
||||
blank-line collapse, trailing-whitespace strip) on the buffer before it
|
||||
is persisted, so `:sync` is **fmt → save → commit → push** and `:w`
|
||||
saves formatted. Implemented in-core 2026-07-11 (`Editor::format_on_save`,
|
||||
default on); this key will drive it. **Open question:** with
|
||||
`save_on_idle` also on, this reformats on every idle pause — reflowing
|
||||
tables / collapsing blanks mid-session. Consider limiting fmt to
|
||||
explicit `:w`/`:sync` and leaving the idle auto-save unformatted.
|
||||
- [ ] `line_numbers` (bool, default `true`) — show the absolute line-number
|
||||
gutter (built always-on in v0.2). Off reclaims the gutter's columns for
|
||||
text; the palette `> line numbers: on/off` command toggles it live.
|
||||
- [ ] `auto_sync` (duration string, default `"10m"`; `"0"` / omitted
|
||||
disables; **min clamp ~`"2m"`** so a palette typo can't drain the
|
||||
battery) — a *max-staleness cap*, not a wall-clock timer:
|
||||
**opportunistic, rate-limited** Publish. Push when already awake + dirty
|
||||
(coalesced into the idle-pause, ≤ once per `auto_sync`) and once on the
|
||||
way into sleep if dirty; **never wake from deep sleep purely to sync**.
|
||||
Wi-Fi energy is a `1/T` curve whose knee sits at 5–10 min, and
|
||||
`save_on_idle` already owns local data safety — so 10 min halves the
|
||||
sync energy of a 5-min default for no real risk. Full derivation:
|
||||
[`tradeoff-curves/wifi-auto-sync.md`](tradeoff-curves/wifi-auto-sync.md).
|
||||
The **schema + defaults live here in v0.5**; the periodic side rides the
|
||||
better-git work (v0.7) and must interact with light / deep sleep (v0.8).
|
||||
- [ ] Open question: because the file is committed, these prefs **sync to
|
||||
every device** that clones the repo — a per-device sync cadence may
|
||||
instead want a card-local override (in `typoena.conf`). Decide before
|
||||
build.
|
||||
- [ ] **Palette command mode** — typing `>` at the `Ctrl-P` palette switches it
|
||||
from file search to a command list (VS Code-style). The v0.5 commands edit
|
||||
the `.typoena.toml` prefs above — e.g. `> save on idle: on/off` and
|
||||
`> auto sync: 10m` — writing the value back to the file and applying it
|
||||
live. This command list is the discoverable surface that later actions
|
||||
(`:fmt`, theme, font) also register into.
|
||||
|
||||
## v0.6 — Markdown affordances — [~]
|
||||
|
||||
**Status:** render affordances done early; the 80-col ruler and the snippet
|
||||
engine remain (snippets are net-new scope, added 2026-07-08).
|
||||
|
||||
- [x] Heading lines bolded in render (faux-bold double-strike)
|
||||
- [x] List continuation on Enter inside `- ` / `1. ` (with empty-item exit)
|
||||
- [x] Soft-wrap at word boundaries
|
||||
- [ ] Optional column ruler at 80
|
||||
- [ ] **Snippets** — trigger-driven text expansion for Markdown authoring
|
||||
(Zed-inspired, but no completion popup: e-ink's ~630 ms refresh rules out
|
||||
a live filtering menu, and it fights the distraction-free premise). Shape,
|
||||
mirroring the existing `list_marker` insert-transform:
|
||||
- [ ] Tab in Insert mode triggers expansion: if the word immediately before
|
||||
the caret matches a snippet prefix, expand it; otherwise insert spaces
|
||||
as today (`expand_snippet(word) -> Option<(body, stops)>`, alongside
|
||||
`list_marker`).
|
||||
- [ ] A snippet body is literal text plus numbered empty tab stops `$1 … $n`
|
||||
and a final `$0`. There is no placeholder text (`${1:label}`) — the
|
||||
editor has no selection/overtype model, so a placeholder would just be
|
||||
text to delete. There are no dynamic or computed values either (e.g. no
|
||||
`date` — there's no RTC; the wall clock is valid only after Wi-Fi+SNTP,
|
||||
so it'd stamp 1970 on a cold boot).
|
||||
- [ ] After expansion the caret lands on `$1`; Tab advances to the next stop,
|
||||
forward only (no Shift-Tab). Stored stop offsets shift with edits at the
|
||||
caret (all pending stops are always after it). The session auto-aborts
|
||||
on Esc, a mode change, or a motion that leaves the stops.
|
||||
- [ ] On a typing pause (same throttle as the insert cursor / word-count
|
||||
refresh — the panel never repaints per keystroke), if the word before
|
||||
the caret is a snippet prefix, the side panel shows the hint (the target
|
||||
expansion). Quiet while typing; the hint appears on pause.
|
||||
- [ ] The snippet table is hard-coded in the binary to start; a git-syncable
|
||||
file on SD (`/sd/repo/.snippets`) is a later option, deferred while SD
|
||||
is still blocked.
|
||||
- [ ] Starter set: link `[$1]($2)$0`, image `$0`, fenced code block,
|
||||
etc.
|
||||
Heading bolding, list continuation, and soft-wrap, plus the trigger-driven
|
||||
snippet engine (net-new scope, added 2026-07-08): a tab-stop session reached
|
||||
inline (prefix + Tab) and from the `$` palette, fed by a git-synced,
|
||||
Zed-compatible `.typoena.snippets.json`; the `Cmd-P` palette generalised into a
|
||||
`files`/`>` commands/`$` snippets split (retiring `:e`); and a `just init`
|
||||
catalog. **COMPLETE in core 2026-07-12** (firmware 0.6.0), on-device smoke-test
|
||||
pending. Detail: [v0.6-markdown.md](v0.6-markdown.md).
|
||||
|
||||
## v0.7 — Search + better git — [~]
|
||||
|
||||
**Status:** the **`:gl` pull command landed in the editor** (2026-07-11,
|
||||
host-tested) — `Effect::Pull` + a firmware stub; the on-device fetch +
|
||||
fast-forward is still to build. Search not started.
|
||||
|
||||
- [ ] `/` forward search, `n N`
|
||||
- [~] `:gl` — pull: fetch + **fast-forward only**, refuse on divergence and
|
||||
surface it (renamed from the planned `:Gpull`). Editor command +
|
||||
`Effect::Pull` done 2026-07-11 (host-tested); the git-thread
|
||||
fetch/fast-forward in `git_sync` remains (only push is wired today).
|
||||
`/` forward search (`n`/`N`) and `:gl` pull (fetch + fast-forward only). The
|
||||
`:gl` editor command landed 2026-07-11; the on-device fetch and search are still
|
||||
to build. Detail: [v0.7-search-and-git.md](v0.7-search-and-git.md).
|
||||
|
||||
## v0.8 — Power: battery + sleep — [ ]
|
||||
|
||||
- [ ] Measure idle / typing / push current draw on bench
|
||||
- [ ] 18650 + IP5306 charge board, soft power switch
|
||||
- [ ] Light sleep on idle > 30 s (keyboard interrupt wakes)
|
||||
- [ ] Deep sleep on lid close (reed switch); restore cursor + buffer
|
||||
- [ ] Battery indicator in the side panel
|
||||
Bench current-draw measurement, 18650 + charge board, light/deep sleep, and a
|
||||
battery indicator. **Not started.**
|
||||
Detail: [v0.8-battery-and-sleep.md](v0.8-battery-and-sleep.md).
|
||||
|
||||
## v0.9 — Robustness — [ ]
|
||||
|
||||
- [ ] Crash-safe writes (write to `.tmp`, fsync, rename)
|
||||
- [ ] Recover from interrupted push (re-attempt on next save)
|
||||
- [ ] SD card removal / reinsert handling
|
||||
- [ ] Wi-Fi reconnect with backoff
|
||||
- [ ] On-device provisioning + settings screen: SSID, PAT rotation, default
|
||||
remote, commit author (replaces the v0.1 dev-only NVS-flashing path —
|
||||
first release usable by someone who is not the firmware author)
|
||||
Crash-safe writes, interrupted-push recovery, SD removal handling, Wi-Fi
|
||||
reconnect, and on-device provisioning (the first release usable by a non-author).
|
||||
**Not started.** Detail: [v0.9-robustness.md](v0.9-robustness.md).
|
||||
|
||||
## v1.0 — Polish — [ ]
|
||||
|
||||
- [ ] Boot time ≤ 3 s to usable cursor — currently ~4.26 s; the ~1.9 s cold-boot
|
||||
full refresh is a hard e-ink floor, so ≤ 3 s is marginal (see
|
||||
[`notes/boot-time-budget.md`](notes/boot-time-budget.md))
|
||||
- [ ] Font selection (at least one serif + one mono) with adjustable font
|
||||
size, switchable at runtime and persisted across reboots
|
||||
- [ ] Theme: light / dark (inverted e-ink), switchable at runtime and
|
||||
persisted across reboots
|
||||
- [ ] Enclosure design files in `hardware/`
|
||||
- [ ] User guide
|
||||
≤ 3 s boot, runtime-switchable fonts, enclosure files, and a user guide
|
||||
(light/dark theme landed early, in v0.5). **Not started.** Detail:
|
||||
[v1.0-polish.md](v1.0-polish.md).
|
||||
|
||||
## v1.x — Stretch / nice-to-have
|
||||
|
||||
- 10.3" panel upgrade via IT8951
|
||||
- Multiple remotes / repos
|
||||
- Stats: words today, streak
|
||||
- BLE-HID fallback for wireless keyboards
|
||||
Post-1.0 ideas, not committed to any release (10.3" panel, multiple remotes,
|
||||
writing stats, BLE-HID fallback). Detail: [v1.x-stretch.md](v1.x-stretch.md).
|
||||
|
||||
@@ -75,10 +75,12 @@ The big rocks are physics or protocol, not slack:
|
||||
- **TLS handshake ~2.4 s** and **push negotiate/upload ~4.4 s** are inherent to
|
||||
libgit2-over-mbedTLS on this part; the payload is tiny, so there's little to
|
||||
shave.
|
||||
- **stage + commit ~3.1 s** is the one soft spot: staging `notes.md` directly
|
||||
instead of `add_all(["*"])` would skip the SD/FAT tree walk (likely →
|
||||
sub-second), at the cost of the file-agnostic design that a future multi-file
|
||||
publish wants. Deferred, on purpose.
|
||||
- **stage + commit ~3.1 s** is the one soft spot: staging over the editor's dirty
|
||||
set (`add_path`) instead of `add_all(["*"])` would skip the SD/FAT tree walk
|
||||
(likely → sub-second) *without* losing multi-file — the dirty set is the file
|
||||
list. Whether the walk actually dominates the ~4 s commit is now being measured
|
||||
by the `commit split —` log line; the cost model and the rule it decides live in
|
||||
[`../tradeoff-curves/sync-commit-staging.md`](../tradeoff-curves/sync-commit-staging.md).
|
||||
|
||||
**Conclusion:** ~16 s cold / ~10 s warm is close to the floor for "commit to FAT +
|
||||
one TLS push over Wi-Fi with a fresh clock." It reads as slow only if you wait on
|
||||
|
||||
@@ -0,0 +1,164 @@
|
||||
# Editor freeze — SPI-DMA out-of-memory during a background `:sync`
|
||||
|
||||
> Date: 2026-07-11 · Build at time of failure: `07-11 18:02Z @229c259-dirty`
|
||||
> Status: **Safety net shipped** (paints are non-fatal, the appliance no longer
|
||||
> bricks) — pending hardware re-test. **Root cause not yet eradicated**: a paint
|
||||
> that lands during a sync still fails and drops the frame; the permanent fix (a
|
||||
> persistent internal DMA scratch buffer) is specced below and tracked in the
|
||||
> follow-ups.
|
||||
>
|
||||
> Context: editor loop [`../../firmware/src/main.rs`](../../firmware/src/main.rs),
|
||||
> EPD driver [`../../firmware/src/epd.rs`](../../firmware/src/epd.rs), git
|
||||
> transport [`../../firmware/src/git_sync.rs`](../../firmware/src/git_sync.rs).
|
||||
> Display medium [ADR-003](../adr.md#adr-003-display-medium--e-ink-gdey0579t93-panel);
|
||||
> concurrency model (dedicated git thread) [ADR-006](../adr.md).
|
||||
|
||||
## Summary
|
||||
|
||||
First-ever field freeze. After an hour of clean editing, the user ran `:sync`;
|
||||
the commit and push **succeeded** (`push accepted by remote`, commit
|
||||
`48a2c0a8`), but partway through the push the panel stopped updating. Keystrokes
|
||||
kept being logged, so the device wasn't hung — but nothing repainted again.
|
||||
|
||||
The editor task had died. A screen refresh that happened to run **while Wi-Fi +
|
||||
TLS were up for the push** failed to allocate an internal DMA buffer
|
||||
(`ESP_ERR_NO_MEM`), and that error propagated through a `?` straight out of
|
||||
`main()`. ESP-IDF's `main_task` returned from `app_main()`; the editor loop lived
|
||||
on that task, so it stopped. The USB-keyboard and git threads are **separate
|
||||
FreeRTOS tasks** (ADR-006) and kept running — hence keys still logged while the
|
||||
panel was frozen. A zombie, not a crash.
|
||||
|
||||
## Symptom
|
||||
|
||||
```
|
||||
I (305749) firmware::git_sync: verifying github.com TLS chain against embedded GitHub CA bundle
|
||||
I (305979) firmware::usb_kbd: key: HalfPageUp ← scroll arrives → editor loop paints
|
||||
E (306259) spi_master: setup_dma_priv_buffer(1206): Failed to allocate priv TX buffer
|
||||
Error: ESP_ERR_NO_MEM
|
||||
I (306259) main_task: Returned from app_main() ← editor task exits
|
||||
...
|
||||
I (310119) firmware::git_sync: push accepted by remote ← git thread unaffected; commit landed
|
||||
I (311069) firmware::usb_kbd: key: Escape ← keys still logged, no refresh ever again
|
||||
I (312729) firmware::usb_kbd: key: HalfPageDown
|
||||
```
|
||||
|
||||
## Root cause
|
||||
|
||||
The EPD is on SPI2 configured `Dma::Auto(4096)` (`main.rs`), and the driver hands
|
||||
frame data to `spi.write()` as ordinary Rust `Vec<u8>` buffers (`epd.rs`
|
||||
`write_frame_bank` / `data`). With PSRAM added to the heap allocator, those
|
||||
`Vec`s can be allocated **from PSRAM**, which is **not DMA-capable**. When a
|
||||
transfer buffer isn't DMA-capable, esp-idf's `spi_master` bounces it through a
|
||||
temporary internal buffer it mallocs on the fly (`setup_dma_priv_buffer`, using
|
||||
`MALLOC_CAP_DMA` → **internal RAM only**).
|
||||
|
||||
For the whole session that bounce allocation succeeded, because internal RAM was
|
||||
plentiful. The instant `:sync` brought up **Wi-Fi + TLS**, they consumed the
|
||||
small internal pool. The next paint's bounce allocation returned
|
||||
`ESP_ERR_NO_MEM`. The real defect is not the low-memory moment itself — it's that
|
||||
the paint's `?` made a **transient, retryable** I/O failure **fatal to the whole
|
||||
appliance**.
|
||||
|
||||
Two things made this easy to misread:
|
||||
|
||||
- **"8.4 MB free heap" is a red herring.** That figure is dominated by PSRAM.
|
||||
The starved pools are the tiny internal ones (~265 KB / 21 KB / 32 KB), and DMA
|
||||
bounce buffers can only come from those.
|
||||
- **It looked like a git/sync bug, but the push was fine.** The git thread
|
||||
finished and the remote accepted the commit. Only the UI task died.
|
||||
|
||||
## Timeline
|
||||
|
||||
1. Hour of normal editing — hundreds of ~630 ms partial refreshes, no issue
|
||||
(internal RAM never under pressure; Wi-Fi off — the radio is lazy, ADR-006).
|
||||
2. `:sync` → save → git thread brings up Wi-Fi, SNTP, TLS, commits `48a2c0a8`,
|
||||
starts the push. Internal RAM now under heavy Wi-Fi/TLS load.
|
||||
3. User keeps scrolling (`HalfPageUp`) during the push. Each scroll triggers a
|
||||
full-area partial refresh → SPI-DMA transfer → bounce-buffer malloc.
|
||||
4. One such malloc fails (`ESP_ERR_NO_MEM`); the `?` in the refresh call
|
||||
propagates out of `main()`; `main_task` returns from `app_main()`.
|
||||
5. Push completes normally; Wi-Fi torn down; internal RAM freed — but the editor
|
||||
task is already gone, so nothing repaints. Keys log into the void.
|
||||
|
||||
## What it was *not*
|
||||
|
||||
- **Not out of heap** — plenty of PSRAM free; it was internal *DMA-capable* RAM
|
||||
specifically.
|
||||
- **Not a git or TLS bug** — the sync succeeded end to end.
|
||||
- **Not an editor-core bug** — `editor`/`keymap` never ran; the failure is in the
|
||||
firmware paint path's error handling.
|
||||
- **Not a panel/wiring fault** — the same paint path worked for an hour and works
|
||||
again after Wi-Fi is down.
|
||||
|
||||
## Remediation shipped — paints are non-fatal
|
||||
|
||||
A screen refresh is idempotent and retryable: the editor buffer is the source of
|
||||
truth, so a dropped frame costs nothing and the next paint recovers. This is the
|
||||
exact contract already written into `save_note` ("errors are logged, never
|
||||
propagated"). Every paint site in the editor loop (`main.rs`) now:
|
||||
|
||||
- logs the failure and **drops the frame** instead of `?`-propagating it;
|
||||
- leaves `shown` untouched so the next paint repaints the same diff;
|
||||
- sets a `force_full` flag so the next paint is a **full refresh**, which
|
||||
rewrites both RAM banks and recovers from a partial that may have died
|
||||
mid-transfer and left the `0x24`/`0x26` banks inconsistent.
|
||||
|
||||
Effect: a paint that lands during a sync is dropped (panel goes stale for the
|
||||
~15 s of the push), then self-heals the moment the push finishes and internal RAM
|
||||
frees. A permanent brick becomes a brief stale window. The boot-time first render
|
||||
stays `?`-fatal on purpose — it runs before Wi-Fi, can't hit this, and a dead
|
||||
panel at boot is a legitimate hard fault.
|
||||
|
||||
## Root-cause eradication (specced, not yet built)
|
||||
|
||||
The safety net stops the brick but not the underlying **contention**: paints
|
||||
during a sync still fail and drop. To keep the editor fully live while a push
|
||||
runs — the whole point of the async git thread (ADR-006) — remove the on-the-fly
|
||||
DMA allocation entirely.
|
||||
|
||||
**Fix: a persistent, internal, DMA-capable scratch buffer owned by `Epd`.**
|
||||
|
||||
- Allocate it **once at `Epd` construction** (boot, before Wi-Fi is ever up, when
|
||||
internal RAM is plentiful) via
|
||||
`heap_caps_malloc(SPI_CHUNK, MALLOC_CAP_DMA | MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT)`.
|
||||
- `data()` already chunks writes to `SPI_CHUNK` (4096 B), so a **4 KB** scratch
|
||||
buffer is sufficient: copy each chunk into it and hand that DMA-capable slice to
|
||||
`spi.write()`.
|
||||
- Because the source is now DMA-capable, `spi_master` DMAs directly from it —
|
||||
`setup_dma_priv_buffer` is never invoked, there is **no per-refresh
|
||||
allocation**, and paints no longer compete with Wi-Fi/TLS for internal RAM.
|
||||
Refreshes then succeed during a sync, and `force_full` recovery is only ever
|
||||
exercised by genuine faults, not by normal syncing.
|
||||
|
||||
**Cost:** ~4 KB of internal RAM reserved for the life of the device; one small
|
||||
`unsafe` block for the alloc + slice; a `memcpy` per 4 KB chunk (negligible
|
||||
against the ~630 ms panel waveform).
|
||||
|
||||
**Alternatives considered and rejected:**
|
||||
|
||||
- *Trim Wi-Fi's internal buffer counts* to leave headroom — fragile tuning that
|
||||
risks Wi-Fi stability/throughput and only widens the margin instead of removing
|
||||
the failure mode.
|
||||
- *Force the whole ~13.6 KB frame buffer internal* (custom allocator / full
|
||||
`heap_caps` buffer) — larger reservation and more churn than the 4 KB
|
||||
chunk-scratch, for no extra benefit since `data()` already chunks.
|
||||
- *Serialize paints against sync* (skip painting while a push is in flight) —
|
||||
defeats the async-git design and freezes the panel for the whole push; the
|
||||
safety net already makes this unnecessary.
|
||||
|
||||
**Verification when built:** reproduce the exact failing scenario — edit,
|
||||
`:sync`, and keep scrolling/paging through the *entire* push. Expect: every
|
||||
refresh succeeds (no `refresh … FAILED` warnings, no dropped frames), min-ever
|
||||
internal heap stays comfortably above zero throughout, and `force_full` is not
|
||||
triggered by the sync.
|
||||
|
||||
## Follow-ups
|
||||
|
||||
- [x] Make all editor-loop paints non-fatal + `force_full` recovery (`main.rs`);
|
||||
release build green.
|
||||
- [ ] Reflash and hardware-verify the safety net against the repro (edit →
|
||||
`:sync` → scroll through the push): panel must recover, not freeze.
|
||||
- [ ] Implement the persistent internal DMA scratch buffer in `Epd` (eradication
|
||||
above) if the stale-during-sync window proves annoying in real use.
|
||||
- [ ] After eradication, confirm refreshes succeed *during* a push and drop the
|
||||
stale window entirely.
|
||||
@@ -12,3 +12,4 @@
|
||||
| ---------- | ------------------------------------------------------------------------ | ------ |
|
||||
| 2026-07-05 | [Spike 3 (SD) — card rejects CMD59 (SPI-mode CRC)](2026-07-05-spike3-sd-cmd59.md) | Paused — awaiting a compliant microSD; wiring + firmware proven |
|
||||
| 2026-07-05 | [Spike 7 (git push) — ADR-004 kill-switch fired: gix can't push over HTTPS](2026-07-05-spike7-gix-https-push.md) | Turned — pivoted to libgit2; git mechanics proven on desktop, device build next |
|
||||
| 2026-07-11 | [Editor freeze — SPI-DMA OOM during a background `:sync`](2026-07-11-editor-freeze-spi-dma-oom-during-sync.md) | Safety net shipped (paints non-fatal); root-cause eradication specced, not yet built |
|
||||
|
||||
@@ -11,3 +11,4 @@
|
||||
| --- | --- |
|
||||
| [`wifi-auto-sync.md`](wifi-auto-sync.md) | `auto_sync` interval vs Wi-Fi energy (a `1/T` hyperbola) — why the default is 10 min and opportunistic, not a wall-clock timer. |
|
||||
| [`epd-refresh-latency.md`](epd-refresh-latency.md) | E-ink refresh latency vs rows driven — the full / full-area-partial / windowed-Y cost model behind typing responsiveness and the boot splash→editor swap. |
|
||||
| [`sync-commit-staging.md`](sync-commit-staging.md) | Commit-staging strategy vs working-tree size — RESOLVED: every index-based path is O(N_tree) and fails on the real repo (611 s / OOM); the O(depth) TreeBuilder splice is benched at ~2–2.8 s and ships. Holds the full measurement trail plus the firmware plumbing plan (merged from the retired handoff note). |
|
||||
|
||||
640
docs/tradeoff-curves/sync-commit-staging.md
Normal file
640
docs/tradeoff-curves/sync-commit-staging.md
Normal file
@@ -0,0 +1,640 @@
|
||||
# Commit-staging cost vs working-tree size
|
||||
|
||||
> **Decision (RESOLVED 2026-07-12, real-repo bench):** neither `add_all(["*"])`
|
||||
> nor an index-free `read_tree`+`write_tree` is viable on the real
|
||||
> `jcalixte/notes` clone — **both are O(N_tree)** and blow up on the 570 MB pack /
|
||||
> 1179-file tree (611 s hash one end, 77 s tree-read + OOM the other; see
|
||||
> [Real-repo run](#real-repo-run-2026-07-12-jcalixtenotes-570-mb-pack--the-index-is-the-wrong-primitive)
|
||||
> below). The commit must be rebuilt with an **O(depth) TreeBuilder walk** —
|
||||
> patch only the edited file's ancestor subtree chain onto HEAD's tree, never
|
||||
> materialise all 1179 entries.
|
||||
>
|
||||
> **Final state (2026-07-13, after four localization rounds):** the splice walk
|
||||
> is benched and the block is lifted. It first measured 6.5 s (each loose-object
|
||||
> write cost ~1.5 s); the root cause was FatFS's lseek cluster-chain walk, and
|
||||
> the fast-seek fix cut it to **2.8 s cold / ~2 s steady-state**. The `esp_map.c`
|
||||
> window cache was **removed entirely** (0 hits across four instrumented runs —
|
||||
> `mwindow` absorbs any true repetition above `p_mmap`). The sub-second bar
|
||||
> failed, but ~2–2.8 s against 611 s/OOM for every alternative ships: a full
|
||||
> cold real-repo `:sync` lands at ~9–10 s. **Decision: wire the splice in —
|
||||
> done 2026-07-13** ([how it landed](#the-fix--wiring-the-odepth-splice-into-the-firmware),
|
||||
> merged from the retired `notes/sync-commit-handoff.md`; on-device `:sync`
|
||||
> against the real repo still pending). Shrinking the repo is **not** an option
|
||||
> ([`../notes/git-sync-images-and-repo-size.md`](../notes/git-sync-images-and-repo-size.md):
|
||||
> the images are load-bearing for another app), which is exactly why the O(depth)
|
||||
> mechanism is the only lever left. This note records the cost model and the full
|
||||
> measurement trail that got here.
|
||||
>
|
||||
> Tradeoff-curves index: [`README.md`](README.md). Docs index:
|
||||
> [`../README.md`](../README.md). The six-step kaizen retelling of this whole
|
||||
> investigation (problem → factor analysis → chosen fix → evaluation):
|
||||
> [`../kaizen/real-repo-sync.md`](../kaizen/real-repo-sync.md). Where the whole sync goes:
|
||||
> [`../notes/sync-latency.md`](../notes/sync-latency.md). Sibling curve on the
|
||||
> radio cost of *how often* we sync: [`wifi-auto-sync.md`](wifi-auto-sync.md).
|
||||
|
||||
## The model
|
||||
|
||||
`:sync` commits the working tree on the SD/FAT card before it pushes. The commit
|
||||
is two kinds of work against the card over SPI (10 MHz today, ADR-012):
|
||||
|
||||
```
|
||||
stage write
|
||||
─────────────────────────────── ─────────────────────────────
|
||||
add_all(["*"]) + update_all(["*"]) index.write + write_tree + commit-obj
|
||||
→ stat() every file in the tree, → serialise the index and three loose
|
||||
hash the ones whose stat moved objects, each a FAT create+write+fsync
|
||||
cost ∝ tree size (O(N_tree)) cost ∝ churn (O(N_changed)) + fixed
|
||||
```
|
||||
|
||||
The two have different curves against **N = files in `/sd/repo`**:
|
||||
|
||||
- **Walk** rises with N. `add_all(["*"])` visits the whole working tree every
|
||||
sync regardless of how little changed, and each visit is a FAT `stat` (and a
|
||||
re-hash when the entry looks dirty) over SPI. This is the term explicit-path
|
||||
staging removes: the editor already knows which buffers are dirty and which
|
||||
were `:delete`d, so `index.add_path(p)` / `index.remove_path(p)` over that set
|
||||
touches `N_changed` files (≈1 for a writing appliance), not `N`.
|
||||
- **Write** is flat in N. A text commit is the index + a blob + a tree + a commit
|
||||
object — a handful of small FAT writes whose cost is set by SPI clock and
|
||||
`fsync`, not by tree size. Explicit-path staging cannot shave this; only a
|
||||
faster card bus (SD 10 → 20 MHz on a clean PCB, `persistence.rs`) does.
|
||||
|
||||
```
|
||||
Commit latency vs working-tree size two staging strategies
|
||||
|
||||
ms
|
||||
| walk-all: add_all(["*"])
|
||||
4000 | . * stat()s every file in the
|
||||
| . * tree each sync → O(N)
|
||||
| . *
|
||||
3000 | . *
|
||||
| . *
|
||||
2000 | . *
|
||||
| *
|
||||
1000 |····································· explicit-path: add_path(dirty)
|
||||
| FAT object-write floor → O(churn); flat in N. The gap
|
||||
0 +----+----+----+----+----+----+----+---→ up to walk-all is the avoidable
|
||||
10 50 100 200 400 800 1179 per-sync tree walk.
|
||||
└── jcalixte/notes today (N files)
|
||||
```
|
||||
|
||||
The gap between the lines at a given N is exactly what switching buys, and it
|
||||
**grows without bound** as the notes tree fills.
|
||||
|
||||
### The real operating point (measured 2026-07-12, `jcalixte/notes`)
|
||||
|
||||
The device syncs into a clone of the actual notes repo, not a `notes.md` toy. Its
|
||||
working tree is **not small**:
|
||||
|
||||
| | count | working-tree bytes |
|
||||
| --- | ---: | ---: |
|
||||
| Markdown (`.md`) | 875 | ~1.5 MB |
|
||||
| Images (png/jpg/webp/bmp/gif) | ~260 | **~150 MB** |
|
||||
| Other (json/ts/pdf/…) | ~44 | ~20 MB |
|
||||
| **Total (N)** | **1179 files, 158 dirs** | **~170 MB** |
|
||||
| `.git` history | | ~570 MB |
|
||||
|
||||
So `add_all(["*"])` walks **1179 files across 158 directories every sync** — and
|
||||
~260 of them are images that a text edit never changes. That does two things the
|
||||
toy-repo baseline hides:
|
||||
|
||||
1. **The walk term is large and paid on every sync** — 1179 `stat`s + 158 dir
|
||||
reads over SPI, for a one-line note change. This is the O(N) cost the curve
|
||||
above predicts, at N ≈ 1179 rather than N ≈ 2.
|
||||
2. **Re-hash risk.** libgit2 decides a file is unchanged from `stat` metadata
|
||||
(mtime/size). FAT's coarse mtime and lack of a stable inode can make entries
|
||||
look racy, forcing a content re-hash. If even a slice of the ~150 MB of images
|
||||
gets re-hashed over a 10 MHz SPI bus, the commit balloons far past 4 s. The
|
||||
`walk` timer will show it; explicit-path staging sidesteps it entirely by never
|
||||
visiting the images.
|
||||
|
||||
## Measurement (2026-07-12, toy `notes.md` tree, N ≈ 2)
|
||||
|
||||
Split from two back-to-back `:sync`es on the small test repo (commits `95ac56ef`
|
||||
cold, `ab260bde` warm), via the `commit split —` log lines:
|
||||
|
||||
| Sub-phase | Kind | Cold (ms) | Warm (ms) |
|
||||
| --- | --- | ---: | ---: |
|
||||
| `walk(add_all+update_all)` | scan (O(N)) + likely 1 blob write | 1402 | 1456 |
|
||||
| `index.write` | FAT write | 204 | 204 |
|
||||
| `write_tree` | **1 tree object → FAT** | 710 | 715 |
|
||||
| `parent-load` | FAT read | 102 | 105 |
|
||||
| `commit-obj` | **1 commit object + ref → FAT** | 914 | 924 |
|
||||
| **commit total** | | **3332** | **3404** |
|
||||
|
||||
### It is not the card — it's libgit2 (`sd_bench`, 2026-07-12)
|
||||
|
||||
My first read of the table was "a loose-object write to this SD card costs
|
||||
~700–900 ms." **That was wrong.** `sd_bench` (`firmware/src/bin/sd_bench.rs`) times
|
||||
the raw FAT primitives on the same card at the same 10 MHz:
|
||||
|
||||
| Raw FAT op (200-byte payload) | p50 |
|
||||
| --- | ---: |
|
||||
| create + write + close | 21.7 ms |
|
||||
| rename | 12.8 ms |
|
||||
| stat (hit / miss) | ~5 ms |
|
||||
| remove | 14.9 ms |
|
||||
| **loose-object composite** (stat + create + write + rename) | **86 ms** |
|
||||
|
||||
The card does a *complete* loose-object write in **~86 ms**. Yet `write_tree`
|
||||
(one tree object) took **710 ms** and `commit-obj` **914 ms** — an **~8× gap that
|
||||
is pure libgit2 overhead, not FAT I/O.** So the earlier "object-write floor / SD
|
||||
write amplification / better card / SPI-clock" framing is refuted: **the SD card is
|
||||
not the bottleneck.** fsync is still confirmed off; the extra ~600 ms/op is CPU or
|
||||
repeated `.git` I/O *inside* libgit2 (candidates: ODB refresh scanning
|
||||
`objects/`, the treebuilder's per-entry `git_odb_exists`, ref-lock + reflog writes,
|
||||
config/attributes re-reads). `git_bench` (`firmware/src/bin/git_bench.rs`) localizes
|
||||
it — see below.
|
||||
|
||||
### It's the pack, read through an un-caching emulated mmap (`git_bench`, 2026-07-12)
|
||||
|
||||
`git_bench` times the git2 primitives in isolation on `/sd/repo` (git ops on the
|
||||
same 96 KB thread the real service uses — the main-task stack overflows on
|
||||
`index.write`, which is itself the reason the service has a dedicated thread):
|
||||
|
||||
| git2 op | p50 | note |
|
||||
| --- | ---: | --- |
|
||||
| `Repository::open` | 100 ms | one-time |
|
||||
| `odb.write(blob)` (unique) | **45 ms** | writes a fresh object; touches no existing object |
|
||||
| `repo.index()` open | ~0 ms | cached |
|
||||
| `index.write()` | 376 ms | index + `index.lock` rename + tree-cache |
|
||||
| `write_tree` [unchanged] | ~0 ms | tree exists → freshen-skips the write |
|
||||
| **`write_tree` [changed]** | **1136 ms** | writes ONE 45 ms object |
|
||||
| **`commit(None)` orphan obj** | **563 ms** | writes ONE 45 ms object, no ref/reflog |
|
||||
|
||||
Writing a fresh object is 45 ms; the ops that wrap one are 8–25×. The cause, from
|
||||
the vendored source: `git_odb_write` calls `git_odb__freshen` (odb.c:1011), which
|
||||
on a not-found object runs **`git_odb_refresh`** (re-reads the pack dir + reloads
|
||||
pack indexes), and existence checks (`freshen(tree)` in `commit.c:169`, base-object
|
||||
lookups in `write_tree`) hit the **pack**. Pack access goes through our
|
||||
`p_mmap` (`esp_map.c`), which **`malloc`s and `read()`s the mapped range from the
|
||||
card on every call — no cache** — with a 32 MB window on this 32-bit target. So
|
||||
each write re-reads pack bytes from SD; `odb.write` of a fresh blob is 45 ms only
|
||||
because it touches no packed object.
|
||||
|
||||
**This scales with pack size.** The toy repo's pack is tiny; the real
|
||||
`jcalixte/notes` clone has a **570 MB pack**, and provisioning rsyncs a full clone
|
||||
onto the card — so a real-repo commit has **never been benchmarked** and, on this
|
||||
mechanism, will be far worse than the ~3.3 s toy number. That is the single biggest
|
||||
open risk in sync.
|
||||
|
||||
### Real-repo run (2026-07-12, `jcalixte/notes`, 570 MB pack) — the index is the wrong primitive
|
||||
|
||||
`git_bench` was finally run against a full clone of the real repo (1179 files, 158
|
||||
dirs, 570 MB pack). It settles the design: **any index-based commit is O(N_tree)
|
||||
and does not fit this device.** Two independent walls:
|
||||
|
||||
| op | result | reading |
|
||||
| --- | ---: | --- |
|
||||
| `Repository::open` | 88 ms | fine |
|
||||
| odb open (implicit) | ~6 s cold | maps the 1.7 MB pack `.idx` once (16 miss / 1790 KB) |
|
||||
| `odb.write(blob)` | **142 ms** p50 | the mmap cache win **holds** (was 862 ms uncached) ✅ |
|
||||
| `repo.index()` load (1179 entries) | 514 ms max | the on-disk index we were trying to avoid |
|
||||
| `index.write()` | **min 360 ms / p50 12.8 s / max 611 s** | ⚠️ hangs — see root cause |
|
||||
| **seed `read_tree(HEAD)` (cold, 1×)** | **~77 s** | ⚠️ reads all ~158 tree objects, 22.7 MB of pack windows |
|
||||
| `Index::new + read_tree` (warm) | 447 ms p50 | windows still mapped → pure CPU |
|
||||
| **index-free `stage→tree`** | **💥 crash** | `zlib (5)`: `deflateInit` failed, **508 KB heap left** |
|
||||
|
||||
**Wall 1 — `index.write()` hashes the whole working tree (up to 611 s).**
|
||||
`git_index_write` unconditionally calls `truncate_racily_clean` (index.c:822),
|
||||
which runs `git_diff_index_to_workdir` over **every** entry flagged "racy" and
|
||||
re-hashes its file. On a fresh FAT clone the mtime granularity is 2 s and
|
||||
`index.stamp.mtime <= entry.mtime` for ~all 1179 entries (index.h:117), so the
|
||||
whole tree looks racy → it re-hashes ~170 MB (mostly the 150 MB of images) over
|
||||
10 MHz SPI. The 611 s → 12.8 s → 360 ms decay across three iterations is the
|
||||
signature: each write bumps the index mtime, shrinking the racy set. **Implication:
|
||||
the shipping `stage_and_commit` calls `index.write()`, so `:sync` on the real repo
|
||||
effectively bricks on the first commit** — the user sees a 10-minute freeze,
|
||||
resets, the index mtime never advances, and it re-hashes forever. The real repo has
|
||||
almost certainly never completed a sync on device (only the toy `typoena-test` has).
|
||||
|
||||
**Wall 2 — the index-free path is still O(N_tree), and the mmap cache OOMs.**
|
||||
Skipping `index.write()` entirely (fresh `Index::new()`, stamp = 0, so
|
||||
`truncate_racily_clean` can never fire) removes Wall 1. But to seed the in-memory
|
||||
index, `read_tree(HEAD)` materialises all 1179 entries and reads every tree object
|
||||
from the 570 MB pack — **77 s cold** (447 ms only once the windows are resident).
|
||||
`write_tree_to` is O(changed), but you pay O(N_tree) to build the cache it needs, so
|
||||
the index-free path only trades a 611 s hash for a 77 s tree-read. Worse, that
|
||||
`read_tree` drove the `esp_map.c` cache to **7.4 MB resident** — past its own 4 MB
|
||||
soft cap — which left 508 KB of heap and made `repo.blob()`'s zlib `deflateInit`
|
||||
fail. **The one write we cannot skip crashed.** Root cause of the OOM: our cache
|
||||
holds pack windows *after* libgit2 `p_munmap`s them (refcount 0, freed only lazily
|
||||
on the next `p_mmap`), which **defeats `GIT_OPT_SET_MWINDOW_MAPPED_LIMIT`** —
|
||||
libgit2 thinks it released the memory; we didn't.
|
||||
|
||||
**Conclusion — use an O(depth) TreeBuilder walk.** "Replace K files in a 1179-file
|
||||
tree" should touch `O(depth × K)` objects, not `O(N_tree)`. Walk HEAD's tree down
|
||||
the edited path (`tree.get_name`/`get_path` → read ~depth subtree objects), then
|
||||
rebuild bottom-up with `repo.treebuilder(Some(&subtree))` → `insert`/`remove` →
|
||||
`write()`, and `commit` the new root. That never materialises the 1179 entries,
|
||||
never re-hashes anything, never visits the 150 MB of images, reads only a handful
|
||||
of tree windows (so the cache stays small and zlib keeps its heap), and — crucially
|
||||
— **carries the image entries forward untouched from HEAD's tree, so the device
|
||||
does not even need the images in its working tree.** The `esp_map.c` cache still
|
||||
needs an evict-on-`munmap` fix (drop the cap, free past a low-water mark) so it can
|
||||
never again starve a downstream `git__malloc`, but with the TreeBuilder walk the
|
||||
pressure it was under largely disappears.
|
||||
|
||||
### Splice bench (2026-07-12, second real-repo run) — the walk is right, the loose-object write is the new wall
|
||||
|
||||
The O(depth) splice op was added to `git_bench` (1 blob + 3 tree writes onto the
|
||||
depth-3 path `.claude/commands/bsky.md`, run FIRST so its first iteration is
|
||||
cold; the index ops moved last so their OOM can't cost the new data — it did
|
||||
crash again, after everything was logged):
|
||||
|
||||
| op | result | reading |
|
||||
| --- | ---: | --- |
|
||||
| `splice stage→tree` (1 blob + 3 trees) | **6.5 s p50, warm ≈ cold** | O(depth) confirmed — cost is 4 loose writes × ~1.6 s |
|
||||
| `commit(None)` orphan obj | 1.7 s p50 | one more loose write |
|
||||
| `odb.write(blob)` | **1.5 s p50** | ⚠️ was 142 ms in the previous run |
|
||||
| `repo.index()` load | 524 ms max | matches previous run |
|
||||
| seed `read_tree(HEAD)` cold (now timed) | 81.6 s | reproduces the 77 s |
|
||||
| `index-free stage→tree` | 💥 crash, 508 KB heap | reproduces the zlib OOM exactly |
|
||||
|
||||
Three readings:
|
||||
|
||||
1. **The splice mechanism is validated as a mechanism.** Pack reads stayed flat
|
||||
(~40 KB per write; 6.4 MB heap free through splice + commit + odb.write), so
|
||||
it really is O(depth) and it cannot OOM. The 6.5 s is not tree-walk cost.
|
||||
2. **The wall moved to the loose-object write: ~1.5 s each, ×4 per splice.**
|
||||
The isolated `odb.write(blob)` — one tiny orphan blob — took 1.5 s where the
|
||||
raw FAT composite is 86 ms. Projected full commit (splice 6.5 s + commit-obj
|
||||
1.7 s + ref/reflog update) ≈ **8–9 s**: enormously better than 611 s, still
|
||||
far off the bar.
|
||||
3. **The mmap cache scored 0 hits over the entire run** — the documented
|
||||
862→142 ms `odb.write` win did **not reproduce** (same `esp_map.c`, same
|
||||
card). Either the earlier run's conditions differed (orphan-object
|
||||
population? FAT allocation state?) or the win was misattributed. Whatever the
|
||||
1.5 s is, it is *not* SD data volume: each write moves ~40 KB read + ~1 KB
|
||||
written.
|
||||
|
||||
### ROOT CAUSE FOUND (2026-07-12, `sd_bench` seek op): FatFS lseek walks the cluster chain
|
||||
|
||||
Two `sd_bench` re-runs on the ~740 MB-full card settled it:
|
||||
|
||||
1. **Free-cluster-scan hypothesis: refuted.** Raw FAT write ops are unchanged on
|
||||
the full card — loose-object composite **77 ms p50** (was 86 ms), create
|
||||
20 ms, rename 10 ms. The card is exonerated a second time.
|
||||
2. **Long seeks are the cost.** A new op opens the repo's largest packfile
|
||||
(263 MB — the "570 MB pack" was actually the whole `.git`) read-only and does
|
||||
seek+read(4 KB): **@offset 0 = 5.8 ms; @end = 198.7 ms** — dead constant
|
||||
across 20 iters. Without `CONFIG_FATFS_USE_FASTSEEK`, FatFS resolves lseek by
|
||||
walking the file's FAT cluster chain over SPI: forward from the current
|
||||
position, **from the chain head on any backward seek**. 263 MB ≈ 16.8k
|
||||
clusters ≈ ~67 KB of FAT reads ≈ ~190 ms per long walk.
|
||||
|
||||
**Why FAT behaves this way:** FAT has no extent map or inode — a file is a
|
||||
singly-linked list of clusters, and the only way to find "byte 260,000,000" is
|
||||
to follow that list entry by entry through the allocation table. FatFS walks
|
||||
forward from the current position when it can, but a backward seek restarts
|
||||
from the chain head ([FatFS `f_lseek` docs, elm-chan.org](http://elm-chan.org/fsw/ff/doc/lseek.html)).
|
||||
The fast-seek feature fixes exactly this: a pre-computed **cluster link map
|
||||
table (CLMT)** per file object, "(fragments + 1) × 2" words, after which "no
|
||||
FAT access is occured in subsequent f_read/f_write/f_lseek" (same page). On
|
||||
esp-idf it's `CONFIG_FATFS_USE_FASTSEEK` — the official docs recommend it "for
|
||||
read-heavy workloads with long backward seeks" and note it does not apply to
|
||||
files opened in write mode
|
||||
([ESP-IDF FatFS docs](https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/storage/fatfs.html)).
|
||||
|
||||
The budget closes: each loose write does ~8–10 small (~4 KB) `p_mmap`s (freshen
|
||||
→ trailer/idx probes) interleaved with low-offset reads, so ~8 of them pay a
|
||||
fresh ~190 ms walk → **~1.5 s per object**. It also explains everything the
|
||||
cache couldn't: warm ≈ cold (the walk is paid inside `lseek` before any data
|
||||
moves, and the maps are below the 64 KB cache floor), the 142 ms vs 1.5 s
|
||||
run-1/run-2 discrepancy (run 1's `odb.write` bench ran first and hammered only
|
||||
the trailer — the file position stayed there, so its seeks were forward/no-ops),
|
||||
and a large slice of the 81.6 s `read_tree` (133 windows × backward seeks ≈ 25 s
|
||||
of walking on top of the 25 MB of data).
|
||||
|
||||
**Fix (config, not code): `CONFIG_FATFS_USE_FASTSEEK=y` +
|
||||
`CONFIG_FATFS_FAST_SEEK_BUFFER_SIZE=256`** (landed in `sdkconfig.defaults`
|
||||
2026-07-12). Fast seek builds an in-memory cluster-link map per read-mode file —
|
||||
exactly how the pack is opened — making lseek O(1); write-mode files fall back
|
||||
to the walk transparently. 256 words = 1 KB per open read-only file, covering
|
||||
~127 fragments (default 64 covers ~31; a fragmented pack would silently fall
|
||||
back to slow seeks, so the headroom matters).
|
||||
|
||||
**A/B measured (same evening): a 2.3× partial win, not a full one.**
|
||||
|
||||
| op | fast-seek off | fast-seek on |
|
||||
| --- | ---: | ---: |
|
||||
| `splice stage→tree` | 6.5 s | **2.81 s** |
|
||||
| `odb.write(blob)` | 1.5 s | **416 ms** |
|
||||
| `commit(None)` | 1.7 s | **1.72 s — unchanged** |
|
||||
|
||||
`odb.write` dropped by almost exactly the ~6 chain walks the model predicted —
|
||||
the seek theory holds — but two residuals remain: **~400 ms per loose write**
|
||||
(vs the 77 ms raw-FAT floor) and **`commit(None)`'s ~1.3 s premium over a plain
|
||||
write, which was never seek-bound at all**. Prime suspect for the commit
|
||||
premium: strict object creation makes `git_commit_create` validate its parent +
|
||||
tree OIDs with pack header resolves, and `git_treebuilder_insert` does the same
|
||||
per inserted entry — `git_bench` grew `odb.read_header(packed)` /
|
||||
`odb.exists(missing)` probes and strict-off re-benches to test it.
|
||||
|
||||
### Second localization round (2026-07-12, run 3b + sd_bench re-run)
|
||||
|
||||
**Fast-seek verified on the metal:** re-running the sd_bench seek op with
|
||||
`CONFIG_FATFS_USE_FASTSEEK=y` dropped `pack seek+read 4KB @end` from
|
||||
**198.7 ms → 20.4 ms** (the CLMT fits the pack in the 256-word buffer — the
|
||||
pack is not too fragmented). A far seek is now ~15 ms, i.e. effectively fixed.
|
||||
|
||||
**The strict-creation theory is refuted; the probes found the real unit cost:**
|
||||
|
||||
| op | p50 | reading |
|
||||
| --- | ---: | --- |
|
||||
| `odb.read_header(packed)` | **470 ms** | ONE pack header resolve costs ~½ s |
|
||||
| `odb.exists(missing)` | **968 ms** (±0.1 ms) | miss path (scan → refresh → rescan) ≈ 2× |
|
||||
| `commit(None)` strict OFF | 1.80 s | vs 1.93 s strict on — validation is NOT the premium |
|
||||
| `splice` strict OFF | 5.7 s | noise-worse; also not validation |
|
||||
|
||||
The ±0.1 ms constancy of `exists(missing)` = a fixed, deterministic SD-op
|
||||
sequence. The map counters identify it: **~7–8 small (~4 KB) `p_mmap` reads per
|
||||
op** — pack trailer probes, idx fanout reads and delta-base windows, repeated
|
||||
at the *same offsets* on every freshen/refresh. Post-fast-seek those cost
|
||||
~20 ms each (~150 ms/op); the rest of `read_header`'s 470 ms is CPU-side
|
||||
delta-chain inflation on the 160 MHz core plus repeated re-reads. Two other
|
||||
observations from run 3b: the loose-object orphan population from bench runs
|
||||
is creeping costs upward (splice 2.81 s → 3.21 s between consecutive
|
||||
fast-seek runs — a re-provision resets it), and the mmap cache STILL scored 0
|
||||
hits — because its 64 KB map-length floor excluded exactly these hot small
|
||||
maps.
|
||||
|
||||
**Fix built (esp_map.c v2):** cache admission re-keyed from
|
||||
map length to **file size ≥ 1 MB** — the pack/idx's small repeated windows now
|
||||
cache (RAM hits after first touch) while small mutable working-tree files stay
|
||||
excluded — plus **evict-on-`p_munmap` down to a 2 MB low-water mark**, fixing
|
||||
the 7.4 MB OOM from the first real-repo run (released windows are actually
|
||||
returned to `git__malloc`, so `MWINDOW_MAPPED_LIMIT` stays honest). Expected:
|
||||
`read_header` collapses toward CPU-only, `odb.write` toward ~150–250 ms,
|
||||
splice at or under the sub-second bar, and no end-of-run zlib OOM.
|
||||
|
||||
### Final bench (run 4, esp_map v2) — memory fix works, cache theory dead, bar failed
|
||||
|
||||
Run 4 (2026-07-12 evening, same card state as run 3b plus its orphans):
|
||||
|
||||
| op | run 3b (fast-seek) | run 4 (+ esp_map v2) |
|
||||
| --- | ---: | ---: |
|
||||
| `splice stage→tree` (cold, first op) | 2.81 s | **2.83 s — unchanged** |
|
||||
| `splice` again (warm, strict-off phase) | 3.21 s | 1.95 s |
|
||||
| `commit(None)` | 1.72 s | 713 ms |
|
||||
| `odb.write(blob)` | 416 ms | 366 ms |
|
||||
| `odb.read_header(packed)` | 470 ms | 412 ms |
|
||||
| `odb.exists(missing)` | 968 ms | 852 ms |
|
||||
| mmap-cache hits | 0 | **0** (313 misses) |
|
||||
| cache resident / heap free | grew to 7.4 MB → zlib OOM | **1833 KB flat / 6.4 MB free all run** |
|
||||
|
||||
Three findings:
|
||||
|
||||
1. **The memory discipline is verified.** Resident sits at 1833 KB through
|
||||
every phase (under the 2 MB low-water, so nothing is being churned) and
|
||||
heap never drops below 6.2 MB. The one uncaptured datum is the index-free
|
||||
`read_tree` tail (the section that OOM'd runs 1–3) — the monitor was cut
|
||||
before it ran. Not blocking: the shipping splice path never calls
|
||||
`read_tree`; the tail would only re-confirm eviction under burst.
|
||||
2. **The repeated-small-window theory is REFUTED — theory #3 down** (after
|
||||
strict-creation and free-cluster-scan). v2 demonstrably admits and retains
|
||||
the small maps now — the 1833 KB resident *is* them, held below low-water so
|
||||
nothing is evicted before reuse — and still scored 0 hits in 313 misses.
|
||||
So the ~8 small reads per loose write hit **unique (offset, len) every
|
||||
time**: `mwindow` was already absorbing any true repetition above `p_mmap`,
|
||||
and what reaches the emulation layer is distinct data (different objects,
|
||||
different delta bases). A window cache cannot help. The residual
|
||||
~360 ms/loose-write ≈ 8 distinct small SD round-trips × ~45 ms each
|
||||
(post-fast-seek) — I/O count, not I/O size or seek cost.
|
||||
3. **Within-run drift cuts both ways, so cross-run tables are mushy.** In this
|
||||
single run `commit(None)` degraded 713 ms → 1.79 s between the early and
|
||||
late (strict-off) phases, while splice *improved* 2.83 → 1.95 s. Two
|
||||
competing effects: first-touch warm-up fading (CLMT build, first pack
|
||||
reads — helps later ops) and orphan loose objects accumulating in
|
||||
`.git/objects/xx/` slowing every freshen existence check (FAT directory
|
||||
lookups are linear scans — hurts later ops). Steady-state on a clean
|
||||
objects dir ≈ **~2 s per splice+commit**.
|
||||
|
||||
**Run 5 (confirmation, cache removed entirely):** esp_map.c stripped back to
|
||||
plain malloc-read/free-at-munmap (stats counters kept). Read pattern
|
||||
**byte-identical** to run 4 at every checkpoint (118 maps / 2314 KB after
|
||||
splice, 148/2434, 163/2494, 208/2674) — except the strict-off phase did **15
|
||||
fewer reads (~60 KB)** than v2: the low-water eviction had been kicking out
|
||||
buffers mwindow still wanted, forcing re-reads. The cache was marginally
|
||||
worse than nothing. Warm splice identical (1953 vs 1949 ms); the cold-op
|
||||
+10–15 % drift (splice 2.83 → 3.26 s) is the known orphan-creep signature,
|
||||
not the removal. Run 5 also reframes run 4's "resident": **1854 KB `live` is
|
||||
mwindow's open-window working set** (pinned mappings under the bench's 4 MB
|
||||
mapped limit), not retained cache buffers — the cache had been retaining
|
||||
essentially nothing. Removal CONFIRMED free; simpler emulation ships.
|
||||
|
||||
**Verdict: the sub-second bar FAILED — wire the splice in anyway.** The bar
|
||||
was aspirational; measured reality is ~2–2.8 s to commit on the real
|
||||
263 MB-pack repo versus 611 s (or a hard OOM) for every alternative benched.
|
||||
That puts a full real-repo `:sync` at roughly **9–10 s cold**, which ships.
|
||||
The remaining ~2 s has survived four localization rounds; the next suspect —
|
||||
FAT *directory-op* cost in the freshen/refresh path (open/stat/rename by path
|
||||
walk FAT directories linearly; consistent with the orphan-creep signal) — is
|
||||
one instrumentation pass for later (log each `p_mmap` miss + bench dir ops in
|
||||
`sd_bench`), not a prerequisite for the plumbing.
|
||||
|
||||
### The walk is ~1.4 s even at N ≈ 2
|
||||
|
||||
Mostly fixed cost — the worktree-diff setup and the second (`update_all`) pass —
|
||||
not per-file `stat` (one raw `stat` is ~5 ms, so N ≈ 2 can't be the 1.4 s). The
|
||||
O(N) slope only bites on the real `jcalixte/notes` clone (N ≈ 1179), which this run
|
||||
did **not** exercise. That slope is still unmeasured.
|
||||
|
||||
For orientation: `publish(commit+push)` was 9846 ms cold, so the **network half is
|
||||
~6.5 s** — still the biggest single block of a warm sync (10.1 s total), a separate
|
||||
floor ([`../notes/sync-latency.md`](../notes/sync-latency.md)).
|
||||
|
||||
## The verdict
|
||||
|
||||
The real-repo run (above) overturned the earlier ranking. Both index strategies
|
||||
are O(N_tree) and fail on the 570 MB-pack clone, and the repo cannot be shrunk. The
|
||||
work, ranked:
|
||||
|
||||
1. **Rewrite the commit as an O(depth) TreeBuilder walk (the fix — build this).**
|
||||
Rebuild only the edited path's ancestor subtree chain onto HEAD's tree; never
|
||||
materialise the 1179-entry index, never `index.write()`, never `read_tree` the
|
||||
whole tree. This is the ONLY mechanism that fits: O(depth × dirty) reads/writes,
|
||||
flat in repo size, small heap, images carried forward untouched. Replaces
|
||||
`stage_and_commit`'s `add_all`/`update_all`/`index.write`/`write_tree`. Needs the
|
||||
editor's dirty set (+ deleted set) plumbed to the git service — the editor
|
||||
already knows both. **Benched to completion 2026-07-12: 6.5 s → 2.8 s cold /
|
||||
~2 s steady-state after the fast-seek fix (run 4). The sub-second bar failed
|
||||
but the block is lifted — wire it in** (see the final-bench section above:
|
||||
the residual is unique small SD round-trips, not something a cache or seek
|
||||
fix can remove).
|
||||
2. **Fix the `esp_map.c` cache so it can't OOM — RESOLVED BY REMOVAL (run 5).**
|
||||
The cache never scored a hit in four instrumented real-repo runs
|
||||
(`mwindow` absorbs true repetition above `p_mmap`; only new ranges reach
|
||||
the emulation), and the 7.4 MB OOM it was patched to avoid was caused by
|
||||
the cache itself holding buffers past `p_munmap`. esp_map.c is now the
|
||||
plain malloc-read/free-at-munmap emulation: honest with
|
||||
`MWINDOW_MAPPED_LIMIT` by construction, ~120 lines lighter, and run 5
|
||||
confirmed removal is I/O-neutral (even 15 reads *better* than v2, whose
|
||||
low-water eviction fought mwindow). Stats counters kept to spot any future
|
||||
workload that does repeat ranges.
|
||||
3. **Retired: `add_all`/explicit-path *index* staging.** Explicit-path `add_path`
|
||||
still goes through the index and `index.write` → `truncate_racily_clean`, so it
|
||||
hits Wall 1 just the same. The TreeBuilder walk supersedes it entirely; the
|
||||
"explicit-path staging" idea survives only as "the editor's dirty set feeds the
|
||||
walk."
|
||||
4. **Retired: SD clock / better card.** The card does a full object write in
|
||||
~86 ms; raw I/O is not the bottleneck. Do not spend the PCB's 20 MHz budget
|
||||
expecting a commit-latency win.
|
||||
5. **Kept: the mmap cache + mwindow tuning** (`GIT_OPT_SET_MWINDOW_*`, 256 KB
|
||||
window / 4 MB mapped limit). It fixed `odb.write` and the push read path; #2 just
|
||||
makes it well-behaved under memory pressure.
|
||||
|
||||
**Recommendation:** build #1 (the O(depth) TreeBuilder walk) — #2 resolved
|
||||
itself by removal. The concrete plumbing plan (exact call sites, dirty-set
|
||||
threading, FD budget) is the
|
||||
[next section](#the-fix--wiring-the-odepth-splice-into-the-firmware).
|
||||
|
||||
## The fix — wiring the O(depth) splice into the firmware
|
||||
|
||||
> Merged from `docs/notes/sync-commit-handoff.md` (written 2026-07-12, retired
|
||||
> 2026-07-13 once the bench phase closed). The handoff's bench half is done —
|
||||
> the splice op lives in `git_bench` and the numbers above are its output.
|
||||
> **The firmware plumbing below SHIPPED 2026-07-13** (compile-verified both
|
||||
> feature flavors; on-device `:sync` against the real repo still pending) —
|
||||
> each item now records how it landed rather than what to do.
|
||||
|
||||
### The splice walk
|
||||
|
||||
Rebuild only the edited file's ancestor subtree chain onto HEAD's tree. Never
|
||||
materialise the 1179-entry index; never `index.write()`; never `read_tree` the
|
||||
whole tree. Cost is O(depth × dirty_files), flat in repo size, tiny heap, and it
|
||||
carries every unchanged entry (all 260 images, the other 1176 files) forward
|
||||
untouched — which means **the device doesn't even need the images in its
|
||||
working tree.**
|
||||
|
||||
Shipped as `git_sync::splice` (git2 0.20: `Repository::treebuilder(Option<&Tree>)`
|
||||
+ `TreeBuilder::{insert,remove,write}`). Signature:
|
||||
|
||||
```rust
|
||||
fn splice(repo: &Repository, base: Option<&Tree>, path: &[&str], blob: Option<Oid>) -> Result<Oid>
|
||||
```
|
||||
|
||||
`Some(blob)` inserts/replaces, `None` removes; `base: None` synthesizes a
|
||||
missing intermediate directory on the way down, and a directory emptied by a
|
||||
remove is pruned on the way up (the empty tree is never re-inserted).
|
||||
`stage_and_commit` folds every dirty path through it (threading the running
|
||||
root tree), then `commit(Some("HEAD"), …)` exactly as before — the
|
||||
`tree unchanged → nothing to publish` check and the `commit split —` timing
|
||||
log survive. The benched reference is `git_bench`'s `splice stage→tree` op.
|
||||
|
||||
### `firmware/src/git_sync.rs` — how it landed
|
||||
|
||||
- **mwindow options set at service start** (top of `run_git_service`, before
|
||||
any `Repository::open`): `set_mwindow_size(256 KB)` +
|
||||
`set_mwindow_mapped_limit(4 MB)`. Without them the 32-bit defaults (32 MB
|
||||
window / 256 MB mapped limit, mwindow.c:16) would `git__malloc` a 32 MB
|
||||
window on the first pack access and die on the 8 MB PSRAM heap. Run 5
|
||||
sharpened the stakes: the ~1.85 MB bench "resident" **is** mwindow's live
|
||||
open-window set, so these opts are the only thing bounding it.
|
||||
- `stage_and_commit(repo, paths)` is the splice walk; `add_all`/`update_all`/
|
||||
`index.write`/`write_tree` are gone. **The request carries one path set, not
|
||||
`{changed, deleted}`:** the working tree is the source of truth, so at commit
|
||||
time a recorded path that exists on the card is spliced in from disk and a
|
||||
missing one is spliced out. An unchanged path is a no-op — over-reporting is
|
||||
free, which is what makes the retry/journal semantics below simple.
|
||||
- **Stranded-commit recovery (new):** when the splice yields the parent's tree
|
||||
(nothing to commit), the service now compares HEAD against
|
||||
`refs/remotes/origin/<branch>` and still pushes if origin lacks HEAD — a
|
||||
previous cycle that committed and then failed mid-push used to strand that
|
||||
commit forever (the old path reported "up to date" and never retried).
|
||||
- **Radio-free up-to-date (new):** an empty dirty set + origin already at HEAD
|
||||
short-circuits before Wi-Fi even comes up — `:sync` with nothing to do
|
||||
answers in ~150 ms instead of a Wi-Fi/TLS round.
|
||||
- `reconcile_onto_origin` now `ResetType::Soft` (ref move only) — there is no
|
||||
index to reset, and a Mixed reset's index write is exactly the racy-clean
|
||||
wall the splice avoids. Side win: a remote-only added file is now *carried*
|
||||
by the replay (origin's tree is the splice base) where the old `add --all`
|
||||
replay dropped it.
|
||||
- The macOS-cruft filter (`skip_macos_cruft`) is gone with the walk — the
|
||||
splice only touches paths the editor recorded, so `._*`/`.DS_Store` can't
|
||||
sneak in the way they once did (07d87772, the Spike-14-era lesson).
|
||||
- **Deliberate behavior change (now in effect):** only paths the editor
|
||||
saved/deleted are ever committed. Files changed on the card outside the
|
||||
editor (card mounted on a Mac) were swept in by `add_all` before; they will
|
||||
now never be committed, and the working tree will show a permanent diff
|
||||
against HEAD if inspected on a desktop. Correct for the appliance; recorded
|
||||
here so it reads as intent, not accident.
|
||||
|
||||
### Dirty-set source — `firmware/src/persistence.rs` + `main.rs` (how it landed)
|
||||
|
||||
- `Storage` owns the record: `save_path`/`delete_path` note their repo-relative
|
||||
path in a `RefCell<Dirty>` (paths outside `/sd/repo` are skipped), and the
|
||||
set is **journaled to `/sd/.typoena-dirty`** — atomic write, rewritten only
|
||||
when the set actually grows. Without the journal a power pull would strand
|
||||
every file saved-but-not-published that session: nothing walks the tree
|
||||
anymore, so an unrecorded change would never reach the remote. The journal
|
||||
is loaded at mount and its paths ride the next `:sync`.
|
||||
- Lifecycle: `take_dirty()` snapshots pending → in-flight for one publish
|
||||
(journal keeps carrying the union); the outcome settles it —
|
||||
`publish_succeeded()` forgets the snapshot and shrinks the journal,
|
||||
`publish_failed()` returns it to pending for the next `:sync`. A save landing
|
||||
*while* a publish runs re-enters pending and rides the next one. Recording
|
||||
happens *before* the file write, so a crash between the two only
|
||||
over-approximates (a no-op splice), never under-records.
|
||||
- `Effect::Publish` in `main.rs` sends `PublishRequest { paths: take_dirty() }`;
|
||||
the outcome handler in the idle branch calls the matching settle method.
|
||||
- **FD budget:** a git build now mounts `Storage::mount_for_git()` (16 FDs) in
|
||||
`boot_storage` — libgit2 keeps the pack + `.idx` descriptors open and opens
|
||||
loose objects on top, which overruns the editor's 4-FD budget. The light
|
||||
build keeps the editor's own budget.
|
||||
|
||||
### `esp_map.c` — nothing left to do
|
||||
|
||||
The handoff's third work item (an evict-on-`p_munmap` cache fix) is superseded:
|
||||
run 5 removed the cache outright and `esp_map.c` is already the plain
|
||||
malloc-read/free-at-munmap emulation (see verdict item 2). One surviving
|
||||
operational note: editing `components/libgit2/*.c` won't rebuild via plain
|
||||
`cargo build` — first
|
||||
`rm -rf firmware/target/xtensa-esp32s3-espidf/release/.fingerprint/esp-idf-sys-*`
|
||||
(the `esp-idf-component-rebuild` lesson).
|
||||
|
||||
### How to bench / flash
|
||||
|
||||
`git_bench.rs` runs git ops on the 96 KB `GIT_STACK` thread (the main task stack
|
||||
overflows on these ops — that's why the real service has a dedicated thread). It's
|
||||
Rust-only, so a plain rebuild picks it up (no fingerprint dance unless you also
|
||||
touched `esp_map.c`).
|
||||
|
||||
```
|
||||
just flash-gitbench
|
||||
# = . ~/export-esp.sh && LIBGIT2_SRC=<repo>/firmware/components/libgit2/vendor \
|
||||
# LIBGIT2_NO_VENDOR=1 PKG_CONFIG_ALLOW_CROSS=1 \
|
||||
# PKG_CONFIG_LIBDIR=<repo>/firmware/pkgconfig \
|
||||
# cargo run --release --bin git_bench --features git
|
||||
```
|
||||
|
||||
Bench on the **real repo** clone (`/sd/repo` = full `jcalixte/notes`), not the
|
||||
toy — the toy pack understates everything by ~2 orders of magnitude.
|
||||
|
||||
### Still open (none block the plumbing)
|
||||
|
||||
- The residual ~360 ms/loose-write ≈ 8 unique small SD round-trips; next
|
||||
suspect is FAT **directory-op** cost in the freshen/refresh path. One
|
||||
instrumentation pass for later (log each `p_mmap` miss + bench dir ops in
|
||||
`sd_bench`).
|
||||
- Ref/reflog-update cost on the real repo — the bench's `commit(None)` writes
|
||||
no ref, so the shipping commit's last leg is unmeasured.
|
||||
- The push's ~6.5 s network floor
|
||||
([`../notes/sync-latency.md`](../notes/sync-latency.md)) — a separate curve.
|
||||
|
||||
## Adjacent lever: should the images be on the card at all?
|
||||
|
||||
Explicit-path staging makes the walk skip the images, but they still cost 150 MB
|
||||
of SD space, inflate the 570 MB clone, and slow provisioning + the pull-before-push
|
||||
paths. Whether the device should carry image blobs at all — vs. markdown-only, or
|
||||
Git-LFS-style pointers — is a separate decision tracked in
|
||||
[`../notes/git-sync-images-and-repo-size.md`](../notes/git-sync-images-and-repo-size.md).
|
||||
That lever shrinks N and the clone; this one stops the walk from paying for N. They
|
||||
compose.
|
||||
|
||||
## What this does *not* touch
|
||||
|
||||
The network half of `:sync` (TLS handshake + push round-trips, ~6.5 s of the warm
|
||||
path) is a separate floor covered in [`sync-latency.md`](../notes/sync-latency.md);
|
||||
this curve is only about the local commit. Radio *frequency* (how often we pay any
|
||||
sync at all) is [`wifi-auto-sync.md`](wifi-auto-sync.md).
|
||||
147
docs/typoena-snippets.md
Normal file
147
docs/typoena-snippets.md
Normal file
@@ -0,0 +1,147 @@
|
||||
# `.typoena.snippets.json` — snippet library
|
||||
|
||||
> The git-tracked file that holds your trigger-driven text expansions for
|
||||
> Markdown authoring. Hand-editable (and Zed-compatible, so you can paste your
|
||||
> existing snippets straight in), synced across devices like your notes. Landed
|
||||
> in **v0.6** (see [`macroplan.md`](macroplan.md)). The editing surfaces — inline
|
||||
> Tab-expansion and the `$` palette — are specified in
|
||||
> [`v0.6-markdown.md`](v0.6-markdown.md).
|
||||
>
|
||||
> **Three files, three concerns, don't confuse them.** `.typoena.snippets.json`
|
||||
> is *content* (your templates). [`.typoena.toml`](typoena-toml.md) is *behaviour*
|
||||
> (auto-save, gutter). `/sd/typoena.conf` is *secrets* (Wi-Fi, PAT), gitignored
|
||||
> and never committed. The first two live in the repo and sync; the third is
|
||||
> per-device.
|
||||
|
||||
## Location
|
||||
|
||||
```
|
||||
/sd/repo/.typoena.snippets.json
|
||||
```
|
||||
|
||||
It sits in the Tracked repo beside [`.typoena.toml`](typoena-toml.md), so it is
|
||||
**committed and pushed** like any note and **syncs to every device** that clones
|
||||
the repo. Your snippet library follows you. It is read **once at boot**; a
|
||||
**missing, empty, or malformed file is fine** — you simply have no snippets, and
|
||||
the editor runs unchanged.
|
||||
|
||||
## Format
|
||||
|
||||
Deliberately **Zed's snippet JSON shape**, so the contents of a Zed
|
||||
`snippets/markdown.json` paste in unmodified:
|
||||
|
||||
```json
|
||||
{
|
||||
"Markdown link": {
|
||||
"prefix": "link",
|
||||
"body": "[$1]($2)$0",
|
||||
"description": "Inline link"
|
||||
},
|
||||
"Book notes": {
|
||||
"prefix": "fiche",
|
||||
"body": ["# $1", "", "## $2 — $3", "", "## What the book is about", ""],
|
||||
"description": "Fiche de lecture"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- The top-level key is the **display name** (what the `$` palette shows).
|
||||
- `prefix` — the word that triggers inline Tab-expansion.
|
||||
- `body` — a **string**, or an **array of lines** joined with `\n` (Zed's form;
|
||||
it sidesteps embedded-newline escaping and reads cleanly for multi-line
|
||||
templates).
|
||||
- `description` — optional but recommended: the `$` palette fuzzy-matches it and
|
||||
shows it, so it is how you find a snippet you don't remember the prefix for.
|
||||
|
||||
### Tab stops
|
||||
|
||||
A body is literal text plus numbered stops:
|
||||
|
||||
- `$1 … $n` — empty stops the caret visits in order.
|
||||
- `$0` — the final resting place (defaults to the end of the insertion if absent).
|
||||
- `${n:label}` — **accepted, but the label is stripped** to a bare `$n`. The
|
||||
editor has no selection/overtype model, so a label would just be text to
|
||||
delete; on a device with no completion popup it could never be shown as a
|
||||
prompt anyway. The **headings and structure carry the template** — the labels
|
||||
were only hints. This is what lets a Zed file with `${1:Titre}` load as-is.
|
||||
- **No dynamic or computed values** (no `date`, no `clipboard`). There is no RTC
|
||||
— the wall clock is valid only after Wi-Fi + SNTP, so a `date` snippet would
|
||||
stamp 1970 on a cold boot. A stop is empty or it is literal; nothing else.
|
||||
|
||||
## The two surfaces
|
||||
|
||||
Every snippet works both ways — there is **no hidden two-tier rule** where some
|
||||
are "inline only" and some are "palette only". Inline Tab is the fast path you
|
||||
reach for once a prefix is in muscle memory; the `$` palette is discovery.
|
||||
|
||||
### Inline Tab-expansion (Insert mode)
|
||||
|
||||
Type a prefix, press **Tab**. If the word immediately before the caret matches a
|
||||
snippet prefix, it expands; otherwise Tab inserts spaces as it does today. (Tab
|
||||
arrives as an ordinary character, so this is a check inside the Insert-mode
|
||||
handler, alongside the existing list-continuation transform.)
|
||||
|
||||
On a **typing pause** — the same throttle as the word-count / cursor refresh, so
|
||||
never a per-keystroke e-ink flash — if the word before the caret is a prefix, the
|
||||
right side panel shows a quiet hint (`» Book notes`), on the row above the mode
|
||||
line. The panel is ~15 columns, so the hint is the **snippet name**, not the whole
|
||||
body; the full preview is what the `$` palette is for. (The marker is `»`, not a
|
||||
tab glyph: the panel font is ISO-8859-15, which has no `↹`.)
|
||||
|
||||
### `$` palette (browse + insert)
|
||||
|
||||
Open the palette (`Cmd-P`) and type **`$`** — the same sigil mechanism as `>` for
|
||||
commands. The query after the `$` fuzzy-matches name, prefix, and description;
|
||||
`Ctrl-N`/`Ctrl-P` move the selection; **Enter inserts the body at the caret** and
|
||||
starts the tab-stop session (dropping you into Insert at `$1`). The empty-palette
|
||||
placeholder legends the sigils: `Go to file · > settings · $ snippets`.
|
||||
|
||||
## The tab-stop session
|
||||
|
||||
Identical whether the snippet was expanded inline or inserted from the palette:
|
||||
|
||||
- After insertion the caret lands on **`$1`** (or the end, if the body has no
|
||||
stops), in **Insert** mode.
|
||||
- **Tab advances** to the next stop, **forward only** (no Shift-Tab). The last
|
||||
Tab lands on `$0` / the end and ends the session.
|
||||
- Pending stop offsets sit **after the caret** and shift with the edits you make
|
||||
at each stop, so typing at `$1` keeps `$2 … $n` correctly placed.
|
||||
- The session **auto-aborts** on Esc, a mode change, or a motion that leaves the
|
||||
stop range — after which the buffer is just text and Tab inserts spaces again.
|
||||
|
||||
## Parsing
|
||||
|
||||
The parse lives in the host-testable `editor` crate (`Snippets::parse`), using
|
||||
`serde_json` — JSON string escapes (`\n`, `\"`, `\uXXXX`) are a foot-gun to
|
||||
hand-roll, and `serde_json` is battle-tested; the editor crate is `std`, so it
|
||||
compiles for xtensa via esp-idf. This is the **one new dependency** the feature
|
||||
adds. The firmware reads the file at boot and hands the parsed list to
|
||||
`Editor::set_snippets`, mirroring how `.typoena.toml` is read and applied via
|
||||
`set_prefs`. A parse error is **non-fatal**: log it and boot with no snippets,
|
||||
rather than refusing to start over a stray comma.
|
||||
|
||||
## Editing it
|
||||
|
||||
- **On your computer (the normal path).** It's plain JSON in your notes repo —
|
||||
edit it in your real editor, copy entries over from Zed, commit, and it reaches
|
||||
the device on the next clone/sync. This is deliberately where the heavy editing
|
||||
happens; the appliance is for writing, not for maintaining a JSON library.
|
||||
- **First-time setup.** [`just init`](../firmware/README.md#provisioning-an-sd-card)
|
||||
seeds this file from a curated catalog (`firmware/snippets-catalog/`) — you pick
|
||||
which snippet groups you want and it `jq`-merges the selected subset into
|
||||
`repo/.typoena.snippets.json` (committed on the device's first `:sync`). It
|
||||
writes **only if the file is absent**, so re-running `init` on a card whose
|
||||
clone already carries your library never overwrites it. See
|
||||
[`v0.6-markdown.md`](v0.6-markdown.md) for the catalog.
|
||||
- **On-device hand-edit — deferred.** The palette hides dotfiles, and `:e` was
|
||||
dropped in v0.6, so there is no in-editor path to this file yet. When one is
|
||||
wanted it returns as a discoverable `> edit snippets` command that opens the
|
||||
file directly, rather than resurrecting a general `:e`.
|
||||
|
||||
## See also
|
||||
|
||||
- [`v0.6-markdown.md`](v0.6-markdown.md) — the editing surfaces, the `$`/`>`
|
||||
palette model, and the setup-recipe snippet catalog.
|
||||
- [`typoena-toml.md`](typoena-toml.md) — the sibling prefs file this is kept
|
||||
separate from, and the `>` command palette snippets share the surface with.
|
||||
- [`macroplan.md`](macroplan.md) — v0.6 scope.
|
||||
176
docs/typoena-toml.md
Normal file
176
docs/typoena-toml.md
Normal file
@@ -0,0 +1,176 @@
|
||||
# `.typoena.toml` — editor preferences
|
||||
|
||||
> The git-tracked file that controls how the editor behaves — auto-save,
|
||||
> format-on-save, the line-number gutter, and the panel theme. Hand-editable, or
|
||||
> changed live from the `Cmd-P` palette (booleans flip; the theme and auto-sync
|
||||
> interval rotate through preset options on **Enter**). Landed in **v0.5** (see
|
||||
> [`macroplan.md`](macroplan.md)).
|
||||
>
|
||||
> **Not to be confused with `/sd/typoena.conf`** — that holds the device
|
||||
> *secrets* (Wi-Fi, PAT, remote URL, commit author), is gitignored, and is never
|
||||
> committed. `.typoena.toml` is *behaviour*, shared across devices; `typoena.conf`
|
||||
> is *secrets*, per-device. See [v0.1 product](v0.1-mvp-product.md).
|
||||
|
||||
## Location
|
||||
|
||||
```
|
||||
/sd/repo/.typoena.toml
|
||||
```
|
||||
|
||||
It lives inside the Tracked repo (`/sd/repo`), so it is **committed and pushed**
|
||||
like any note — which means the preferences **sync to every device** that clones
|
||||
the repo. That is deliberate: your editor behaviour follows you. (A per-device
|
||||
override for the one genuinely device-specific key, `auto_sync`, may layer on top
|
||||
later via `typoena.conf` — worth it only once `auto_sync` actually does something
|
||||
in v0.7. See the [auto_sync](#auto_sync) note.)
|
||||
|
||||
The file is read **once at boot**, before the first screen is drawn (so
|
||||
`line_numbers` shapes the opening frame). A **missing, empty, or partial file is
|
||||
fine** — every absent key falls back to its default below, so a fresh card just
|
||||
works with no config present.
|
||||
|
||||
## Keys
|
||||
|
||||
| Key | Type | Default | Options | Effect |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `save_on_idle` | bool | `true` | `true` / `false` | Auto-save the current buffer on the idle typing-pause, so `:w` is optional. |
|
||||
| `format_on_save` | bool | `true` | `true` / `false` | Run `:fmt` on the buffer before an explicit `:w`/`:sync`. |
|
||||
| `line_numbers` | bool | `true` | `true` / `false` | Show the absolute line-number gutter. Off reclaims its columns for text. |
|
||||
| `theme` | string | `"light"` | `light` / `dark` | Panel colour polarity. `dark` inverts the whole frame to white-on-black. |
|
||||
| `auto_sync` | string | `"10m"` | `2m` / `5m` / `10m` / `15m` / `30m` | Max-staleness cap for opportunistic auto-publish. **Value only — no behaviour yet** (rides v0.7). |
|
||||
|
||||
The **Options** column is what the palette rotates through on **Enter**; a
|
||||
boolean is just the two-option case. Hand-editing a string key can still set any
|
||||
value — the palette only cycles the presets.
|
||||
|
||||
### Example
|
||||
|
||||
```toml
|
||||
# Typoena editor preferences — hand-editable, git-tracked.
|
||||
# Edit here, or change live from the Cmd-P palette (type `>`).
|
||||
save_on_idle = true
|
||||
format_on_save = true
|
||||
line_numbers = true
|
||||
theme = "light"
|
||||
auto_sync = "10m"
|
||||
```
|
||||
|
||||
### `save_on_idle`
|
||||
|
||||
When on, the firmware quietly persists a dirty, named buffer once typing has
|
||||
paused (~1.5 s), so a power pull can't cost more than the last couple of seconds
|
||||
of writing. It is a **safety net, not an action**:
|
||||
|
||||
- **Silent.** No snackbar, no forced screen refresh. A visible confirmation on
|
||||
every pause would cost a ~630 ms e-ink flash purely to say "saved" — exactly
|
||||
the gratuitous flashing the panel avoids elsewhere. `:w` remains the *loud*
|
||||
save (it posts `saved`).
|
||||
- **Unformatted.** The idle save never runs `:fmt` — see the
|
||||
[format_on_save](#format_on_save) note for why.
|
||||
- Fires **once per typing burst**; a failed save doesn't retry-storm (it's kept
|
||||
in RAM and re-attempted on the next burst, or on `:w`).
|
||||
|
||||
### `format_on_save`
|
||||
|
||||
Runs `:fmt` — table alignment, blank-line collapse, trailing-whitespace strip —
|
||||
on the buffer *before* it is persisted, so `:sync` is **fmt → save → commit →
|
||||
push** and `:w` saves formatted.
|
||||
|
||||
**Formatting only happens on an explicit `:w`/`:sync`.** The `save_on_idle`
|
||||
auto-save is deliberately left unformatted: if it reformatted on every idle
|
||||
pause, tables would reflow and blank lines collapse *mid-session*, with the caret
|
||||
jumping under you every time you paused to think. Formatting is a deliberate act;
|
||||
the safety-net save is not.
|
||||
|
||||
### `line_numbers`
|
||||
|
||||
Shows the absolute line-number gutter (built always-on in v0.2). Turning it off
|
||||
returns the gutter's columns to the text, so prose gets the full writing width.
|
||||
Applied **live** — toggling it from the palette redraws immediately with (or
|
||||
without) the gutter.
|
||||
|
||||
### `theme`
|
||||
|
||||
Panel colour polarity: `light` (the native black-ink-on-white-paper) or `dark`
|
||||
(white-on-black). On the 1-bit e-paper panel this is not a palette swap but a
|
||||
**whole-frame invert** applied at the very end of the render, so text, selection,
|
||||
caret, side panel and command palette all flip together and each stays legible.
|
||||
Any value other than `dark` reads as light. Applied **live** — cycling it from
|
||||
the palette repaints inverted at once.
|
||||
|
||||
> **On e-paper, `dark` is not free.** Partial refreshes over a mostly-black field
|
||||
> ghost more than over white, and the panel is tuned for white-background reading.
|
||||
> It works, but expect a slightly muddier refresh than `light` — verify on-device.
|
||||
|
||||
### `auto_sync`
|
||||
|
||||
A duration string that will one day cap how stale the published copy is allowed
|
||||
to get — an *opportunistic, rate-limited* push, not a wall-clock timer. The
|
||||
palette rotates it through the presets `2m` / `5m` / `10m` / `15m` / `30m`
|
||||
(hand-editing can still set any string, e.g. `"0"`/empty to disable). **The value
|
||||
is only stored and displayed in v0.5 — nothing reads it yet:** the periodic push
|
||||
rides the better-git work in v0.7 and must interact with sleep in v0.8, so
|
||||
cycling the interval today changes what will be honoured *then*, not now.
|
||||
Rationale for the `"10m"` default:
|
||||
[`tradeoff-curves/wifi-auto-sync.md`](tradeoff-curves/wifi-auto-sync.md).
|
||||
|
||||
## Editing it
|
||||
|
||||
Two ways, both landing in the same file:
|
||||
|
||||
1. **By hand** — it's plain text on the card; edit it on your computer and reboot
|
||||
to apply. (The palette hides dotfiles, but you can still open it in-editor with
|
||||
`:e repo/.typoena.toml`.)
|
||||
2. **Live, from the device** — open the settings list either way:
|
||||
- **`:settings`** — drops you straight into it, or
|
||||
- **`Cmd-P`** then type **`>`** — switches the file palette to the command
|
||||
list (VS Code semantics).
|
||||
|
||||
Every pref appears carrying its current state:
|
||||
|
||||
```
|
||||
> save on idle: on
|
||||
format on save: on
|
||||
line numbers: on
|
||||
theme: light
|
||||
auto sync: 10m
|
||||
```
|
||||
|
||||
`Ctrl-N`/`Ctrl-P` move the selection; **Enter** advances the selected pref to
|
||||
its next value, applies it at once, writes the change back to `.typoena.toml`,
|
||||
and confirms the new state on the snackbar (e.g. `theme: dark - saved`). A
|
||||
boolean flips; the theme and auto-sync interval **rotate through their preset
|
||||
options and wrap** — same key, so the palette is uniformly "press Enter to
|
||||
change". **The list stays open** so you can change several prefs in a row;
|
||||
**Esc** (or `Cmd-P`) closes it. Each change rides the next `:sync` to your
|
||||
other devices.
|
||||
|
||||
`auto_sync` is a value command now, but has no behaviour to drive until v0.7 —
|
||||
cycling it sets the interval that the future periodic push will honour.
|
||||
|
||||
## Parsing
|
||||
|
||||
The reader is a deliberately tiny **line-based** parser, not a general TOML
|
||||
library — the file is flat `key = value` pairs (a bool, or a quoted string) with
|
||||
`#` comments, so a full TOML crate isn't worth pulling onto the firmware build.
|
||||
It lives in the host-testable `editor` crate (`Prefs::parse` / `Prefs::to_toml`).
|
||||
Rules:
|
||||
|
||||
- A `#` starts a comment to end of line (whole-line or trailing).
|
||||
- Blank lines and lines without `=` are ignored.
|
||||
- An **unrecognized key** is ignored; an **unparseable value** (e.g.
|
||||
`save_on_idle = yes`) leaves *that key* at its default rather than reading as
|
||||
`false`.
|
||||
- Any key not present falls back to its default, so partial files are valid.
|
||||
|
||||
Because `Prefs::to_toml` round-trips with `Prefs::parse`, a palette edit rewrites
|
||||
the whole file in canonical form (with the header comment) — hand-added comments
|
||||
elsewhere in the file are not preserved across a palette toggle.
|
||||
|
||||
## See also
|
||||
|
||||
- [`macroplan.md`](macroplan.md) — v0.5 scope and the decisions behind these keys.
|
||||
- [`v0.1-mvp-product.md`](v0.1-mvp-product.md) — the `typoena.conf` device secrets
|
||||
this file is kept separate from.
|
||||
- [`tradeoff-curves/wifi-auto-sync.md`](tradeoff-curves/wifi-auto-sync.md) — why
|
||||
`auto_sync` defaults to 10 minutes.
|
||||
39
docs/v0.2-navigation.md
Normal file
39
docs/v0.2-navigation.md
Normal file
@@ -0,0 +1,39 @@
|
||||
# v0.2 — Vim navigation
|
||||
|
||||
> Part of the [Typoena macro plan](macroplan.md). Requirements and targets:
|
||||
> [qfd.md](qfd.md). Load-bearing decisions: [adr.md](adr.md).
|
||||
|
||||
**Status:** COMPLETE 2026-07-11. Navigation done in core; the **UTF-8-correct
|
||||
buffer** and **`Ctrl-d/u` half-page scroll** landed and are hardware-verified,
|
||||
and the **absolute line-number gutter** is built, host-tested, and **confirmed
|
||||
on the panel (Spike 13) 2026-07-11** — a single-line edit repaints only the rows
|
||||
at/below the change and forces no extra full refresh. Shipped early beyond scope:
|
||||
a read-only **View** mode and the full `d`/`c` operator + text-object grammar
|
||||
(see [v0.3](v0.3-editing.md) / [v0.4](v0.4-visual-and-ex.md)).
|
||||
|
||||
- [x] Mode state machine (Normal / Insert / View), mode indicator in the status strip
|
||||
- [x] Movement: `h j k l`, `w b e`, `0 $`, `gg G`, `Ctrl-d Ctrl-u`. `Ctrl-d/u`
|
||||
step **display** (soft-wrapped) rows, not logical lines — half a page is
|
||||
half the visible window however prose wraps; decoded as `HalfPageDown/Up`
|
||||
intents in the keymap, caret moves and the viewport follows.
|
||||
- [x] `i a o O A` to enter Insert
|
||||
- [x] `Esc` returns to Normal
|
||||
- [x] Line numbers in the left gutter: **absolute**, built + host-tested
|
||||
2026-07-11, **confirmed on the panel (Spike 13) 2026-07-11** — numbered on a
|
||||
logical line's first display row, blank on wrapped continuation rows; the
|
||||
gutter width tracks the buffer's line count (2 digits + separator, widening
|
||||
past 99 lines) and steals its columns from the soft-wrap. **Always on** in
|
||||
v0.2; the on/off toggle rides the [v0.5](v0.5-palette-and-multi-file.md)
|
||||
`.typoena.toml` prefs.
|
||||
Relative numbering was dropped (2026-07-11): renumbering the whole gutter on
|
||||
every `j`/`k` burns the e-ink ghosting budget for no proportionate gain,
|
||||
whereas absolute renumbers only the rows below an edit — the on-panel check
|
||||
confirmed a single-line edit repaints only rows at/below it with no extra
|
||||
full refresh.
|
||||
- [x] Groundwork — UTF-8-correct buffer: caret motions and edits step by
|
||||
character, not byte (dropped the ASCII == byte-offset assumption), so every
|
||||
motion stays correct with accented input. **Done 2026-07-11** alongside
|
||||
extracting the editor into a host-testable crate — char-step
|
||||
motions/deletes, byte-vs-char split in `layout`/`caret_rc`, `word_end`/`de`
|
||||
fixed; 15 host tests. Render font is ISO-8859-15 (Latin-9), so accented
|
||||
glyphs display.
|
||||
25
docs/v0.2.5-international-input.md
Normal file
25
docs/v0.2.5-international-input.md
Normal file
@@ -0,0 +1,25 @@
|
||||
# v0.2.5 — International input
|
||||
|
||||
> Part of the [Typoena macro plan](macroplan.md). Requirements and targets:
|
||||
> [qfd.md](qfd.md). Load-bearing decisions: [adr.md](adr.md).
|
||||
|
||||
**Status:** DONE in core, **hardware-verified 2026-07-11** (typed ç é è ñ on the
|
||||
bench, no crash). US-International dead-key accent composition lives in the
|
||||
`keymap` crate — a `Composer` downstream of the decoder — wired into
|
||||
`usb_kbd.rs` so the editor still receives a single `Key::Char`. Builds on the
|
||||
[v0.2](v0.2-navigation.md) UTF-8-correct buffer and the ISO-8859-15 render font.
|
||||
Host-tested.
|
||||
|
||||
- [x] Dead keys — grave, acute, circumflex, diaeresis, tilde — compose with
|
||||
the next letter: à é ê ë ñ, ç (via `'`+c), both cases
|
||||
- [x] `'`+space emits a literal apostrophe (the everyday apostrophe path); a
|
||||
dead key followed by a non-composing letter emits the accent then the
|
||||
letter
|
||||
- [x] A non-character event (Enter, Backspace, arrows) flushes any pending
|
||||
accent as its literal first
|
||||
- [ ] ~~Pending-accent indicator in the side-panel status strip~~ — **DROPPED
|
||||
(2026-07-11 decision):** at typing speed it would be stale before the
|
||||
~630 ms panel repaint, so it conveys nothing. Left unbuilt on purpose.
|
||||
- [x] Bonus (2026-07-11): the physical **Esc key** (HID 0x29) now types
|
||||
`` ` ``/`~` — Esc comes from the Caps tap — so grave/tilde accents and
|
||||
Markdown code fences are reachable on a 60% board without a Fn layer.
|
||||
30
docs/v0.3-editing.md
Normal file
30
docs/v0.3-editing.md
Normal file
@@ -0,0 +1,30 @@
|
||||
# v0.3 — Vim editing
|
||||
|
||||
> Part of the [Typoena macro plan](macroplan.md). Requirements and targets:
|
||||
> [qfd.md](qfd.md). Load-bearing decisions: [adr.md](adr.md).
|
||||
|
||||
**Status:** COMPLETE in core 2026-07-11, host-tested (65 editor + 28 keymap
|
||||
tests) and **partially smoke-tested on the panel 2026-07-11**. The three
|
||||
remaining pieces landed together: a single unnamed **register** with
|
||||
`y`/`yy`/`p`/`P` (and `x`/`d`/`c` filling it, so `dd`…`p` moves a line),
|
||||
**undo/redo** (`u`/`Ctrl-r`, snapshot-based, bounded to 100 groups in PSRAM — a
|
||||
whole Insert session undoes as one group), and **`.` repeat** (keystroke-recorded,
|
||||
so it replays an insert session like `ciwfoo<Esc>`). The `d`/`c` operator grammar
|
||||
and text objects had already landed ahead of schedule. On device, `dd`, `yy`, and
|
||||
`Ctrl-r` confirmed good; the one issue found was that a **multi-line paste near
|
||||
the bottom left its later lines below the fold** — `adjust_scroll` only kept the
|
||||
caret's (first) pasted line visible. Fixed by a `reveal()` that scrolls the end of
|
||||
the pasted block into view while the caret stays on its first line (reflash to
|
||||
re-confirm on panel).
|
||||
|
||||
- [x] `x dd`, `dw dd d$` (✓); `yy p P` (✓) and `.` repeat (✓) — register + a
|
||||
keystroke-recorded last-change both landed 2026-07-11
|
||||
- [x] Undo / redo (`u`, `Ctrl-r`) — snapshot history bounded to 100 groups in
|
||||
PSRAM; one Insert session = one undo group
|
||||
- [x] Numeric prefixes (`3dd`, `5j`)
|
||||
- [x] Ahead of schedule: `c` change operator + text objects
|
||||
(`ciw`, `di(`, `ca"`, … — inner/around, nesting-aware)
|
||||
|
||||
Known limits (deferred): `.` drops a *leading* count (`3x` then `.` deletes one;
|
||||
a count inside an operator like `d2w` is kept); no named registers; `.` after an
|
||||
aborted operator (`d<Esc>`) is a no-op.
|
||||
34
docs/v0.4-visual-and-ex.md
Normal file
34
docs/v0.4-visual-and-ex.md
Normal file
@@ -0,0 +1,34 @@
|
||||
# v0.4 — Visual mode + ex commands
|
||||
|
||||
> Part of the [Typoena macro plan](macroplan.md). Requirements and targets:
|
||||
> [qfd.md](qfd.md). Load-bearing decisions: [adr.md](adr.md).
|
||||
|
||||
**Status:** COMPLETE in core 2026-07-11, host-tested (83 editor tests), on-device
|
||||
smoke-test pending. Charwise **Visual** (`v`) and linewise **VisualLine** (`V`)
|
||||
selection landed with `y`/`d`/`c` on the span: charwise is vim-inclusive of the
|
||||
char under the further caret, linewise spans whole logical lines and fills the
|
||||
register linewise (so `Vy`…`p` copies a line, `Vd` deletes it like `dd`). Motions
|
||||
(`h j k l`, `w b e`, `0 $`, `gg G`, `Ctrl-d/u`) and counts extend the selection;
|
||||
`v`/`V` toggle/switch submode, `Esc` cancels. The selection renders as
|
||||
reverse-video cells (black fill, glyphs redrawn white) — the only selection
|
||||
affordance on a 1-bit panel — with the caret cell punched back to *normal* video
|
||||
so the active end stands out. The Normal-mode motions were factored into a shared
|
||||
`move_by` helper so Normal and Visual can't drift.
|
||||
|
||||
**DECISION (2026-07-07, resolved 2026-07-11):** `v`/`V` = **Visual** selection
|
||||
(vim-standard). The read-only **View** (reading/scroll) mode that used to sit on
|
||||
`v`/`V` moved to **`gr`** (go-read) — a `g`-prefixed gesture reusing the existing
|
||||
pending-`g` machinery, no vim clash. View mode stays; `v`/`V` are now Visual.
|
||||
|
||||
- [x] Visual char (`v`) and line (`V`) modes, `y d c` on selections — landed
|
||||
2026-07-11 (18 new tests). Known limits (deferred): no `o` swap-ends, no
|
||||
`x`/`s` operator aliases, no Visual `.` repeat, no `:'<,'>` range commands.
|
||||
- [~] `:` command line (mechanism ✓; `:w`/`:wq`/`:x` save, `:fmt`/`:sync`/`:gl`
|
||||
wired; `:q` deliberately dropped — nothing to quit to). Command-line
|
||||
editing added 2026-07-11: Ctrl-W deletes the previous word, Cmd-Backspace
|
||||
clears the line. **`:e <path>` deferred to [v0.5](v0.5-palette-and-multi-file.md)** — opening another file
|
||||
needs host file-IO + buffer switching, which is v0.5's multi-file work
|
||||
(gated behind Spikes 11/14); half-building it here ahead of its
|
||||
dirty-buffer handling wasn't worth it.
|
||||
- [x] Ahead of schedule / unscheduled: `:fmt` Markdown formatter
|
||||
(table alignment, blank-line collapse, trailing-whitespace strip)
|
||||
286
docs/v0.5-palette-and-multi-file.md
Normal file
286
docs/v0.5-palette-and-multi-file.md
Normal file
@@ -0,0 +1,286 @@
|
||||
# v0.5 — File palette + multi-file
|
||||
|
||||
> Part of the [Typoena macro plan](macroplan.md). Requirements and targets:
|
||||
> [qfd.md](qfd.md). Load-bearing decisions: [adr.md](adr.md). Prefs reference:
|
||||
> [typoena-toml.md](typoena-toml.md).
|
||||
|
||||
**Status:** buffer **foundation** landed in core 2026-07-11 (slice 1 of 4),
|
||||
host-tested; the palette + transient panel (Spike 11) and delete → git-staging
|
||||
(Spike 14) remain the on-device gates. The single-file `Effect` return became a
|
||||
drained **effect queue** (`Save{path,contents}` / `Load{path}` / `Publish` /
|
||||
`Pull`), so one action can ask the host for several steps in order — opening a
|
||||
non-resident file queues a `Save` of the outgoing dirty buffer *then* a `Load` of
|
||||
the target. The multi-buffer state deliberately avoids a rope-per-buffer rewrite:
|
||||
the active buffer keeps its fields inline on `Editor`, inactive buffers park in a
|
||||
small LRU `Vec<Buffer>` (≤ 3 resident = active + 2), and a switch marshals fields
|
||||
in/out so the ~3k-line editing engine is untouched. A dirty parked buffer is
|
||||
saved before it is evicted (nothing leaves RAM unsaved); `:e <path>` opens by
|
||||
prefix (`/sd/repo` → Tracked, `/sd/local` → Local); `:sync` is refused in-core in
|
||||
a Local buffer. Firmware drains the queue **to empty** each batch (a `Load` can
|
||||
cascade an eviction `Save`), and `persistence::{load_path,save_path}` generalise
|
||||
the atomic save off the hard-coded `notes.md`.
|
||||
|
||||
**Slice 2 of 4 landed in core 2026-07-11**, host-tested: the `Cmd-P` file
|
||||
**palette** — a modal transient panel over the writing column with a bare
|
||||
fuzzy-search input (no `>` prefix: `>` is reserved for the command palette,
|
||||
slice 4 — VS Code semantics), the ranked list, and the selected row in reverse
|
||||
video. A pure host-testable fuzzy matcher (`fuzzy_score`: subsequence match,
|
||||
boundary + consecutive-run bonuses, no penalties) ranks results; an in-core MRU
|
||||
floats recently-opened files to the top on an empty query and is **shared with
|
||||
`:e`** (both flow through `open_path`). The host feeds the file list once at boot
|
||||
(`set_file_list`, enumerating `/sd/repo` + `/sd/local`, dotfiles skipped);
|
||||
`Ctrl-n`/`Ctrl-p` (fzf-style; `Ctrl-d`/`Ctrl-u` too) move the selection — the
|
||||
60 % board has no arrow keys — Enter opens via the same park/evict path as `:e`,
|
||||
Esc (or `Cmd-P` again) closes. Same slice: **`Ctrl-n`/`Ctrl-p` also work as
|
||||
down/up line motions in Normal mode** (vim `CTRL-N`≡`j`, `CTRL-P`≡`k`,
|
||||
count-aware), which is why the palette opener moved to `Cmd-P` alone. Scope
|
||||
shows as the inline `repo/…` vs `local/…` label rather than the planned
|
||||
`[git]`/`[local]` badge — it also disambiguates subpaths, not just scope. 111
|
||||
editor tests + 28 keymap tests pass; the no-git firmware binary builds clean.
|
||||
The transient-panel refresh (**Spike 11**) is **CONFIRMED ON DEVICE 2026-07-12 —
|
||||
no ghosting** (user flashed it and eyeballed the full-area partial the palette
|
||||
forces); Cmd-P opens it on-device too. Remaining v0.5 slice: 4 prefs +
|
||||
palette command mode.
|
||||
|
||||
**Slice 3 (`:enew` + delete) COMPLETE + CONFIRMED ON DEVICE 2026-07-12**
|
||||
(committed `c9c0716`). `:enew <name>` creates a new file: empty, active, marked **dirty**
|
||||
so eviction/`:w` persists it, and added to the in-core file list so the palette
|
||||
finds it without a disk re-enumeration — no card IO until it is saved. `:delete`
|
||||
unlinks the **current** file (a new `Effect::Delete` the host services), then
|
||||
switches to the most-recently-parked buffer or an empty scratch; the discarded
|
||||
buffer is never saved even when dirty. **Scope for a new file is read from the
|
||||
path, not a modal prompt** — `local/x` / `repo/x` (the palette label form) select
|
||||
the scope, a bare name uses the current buffer's scope. Same change made the
|
||||
**`/sd` prefix optional everywhere** in `resolve_path`: `/sd/repo/x`, `/repo/x`,
|
||||
and `repo/x` all name one file and nothing resolves outside `/sd` (the writer
|
||||
can't reach anything else). **Spike 14 (delete → git-staging) DID need a firmware
|
||||
fix.** The first on-device test found `add_all(["*"])` alone does **not** stage a
|
||||
deletion on this libgit2 build (the tree came back unchanged, so the second push
|
||||
was a silent "up to date" no-op — the "delete didn't work" report). Fix:
|
||||
`stage_and_commit` now runs `add_all` **then `update_all(["*"])`** (`git add -u`),
|
||||
which removes index entries whose working-tree file is gone — together they are
|
||||
`git add -A`. Also, `:delete` gave no clear feedback, so the snackbar now names
|
||||
the scoped file and, for a Tracked file, that it is local until `:sync`
|
||||
(`deleted repo/notes.md - :sync to publish`). Deferred to later: greying the
|
||||
Publish affordance for a Local buffer, and the multi-file publish count. 123
|
||||
editor tests + 28 keymap tests pass; the no-git firmware binary builds clean. The
|
||||
`update_all` fix (behind `--features git`, unbuildable locally) was **verified on
|
||||
device 2026-07-12** — `:enew test.txt` → `:sync` → `:delete` → `:sync` removed
|
||||
test.txt from origin.
|
||||
|
||||
**Slice 4 (`.typoena.toml` prefs + palette `>` command mode) COMPLETE in core
|
||||
2026-07-12, HOST-TESTED not yet on-device.** A `Prefs` type (host-testable
|
||||
line-based TOML parse/serialize — flat `key = value` bools + one string with `#`
|
||||
comments, no crate pulled onto xtensa) lives on `Editor`; the host reads
|
||||
`/sd/repo/.typoena.toml` at boot and applies it before the first render, and a
|
||||
missing/partial file falls back to per-key defaults. Keys: `save_on_idle`,
|
||||
`format_on_save`, `line_numbers` (all bool, default on) and `auto_sync`
|
||||
(string, default `"10m"`, **schema + default only** — nothing reads it yet).
|
||||
`line_numbers` is live: `gutter_cols()` returns 0 when off, so the text reclaims
|
||||
the gutter's columns (the `gutter - 1` field width made saturating to avoid the
|
||||
underflow). The palette `>` command mode (VS Code semantics — a leading `>` in
|
||||
the query switches file search to the command list) exposes the three booleans
|
||||
as live toggles; Enter flips the pref, applies it at once, queues a new
|
||||
`Effect::SavePrefs` (the editor serializes; the host does the atomic write to
|
||||
`.typoena.toml`, which rides the next `:sync` to other devices), and confirms
|
||||
the new state on the snackbar. **The list stays open after a toggle** so several
|
||||
prefs flip in one visit (Esc/`Cmd-P` closes); **`:settings` opens the palette
|
||||
straight into `>` mode** as a one-command shortcut (both requested by the user
|
||||
2026-07-12, chosen over a separate settings modal — same surface, no duplicate
|
||||
machinery). Committed `c535864`. **Three "decide before build" calls:** (1) the
|
||||
idle auto-save is **unformatted** — `:fmt` runs only on explicit `:w`/`:sync`, so
|
||||
tables/blank-lines are never reflowed mid-session; (2) the per-device `auto_sync`
|
||||
override (card-local `typoena.conf`) is **deferred** — auto_sync is inert in
|
||||
v0.5, so there is nothing yet to override; (3) `> auto sync: <dur>` as a palette
|
||||
command is **deferred to v0.7** — a control that changes a value nothing reads
|
||||
would be a dead switch. `save_on_idle` is honoured host-side: a silent idle
|
||||
auto-save (no snackbar, no forced e-ink flash — a safety net, not an action)
|
||||
fires once per typing burst after a 1.5 s pause. 141 editor tests + 28 keymap
|
||||
tests pass; the no-git firmware binary builds clean. Firmware bumped **0.4.0 →
|
||||
0.5.0** (the v0.5 feature set is met). **Boot-read of the prefs file CONFIRMED ON
|
||||
DEVICE 2026-07-12** — a `.typoena.toml` in `typoena-test` with non-default values
|
||||
(`save_on_idle=false`, `line_numbers=false`, `auto_sync="5m"`) logged back
|
||||
`prefs: Prefs { save_on_idle: false, format_on_save: true, line_numbers: false,
|
||||
auto_sync: "5m" }` at boot, a byte-exact parse (comments skipped, bools + quoted
|
||||
string read). **Full gate CLOSED 2026-07-12:** the palette `>` live-toggle
|
||||
round-trip is confirmed — origin's `.typoena.toml` went `line_numbers` false →
|
||||
true via a *device*-authored publish (`3c79f38`), proving toggle → `SavePrefs` →
|
||||
atomic write → `git add -A` → push — and the `save_on_idle` autosave works on
|
||||
device too. **v0.5 slice 4 fully DONE + on-device confirmed.**
|
||||
|
||||
**Amendment 2026-07-12 — non-boolean prefs (`theme`, `auto_sync`).** Two of the
|
||||
"decide before build" calls above are **superseded**: `auto_sync` is now a live
|
||||
palette command, and a new `theme` (`light`/`dark`) key ships. The generalising
|
||||
idea is that a boolean toggle is just the two-option case of *rotate through a
|
||||
preset list on Enter* — so the palette gains one uniform gesture: **Enter
|
||||
advances the selected pref to its next value and wraps** (a bool flips; a string
|
||||
pref cycles its options). `theme` rotates `light`↔`dark` and is applied by a
|
||||
single whole-frame invert at the end of the render ([`Frame::invert`]), so text,
|
||||
selection, caret, panel and palette all flip together; `auto_sync` rotates
|
||||
`2m`/`5m`/`10m`/`15m`/`30m`. Decision (3) — "a value control that changes nothing
|
||||
readable would be a dead switch" — is knowingly overridden: cycling `auto_sync`
|
||||
persists and displays the interval, but **still drives no behaviour until v0.7**;
|
||||
we accept a set-ahead control so the surface is ready and the value syncs now.
|
||||
Decision (2) (per-device `typoena.conf` override) stays deferred. Kept simple: no
|
||||
enum machinery — both string prefs share a `next_option(current, &OPTIONS)`
|
||||
helper, and hand-editing the TOML can still set any value (the palette only
|
||||
cycles presets; an off-list value snaps to the head on the next Enter). Editor
|
||||
tests cover the rotate/wrap/snap, the live theme invert, and the round-trip.
|
||||
|
||||
**Trailing-newline handling — a saved note ends with a *visible* blank line
|
||||
(commit `d14d9e7`, 2026-07-12; host-tested, on-device gate open).** Two
|
||||
adjustments to how the buffer meets the file. First, format-on-save no longer
|
||||
strips *every* trailing blank line — it collapses a run to **at most one** and
|
||||
keeps it, so a writer who presses Enter to open the next line doesn't have that
|
||||
line (and the caret) yanked away on save (the caret used to jump up to the last
|
||||
non-empty line). Second, persistence treats the file's POSIX terminator as
|
||||
content the editor *shows*: `load_path` reads the file **verbatim** and
|
||||
`save_path` writes the buffer, appending a final newline **only if one is
|
||||
missing** (guarded, not unconditional). Because the editor lays out
|
||||
`rows = #\n + 1`, that terminating newline renders as a **visible trailing empty
|
||||
line** the caret can land on — open a note and the blank line the newline stands
|
||||
for is there. The two are an identity round-trip for any device-written file
|
||||
(all end in `\n`); the file stays git-clean (exactly one terminator — vim and
|
||||
GitHub show no phantom blank line); and a trailing blank line the writer leaves
|
||||
is mirrored, never doubled. This replaced an interim model that stripped the
|
||||
terminator on load and hid it (the file was correct, the newline just wasn't
|
||||
shown). On-device check: reflash → open a note (trailing empty line visible) →
|
||||
`:w` (caret stays on it) → reopen (still there). `Prefs::to_toml` ends in a
|
||||
newline for the same reason; the guarded save leaves the prefs file with exactly
|
||||
one.
|
||||
|
||||
**Amendment 2026-07-13 — recursive enumeration + a 2-char search threshold.**
|
||||
Loading a real repo (`jcalixte/notes`) exposed that `enumerate_files` listed
|
||||
only the **top-level** files of `/sd/repo` and `/sd/local` — a nested notes tree
|
||||
showed a single file in the palette (subpaths always *opened* fine via
|
||||
`:e repo/sub/x.md`; only the listing was flat). The enumeration is now a
|
||||
recursive walk: dot entries are skipped at every level (so `.git` is never
|
||||
descended into), each directory is read fully before recursing (one FatFS dir
|
||||
handle open at a time — the `remove_dir_recursive` pattern, kind to the
|
||||
FD-bounded mount), depth is capped at 8, and the boot-time walk logs its file
|
||||
count and duration (`file walk: N files in Xms`) so the FAT dir-IO cost on a
|
||||
big repo is measurable, not assumed. With the list now card-sized, the palette
|
||||
gained a **search threshold**: below 2 typed chars the result list is the
|
||||
**recents (MRU) only** — quick-switch (`Cmd-P`, `Enter`) stays one keystroke
|
||||
away — and the full fuzzy-ranked list appears from 2 chars on
|
||||
(`PALETTE_MIN_QUERY`). A fresh boot with no opens yet shows `(type to search)`.
|
||||
`>` commands and `$` snippets are short curated lists; the threshold does not
|
||||
apply to them.
|
||||
|
||||
**TODO (on-device, next time the device is on the bench)** — two measurements
|
||||
from the same boot log, both already instrumented:
|
||||
|
||||
- [ ] **Re-measure the walk time** after the d_type fix (`2660a3e` — dirent
|
||||
`file_type()` instead of a per-entry `metadata()` stat, which cost
|
||||
~32 ms/file and made run 1 take 35 s for 1098 files). Read the
|
||||
`file walk: N files in Xms` line. Only if it's still slow does the
|
||||
async-walk idea come back on the table.
|
||||
- [ ] **Read the file-list DRAM cost** from the new
|
||||
`file list: internal heap <before> -> <after> (<N> KB consumed)` line
|
||||
(the build is bracketed with `MALLOC_CAP_INTERNAL` readings in
|
||||
`main.rs`). The 1098 path Strings are each below the 16 KB SPIRAM
|
||||
malloc threshold, so they all land in internal DRAM — estimated
|
||||
60–70 KB, competing with Wi-Fi/TLS. Decision rule: **~60–70 KB
|
||||
confirms** interning the paths into one shared buffer (a single
|
||||
>16 KB alloc goes to PSRAM; only a ~9 KB offset index stays in DRAM);
|
||||
**well under that (≤~30 KB)** kills the idea — the next DRAM suspect
|
||||
is then parked-buffer text.
|
||||
|
||||
- [x] `Cmd-P` opens fuzzy file palette over **both** `/sd/repo/` and
|
||||
`/sd/local/` — **landed and CONFIRMED ON DEVICE 2026-07-12** (Spike 11: no
|
||||
ghosting on the transient panel); scope shows as the inline
|
||||
`repo/…` / `local/…` label instead of a `[git]`/`[local]` badge.
|
||||
- [~] Open, switch, close buffers (keep ≤ 3 in memory) — **open + switch + the
|
||||
≤ 3 LRU-resident model with dirty-aware save-before-evict done in core**
|
||||
(host-tested); `:e <path>` **and the palette** drive it today. Explicit
|
||||
**close** still to come.
|
||||
- [x] `:e` and palette share the same recent-files list — both open via
|
||||
`open_path`, which pushes to the in-core MRU that orders the palette.
|
||||
- [x] `:enew` creates a new file — **done in core (host-tested) 2026-07-12.**
|
||||
Scope is read from the path (`local/x` / `repo/x` select it, the palette
|
||||
label form; a bare name uses the current scope) rather than a modal
|
||||
prompt — the resolved scope is echoed in the snackbar. The `/sd` prefix is
|
||||
optional throughout (`/sd/repo/x` = `/repo/x` = `repo/x`).
|
||||
- [x] Delete a file — **core done (host-tested) 2026-07-12;** `:delete` unlinks
|
||||
the current file via `Effect::Delete`. For a Tracked file the removal reaches
|
||||
the next `:sync` Publish's staged set. **Spike 14 (on-device) found the
|
||||
staging incomplete:** `add_all(["*"])` alone did not stage the deletion, so
|
||||
`stage_and_commit` now also runs `update_all(["*"])` (`git add -u`) — the
|
||||
two together are `git add -A`. A Local file is just unlinked. The snackbar
|
||||
now confirms the delete and flags that a Tracked file needs `:sync`.
|
||||
**Verified on device 2026-07-12** — the `:enew`→`:sync`→`:delete`→`:sync`
|
||||
cycle removed test.txt from origin.
|
||||
- [~] `Ctrl-G` is disabled / hidden when the current buffer is local-scope —
|
||||
**`:sync` / Publish is blocked in-core for a Local buffer** (posts "Publish
|
||||
unavailable (Local)"); the side-panel affordance that hides/greys the
|
||||
gesture is the remaining half.
|
||||
- [ ] The side panel briefly shows file count on `Ctrl-G` when the publish bundles
|
||||
more than one dirty Tracked file (e.g. `"publishing 3 files: abc1234"`),
|
||||
so workspace-scoped behaviour stays visible to the user
|
||||
- [x] **Preferences file** `/sd/repo/.typoena.toml` — a git-tracked,
|
||||
hand-editable TOML file for editor behaviour, deliberately **distinct from
|
||||
the `/sd/typoena.conf` card secrets** (Wi-Fi / PAT / remote / author,
|
||||
gitignored, never committed — see [v0.1](v0.1-mvp-product.md)). Read at boot; a missing file or
|
||||
key falls back to the defaults below. **Core done 2026-07-12** (a `Prefs`
|
||||
type on `Editor`, host-testable parse/serialize, applied via
|
||||
`Editor::set_prefs` before the first render); full reference:
|
||||
[`typoena-toml.md`](typoena-toml.md). Keys:
|
||||
- [x] `save_on_idle` (bool, default `true`) — auto-save the current buffer on
|
||||
the idle typing-pause, so `:w` becomes optional rather than required.
|
||||
**Honoured host-side** as a *silent* save (no snackbar, no forced e-ink
|
||||
flash — a safety net, not an action), unformatted, once per typing burst
|
||||
after a 1.5 s pause.
|
||||
- [x] `format_on_save` (bool, default `true`) — run `:fmt` (table alignment,
|
||||
blank-line collapse, trailing-whitespace strip) on the buffer before it
|
||||
is persisted, so `:sync` is **fmt → save → commit → push** and `:w`
|
||||
saves formatted. Implemented in-core 2026-07-11 (`Editor`), now **driven
|
||||
by this key**. **Open question RESOLVED (2026-07-12):** fmt runs only on
|
||||
an explicit `:w`/`:sync`; the `save_on_idle` auto-save is deliberately
|
||||
**unformatted**, so tables/blank lines are never reflowed mid-session
|
||||
(the caret would jump under you on every thinking pause).
|
||||
- [x] `line_numbers` (bool, default `true`) — show the absolute line-number
|
||||
gutter (built always-on in v0.2). Off reclaims the gutter's columns for
|
||||
text (`gutter_cols()` → 0); the palette `> line numbers: on/off` command
|
||||
toggles it live. **Done 2026-07-12.**
|
||||
- [x] `theme` (string, default `"light"`; options `light`/`dark`) — panel
|
||||
colour polarity. `dark` is a single whole-frame invert at the end of the
|
||||
render (`Frame::invert`), so everything flips together and stays legible.
|
||||
Palette `> theme` rotates `light`↔`dark` live. **Added 2026-07-12
|
||||
(amendment); host-tested, on-device check pending.** Caveat: `dark`
|
||||
partial-refreshes ghost more on e-paper than `light` — verify on panel.
|
||||
- [ ] `auto_sync` (duration string, default `"10m"`; `"0"` / omitted
|
||||
disables; **min clamp ~`"2m"`** so a palette typo can't drain the
|
||||
battery) — a *max-staleness cap*, not a wall-clock timer:
|
||||
**opportunistic, rate-limited** Publish. Push when already awake + dirty
|
||||
(coalesced into the idle-pause, ≤ once per `auto_sync`) and once on the
|
||||
way into sleep if dirty; **never wake from deep sleep purely to sync**.
|
||||
Wi-Fi energy is a `1/T` curve whose knee sits at 5–10 min, and
|
||||
`save_on_idle` already owns local data safety — so 10 min halves the
|
||||
sync energy of a 5-min default for no real risk. Full derivation:
|
||||
[`tradeoff-curves/wifi-auto-sync.md`](tradeoff-curves/wifi-auto-sync.md).
|
||||
The **schema + default (`"10m"`) live here in v0.5** and round-trip
|
||||
through `Prefs`; **nothing reads the value yet** — the periodic side
|
||||
rides the better-git work (v0.7) and must interact with light / deep
|
||||
sleep (v0.8). Marked `[~]`: parsed and preserved, no behaviour.
|
||||
**Amended 2026-07-12:** now a palette preset command — Enter rotates it
|
||||
through `2m`/`5m`/`10m`/`15m`/`30m` (the `~2m` min is baked into the
|
||||
preset list). Set-ahead only: still read by nothing until v0.7.
|
||||
- [x] Open question RESOLVED (2026-07-12): the per-device sync cadence override
|
||||
(a card-local `typoena.conf` layer over the committed prefs) is
|
||||
**deferred** — `auto_sync` is inert in v0.5, so there is nothing yet to
|
||||
override; revisit when v0.7 makes the periodic push real.
|
||||
- [x] **Palette command mode** — typing `>` at the `Cmd-P` palette switches it
|
||||
from file search to a command list (VS Code-style). **Done in core
|
||||
2026-07-12.** The v0.5 commands toggle the three boolean `.typoena.toml`
|
||||
prefs — `> save on idle`, `> format on save`, `> line numbers` — each label
|
||||
carrying its live state; Enter flips the pref, applies it at once, queues
|
||||
`Effect::SavePrefs` (persist to the file), and confirms on the snackbar.
|
||||
**The list stays open after a toggle** (flip several, Esc/`Cmd-P` closes),
|
||||
and **`:settings` opens it directly** — both added 2026-07-12 as the "change
|
||||
config from the device" surface (chosen over a separate settings modal).
|
||||
This command list is the discoverable surface later actions (`:fmt`, font)
|
||||
also register into. **Amended 2026-07-12:** the palette now also carries the
|
||||
non-boolean prefs `> theme` (`light`/`dark`, live whole-frame invert) and
|
||||
`> auto sync` (`2m`..`30m`), both cycled by the same Enter-rotates-to-next
|
||||
gesture. `auto_sync` is exposed **set-ahead** (no behaviour until v0.7),
|
||||
knowingly overriding the earlier "dead switch" call.
|
||||
112
docs/v0.6-markdown.md
Normal file
112
docs/v0.6-markdown.md
Normal file
@@ -0,0 +1,112 @@
|
||||
# v0.6 — Markdown affordances
|
||||
|
||||
> Part of the [Typoena macro plan](macroplan.md). Requirements and targets:
|
||||
> [qfd.md](qfd.md). Load-bearing decisions: [adr.md](adr.md).
|
||||
|
||||
**Status:** render affordances done early. The snippet engine (tab-stop core,
|
||||
inline Tab-expansion, hint-on-pause, `$` palette) and the `>` command-palette
|
||||
generalisation are **done in core, host-tested** (187 editor tests); the
|
||||
remaining work is the **firmware boot-read + `just init` catalog** (slice 5, the
|
||||
on-device gate). Snippets are net-new scope, added 2026-07-08; reshaped
|
||||
2026-07-12 from a hard-coded table into a git-synced, Zed-compatible library with
|
||||
a `$` palette launcher — see [`typoena-snippets.md`](typoena-snippets.md) for the
|
||||
file format.
|
||||
|
||||
- [x] Heading lines bolded in render (faux-bold double-strike)
|
||||
- [x] List continuation on Enter inside `- ` / `1. ` (with empty-item exit)
|
||||
- [x] Soft-wrap at word boundaries
|
||||
## Snippets
|
||||
|
||||
Trigger-driven text expansion for Markdown authoring (Zed-inspired, but **no
|
||||
completion popup**: e-ink's ~630 ms refresh rules out a live filtering menu, and
|
||||
it fights the distraction-free premise). The library is a git-synced,
|
||||
Zed-compatible JSON file — full file-format reference in
|
||||
[`typoena-snippets.md`](typoena-snippets.md). This section is the *editor*
|
||||
behaviour.
|
||||
|
||||
- [x] **The tab-stop engine** — the shared core both surfaces drive. A body is
|
||||
literal text plus numbered stops `$1 … $n` and a final `$0`; on insertion
|
||||
the caret lands on `$1` (or the end), in Insert. Tab advances to the next
|
||||
stop, **forward only** (no Shift-Tab); pending stops sit after the caret
|
||||
and shift with edits there. The session auto-aborts on Esc, a mode change,
|
||||
or a motion that leaves the stops. `${n:label}` parses to a bare `$n` (the
|
||||
label is stripped — no selection model to fill); no dynamic values (no RTC,
|
||||
so no `date`).
|
||||
- [x] **Inline Tab-expansion (Insert mode).** If the word immediately before the
|
||||
caret matches a snippet prefix, Tab expands it and starts the tab-stop
|
||||
session; otherwise Tab inserts spaces as today. A check in the Insert
|
||||
handler alongside the existing `list_marker` transform
|
||||
(`expand_snippet(word) -> Option<Snippet>`). Tab already arrives as
|
||||
`Key::Char('\t')`, so no new key event.
|
||||
- [x] **Hint-on-pause.** On the typing pause (same throttle as the word-count /
|
||||
cursor refresh — never a per-keystroke repaint), if the word before the
|
||||
caret is a prefix, the right side panel shows a quiet hint (`» name`, on the
|
||||
row above the mode line, sharing the slot with the `NO KBD` flag — the two
|
||||
never co-occur since the hint means you're typing). The panel is ~15 cols,
|
||||
so the hint is the snippet **name**, not the whole body — the full preview is
|
||||
the `$` palette's job. Snapshotted in `refresh_stats` so it rides the pause,
|
||||
and the firmware's Insert-pause repaint already carries it (no firmware
|
||||
change). Latin-9 has no `↹` glyph, hence the `»` marker.
|
||||
- [x] **`$` palette (browse + insert).** `Cmd-P` then `$` switches the palette to
|
||||
the snippet list (the same sigil mechanism as `>`). Fuzzy-matches name /
|
||||
prefix / description; Enter inserts the body at the caret and starts the
|
||||
tab-stop session. Lists **all** snippets — the fuzzy filter handles clutter,
|
||||
so there's no hidden "inline-only vs palette-only" split. Rows read
|
||||
`Name [prefix]`, so browsing also teaches the inline trigger.
|
||||
- [x] **Boot wiring.** The host reads `/sd/repo/.typoena.snippets.json` at boot
|
||||
and calls `Editor::set_snippets` (mirroring `set_prefs`); a missing or
|
||||
malformed file is non-fatal (no snippets, editor runs). Parse lives in the
|
||||
host-testable `editor` crate via `serde_json` — the one new dependency,
|
||||
confirmed to build for xtensa (`cargo check`, firmware 0.6.0). On-device
|
||||
smoke-test still pending.
|
||||
|
||||
## The palette, generalised (`Cmd-P` · `>` · `$`) — done in core
|
||||
|
||||
v0.5 shipped `Cmd-P` = files and `>` = a five-entry settings list (`save_on_idle`,
|
||||
`format_on_save`, `line_numbers` toggles + `theme`/`auto_sync` rotations). v0.6
|
||||
makes the sigils a clean split by verb, and the empty-palette placeholder legends
|
||||
them: `Go to file · > settings · $ snippets`.
|
||||
|
||||
- **bare `Cmd-P`** → *navigate*: go to file (unchanged).
|
||||
- **`>`** → *act on the editor* — the command palette, a real action registry (the
|
||||
`PALETTE_CMDS` list, actions first then settings). The pref toggles are just its
|
||||
stateful entries, not a special section. Dispatch is by `PaletteCmd::kind`: a
|
||||
**toggle** flips and the list **stays open**; a **one-shot** (`format`,
|
||||
`publish` — the latter shares `run_publish` with `:sync`) runs and **closes**; a
|
||||
**parameterised** command (`new file...`) morphs the palette into a second
|
||||
**input step** (the box becomes a filename prompt → Enter creates it, scope read
|
||||
from a `repo/`/`local/` prefix as `:enew` does; backspacing past the start steps
|
||||
back to the `>` list). This **retired `:e`** — bare `Cmd-P` covers file-opening;
|
||||
dotfiles can get a dedicated `> edit ...` command if/when wanted.
|
||||
- **`$`** → *insert content* — the snippet launcher above.
|
||||
|
||||
Labels avoid `…`/`↹` — the palette/panel fonts are ISO-8859-15, which has neither
|
||||
glyph — so `new file...` uses ASCII dots and the pause hint leads with `»`.
|
||||
|
||||
## First-time setup — snippet catalog (`just init`) — done
|
||||
|
||||
[`just init`](../firmware/README.md#provisioning-an-sd-card) seeds the two
|
||||
git-tracked config files into `repo/` (so they commit + sync on the device's
|
||||
first `:sync`), **writing only a file that is absent** so a re-`init` never
|
||||
clobbers a synced library: a starter [`.typoena.toml`](typoena-toml.md) (the five
|
||||
keys at their defaults) and a [`.typoena.snippets.json`](typoena-snippets.md)
|
||||
assembled from a **curated catalog** (`firmware/snippets-catalog/`, `jq`-merged).
|
||||
The catalog is grouped and opt-in — you pick the groups (each `[Y/n]`, all-yes on
|
||||
a non-interactive run) rather than getting one writer's whole personal set. Not
|
||||
every Zed snippet is worth proposing: the Slidev/blog-pipeline ones (`@[youtube]`,
|
||||
`<v-clicks>`, frontmatter, mermaid) and hyper-specific personal ones are left out,
|
||||
since a distraction-free prose appliance can't render them and you'd never miss
|
||||
them. Prefixes are **English** (except `edanso`, the mnemonic for `œ`), bodies
|
||||
translated from the source Zed snippets. **Three groups, 17 snippets:**
|
||||
|
||||
- [x] **Symbols** (inline, the keyboard can't type them): `arrow`→`→`,
|
||||
`neq`→`≠`, `times`→`×`, `middot`→`·`, `deg`→`°`, `euro`→`€`, `edanso`→`œ`
|
||||
(dead keys in v0.2.5 cover accents, *not* these). **Caveat:** `→` (U+2192)
|
||||
and `≠` (U+2260) are outside ISO-8859-15, so they store correctly and sync
|
||||
but render as a missing-glyph box on the device panel until the font is
|
||||
extended; the other five are Latin-9 and draw fine. Kept deliberately.
|
||||
- [x] **Structure**: `todo`→`- [ ] `, `link`→`[$1]($2)$0`, `img`→`$0`,
|
||||
`table` (2-col), `code` (fenced block). `img`/`code` are net-new.
|
||||
- [x] **Prose / PKM templates** (`${n:label}` stripped to `$n`): `booknotes`,
|
||||
`reference` (the English reference block — `refangl` folded in), `bias`,
|
||||
`capture`, `standard`. (`5w1h` dropped.)
|
||||
14
docs/v0.7-search-and-git.md
Normal file
14
docs/v0.7-search-and-git.md
Normal file
@@ -0,0 +1,14 @@
|
||||
# v0.7 — Search + better git
|
||||
|
||||
> Part of the [Typoena macro plan](macroplan.md). Requirements and targets:
|
||||
> [qfd.md](qfd.md). Load-bearing decisions: [adr.md](adr.md).
|
||||
|
||||
**Status:** the **`:gl` pull command landed in the editor** (2026-07-11,
|
||||
host-tested) — `Effect::Pull` + a firmware stub; the on-device fetch +
|
||||
fast-forward is still to build. Search not started.
|
||||
|
||||
- [ ] `/` forward search, `n N`
|
||||
- [~] `:gl` — pull: fetch + **fast-forward only**, refuse on divergence and
|
||||
surface it (renamed from the planned `:Gpull`). Editor command +
|
||||
`Effect::Pull` done 2026-07-11 (host-tested); the git-thread
|
||||
fetch/fast-forward in `git_sync` remains (only push is wired today).
|
||||
12
docs/v0.8-battery-and-sleep.md
Normal file
12
docs/v0.8-battery-and-sleep.md
Normal file
@@ -0,0 +1,12 @@
|
||||
# v0.8 — Power: battery + sleep
|
||||
|
||||
> Part of the [Typoena macro plan](macroplan.md). Requirements and targets:
|
||||
> [qfd.md](qfd.md). Load-bearing decisions: [adr.md](adr.md).
|
||||
|
||||
**Status:** not started.
|
||||
|
||||
- [ ] Measure idle / typing / push current draw on bench
|
||||
- [ ] 18650 + IP5306 charge board, soft power switch
|
||||
- [ ] Light sleep on idle > 30 s (keyboard interrupt wakes)
|
||||
- [ ] Deep sleep on lid close (reed switch); restore cursor + buffer
|
||||
- [ ] Battery indicator in the side panel
|
||||
14
docs/v0.9-robustness.md
Normal file
14
docs/v0.9-robustness.md
Normal file
@@ -0,0 +1,14 @@
|
||||
# v0.9 — Robustness
|
||||
|
||||
> Part of the [Typoena macro plan](macroplan.md). Requirements and targets:
|
||||
> [qfd.md](qfd.md). Load-bearing decisions: [adr.md](adr.md).
|
||||
|
||||
**Status:** not started.
|
||||
|
||||
- [ ] Crash-safe writes (write to `.tmp`, fsync, rename)
|
||||
- [ ] Recover from interrupted push (re-attempt on next save)
|
||||
- [ ] SD card removal / reinsert handling
|
||||
- [ ] Wi-Fi reconnect with backoff
|
||||
- [ ] On-device provisioning + settings screen: SSID, PAT rotation, default
|
||||
remote, commit author (replaces the v0.1 dev-only NVS-flashing path —
|
||||
first release usable by someone who is not the firmware author)
|
||||
16
docs/v1.0-polish.md
Normal file
16
docs/v1.0-polish.md
Normal file
@@ -0,0 +1,16 @@
|
||||
# v1.0 — Polish
|
||||
|
||||
> Part of the [Typoena macro plan](macroplan.md). Requirements and targets:
|
||||
> [qfd.md](qfd.md). Load-bearing decisions: [adr.md](adr.md).
|
||||
|
||||
**Status:** not started.
|
||||
|
||||
- [ ] Boot time ≤ 3 s to usable cursor — currently ~4.26 s; the ~1.9 s cold-boot
|
||||
full refresh is a hard e-ink floor, so ≤ 3 s is marginal (see
|
||||
[`notes/boot-time-budget.md`](notes/boot-time-budget.md))
|
||||
- [ ] Font selection (at least one serif + one mono) with adjustable font
|
||||
size, switchable at runtime and persisted across reboots
|
||||
- [ ] Theme: light / dark (inverted e-ink), switchable at runtime and
|
||||
persisted across reboots
|
||||
- [ ] Enclosure design files in `hardware/`
|
||||
- [ ] User guide
|
||||
11
docs/v1.x-stretch.md
Normal file
11
docs/v1.x-stretch.md
Normal file
@@ -0,0 +1,11 @@
|
||||
# v1.x — Stretch / nice-to-have
|
||||
|
||||
> Part of the [Typoena macro plan](macroplan.md). Requirements and targets:
|
||||
> [qfd.md](qfd.md). Load-bearing decisions: [adr.md](adr.md).
|
||||
|
||||
Post-1.0 ideas, not committed to any release:
|
||||
|
||||
- 10.3" panel upgrade via IT8951
|
||||
- Multiple remotes / repos
|
||||
- Stats: words today, streak
|
||||
- BLE-HID fallback for wireless keyboards
|
||||
@@ -8,3 +8,8 @@ description = "Modal (vim-style) text-editor core: the buffer, motions, edits, a
|
||||
embedded-graphics = "0.8"
|
||||
display = { path = "../display" }
|
||||
keymap = { path = "../keymap" }
|
||||
# Snippet library parse (v0.6): the `.typoena.snippets.json` file uses Zed's JSON
|
||||
# shape. serde_json handles the string-escape corner cases a hand-rolled reader
|
||||
# would get wrong; the crate is `std` (esp-idf provides std on xtensa).
|
||||
serde = { version = "1", features = ["derive"] }
|
||||
serde_json = "1"
|
||||
|
||||
4043
editor/src/lib.rs
4043
editor/src/lib.rs
File diff suppressed because it is too large
Load Diff
@@ -3,7 +3,7 @@ target = "xtensa-esp32s3-espidf"
|
||||
|
||||
[target.'cfg(target_os = "espidf")']
|
||||
linker = "ldproxy"
|
||||
runner = "espflash flash --monitor"
|
||||
runner = "espflash flash --monitor --baud 921600"
|
||||
rustflags = [ "--cfg", "espidf_time64"]
|
||||
|
||||
[unstable]
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "firmware"
|
||||
version = "0.3.0"
|
||||
version = "0.6.0"
|
||||
authors = ["Julien Calixte <juliencalixte@gmail.com>"]
|
||||
edition = "2024"
|
||||
resolver = "2"
|
||||
@@ -36,6 +36,22 @@ name = "sd_fat"
|
||||
path = "src/bin/sd_fat.rs"
|
||||
harness = false
|
||||
|
||||
# SD/FAT primitive-op micro-benchmark — attributes the ~700 ms/loose-object write
|
||||
# floor (see docs/tradeoff-curves/sync-commit-staging.md). Pure SD, no git feature.
|
||||
# Flash with `just flash-bench`.
|
||||
[[bin]]
|
||||
name = "sd_bench"
|
||||
path = "src/bin/sd_bench.rs"
|
||||
harness = false
|
||||
|
||||
# git-level micro-benchmark — localizes the ~700 ms/object libgit2 overhead (see
|
||||
# docs/tradeoff-curves/sync-commit-staging.md). Needs the `git` feature.
|
||||
[[bin]]
|
||||
name = "git_bench"
|
||||
path = "src/bin/git_bench.rs"
|
||||
harness = false
|
||||
required-features = ["git"]
|
||||
|
||||
# Spike 9 — boot splash. Standalone bench program that paints the Typoena
|
||||
# wordmark-in-a-circle on the EPD. No git, no SD, no Wi-Fi. Flash with
|
||||
# `just flash-splash`.
|
||||
|
||||
@@ -41,6 +41,12 @@ file(GLOB LG2_SRCS
|
||||
"${LG2}/deps/pcre/*.c"
|
||||
"${LG2}/deps/zlib/*.c"
|
||||
)
|
||||
# streams/mbedtls.c is replaced by our patched copy: its wrap() error path
|
||||
# double-freed the socket stream (tlsf abort on device when ssl_setup failed
|
||||
# under memory pressure). See esp_mbedtls_stream.c's header for the one-hunk
|
||||
# delta; keep the copy in lockstep on submodule bumps.
|
||||
list(REMOVE_ITEM LG2_SRCS "${LG2}/src/libgit2/streams/mbedtls.c")
|
||||
|
||||
list(APPEND LG2_SRCS
|
||||
"${LG2}/src/util/allocators/failalloc.c"
|
||||
"${LG2}/src/util/allocators/stdalloc.c"
|
||||
@@ -48,6 +54,7 @@ list(APPEND LG2_SRCS
|
||||
"${LG2}/src/util/hash/mbedtls.c" # SHA1 + SHA256 via mbedtls
|
||||
"${CMAKE_CURRENT_LIST_DIR}/esp_map.c" # p_mmap via malloc+read (no <sys/mman.h>)
|
||||
"${CMAKE_CURRENT_LIST_DIR}/esp_stubs.c" # getuid/readlink/utimes/... stubs
|
||||
"${CMAKE_CURRENT_LIST_DIR}/esp_mbedtls_stream.c" # streams/mbedtls.c + double-free fix
|
||||
# NOTE: unix/map.c replaced by esp_map.c — picolibc has no <sys/mman.h>.
|
||||
# NOTE: unix/process.c deliberately excluded — needs fork()/sys/wait.h,
|
||||
# only used by the SSH-exec transport we don't enable.
|
||||
|
||||
@@ -7,9 +7,19 @@
|
||||
* Allocations go through git__malloc, so with PSRAM in the heap they land in
|
||||
* the 8 MB external RAM rather than the ~340 KB internal DRAM.
|
||||
*
|
||||
* Limitation: writable/shared mappings are not written back (libgit2 does not
|
||||
* mmap for writing in the paths we exercise). If that ever changes it will
|
||||
* surface at runtime, not here.
|
||||
* There is deliberately NO cache here. A window cache was built and removed
|
||||
* on 2026-07-12: across four instrumented real-repo bench runs it scored
|
||||
* exactly 0 hits, because libgit2's mwindow layer reuses its open windows and
|
||||
* only genuinely new (offset, len) ranges ever reach p_mmap — and the one
|
||||
* memory bug it "fixed" (7.4 MB resident starving zlib) was caused by the
|
||||
* cache itself holding buffers past p_munmap. Free-at-munmap keeps
|
||||
* GIT_OPT_SET_MWINDOW_MAPPED_LIMIT honest by construction: when libgit2
|
||||
* releases a window, the memory really is back in git__malloc's pool. Full
|
||||
* trail: docs/tradeoff-curves/sync-commit-staging.md (final-bench section).
|
||||
* The stats counters below are kept so a future workload that *does* repeat
|
||||
* ranges (push? reconcile?) can be spotted before anyone rebuilds a cache.
|
||||
*
|
||||
* Limitation: writable/shared mappings are not written back.
|
||||
*/
|
||||
|
||||
#include "git2_util.h"
|
||||
@@ -18,6 +28,7 @@
|
||||
#include <unistd.h>
|
||||
#include <string.h>
|
||||
#include <errno.h>
|
||||
#include <stdint.h>
|
||||
|
||||
int git__page_size(size_t *page_size)
|
||||
{
|
||||
@@ -31,10 +42,49 @@ int git__mmap_alignment(size_t *alignment)
|
||||
return 0;
|
||||
}
|
||||
|
||||
/* Diagnostics, read from the bench via esp_map_stats(). The signature predates
|
||||
* the cache removal: `hits` is always 0, `misses` counts every mapping, and
|
||||
* `cached_kb` now reports the LIVE mapped bytes (the mwindow working set). */
|
||||
static uint32_t g_maps;
|
||||
static uint64_t g_read_bytes;
|
||||
static size_t g_live_bytes;
|
||||
|
||||
void esp_map_stats(uint32_t *hits, uint32_t *misses, uint32_t *read_kb, uint32_t *cached_kb)
|
||||
{
|
||||
if (hits) *hits = 0;
|
||||
if (misses) *misses = g_maps;
|
||||
if (read_kb) *read_kb = (uint32_t)(g_read_bytes / 1024);
|
||||
if (cached_kb) *cached_kb = (uint32_t)(g_live_bytes / 1024);
|
||||
}
|
||||
|
||||
static int read_range(int fd, off64_t offset, size_t len, unsigned char *data)
|
||||
{
|
||||
size_t got = 0;
|
||||
|
||||
if (lseek(fd, offset, SEEK_SET) < 0) {
|
||||
git_error_set(GIT_ERROR_OS, "failed to seek for mmap emulation");
|
||||
return -1;
|
||||
}
|
||||
while (got < len) {
|
||||
ssize_t n = read(fd, data + got, len - got);
|
||||
if (n < 0) {
|
||||
if (errno == EINTR)
|
||||
continue;
|
||||
git_error_set(GIT_ERROR_OS, "failed to read for mmap emulation");
|
||||
return -1;
|
||||
}
|
||||
if (n == 0)
|
||||
break; /* short file: zero-fill the tail, like a real mapping */
|
||||
got += (size_t)n;
|
||||
}
|
||||
if (got < len)
|
||||
memset(data + got, 0, len - got);
|
||||
return 0;
|
||||
}
|
||||
|
||||
int p_mmap(git_map *out, size_t len, int prot, int flags, int fd, off64_t offset)
|
||||
{
|
||||
unsigned char *data;
|
||||
size_t got = 0;
|
||||
|
||||
GIT_UNUSED(prot);
|
||||
GIT_UNUSED(flags);
|
||||
@@ -45,29 +95,13 @@ int p_mmap(git_map *out, size_t len, int prot, int flags, int fd, off64_t offset
|
||||
|
||||
data = git__malloc(len);
|
||||
GIT_ERROR_CHECK_ALLOC(data);
|
||||
|
||||
if (lseek(fd, offset, SEEK_SET) < 0) {
|
||||
git_error_set(GIT_ERROR_OS, "failed to seek for mmap emulation");
|
||||
if (read_range(fd, offset, len, data) < 0) {
|
||||
git__free(data);
|
||||
return -1;
|
||||
}
|
||||
|
||||
while (got < len) {
|
||||
ssize_t n = read(fd, data + got, len - got);
|
||||
if (n < 0) {
|
||||
if (errno == EINTR)
|
||||
continue;
|
||||
git_error_set(GIT_ERROR_OS, "failed to read for mmap emulation");
|
||||
git__free(data);
|
||||
return -1;
|
||||
}
|
||||
if (n == 0)
|
||||
break; /* short file: zero-fill the tail, like a real mapping */
|
||||
got += (size_t)n;
|
||||
}
|
||||
|
||||
if (got < len)
|
||||
memset(data + got, 0, len - got);
|
||||
g_maps++;
|
||||
g_read_bytes += len;
|
||||
g_live_bytes += len;
|
||||
|
||||
out->data = data;
|
||||
out->len = len;
|
||||
@@ -77,7 +111,12 @@ int p_mmap(git_map *out, size_t len, int prot, int flags, int fd, off64_t offset
|
||||
int p_munmap(git_map *map)
|
||||
{
|
||||
GIT_ASSERT_ARG(map);
|
||||
|
||||
git__free(map->data);
|
||||
if (g_live_bytes >= map->len)
|
||||
g_live_bytes -= map->len;
|
||||
else
|
||||
g_live_bytes = 0;
|
||||
map->data = NULL;
|
||||
map->len = 0;
|
||||
return 0;
|
||||
|
||||
498
firmware/components/libgit2/esp_mbedtls_stream.c
Normal file
498
firmware/components/libgit2/esp_mbedtls_stream.c
Normal file
@@ -0,0 +1,498 @@
|
||||
/*
|
||||
* Copyright (C) the libgit2 contributors. All rights reserved.
|
||||
*
|
||||
* This file is part of libgit2, distributed under the GNU GPL v2 with
|
||||
* a Linking Exception. For full terms see the included COPYING file.
|
||||
*/
|
||||
|
||||
/*
|
||||
* esp_mbedtls_stream.c — verbatim copy of the vendored
|
||||
* src/libgit2/streams/mbedtls.c (v1.9.4) with ONE fix; it replaces the
|
||||
* vendored file in CMakeLists.txt (same pattern as esp_map.c).
|
||||
*
|
||||
* THE FIX (2026-07-13): mbedtls_stream_wrap()'s `out_err` path closed and
|
||||
* freed `st->io` — the caller's socket stream — but every caller frees that
|
||||
* stream on error too (git_mbedtls_stream_new does close+free right after),
|
||||
* and wrap's OTHER error paths (calloc/strdup failures) do NOT free it, so
|
||||
* the caller cannot compensate either way. When mbedtls_ssl_setup failed on
|
||||
* the device (internal-RAM exhaustion during the first real-repo push), the
|
||||
* double git__free tripped tlsf ("block already marked as free") and reset
|
||||
* the chip instead of surfacing a clean error. Delta from vendor: the
|
||||
* `out_err` label no longer touches st->io — on error, ownership of `in`
|
||||
* stays with the caller, consistently — and it frees the git__malloc'd
|
||||
* st->ssl struct the vendored path leaked.
|
||||
*
|
||||
* Keep this file in lockstep with the vendored one on submodule bumps (diff
|
||||
* against it; the delta must stay this one hunk).
|
||||
*/
|
||||
|
||||
#include "streams/mbedtls.h"
|
||||
|
||||
#ifdef GIT_MBEDTLS
|
||||
|
||||
#include <ctype.h>
|
||||
|
||||
#include "runtime.h"
|
||||
#include "stream.h"
|
||||
#include "streams/socket.h"
|
||||
#include "git2/transport.h"
|
||||
#include "util.h"
|
||||
|
||||
#ifndef GIT_DEFAULT_CERT_LOCATION
|
||||
#define GIT_DEFAULT_CERT_LOCATION NULL
|
||||
#endif
|
||||
|
||||
/* Work around C90-conformance issues */
|
||||
#if !defined(__STDC_VERSION__) || (__STDC_VERSION__ < 199901L)
|
||||
# if defined(_MSC_VER)
|
||||
# define inline __inline
|
||||
# elif defined(__GNUC__)
|
||||
# define inline __inline__
|
||||
# else
|
||||
# define inline
|
||||
# endif
|
||||
#endif
|
||||
|
||||
#include <mbedtls/ssl.h>
|
||||
#include <mbedtls/error.h>
|
||||
#include <mbedtls/entropy.h>
|
||||
#include <mbedtls/ctr_drbg.h>
|
||||
|
||||
#undef inline
|
||||
|
||||
#define GIT_SSL_DEFAULT_CIPHERS "TLS1-3-AES-128-GCM-SHA256:TLS1-3-AES-256-GCM-SHA384:TLS1-3-CHACHA20-POLY1305-SHA256:TLS-ECDHE-ECDSA-WITH-AES-128-GCM-SHA256:TLS-ECDHE-RSA-WITH-AES-128-GCM-SHA256:TLS-ECDHE-ECDSA-WITH-AES-256-GCM-SHA384:TLS-ECDHE-RSA-WITH-AES-256-GCM-SHA384:TLS-ECDHE-ECDSA-WITH-CHACHA20-POLY1305-SHA256:TLS-ECDHE-RSA-WITH-CHACHA20-POLY1305-SHA256:TLS-DHE-RSA-WITH-AES-128-GCM-SHA256:TLS-DHE-RSA-WITH-AES-256-GCM-SHA384:TLS-DHE-RSA-WITH-CHACHA20-POLY1305-SHA256:TLS-ECDHE-ECDSA-WITH-AES-128-CBC-SHA256:TLS-ECDHE-RSA-WITH-AES-128-CBC-SHA256:TLS-ECDHE-ECDSA-WITH-AES-128-CBC-SHA:TLS-ECDHE-RSA-WITH-AES-128-CBC-SHA:TLS-ECDHE-ECDSA-WITH-AES-256-CBC-SHA384:TLS-ECDHE-RSA-WITH-AES-256-CBC-SHA384:TLS-ECDHE-ECDSA-WITH-AES-256-CBC-SHA:TLS-ECDHE-RSA-WITH-AES-256-CBC-SHA:TLS-DHE-RSA-WITH-AES-128-CBC-SHA256:TLS-DHE-RSA-WITH-AES-256-CBC-SHA256:TLS-RSA-WITH-AES-128-GCM-SHA256:TLS-RSA-WITH-AES-256-GCM-SHA384:TLS-RSA-WITH-AES-128-CBC-SHA256:TLS-RSA-WITH-AES-256-CBC-SHA256:TLS-RSA-WITH-AES-128-CBC-SHA:TLS-RSA-WITH-AES-256-CBC-SHA"
|
||||
#define GIT_SSL_DEFAULT_CIPHERS_COUNT 28
|
||||
|
||||
static int ciphers_list[GIT_SSL_DEFAULT_CIPHERS_COUNT];
|
||||
|
||||
static bool initialized = false;
|
||||
static mbedtls_ssl_config mbedtls_config;
|
||||
static mbedtls_ctr_drbg_context mbedtls_rng;
|
||||
static mbedtls_entropy_context mbedtls_entropy;
|
||||
|
||||
static bool has_ca_chain = false;
|
||||
static mbedtls_x509_crt mbedtls_ca_chain;
|
||||
|
||||
/**
|
||||
* This function aims to clean-up the SSL context which
|
||||
* we allocated.
|
||||
*/
|
||||
static void shutdown_ssl(void)
|
||||
{
|
||||
if (has_ca_chain) {
|
||||
mbedtls_x509_crt_free(&mbedtls_ca_chain);
|
||||
has_ca_chain = false;
|
||||
}
|
||||
|
||||
if (initialized) {
|
||||
mbedtls_ctr_drbg_free(&mbedtls_rng);
|
||||
mbedtls_ssl_config_free(&mbedtls_config);
|
||||
mbedtls_entropy_free(&mbedtls_entropy);
|
||||
initialized = false;
|
||||
}
|
||||
}
|
||||
|
||||
int git_mbedtls_stream_global_init(void)
|
||||
{
|
||||
int loaded = 0;
|
||||
char *crtpath = GIT_DEFAULT_CERT_LOCATION;
|
||||
struct stat statbuf;
|
||||
|
||||
size_t ciphers_known = 0;
|
||||
char *cipher_name = NULL;
|
||||
char *cipher_string = NULL;
|
||||
char *cipher_string_tmp = NULL;
|
||||
|
||||
mbedtls_ssl_config_init(&mbedtls_config);
|
||||
mbedtls_entropy_init(&mbedtls_entropy);
|
||||
mbedtls_ctr_drbg_init(&mbedtls_rng);
|
||||
|
||||
if (mbedtls_ssl_config_defaults(&mbedtls_config,
|
||||
MBEDTLS_SSL_IS_CLIENT,
|
||||
MBEDTLS_SSL_TRANSPORT_STREAM,
|
||||
MBEDTLS_SSL_PRESET_DEFAULT) != 0) {
|
||||
git_error_set(GIT_ERROR_SSL, "failed to initialize mbedTLS");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
/* configure TLSv1.1 or better */
|
||||
#ifdef MBEDTLS_SSL_MINOR_VERSION_2
|
||||
mbedtls_ssl_conf_min_version(&mbedtls_config, MBEDTLS_SSL_MAJOR_VERSION_3, MBEDTLS_SSL_MINOR_VERSION_2);
|
||||
#endif
|
||||
|
||||
/* verify_server_cert is responsible for making the check.
|
||||
* OPTIONAL because REQUIRED drops the certificate as soon as the check
|
||||
* is made, so we can never see the certificate and override it. */
|
||||
mbedtls_ssl_conf_authmode(&mbedtls_config, MBEDTLS_SSL_VERIFY_OPTIONAL);
|
||||
|
||||
/* set the list of allowed ciphersuites */
|
||||
ciphers_known = 0;
|
||||
cipher_string = cipher_string_tmp = git__strdup(GIT_SSL_DEFAULT_CIPHERS);
|
||||
GIT_ERROR_CHECK_ALLOC(cipher_string);
|
||||
|
||||
while ((cipher_name = git__strtok(&cipher_string_tmp, ":")) != NULL) {
|
||||
int cipherid = mbedtls_ssl_get_ciphersuite_id(cipher_name);
|
||||
if (cipherid == 0) continue;
|
||||
|
||||
if (ciphers_known >= ARRAY_SIZE(ciphers_list)) {
|
||||
git_error_set(GIT_ERROR_SSL, "out of cipher list space");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
ciphers_list[ciphers_known++] = cipherid;
|
||||
}
|
||||
git__free(cipher_string);
|
||||
|
||||
if (!ciphers_known) {
|
||||
git_error_set(GIT_ERROR_SSL, "no cipher could be enabled");
|
||||
goto cleanup;
|
||||
}
|
||||
mbedtls_ssl_conf_ciphersuites(&mbedtls_config, ciphers_list);
|
||||
|
||||
/* Seeding the random number generator */
|
||||
|
||||
if (mbedtls_ctr_drbg_seed(&mbedtls_rng, mbedtls_entropy_func,
|
||||
&mbedtls_entropy, NULL, 0) != 0) {
|
||||
git_error_set(GIT_ERROR_SSL, "failed to initialize mbedTLS entropy pool");
|
||||
goto cleanup;
|
||||
}
|
||||
|
||||
mbedtls_ssl_conf_rng(&mbedtls_config, mbedtls_ctr_drbg_random, &mbedtls_rng);
|
||||
|
||||
/* load default certificates */
|
||||
if (crtpath != NULL && stat(crtpath, &statbuf) == 0 && S_ISREG(statbuf.st_mode))
|
||||
loaded = (git_mbedtls__set_cert_location(crtpath, NULL) == 0);
|
||||
|
||||
if (!loaded && crtpath != NULL && stat(crtpath, &statbuf) == 0 && S_ISDIR(statbuf.st_mode))
|
||||
loaded = (git_mbedtls__set_cert_location(NULL, crtpath) == 0);
|
||||
|
||||
initialized = true;
|
||||
|
||||
return git_runtime_shutdown_register(shutdown_ssl);
|
||||
|
||||
cleanup:
|
||||
mbedtls_ctr_drbg_free(&mbedtls_rng);
|
||||
mbedtls_ssl_config_free(&mbedtls_config);
|
||||
mbedtls_entropy_free(&mbedtls_entropy);
|
||||
|
||||
return -1;
|
||||
}
|
||||
|
||||
static int bio_read(void *b, unsigned char *buf, size_t len)
|
||||
{
|
||||
git_stream *io = (git_stream *) b;
|
||||
return (int) git_stream_read(io, buf, min(len, INT_MAX));
|
||||
}
|
||||
|
||||
static int bio_write(void *b, const unsigned char *buf, size_t len)
|
||||
{
|
||||
git_stream *io = (git_stream *) b;
|
||||
return (int) git_stream_write(io, (const char *)buf, min(len, INT_MAX), 0);
|
||||
}
|
||||
|
||||
static int ssl_set_error(mbedtls_ssl_context *ssl, int error)
|
||||
{
|
||||
char errbuf[512];
|
||||
int ret = -1;
|
||||
|
||||
GIT_ASSERT(error != MBEDTLS_ERR_SSL_WANT_READ);
|
||||
GIT_ASSERT(error != MBEDTLS_ERR_SSL_WANT_WRITE);
|
||||
|
||||
if (error != 0)
|
||||
mbedtls_strerror( error, errbuf, 512 );
|
||||
|
||||
switch(error) {
|
||||
case 0:
|
||||
git_error_set(GIT_ERROR_SSL, "SSL error: unknown error");
|
||||
break;
|
||||
|
||||
case MBEDTLS_ERR_X509_CERT_VERIFY_FAILED:
|
||||
git_error_set(GIT_ERROR_SSL, "SSL error: %#04x [%x] - %s", error, mbedtls_ssl_get_verify_result(ssl), errbuf);
|
||||
ret = GIT_ECERTIFICATE;
|
||||
break;
|
||||
|
||||
default:
|
||||
git_error_set(GIT_ERROR_SSL, "SSL error: %#04x - %s", error, errbuf);
|
||||
}
|
||||
|
||||
return ret;
|
||||
}
|
||||
|
||||
static int ssl_teardown(mbedtls_ssl_context *ssl)
|
||||
{
|
||||
int ret = 0;
|
||||
|
||||
ret = mbedtls_ssl_close_notify(ssl);
|
||||
if (ret < 0)
|
||||
ret = ssl_set_error(ssl, ret);
|
||||
|
||||
mbedtls_ssl_free(ssl);
|
||||
return ret;
|
||||
}
|
||||
|
||||
static int verify_server_cert(mbedtls_ssl_context *ssl)
|
||||
{
|
||||
int ret = -1;
|
||||
|
||||
if ((ret = mbedtls_ssl_get_verify_result(ssl)) != 0) {
|
||||
char vrfy_buf[512];
|
||||
int len = mbedtls_x509_crt_verify_info(vrfy_buf, sizeof(vrfy_buf), "", ret);
|
||||
if (len >= 1) vrfy_buf[len - 1] = '\0'; /* Remove trailing \n */
|
||||
git_error_set(GIT_ERROR_SSL, "the SSL certificate is invalid: %#04x - %s", ret, vrfy_buf);
|
||||
return GIT_ECERTIFICATE;
|
||||
}
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
typedef struct {
|
||||
git_stream parent;
|
||||
git_stream *io;
|
||||
int owned;
|
||||
bool connected;
|
||||
char *host;
|
||||
mbedtls_ssl_context *ssl;
|
||||
git_cert_x509 cert_info;
|
||||
} mbedtls_stream;
|
||||
|
||||
|
||||
static int mbedtls_connect(git_stream *stream)
|
||||
{
|
||||
int ret;
|
||||
mbedtls_stream *st = (mbedtls_stream *) stream;
|
||||
|
||||
if (st->owned && (ret = git_stream_connect(st->io)) < 0)
|
||||
return ret;
|
||||
|
||||
st->connected = true;
|
||||
|
||||
mbedtls_ssl_set_hostname(st->ssl, st->host);
|
||||
|
||||
mbedtls_ssl_set_bio(st->ssl, st->io, bio_write, bio_read, NULL);
|
||||
|
||||
if ((ret = mbedtls_ssl_handshake(st->ssl)) != 0)
|
||||
return ssl_set_error(st->ssl, ret);
|
||||
|
||||
return verify_server_cert(st->ssl);
|
||||
}
|
||||
|
||||
static int mbedtls_certificate(git_cert **out, git_stream *stream)
|
||||
{
|
||||
unsigned char *encoded_cert;
|
||||
mbedtls_stream *st = (mbedtls_stream *) stream;
|
||||
|
||||
const mbedtls_x509_crt *cert = mbedtls_ssl_get_peer_cert(st->ssl);
|
||||
if (!cert) {
|
||||
git_error_set(GIT_ERROR_SSL, "the server did not provide a certificate");
|
||||
return -1;
|
||||
}
|
||||
|
||||
/* Retrieve the length of the certificate first */
|
||||
if (cert->raw.len == 0) {
|
||||
git_error_set(GIT_ERROR_NET, "failed to retrieve certificate information");
|
||||
return -1;
|
||||
}
|
||||
|
||||
encoded_cert = git__malloc(cert->raw.len);
|
||||
GIT_ERROR_CHECK_ALLOC(encoded_cert);
|
||||
memcpy(encoded_cert, cert->raw.p, cert->raw.len);
|
||||
|
||||
st->cert_info.parent.cert_type = GIT_CERT_X509;
|
||||
st->cert_info.data = encoded_cert;
|
||||
st->cert_info.len = cert->raw.len;
|
||||
|
||||
*out = &st->cert_info.parent;
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int mbedtls_set_proxy(git_stream *stream, const git_proxy_options *proxy_options)
|
||||
{
|
||||
mbedtls_stream *st = (mbedtls_stream *) stream;
|
||||
|
||||
return git_stream_set_proxy(st->io, proxy_options);
|
||||
}
|
||||
|
||||
static ssize_t mbedtls_stream_write(git_stream *stream, const char *data, size_t len, int flags)
|
||||
{
|
||||
mbedtls_stream *st = (mbedtls_stream *) stream;
|
||||
int written;
|
||||
|
||||
GIT_UNUSED(flags);
|
||||
|
||||
/*
|
||||
* `mbedtls_ssl_write` can only represent INT_MAX bytes
|
||||
* written via its return value. We thus need to clamp
|
||||
* the maximum number of bytes written.
|
||||
*/
|
||||
len = min(len, INT_MAX);
|
||||
|
||||
if ((written = mbedtls_ssl_write(st->ssl, (const unsigned char *)data, len)) <= 0)
|
||||
return ssl_set_error(st->ssl, written);
|
||||
|
||||
return written;
|
||||
}
|
||||
|
||||
static ssize_t mbedtls_stream_read(git_stream *stream, void *data, size_t len)
|
||||
{
|
||||
mbedtls_stream *st = (mbedtls_stream *) stream;
|
||||
int ret;
|
||||
|
||||
if ((ret = mbedtls_ssl_read(st->ssl, (unsigned char *)data, len)) <= 0)
|
||||
ssl_set_error(st->ssl, ret);
|
||||
|
||||
return ret;
|
||||
}
|
||||
|
||||
static int mbedtls_stream_close(git_stream *stream)
|
||||
{
|
||||
mbedtls_stream *st = (mbedtls_stream *) stream;
|
||||
int ret = 0;
|
||||
|
||||
if (st->connected && (ret = ssl_teardown(st->ssl)) != 0)
|
||||
return -1;
|
||||
|
||||
st->connected = false;
|
||||
|
||||
return st->owned ? git_stream_close(st->io) : 0;
|
||||
}
|
||||
|
||||
static void mbedtls_stream_free(git_stream *stream)
|
||||
{
|
||||
mbedtls_stream *st = (mbedtls_stream *) stream;
|
||||
|
||||
if (st->owned)
|
||||
git_stream_free(st->io);
|
||||
|
||||
git__free(st->host);
|
||||
git__free(st->cert_info.data);
|
||||
mbedtls_ssl_free(st->ssl);
|
||||
git__free(st->ssl);
|
||||
git__free(st);
|
||||
}
|
||||
|
||||
static int mbedtls_stream_wrap(
|
||||
git_stream **out,
|
||||
git_stream *in,
|
||||
const char *host,
|
||||
int owned)
|
||||
{
|
||||
mbedtls_stream *st;
|
||||
int error;
|
||||
|
||||
st = git__calloc(1, sizeof(mbedtls_stream));
|
||||
GIT_ERROR_CHECK_ALLOC(st);
|
||||
|
||||
st->io = in;
|
||||
st->owned = owned;
|
||||
|
||||
st->ssl = git__malloc(sizeof(mbedtls_ssl_context));
|
||||
GIT_ERROR_CHECK_ALLOC(st->ssl);
|
||||
mbedtls_ssl_init(st->ssl);
|
||||
if (mbedtls_ssl_setup(st->ssl, &mbedtls_config)) {
|
||||
git_error_set(GIT_ERROR_SSL, "failed to create ssl object");
|
||||
error = -1;
|
||||
goto out_err;
|
||||
}
|
||||
|
||||
st->host = git__strdup(host);
|
||||
GIT_ERROR_CHECK_ALLOC(st->host);
|
||||
|
||||
st->parent.version = GIT_STREAM_VERSION;
|
||||
st->parent.encrypted = 1;
|
||||
st->parent.proxy_support = git_stream_supports_proxy(st->io);
|
||||
st->parent.connect = mbedtls_connect;
|
||||
st->parent.certificate = mbedtls_certificate;
|
||||
st->parent.set_proxy = mbedtls_set_proxy;
|
||||
st->parent.read = mbedtls_stream_read;
|
||||
st->parent.write = mbedtls_stream_write;
|
||||
st->parent.close = mbedtls_stream_close;
|
||||
st->parent.free = mbedtls_stream_free;
|
||||
|
||||
*out = (git_stream *) st;
|
||||
return 0;
|
||||
|
||||
out_err:
|
||||
/* ESP FIX: do NOT close/free st->io here — on error the caller keeps
|
||||
* ownership of `in` (git_mbedtls_stream_new closes+frees it right after;
|
||||
* the vendored code freed it here too → double free → tlsf abort). */
|
||||
mbedtls_ssl_free(st->ssl);
|
||||
git__free(st->ssl);
|
||||
git__free(st);
|
||||
|
||||
return error;
|
||||
}
|
||||
|
||||
int git_mbedtls_stream_wrap(
|
||||
git_stream **out,
|
||||
git_stream *in,
|
||||
const char *host)
|
||||
{
|
||||
return mbedtls_stream_wrap(out, in, host, 0);
|
||||
}
|
||||
|
||||
int git_mbedtls_stream_new(
|
||||
git_stream **out,
|
||||
const char *host,
|
||||
const char *port)
|
||||
{
|
||||
git_stream *stream;
|
||||
int error;
|
||||
|
||||
GIT_ASSERT_ARG(out);
|
||||
GIT_ASSERT_ARG(host);
|
||||
GIT_ASSERT_ARG(port);
|
||||
|
||||
if ((error = git_socket_stream_new(&stream, host, port)) < 0)
|
||||
return error;
|
||||
|
||||
if ((error = mbedtls_stream_wrap(out, stream, host, 1)) < 0) {
|
||||
git_stream_close(stream);
|
||||
git_stream_free(stream);
|
||||
}
|
||||
|
||||
return error;
|
||||
}
|
||||
|
||||
int git_mbedtls__set_cert_location(const char *file, const char *path)
|
||||
{
|
||||
int ret = 0;
|
||||
char errbuf[512];
|
||||
|
||||
GIT_ASSERT_ARG(file || path);
|
||||
|
||||
if (has_ca_chain)
|
||||
mbedtls_x509_crt_free(&mbedtls_ca_chain);
|
||||
|
||||
mbedtls_x509_crt_init(&mbedtls_ca_chain);
|
||||
|
||||
if (file)
|
||||
ret = mbedtls_x509_crt_parse_file(&mbedtls_ca_chain, file);
|
||||
|
||||
if (ret >= 0 && path)
|
||||
ret = mbedtls_x509_crt_parse_path(&mbedtls_ca_chain, path);
|
||||
|
||||
/* mbedtls_x509_crt_parse_path returns the number of invalid certs on success */
|
||||
if (ret < 0) {
|
||||
mbedtls_x509_crt_free(&mbedtls_ca_chain);
|
||||
mbedtls_strerror( ret, errbuf, 512 );
|
||||
git_error_set(GIT_ERROR_SSL, "failed to load CA certificates: %#04x - %s", ret, errbuf);
|
||||
return -1;
|
||||
}
|
||||
|
||||
mbedtls_ssl_conf_ca_chain(&mbedtls_config, &mbedtls_ca_chain, NULL);
|
||||
has_ca_chain = true;
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
#else
|
||||
|
||||
#include "stream.h"
|
||||
|
||||
int git_mbedtls_stream_global_init(void)
|
||||
{
|
||||
return 0;
|
||||
}
|
||||
|
||||
#endif
|
||||
@@ -80,6 +80,32 @@ flash-sd:
|
||||
monitor-sd:
|
||||
espflash monitor --elf {{elf_sd}}
|
||||
|
||||
# SD primitive-op micro-benchmark — attributes the ~700 ms/loose-object write
|
||||
# floor (docs/tradeoff-curves/sync-commit-staging.md). Writes only to /sd/sdbench.
|
||||
build-bench:
|
||||
{{esp_env}} cargo build --release --bin sd_bench
|
||||
|
||||
# build + flash + monitor the SD bench
|
||||
flash-bench:
|
||||
{{esp_env}} cargo run --release --bin sd_bench
|
||||
|
||||
# serial monitor for the SD bench, with decoded backtraces
|
||||
monitor-bench:
|
||||
espflash monitor --elf target/xtensa-esp32s3-espidf/release/sd_bench
|
||||
|
||||
# git-level micro-benchmark — localizes the ~700 ms/object libgit2 overhead
|
||||
# (docs/tradeoff-curves/sync-commit-staging.md). Read-mostly on /sd/repo.
|
||||
build-gitbench:
|
||||
{{esp_env}} {{git_env}} cargo build --release --bin git_bench --features git
|
||||
|
||||
# build + flash + monitor the git-level bench
|
||||
flash-gitbench:
|
||||
{{esp_env}} {{git_env}} cargo run --release --bin git_bench --features git
|
||||
|
||||
# serial monitor for the git-level bench, with decoded backtraces
|
||||
monitor-gitbench:
|
||||
espflash monitor --elf target/xtensa-esp32s3-espidf/release/git_bench
|
||||
|
||||
# Spike 9 — build the boot splash spike (no .env needed)
|
||||
build-splash:
|
||||
{{esp_env}} cargo build --release --bin splash
|
||||
@@ -112,7 +138,7 @@ build-git-push:
|
||||
# (so the `storage` FAT partition exists) and sets 16 MB flash. Uses espflash
|
||||
# directly, not `cargo run`, so the table is applied only to this binary.
|
||||
flash-git-push: build-git-push
|
||||
espflash flash --monitor --partition-table {{partition_table}} --flash-size 16mb {{elf_git_push}}
|
||||
espflash flash --monitor --baud 921600 --partition-table {{partition_table}} --flash-size 16mb {{elf_git_push}}
|
||||
|
||||
# serial monitor for the git push, with decoded backtraces
|
||||
monitor-git-push:
|
||||
@@ -125,7 +151,7 @@ build-git-sync:
|
||||
# Milestone #2A — flash + monitor. Applies the same custom partition table as
|
||||
# git-push (the `storage` FAT partition holds the persistent clone).
|
||||
flash-git-sync: build-git-sync
|
||||
espflash flash --monitor --partition-table {{partition_table}} --flash-size 16mb {{elf_git_sync}}
|
||||
espflash flash --monitor --baud 921600 --partition-table {{partition_table}} --flash-size 16mb {{elf_git_sync}}
|
||||
|
||||
# serial monitor for the git sync, with decoded backtraces
|
||||
monitor-git-sync:
|
||||
@@ -161,18 +187,21 @@ ports:
|
||||
# The `_`-prefixed recipes are shared internals (hidden from `just --list`).
|
||||
sd_repo_dir := "repo"
|
||||
|
||||
# Full prep of a fresh card: copy the notes repo + write config, then eject.
|
||||
# Run this once per card. <repo-src> is a clone made on this computer.
|
||||
# Full prep of a fresh card: copy the notes repo, seed the git-tracked config
|
||||
# files (first-time only), write the runtime config, then eject. Run this once
|
||||
# per card. <repo-src> is a clone made on this computer.
|
||||
init repo_src sd_volume="":
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
vol="$(just _card "{{sd_volume}}" | tail -n1)"
|
||||
just _load-repo "{{repo_src}}" "$vol"
|
||||
just _seed-configs "$vol"
|
||||
just _write-conf "$vol" "{{repo_src}}"
|
||||
just _eject "$vol"
|
||||
|
||||
# (Re)copy just the notes repo to /sd/repo (full clone, gitignored paths
|
||||
# excluded), then eject — e.g. to refresh the card after big upstream changes.
|
||||
# (Re)copy just the notes repo to /sd/repo: fast-forward the source clone from
|
||||
# its remote first, then rsync it across (full clone, gitignored paths excluded)
|
||||
# and eject — e.g. to refresh the card after the device pushed new notes.
|
||||
load repo_src sd_volume="":
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
@@ -241,7 +270,10 @@ _card sd_volume="":
|
||||
# .gitignore ignores is excluded (node_modules is 3.9 GB, gitignored, never in
|
||||
# .git) — the device needs .git + the checkout (~720 MB), not the JS deps or
|
||||
# secrets like firmware/.env. Copying (not a fresh `git clone` of the local
|
||||
# path) preserves origin → GitHub so on-device fetch/push still work.
|
||||
# path) preserves origin → GitHub so on-device fetch/push still work — except
|
||||
# the URL scheme: the card copy's origin is rewritten to HTTPS at the end,
|
||||
# because the device's libgit2 has no SSH transport (HTTPS+PAT over mbedTLS
|
||||
# only). The source clone is never touched.
|
||||
_load-repo repo_src vol:
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
@@ -259,6 +291,29 @@ _load-repo repo_src vol:
|
||||
echo "warning: origin ($origin) != TW_REMOTE_URL (${TW_REMOTE_URL})"
|
||||
fi
|
||||
|
||||
# Refresh the source clone from its remote before mirroring, so the card
|
||||
# carries the latest — e.g. notes the device pushed since the last load.
|
||||
# Fast-forward-only and best-effort: a dirty tree, diverged history, detached
|
||||
# HEAD, or being offline just warns and copies the current on-disk state
|
||||
# rather than aborting the card prep. Set TW_NO_PULL=1 to skip the pull.
|
||||
case "$origin" in
|
||||
https://*|git@*|ssh://*|git://*)
|
||||
if [ -n "${TW_NO_PULL:-}" ]; then
|
||||
echo "TW_NO_PULL set — skipping pull, copying current on-disk state"
|
||||
else
|
||||
branch="$(git -C "$src" symbolic-ref --quiet --short HEAD || true)"
|
||||
if [ -z "$branch" ]; then
|
||||
echo "warning: '$src' is in detached HEAD — skipping pull, copying current state" >&2
|
||||
elif git -C "$src" pull --ff-only origin "$branch"; then
|
||||
:
|
||||
else
|
||||
echo "warning: 'git pull --ff-only origin $branch' failed (dirty, diverged, or offline) —" >&2
|
||||
echo " copying the current on-disk state instead" >&2
|
||||
fi
|
||||
fi
|
||||
;;
|
||||
esac
|
||||
|
||||
dest="{{vol}}/{{sd_repo_dir}}"
|
||||
echo "copying repo: $src -> $dest"
|
||||
mkdir -p "$dest"
|
||||
@@ -280,6 +335,101 @@ _load-repo repo_src vol:
|
||||
# -P for progress on the ~700 MB copy. Scoped to repo/, never the card root.
|
||||
rsync -rtP --delete --modify-window=1 --exclude-from="$ignore_list" "$src/" "$dest/"
|
||||
|
||||
# The device's libgit2 speaks HTTPS+PAT only (mbedTLS — no SSH transport),
|
||||
# so an SSH origin copied verbatim fails every on-device push/fetch with
|
||||
# "unsupported URL protocol" (found the hard way, 2026-07-13). Rewrite the
|
||||
# CARD copy's origin to the HTTPS equivalent; the source clone keeps its
|
||||
# own URL. Its embedded trust store carries GitHub's roots, hence the
|
||||
# non-github warning.
|
||||
case "$origin" in
|
||||
git@*:*)
|
||||
host="${origin#git@}"; host="${host%%:*}"
|
||||
card_origin="https://$host/${origin#git@*:}"
|
||||
;;
|
||||
ssh://git@*)
|
||||
rest="${origin#ssh://git@}"
|
||||
card_origin="https://${rest%%[:/]*}/${rest#*/}"
|
||||
;;
|
||||
https://*) card_origin="$origin" ;;
|
||||
*) card_origin="" ;;
|
||||
esac
|
||||
if [ -n "$card_origin" ]; then
|
||||
if [ "$card_origin" != "$origin" ]; then
|
||||
echo "rewriting card origin for the device: $origin -> $card_origin"
|
||||
fi
|
||||
git -C "$dest" remote set-url origin "$card_origin"
|
||||
case "$card_origin" in
|
||||
https://github.com/*) ;;
|
||||
*) echo "warning: origin host isn't github.com — the device's embedded trust store" \
|
||||
"only carries GitHub roots, so on-device TLS will fail without its CA" ;;
|
||||
esac
|
||||
elif [ -n "$origin" ]; then
|
||||
echo "warning: origin '$origin' has no HTTPS equivalent I can derive —"
|
||||
echo " on-device push/fetch will fail; set it by hand:"
|
||||
echo " git -C '$dest' remote set-url origin https://github.com/<owner>/<repo>.git"
|
||||
fi
|
||||
|
||||
# First-time setup: seed the two git-tracked config files into the card's repo/
|
||||
# so they commit + sync on the device's first `:sync`. Writes ONLY a file that is
|
||||
# absent — a card whose clone already carries `.typoena.toml` /
|
||||
# `.typoena.snippets.json` keeps its own, so a re-`init` never clobbers a synced
|
||||
# library. The starter `.typoena.toml` mirrors the editor defaults; the snippet
|
||||
# library is assembled from the curated catalog (snippets-catalog/, opt-in per
|
||||
# group) with `jq`. Non-interactive runs take every default (all groups).
|
||||
_seed-configs vol:
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
repo="{{vol}}/{{sd_repo_dir}}"
|
||||
catalog="{{justfile_directory()}}/snippets-catalog"
|
||||
interactive=1; [ -t 0 ] || interactive=0
|
||||
|
||||
# ask_yn "label" → 0 (yes) / 1 (no); default yes, non-interactive → yes.
|
||||
ask_yn() {
|
||||
local ans
|
||||
if [ "$interactive" = 0 ]; then return 0; fi
|
||||
read -r -p " $1 [Y/n]: " ans || ans=""
|
||||
[[ "${ans:-Y}" =~ ^[Yy] ]]
|
||||
}
|
||||
|
||||
# ── starter prefs (.typoena.toml) ────────────────────────────────────────
|
||||
prefs="$repo/.typoena.toml"
|
||||
if [ -e "$prefs" ]; then
|
||||
echo "keeping existing .typoena.toml"
|
||||
elif ask_yn "seed a starter .typoena.toml (editor defaults)?"; then
|
||||
# printf per line (matching _write-conf), so no heredoc indentation trap.
|
||||
{
|
||||
printf '# Typoena editor preferences — hand-editable, git-tracked.\n'
|
||||
printf '# Edit here, or change live from the Cmd-P palette (type `>`).\n'
|
||||
printf 'save_on_idle = true\n'
|
||||
printf 'format_on_save = true\n'
|
||||
printf 'line_numbers = true\n'
|
||||
printf 'theme = "light"\n'
|
||||
printf 'auto_sync = "10m"\n'
|
||||
} > "$prefs"
|
||||
echo "wrote .typoena.toml"
|
||||
fi
|
||||
|
||||
# ── snippet library (.typoena.snippets.json) from the catalog ────────────
|
||||
snips="$repo/.typoena.snippets.json"
|
||||
if [ -e "$snips" ]; then
|
||||
echo "keeping existing .typoena.snippets.json"
|
||||
elif ! command -v jq >/dev/null 2>&1; then
|
||||
echo "warning: jq not found — skipping the snippet catalog (install jq, or" >&2
|
||||
echo " hand-write $snips later)" >&2
|
||||
else
|
||||
echo "snippet catalog — choose groups to include:" >&2
|
||||
picked=()
|
||||
ask_yn "Symbols (→ ≠ × · ° € œ)" && picked+=("$catalog/symbols.json")
|
||||
ask_yn "Structure (todo, link, img, table, code)" && picked+=("$catalog/structure.json")
|
||||
ask_yn "Prose (booknotes, reference, bias, capture, standard)" && picked+=("$catalog/prose.json")
|
||||
if [ "${#picked[@]}" -gt 0 ]; then
|
||||
jq -s 'add' "${picked[@]}" > "$snips"
|
||||
echo "wrote .typoena.snippets.json ($(jq 'length' "$snips") snippets, ${#picked[@]} group(s))"
|
||||
else
|
||||
echo "no groups chosen — skipping .typoena.snippets.json"
|
||||
fi
|
||||
fi
|
||||
|
||||
# Resolve + write <vol>/typoena.conf (Wi-Fi + PAT + git identity). Each value
|
||||
# runs a ladder: firmware/.env (loaded via `set dotenv-load`) → derived from
|
||||
# tools already on the machine (git config, gh, the active Wi-Fi network + its
|
||||
|
||||
@@ -44,9 +44,42 @@ CONFIG_SPIRAM=y
|
||||
CONFIG_SPIRAM_MODE_OCT=y
|
||||
CONFIG_SPIRAM_USE_MALLOC=y
|
||||
|
||||
# FatFS fast seek (sync-latency root cause, 2026-07-12). Without it, lseek
|
||||
# resolves by walking the file's FAT cluster chain over SPI — forward from the
|
||||
# current position, from the CHAIN HEAD on any backward seek. sd_bench measured
|
||||
# ~190 ms per long seek into the 263 MB packfile (5.8 ms at offset 0), and
|
||||
# libgit2 pays ~8 such seeks per loose-object write via p_mmap's lseek+read →
|
||||
# the ~1.5 s/object commit cost (docs/tradeoff-curves/sync-commit-staging.md).
|
||||
# Fast seek builds an in-memory cluster-link map (CLMT) per READ-mode file,
|
||||
# making lseek O(1); write-mode files transparently fall back to the walk. The
|
||||
# buffer is per-open-file, 4 B × size: 256 words = 1 KB, covering ~127 fragments
|
||||
# (the default 64 covers ~31 — a freshly-provisioned pack is near-contiguous,
|
||||
# but don't let card aging silently disable the fix; vfs_fat falls back to slow
|
||||
# seeks when the map doesn't fit).
|
||||
CONFIG_FATFS_USE_FASTSEEK=y
|
||||
CONFIG_FATFS_FAST_SEEK_BUFFER_SIZE=256
|
||||
|
||||
# Silence the legacy-I2C boot warning. esp-idf-hal's i2c.rs is always compiled
|
||||
# and binds the old `driver/i2c.h` API, so the legacy driver's TU (and its
|
||||
# `__attribute__((constructor))` deprecation warning) gets linked into every
|
||||
# image — even though Typoena uses no I2C at all (EPD/SD are SPI, keyboard is
|
||||
# USB). This flag wraps that constructor out. Safe here: we link neither the
|
||||
# new i2c_master driver nor call the old one, so the conflict-abort check it
|
||||
# also removes can never fire.
|
||||
CONFIG_I2C_SKIP_LEGACY_CONFLICT_CHECK=y
|
||||
|
||||
# TLS trust store (Spike 6 — Wi-Fi + TLS, the gate for Spike 7 git push).
|
||||
# The certificate bundle backs esp_crt_bundle_attach so an HTTPS GET to
|
||||
# api.github.com validates against real roots. FULL rather than the common
|
||||
# subset so a less common CA in the chain can't surprise us on the bench.
|
||||
CONFIG_MBEDTLS_CERTIFICATE_BUNDLE=y
|
||||
CONFIG_MBEDTLS_CERTIFICATE_BUNDLE_DEFAULT_FULL=y
|
||||
|
||||
# mbedTLS allocations go to PSRAM. The default (internal-only) needs ~33 KB of
|
||||
# contiguous internal RAM per TLS connection (two ~17 KB I/O buffers + contexts)
|
||||
# at the moment `:sync` pushes — with Wi-Fi, USB host, the editor and libgit2
|
||||
# all resident, mbedtls_ssl_setup failed exactly there on the first real-repo
|
||||
# push (2026-07-13; the failure then tripped the vendored stream double-free —
|
||||
# see components/libgit2/esp_mbedtls_stream.c). TLS buffers are CPU-only data,
|
||||
# so PSRAM is safe; the handshake is network-bound, not memory-bandwidth-bound.
|
||||
CONFIG_MBEDTLS_EXTERNAL_MEM_ALLOC=y
|
||||
|
||||
58
firmware/snippets-catalog/prose.json
Normal file
58
firmware/snippets-catalog/prose.json
Normal file
@@ -0,0 +1,58 @@
|
||||
{
|
||||
"Book notes": {
|
||||
"prefix": "booknotes",
|
||||
"body": ["# $1", "", "## $2 - $3", "", "## What the book is about", ""],
|
||||
"description": "Reading notes (title / author - year)"
|
||||
},
|
||||
"Reference block": {
|
||||
"prefix": "reference",
|
||||
"body": ["___", "", "## References", ""],
|
||||
"description": "Reference block"
|
||||
},
|
||||
"Cognitive bias": {
|
||||
"prefix": "bias",
|
||||
"body": [
|
||||
"# $1",
|
||||
"",
|
||||
"## Misconception",
|
||||
"",
|
||||
"$3",
|
||||
"",
|
||||
"## The reality",
|
||||
"",
|
||||
"$4",
|
||||
"",
|
||||
"___",
|
||||
"",
|
||||
"## References",
|
||||
"",
|
||||
"- You are not so smart - p. $2",
|
||||
""
|
||||
],
|
||||
"description": "Cognitive-bias note"
|
||||
},
|
||||
"Capture": {
|
||||
"prefix": "capture",
|
||||
"body": ["# $1", "", "## Captured inspiration", "", "$2", ""],
|
||||
"description": "Inspiration capture"
|
||||
},
|
||||
"Standard": {
|
||||
"prefix": "standard",
|
||||
"body": [
|
||||
"# $1",
|
||||
"",
|
||||
"## Key points",
|
||||
"",
|
||||
"## Mistakes to avoid",
|
||||
"",
|
||||
"___",
|
||||
"",
|
||||
"## Examples",
|
||||
"",
|
||||
"___",
|
||||
"",
|
||||
"## References"
|
||||
],
|
||||
"description": "Standard / playbook"
|
||||
}
|
||||
}
|
||||
27
firmware/snippets-catalog/structure.json
Normal file
27
firmware/snippets-catalog/structure.json
Normal file
@@ -0,0 +1,27 @@
|
||||
{
|
||||
"Todo item": {
|
||||
"prefix": "todo",
|
||||
"body": "- [ ] ",
|
||||
"description": "Task-list item"
|
||||
},
|
||||
"Link": {
|
||||
"prefix": "link",
|
||||
"body": "[$1]($2)$0",
|
||||
"description": "Inline link"
|
||||
},
|
||||
"Image": {
|
||||
"prefix": "img",
|
||||
"body": "$0",
|
||||
"description": "Image"
|
||||
},
|
||||
"Table": {
|
||||
"prefix": "table",
|
||||
"body": ["| $1 | $2 |", "|-|-|", "| $3 | $4 |"],
|
||||
"description": "Two-column table"
|
||||
},
|
||||
"Code block": {
|
||||
"prefix": "code",
|
||||
"body": ["```$1", "$2", "```"],
|
||||
"description": "Fenced code block"
|
||||
}
|
||||
}
|
||||
37
firmware/snippets-catalog/symbols.json
Normal file
37
firmware/snippets-catalog/symbols.json
Normal file
@@ -0,0 +1,37 @@
|
||||
{
|
||||
"Arrow": {
|
||||
"prefix": "arrow",
|
||||
"body": "→",
|
||||
"description": "Rightwards arrow (→)"
|
||||
},
|
||||
"Not equal": {
|
||||
"prefix": "neq",
|
||||
"body": "≠",
|
||||
"description": "Not-equal sign (≠)"
|
||||
},
|
||||
"Times": {
|
||||
"prefix": "times",
|
||||
"body": "×",
|
||||
"description": "Multiplication sign (×)"
|
||||
},
|
||||
"Middle dot": {
|
||||
"prefix": "middot",
|
||||
"body": "·",
|
||||
"description": "Middle dot (·), inclusive writing"
|
||||
},
|
||||
"Degree": {
|
||||
"prefix": "deg",
|
||||
"body": "°",
|
||||
"description": "Degree sign (°)"
|
||||
},
|
||||
"Euro": {
|
||||
"prefix": "euro",
|
||||
"body": "€",
|
||||
"description": "Euro sign (€)"
|
||||
},
|
||||
"OE ligature": {
|
||||
"prefix": "edanso",
|
||||
"body": "œ",
|
||||
"description": "Ligature œ (e dans l'o)"
|
||||
}
|
||||
}
|
||||
371
firmware/src/bin/git_bench.rs
Normal file
371
firmware/src/bin/git_bench.rs
Normal file
@@ -0,0 +1,371 @@
|
||||
//! git-level micro-benchmark — localizes the ~700 ms/object libgit2 overhead the
|
||||
//! `:sync` commit split showed (2026-07-12), now that `sd_bench` proved the raw
|
||||
//! card does a *full* loose-object write (stat+create+write+rename) in ~86 ms.
|
||||
//! The ~8× gap between that and `write_tree`'s 710 ms lives inside libgit2, not
|
||||
//! FAT — this bench times the git2 ODB/index primitives in isolation to find it.
|
||||
//!
|
||||
//! HEADLINE OP (since the 2026-07-12 real-repo run): `splice stage→tree` — the
|
||||
//! O(depth) TreeBuilder walk that replaces the index-based commit entirely
|
||||
//! (docs/tradeoff-curves/sync-commit-staging.md). It runs FIRST so its first iteration is
|
||||
//! the cold number; acceptance bar: **sub-second cold on the real 570 MB-pack
|
||||
//! clone, heap staying healthy**. The index paths it supersedes run LAST, for
|
||||
//! regression tracking — they previously OOM'd, and a late crash can't cost the
|
||||
//! splice data.
|
||||
//!
|
||||
//! Read-mostly on `/sd/repo`: the only writes are unreferenced ("orphan") loose
|
||||
//! blobs/trees/commits — never reachable from a ref, so never pushed, and
|
||||
//! gc-able. Safe on the test card.
|
||||
//!
|
||||
//! Flash with `just flash-gitbench` (needs the `git` feature; env in the recipe).
|
||||
|
||||
use std::time::Instant;
|
||||
|
||||
use anyhow::{Context, Result};
|
||||
use esp_idf_svc::hal::delay::FreeRtos;
|
||||
use git2::{IndexEntry, IndexTime, ObjectType, Oid, Repository, Signature, Tree};
|
||||
|
||||
use firmware::git_sync::GIT_STACK;
|
||||
use firmware::persistence::{Storage, REPO_DIR};
|
||||
|
||||
const BUILD_TAG: &str = concat!("build ", env!("BUILD_TIME"), " @", env!("BUILD_GIT"));
|
||||
|
||||
/// Iterations per op. Small — some ops write to the card, and the first vs rest
|
||||
/// spread (min vs max) is itself the signal (e.g. cold vs warm, write vs
|
||||
/// freshen-skip). Kept low (3) on the real 570 MB-pack clone so a slow op still
|
||||
/// finishes in seconds.
|
||||
const N: usize = 3;
|
||||
|
||||
fn main() -> Result<()> {
|
||||
esp_idf_svc::sys::link_patches();
|
||||
esp_idf_svc::log::EspLogger::initialize_default();
|
||||
log::info!("Typoena — git-level bench, {BUILD_TAG}");
|
||||
|
||||
// libgit2 nests ~67 KB of GIT_PATH_MAX stack buffers (postmortem #3), so the
|
||||
// git work must run on the same 96 KB stack the real git service uses. On the
|
||||
// small main-task stack `index.write()` overflows → nested panic → boot loop.
|
||||
let handle = std::thread::Builder::new()
|
||||
.name("git_bench".into())
|
||||
.stack_size(GIT_STACK)
|
||||
.spawn(run)
|
||||
.expect("spawn git_bench thread");
|
||||
match handle.join() {
|
||||
Ok(Ok(())) => log::info!("git_bench: done"),
|
||||
Ok(Err(e)) => log::error!("git_bench failed: {e:?}"),
|
||||
Err(_) => log::error!("git_bench thread panicked"),
|
||||
}
|
||||
loop {
|
||||
FreeRtos::delay_ms(1000);
|
||||
}
|
||||
}
|
||||
|
||||
fn run() -> Result<()> {
|
||||
// libgit2 holds the pack + idx (+ commit-graph) fds open and reads loose
|
||||
// objects on top; the editor's default 4-FD budget can't cover read_tree.
|
||||
let _sd = Storage::mount_for_git().context("mounting SD")?;
|
||||
|
||||
// A 32 MB default mwindow window (mwindow.c) would git__malloc > PSRAM on the
|
||||
// real 570 MB pack; small windows keep each p_mmap read cheap, and the
|
||||
// esp_map cache keeps them from being re-read on every freshen→refresh.
|
||||
// SAFETY: process-global libgit2 options, set once before any repo work.
|
||||
unsafe {
|
||||
git2::opts::set_mwindow_size(256 * 1024).ok();
|
||||
git2::opts::set_mwindow_mapped_limit(4 * 1024 * 1024).ok();
|
||||
}
|
||||
|
||||
// Repository open — one-time, but shows the cost of scanning .git (config,
|
||||
// refs, ODB backends/packs) which every later op may implicitly refresh.
|
||||
let t = Instant::now();
|
||||
let repo = Repository::open(REPO_DIR)
|
||||
.with_context(|| format!("opening git repo at {REPO_DIR}"))?;
|
||||
log::info!("Repository::open {:.1} ms", t.elapsed().as_micros() as f64 / 1000.0);
|
||||
log_map_stats("open");
|
||||
|
||||
// 1) THE FIX — `splice stage→tree`, the O(depth) TreeBuilder walk
|
||||
// (docs/tradeoff-curves/sync-commit-staging.md): patch the edited file's ancestor
|
||||
// subtree chain onto HEAD's tree; never materialise the 1179-entry index,
|
||||
// never index.write(), never read_tree the whole tree. Runs FIRST so
|
||||
// iteration #1 is genuinely cold (only `open` has touched the pack).
|
||||
let head_tree = repo
|
||||
.head()?
|
||||
.peel_to_commit()
|
||||
.context("HEAD → commit")?
|
||||
.tree()
|
||||
.context("HEAD tree")?;
|
||||
// A nested path that already exists in HEAD's tree, found by an O(depth)
|
||||
// descent — NOT read_tree, which is itself the 77 s op — so the splice
|
||||
// REPLACES a real file and rebuilds a real ancestor chain, not just the root.
|
||||
let edit_path = find_edit_path(&repo, &head_tree)?;
|
||||
log::info!(
|
||||
"splice: editing {} (depth {})",
|
||||
edit_path.join("/"),
|
||||
edit_path.len()
|
||||
);
|
||||
|
||||
// The blob write is inside the timing: the real commit pays blob + trees, and
|
||||
// it keeps the number comparable to `index-free stage→tree` below. Not
|
||||
// measured here: the ref/reflog update (commit(Some("HEAD"))) — flat FAT
|
||||
// writes, ~350 ms on the toy repo.
|
||||
bench("splice stage→tree", |i| {
|
||||
let data = format!("typoena splice bench edit #{i}\n");
|
||||
let oid = repo.blob(data.as_bytes()).context("write blob")?;
|
||||
let parts: Vec<&str> = edit_path.iter().map(String::as_str).collect();
|
||||
splice(&repo, Some(&head_tree), &parts, Some(oid)).map(|_| ())
|
||||
})?;
|
||||
log_map_stats("splice");
|
||||
|
||||
// 2) commit(None, …) — create a commit OBJECT without moving HEAD or writing a
|
||||
// reflog (update_ref = None → an orphan commit, gc-able). Isolates commit-
|
||||
// object creation from the ref-update + reflog cost; splice + this projects
|
||||
// the full real-repo commit. Reuses the parent's tree (no new tree needed);
|
||||
// unique message each iter forces a real write.
|
||||
let parent = repo.head()?.peel_to_commit().context("HEAD → commit")?;
|
||||
let sig = Signature::now("typoena-bench", "bench@typoena.local").context("sig")?;
|
||||
bench("commit(None) orphan obj", |i| {
|
||||
let msg = format!("typoena git_bench orphan commit #{i}");
|
||||
repo.commit(None, &sig, &sig, &msg, &head_tree, &[&parent])
|
||||
.map(|_| ())
|
||||
.context("commit(None)")
|
||||
})?;
|
||||
log_map_stats("commit");
|
||||
|
||||
// 3) odb.write(blob) in isolation — unique content each iter forces a real
|
||||
// write (no freshen-skip). If ~100 ms the ODB write path is fine and any
|
||||
// slow op above is in the tree/ref layer; if ~1 s the cost is inside the
|
||||
// ODB write itself (deflate/sha/freshen) and the mmap cache regressed.
|
||||
let odb = repo.odb().context("opening odb")?;
|
||||
bench("odb.write(blob)", |i| {
|
||||
let data = format!("typoena git_bench orphan blob #{i} — unique so the write is real\n");
|
||||
odb.write(ObjectType::Blob, data.as_bytes())
|
||||
.map(|_| ())
|
||||
.context("odb.write")
|
||||
})?;
|
||||
log_map_stats("odb.write");
|
||||
|
||||
// 3b) LOCALIZE the commit-vs-blob gap. The fast-seek A/B (2026-07-12) left
|
||||
// `commit(None)` at 1.7 s while `odb.write` dropped to ~0.4 s — commit
|
||||
// additionally VALIDATES its parent + tree OIDs against the odb (strict
|
||||
// object creation → pack header resolves) and freshens the packed tree.
|
||||
// Price the two suspects, then re-bench commit + splice with strict
|
||||
// creation OFF. If commit collapses toward odb.write, validation was the
|
||||
// gap — and git_sync can ship with strict off (every OID it inserts
|
||||
// comes from HEAD's tree or a blob it just wrote).
|
||||
let parent_id = parent.id();
|
||||
let tree_id = head_tree.id();
|
||||
bench("odb.read_header(packed)", |i| {
|
||||
let id = if i % 2 == 0 { tree_id } else { parent_id };
|
||||
odb.read_header(id).map(|_| ()).context("read_header")
|
||||
})?;
|
||||
bench("odb.exists(missing)", |i| {
|
||||
let id = Oid::from_str(&format!("{:040x}", 0xdead_beef_u64 + i as u64))?;
|
||||
let _ = odb.exists(id); // miss → freshen fails → git_odb_refresh path
|
||||
Ok(())
|
||||
})?;
|
||||
log_map_stats("probes");
|
||||
|
||||
// Process-global libgit2 flag; this bench owns the process.
|
||||
git2::opts::strict_object_creation(false);
|
||||
bench("commit(None) [strict off]", |i| {
|
||||
let msg = format!("typoena git_bench strict-off commit #{i}");
|
||||
repo.commit(None, &sig, &sig, &msg, &head_tree, &[&parent])
|
||||
.map(|_| ())
|
||||
.context("commit strict-off")
|
||||
})?;
|
||||
bench("splice [strict off]", |i| {
|
||||
let data = format!("typoena splice strict-off edit #{i}\n");
|
||||
let oid = repo.blob(data.as_bytes()).context("write blob")?;
|
||||
let parts: Vec<&str> = edit_path.iter().map(String::as_str).collect();
|
||||
splice(&repo, Some(&head_tree), &parts, Some(oid)).map(|_| ())
|
||||
})?;
|
||||
git2::opts::strict_object_creation(true);
|
||||
log_map_stats("strict-off");
|
||||
|
||||
// 4) on-disk index LOAD (no write). Times loading all ~1179 entries from the
|
||||
// card and prints the count. We deliberately do NOT bench index.write():
|
||||
// it calls truncate_racily_clean, which diffs the whole working tree
|
||||
// against the index and — because a fresh FAT clone makes every entry look
|
||||
// "racy" (2 s mtime granularity) — re-hashes ~170 MB over SPI, up to ~10 min
|
||||
// on this repo (proven 2026-07-12, index.write max 611 s). The splice
|
||||
// never touches the on-disk index, so that path never runs.
|
||||
bench("repo.index() load", |_| {
|
||||
repo.index().map(|_| ()).context("index open")
|
||||
})?;
|
||||
let n_entries = repo.index().map(|i| i.len()).unwrap_or(0);
|
||||
log::info!("on-disk index has {n_entries} entries");
|
||||
log_map_stats("index load");
|
||||
|
||||
// 5) REFUTED ALTERNATIVE — the index-free in-memory-index commit
|
||||
// (read_tree(HEAD) + add + write_tree_to). It dodges truncate_racily_clean
|
||||
// but is still O(N_tree): the 2026-07-12 real-repo run measured ~77 s for
|
||||
// the cold read_tree and drove the mmap cache to 7.4 MB (zlib OOM). Kept
|
||||
// for regression tracking, run LAST so a crash here can't cost the splice
|
||||
// data above. The cold read_tree is now timed explicitly (the 77 s was
|
||||
// previously visible only via log timestamps); the ops above warmed only
|
||||
// ~depth of the ~158 tree windows, so this is still ~cold.
|
||||
let t = Instant::now();
|
||||
{
|
||||
let mut idx = git2::Index::new().context("Index::new")?;
|
||||
idx.read_tree(&head_tree).context("seed read_tree")?;
|
||||
}
|
||||
log::info!(
|
||||
"seed read_tree(HEAD) cold {:.1} ms",
|
||||
t.elapsed().as_micros() as f64 / 1000.0
|
||||
);
|
||||
log_map_stats("read_tree");
|
||||
|
||||
// Warm repeats: windows resident → pure CPU + cache lookups.
|
||||
bench("Index::new + read_tree", |_| {
|
||||
let mut idx = git2::Index::new().context("Index::new")?;
|
||||
idx.read_tree(&head_tree).context("read_tree")?;
|
||||
Ok(())
|
||||
})?;
|
||||
|
||||
let edit_path_bytes = edit_path.join("/").into_bytes();
|
||||
bench("index-free stage→tree", |i| {
|
||||
let mut idx = git2::Index::new().context("Index::new")?;
|
||||
idx.read_tree(&head_tree).context("read_tree")?;
|
||||
let data = format!("typoena index-free bench edit #{i}\n");
|
||||
let oid = repo.blob(data.as_bytes()).context("write blob")?;
|
||||
idx.add(&blob_entry(&edit_path_bytes, oid)).context("index.add")?;
|
||||
idx.write_tree_to(&repo).map(|_| ()).context("write_tree_to")
|
||||
})?;
|
||||
log_map_stats("index-free");
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// PROTOTYPE of the real fix (destined for `git_sync::stage_and_commit`): return
|
||||
/// a new tree OID equal to `base` with `path` set to `new` — `Some(blob)` to
|
||||
/// add/replace, `None` to delete. Reads ~depth subtree objects, writes ~depth
|
||||
/// trees; every other entry (all 1179 files, the 150 MB of images) is carried
|
||||
/// forward by OID without ever being read. `base = None` builds a fresh subtree
|
||||
/// chain (new file in a new directory). The git_sync version must additionally
|
||||
/// drop a directory entry when a delete empties its subtree; the bench only
|
||||
/// exercises replace.
|
||||
fn splice(repo: &Repository, base: Option<&Tree>, path: &[&str], new: Option<Oid>) -> Result<Oid> {
|
||||
let (head, rest) = path.split_first().context("splice: empty path")?;
|
||||
let mut tb = repo.treebuilder(base).context("treebuilder")?;
|
||||
if rest.is_empty() {
|
||||
match new {
|
||||
Some(oid) => {
|
||||
tb.insert(*head, oid, 0o100644).context("insert blob")?;
|
||||
}
|
||||
None => {
|
||||
let _ = tb.remove(*head); // already absent ⇒ nothing to delete
|
||||
}
|
||||
}
|
||||
} else {
|
||||
let sub = match base.and_then(|b| b.get_name(head)) {
|
||||
Some(e) if e.kind() == Some(ObjectType::Tree) => {
|
||||
Some(repo.find_tree(e.id()).context("loading subtree")?)
|
||||
}
|
||||
_ => None, // no such dir yet (or a blob in the way): build from empty
|
||||
};
|
||||
let new_sub = splice(repo, sub.as_ref(), rest, new)?;
|
||||
tb.insert(*head, new_sub, 0o040000).context("insert subtree")?;
|
||||
}
|
||||
tb.write().context("treebuilder write")
|
||||
}
|
||||
|
||||
/// Find a real file to "edit": descend the first subtree at each level (capped),
|
||||
/// then take the first blob of the deepest tree reached. Reads O(depth) tree
|
||||
/// objects — never `read_tree`/materialise the whole tree (that's the 77 s op
|
||||
/// this bench exists to retire).
|
||||
fn find_edit_path(repo: &Repository, root: &Tree) -> Result<Vec<String>> {
|
||||
let mut path = Vec::new();
|
||||
let mut cur_id = root.id();
|
||||
for _ in 0..6 {
|
||||
let cur = repo.find_tree(cur_id).context("descending tree")?;
|
||||
match cur.iter().find(|e| e.kind() == Some(ObjectType::Tree)) {
|
||||
Some(sub) => {
|
||||
path.push(sub.name().context("non-utf8 tree name")?.to_string());
|
||||
cur_id = sub.id();
|
||||
}
|
||||
None => break,
|
||||
}
|
||||
}
|
||||
let cur = repo.find_tree(cur_id).context("leaf tree")?;
|
||||
let blob = cur
|
||||
.iter()
|
||||
.find(|e| e.kind() == Some(ObjectType::Blob))
|
||||
.context("no blob along the first-subtree chain — pick an edit path manually")?;
|
||||
path.push(blob.name().context("non-utf8 blob name")?.to_string());
|
||||
Ok(path)
|
||||
}
|
||||
|
||||
unsafe extern "C" {
|
||||
/// Counters from the p_mmap emulation in `components/libgit2/esp_map.c`.
|
||||
/// Post cache-removal: `hits` is always 0, `misses` counts every mapping,
|
||||
/// `cached_kb` reports the LIVE mapped bytes (the mwindow working set).
|
||||
fn esp_map_stats(hits: *mut u32, misses: *mut u32, read_kb: *mut u32, cached_kb: *mut u32);
|
||||
}
|
||||
|
||||
/// Log the p_mmap counters — mappings performed, total KB read from the card,
|
||||
/// and KB currently live-mapped (should track mwindow's open windows and stay
|
||||
/// well under MWINDOW_MAPPED_LIMIT now that munmap frees immediately).
|
||||
fn log_map_stats(label: &str) {
|
||||
let (mut hits, mut misses, mut read_kb, mut cached_kb) = (0u32, 0u32, 0u32, 0u32);
|
||||
unsafe { esp_map_stats(&mut hits, &mut misses, &mut read_kb, &mut cached_kb) };
|
||||
let _ = hits; // always 0 since the cache removal; slot kept for ABI stability
|
||||
// Free heap spans PSRAM here; a drop toward 0 during write_tree/commit on the
|
||||
// real repo would point at mwindow/idx allocation pressure (or thrash) as the
|
||||
// cause of an apparent hang, not CPU.
|
||||
let free_kb = unsafe { esp_idf_svc::sys::esp_get_free_heap_size() } / 1024;
|
||||
log::info!(
|
||||
"mmap @ {label:<11} {misses} maps, {read_kb} KB read, {cached_kb} KB live, {free_kb} KB heap free"
|
||||
);
|
||||
}
|
||||
|
||||
/// Announce, time, and summarize an op. The `→ label …` line prints BEFORE the op
|
||||
/// runs, so if an op hangs on the real 570 MB-pack repo we can see which one it
|
||||
/// entered — a bare `summarize` prints only after all N iters, hiding the culprit.
|
||||
fn bench<F: FnMut(usize) -> Result<()>>(label: &str, op: F) -> Result<()> {
|
||||
log::info!("→ {label} …");
|
||||
summarize(label, time_each(op)?);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// A minimal index entry pointing at an already-written blob — for `index.add`,
|
||||
/// which (unlike `add_frombuffer`) needs no repo owner, so it works on a bare
|
||||
/// in-memory index. Only `id`, `path` and `mode` feed the tree write.
|
||||
fn blob_entry(path: &[u8], oid: Oid) -> IndexEntry {
|
||||
IndexEntry {
|
||||
ctime: IndexTime::new(0, 0),
|
||||
mtime: IndexTime::new(0, 0),
|
||||
dev: 0,
|
||||
ino: 0,
|
||||
mode: 0o100644,
|
||||
uid: 0,
|
||||
gid: 0,
|
||||
file_size: 0,
|
||||
id: oid,
|
||||
flags: 0,
|
||||
flags_extended: 0,
|
||||
path: path.to_vec(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Run `op(i)` for `i in 0..N`, returning each call's wall time in microseconds.
|
||||
fn time_each<F: FnMut(usize) -> Result<()>>(mut op: F) -> Result<Vec<u64>> {
|
||||
let mut times = Vec::with_capacity(N);
|
||||
for i in 0..N {
|
||||
let t = Instant::now();
|
||||
op(i)?;
|
||||
times.push(t.elapsed().as_micros() as u64);
|
||||
}
|
||||
Ok(times)
|
||||
}
|
||||
|
||||
/// Log min / p50 / mean / max in ms for a set of per-call microsecond timings.
|
||||
fn summarize(label: &str, mut times: Vec<u64>) {
|
||||
times.sort_unstable();
|
||||
let n = times.len();
|
||||
let mean = times.iter().sum::<u64>() / n as u64;
|
||||
let ms = |us: u64| us as f64 / 1000.0;
|
||||
log::info!(
|
||||
"{label:<26} min {:>6.1} p50 {:>6.1} mean {:>6.1} max {:>6.1} ms",
|
||||
ms(times[0]),
|
||||
ms(times[n / 2]),
|
||||
ms(mean),
|
||||
ms(times[n - 1]),
|
||||
);
|
||||
}
|
||||
212
firmware/src/bin/sd_bench.rs
Normal file
212
firmware/src/bin/sd_bench.rs
Normal file
@@ -0,0 +1,212 @@
|
||||
//! SD/FAT primitive-op micro-benchmark — investigating the ~700 ms-per-loose-
|
||||
//! object write floor found in the `:sync` commit split (2026-07-12, see
|
||||
//! `docs/tradeoff-curves/sync-commit-staging.md`).
|
||||
//!
|
||||
//! The split showed a single small git loose object (`write_tree` = one tree
|
||||
//! object) takes ~710 ms to land on the card, and it is **not** fsync
|
||||
//! (`GIT_OPT_ENABLE_FSYNC_GITDIR` is off). libgit2's loose-object write
|
||||
//! (`odb_loose.c` `loose_backend__write` → `git_filebuf_commit_at`) is, per object:
|
||||
//!
|
||||
//! stat(final) — freshen probe, misses (our `utimes` stub → `stat`)
|
||||
//! open+write+close — a temp file (`GIT_FILEBUF_TEMPORARY`)
|
||||
//! [mkdir objects/xx once per fan-out]
|
||||
//! p_rename — our stub: remove(final) [ENOENT] + rename(temp → final)
|
||||
//!
|
||||
//! i.e. **two directory-mutating writes** (temp create + rename) per object. This
|
||||
//! bench times each FAT primitive in isolation, then a composite that mirrors the
|
||||
//! sequence above, so we can attribute the ~700 ms to specific ops and get a
|
||||
//! baseline to compare an A1/A2 card or a 20 MHz bus against. All writes go to
|
||||
//! `/sd/sdbench` (cleaned up at the end); the pack-seek op additionally opens
|
||||
//! `/sd/repo`'s packfile READ-ONLY — it never writes there.
|
||||
//!
|
||||
//! Flash with `just flash-bench`. Needs no `.env`, no `git` feature (pure SD).
|
||||
|
||||
use std::fs::{self, File};
|
||||
use std::io::{Read, Seek, SeekFrom, Write};
|
||||
use std::time::Instant;
|
||||
|
||||
use anyhow::{Context, Result};
|
||||
use esp_idf_svc::hal::delay::FreeRtos;
|
||||
|
||||
use firmware::persistence::Storage;
|
||||
|
||||
/// Injected by build.rs so serial output identifies the exact build.
|
||||
const BUILD_TAG: &str = concat!("build ", env!("BUILD_TIME"), " @", env!("BUILD_GIT"));
|
||||
|
||||
/// Scratch dir on the card ROOT — outside `/sd/repo`, so a later `:sync` never
|
||||
/// stages it and the user's notes are never touched.
|
||||
const BENCH_DIR: &str = "/sd/sdbench";
|
||||
/// Iterations per op: enough to read min/p50/mean past controller jitter, few
|
||||
/// enough that total write volume stays tiny.
|
||||
const N: usize = 20;
|
||||
/// ~ the size of a small deflated git loose object (blob/tree/commit).
|
||||
const PAYLOAD: [u8; 200] = [b'x'; 200];
|
||||
|
||||
fn main() -> Result<()> {
|
||||
// Required once before any esp-idf-svc call (see esp-idf-template#71).
|
||||
esp_idf_svc::sys::link_patches();
|
||||
esp_idf_svc::log::EspLogger::initialize_default();
|
||||
log::info!("Typoena — SD primitive bench, {BUILD_TAG}");
|
||||
match run() {
|
||||
Ok(()) => log::info!("sd_bench: done"),
|
||||
Err(e) => log::error!("sd_bench failed: {e:?}"),
|
||||
}
|
||||
loop {
|
||||
FreeRtos::delay_ms(1000);
|
||||
}
|
||||
}
|
||||
|
||||
fn run() -> Result<()> {
|
||||
let sd = Storage::mount().context("mounting SD")?;
|
||||
let (max_khz, real_khz) = sd.negotiated_khz();
|
||||
log::info!("bus: max {max_khz} kHz, negotiated {real_khz} kHz — {N} iters, {}-byte payload", PAYLOAD.len());
|
||||
|
||||
// Fresh scratch dir.
|
||||
let _ = fs::remove_dir_all(BENCH_DIR);
|
||||
fs::create_dir_all(BENCH_DIR).with_context(|| format!("creating {BENCH_DIR}"))?;
|
||||
|
||||
// Warm-up: the first write after mount pays one-time settling — don't measure it.
|
||||
{
|
||||
let mut f = File::create(format!("{BENCH_DIR}/warmup"))?;
|
||||
f.write_all(&PAYLOAD)?;
|
||||
}
|
||||
|
||||
// 1) create + write(200B) + close, a fresh unique file each time. The drop at
|
||||
// the block's end is the close (FatFS f_close flushes dir entry + data).
|
||||
summarize("create+write(200B)+close", time_each(|i| {
|
||||
let mut f = File::create(format!("{BENCH_DIR}/c{i}"))?;
|
||||
f.write_all(&PAYLOAD)?;
|
||||
Ok(())
|
||||
})?);
|
||||
|
||||
// 2) rename c{i} -> o{i}. Sources exist from step 1 (untimed setup).
|
||||
summarize("rename", time_each(|i| {
|
||||
fs::rename(format!("{BENCH_DIR}/c{i}"), format!("{BENCH_DIR}/o{i}"))
|
||||
.map_err(Into::into)
|
||||
})?);
|
||||
|
||||
// 3) stat, hit.
|
||||
summarize("stat (hit)", time_each(|i| {
|
||||
fs::metadata(format!("{BENCH_DIR}/o{i}")).map(|_| ()).map_err(Into::into)
|
||||
})?);
|
||||
|
||||
// 4) stat, miss (ENOENT) — the freshen-probe analogue. A read, expected cheap.
|
||||
summarize("stat (miss/ENOENT)", time_each(|i| {
|
||||
let _ = fs::metadata(format!("{BENCH_DIR}/nope{i}"));
|
||||
Ok(())
|
||||
})?);
|
||||
|
||||
// 5) remove o{i}.
|
||||
summarize("remove", time_each(|i| {
|
||||
fs::remove_file(format!("{BENCH_DIR}/o{i}")).map_err(Into::into)
|
||||
})?);
|
||||
|
||||
// 6) Composite: the exact loose-object write sequence libgit2 performs, with a
|
||||
// git-length (38-hex) final name so LFN directory-entry cost is included.
|
||||
// If the model is right this lands near the ~700 ms/object from the split.
|
||||
summarize("loose-object composite", time_each(|i| {
|
||||
let tmp = format!("{BENCH_DIR}/tmp_obj{i}");
|
||||
let fin = format!("{BENCH_DIR}/{i:038x}");
|
||||
let _ = fs::metadata(&fin); // freshen probe, misses
|
||||
{
|
||||
let mut f = File::create(&tmp)?; // temp create + write + close
|
||||
f.write_all(&PAYLOAD)?;
|
||||
}
|
||||
let _ = fs::remove_file(&fin); // p_rename's remove(to) — ENOENT
|
||||
fs::rename(&tmp, &fin)?; // temp -> final
|
||||
Ok(())
|
||||
})?);
|
||||
|
||||
// Clean up so the card is left as we found it.
|
||||
fs::remove_dir_all(BENCH_DIR).with_context(|| format!("removing {BENCH_DIR}"))?;
|
||||
|
||||
// 7) THE ~1.5 s LOOSE-WRITE SUSPECT (git_bench, 2026-07-12 second real-repo
|
||||
// run): lseek inside a huge file. Without CONFIG_FATFS_USE_FASTSEEK,
|
||||
// FatFS resolves lseek by walking the file's FAT cluster chain — forward
|
||||
// from the current position, from the CHAIN HEAD on any backward seek.
|
||||
// The 570 MB pack is ~36k clusters ≈ ~146 KB of FAT reads over SPI per
|
||||
// long walk. `p_mmap` (esp_map.c) does lseek+read per window, and
|
||||
// libgit2's freshen path probes the pack TRAILER (near the end) while
|
||||
// tree windows sit at low offsets — so each loose write pays ~one full
|
||||
// walk. Prediction: "@start" stays ~ms; "@end" costs ~1.5 s per iter.
|
||||
// If so, the fix is CONFIG_FATFS_USE_FASTSEEK=y (fast-seek applies to
|
||||
// read-mode files only — exactly how the pack is opened).
|
||||
match find_pack()? {
|
||||
Some(pack) => {
|
||||
let len = fs::metadata(&pack)?.len();
|
||||
log::info!("pack seek bench: {pack} ({} MB)", len / (1024 * 1024));
|
||||
if len < 1024 * 1024 {
|
||||
log::info!("pack too small to show chain-walk cost — skipping (toy card?)");
|
||||
} else {
|
||||
let mut f = File::open(&pack).with_context(|| format!("opening {pack}"))?;
|
||||
let mut buf = vec![0u8; 4096];
|
||||
// Baseline: rewind + read at the chain head — no walk to resolve.
|
||||
summarize("pack seek+read 4KB @start", time_each(|_| {
|
||||
f.seek(SeekFrom::Start(0))?;
|
||||
f.read_exact(&mut buf)?;
|
||||
Ok(())
|
||||
})?);
|
||||
// Rewind (cheap, measured above), then seek near the end — pays
|
||||
// one full cluster-chain walk per iteration if fast-seek is off.
|
||||
let high = len - 4096;
|
||||
summarize("pack seek+read 4KB @end", time_each(|_| {
|
||||
f.seek(SeekFrom::Start(0))?;
|
||||
f.read_exact(&mut buf)?;
|
||||
f.seek(SeekFrom::Start(high))?;
|
||||
f.read_exact(&mut buf)?;
|
||||
Ok(())
|
||||
})?);
|
||||
}
|
||||
}
|
||||
None => log::info!("no packfile under /sd/repo/.git/objects/pack — skipping seek bench"),
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Largest `*.pack` under the repo's pack dir, if the card carries a clone.
|
||||
/// Skips macOS AppleDouble sidecars (`._pack-*.pack`, 4 KB of Finder metadata) —
|
||||
/// the Spike-14 cruft in its latest disguise.
|
||||
fn find_pack() -> Result<Option<String>> {
|
||||
let Ok(entries) = fs::read_dir("/sd/repo/.git/objects/pack") else {
|
||||
return Ok(None);
|
||||
};
|
||||
let mut best: Option<(u64, String)> = None;
|
||||
for e in entries.flatten() {
|
||||
let p = e.path();
|
||||
let name = p.file_name().and_then(|n| n.to_str()).unwrap_or("");
|
||||
if name.starts_with("._") || !name.ends_with(".pack") {
|
||||
continue;
|
||||
}
|
||||
let len = fs::metadata(&p).map(|m| m.len()).unwrap_or(0);
|
||||
if best.as_ref().is_none_or(|(l, _)| len > *l) {
|
||||
best = Some((len, p.to_string_lossy().into_owned()));
|
||||
}
|
||||
}
|
||||
Ok(best.map(|(_, p)| p))
|
||||
}
|
||||
|
||||
/// Run `op(i)` for `i in 0..N`, returning each call's wall time in microseconds.
|
||||
fn time_each<F: FnMut(usize) -> Result<()>>(mut op: F) -> Result<Vec<u64>> {
|
||||
let mut times = Vec::with_capacity(N);
|
||||
for i in 0..N {
|
||||
let t = Instant::now();
|
||||
op(i)?;
|
||||
times.push(t.elapsed().as_micros() as u64);
|
||||
}
|
||||
Ok(times)
|
||||
}
|
||||
|
||||
/// Log min / p50 / mean / max in ms for a set of per-call microsecond timings.
|
||||
fn summarize(label: &str, mut times: Vec<u64>) {
|
||||
times.sort_unstable();
|
||||
let n = times.len();
|
||||
let mean = times.iter().sum::<u64>() / n as u64;
|
||||
let ms = |us: u64| us as f64 / 1000.0;
|
||||
log::info!(
|
||||
"{label:<26} min {:>6.1} p50 {:>6.1} mean {:>6.1} max {:>6.1} ms",
|
||||
ms(times[0]),
|
||||
ms(times[n / 2]),
|
||||
ms(mean),
|
||||
ms(times[n - 1]),
|
||||
);
|
||||
}
|
||||
@@ -94,6 +94,11 @@ fn write_test(storage: &Storage) -> Result<()> {
|
||||
log::info!("{REPO_DIR} missing — creating it (bench setup) so the write test can run");
|
||||
fs::create_dir_all(REPO_DIR).with_context(|| format!("create {REPO_DIR}"))?;
|
||||
}
|
||||
// End the payload with '\n', like a real editor buffer (whose visible trailing
|
||||
// blank line is exactly that terminating newline). `save` inserts a final
|
||||
// newline only when one is missing and `load` reads verbatim, so a payload that
|
||||
// already ends in '\n' round-trips byte-for-byte — which this exact-equality
|
||||
// check relies on.
|
||||
let payload = format!("typoena spike 3\n{BUILD_TAG}\ndedicated SPI3: SCK14 MOSI15 MISO13 CS10\n");
|
||||
storage.save(&payload).context("Storage::save")?;
|
||||
let back = storage.load().context("Storage::load after save")?;
|
||||
|
||||
@@ -17,15 +17,23 @@
|
||||
//! notes. A `/sd/repo` that isn't a valid repo is a provisioning error
|
||||
//! (`just init`), surfaced as such, not papered over.
|
||||
//! 3. **No synthetic content.** The spike appended a marker line; here the
|
||||
//! editor has already saved the user's `notes.md` before `:sync` signals us,
|
||||
//! so we just stage + commit + push what's on disk.
|
||||
//! editor has already saved the user's buffers before `:sync` signals us,
|
||||
//! so we just commit + push what's on disk.
|
||||
//! 4. **The commit is an O(depth) TreeBuilder splice, not an index pass.**
|
||||
//! The request carries the repo-relative paths saved/deleted since the last
|
||||
//! confirmed publish (`Storage`'s journaled dirty set); `stage_and_commit`
|
||||
//! patches exactly those onto HEAD's tree. The index pipeline it replaced
|
||||
//! (`add_all` → `index.write` → `write_tree`) is O(N_tree) and measured up
|
||||
//! to 611 s on the real 1179-file / 570 MB-pack clone — see
|
||||
//! docs/tradeoff-curves/sync-commit-staging.md for the whole trail.
|
||||
//!
|
||||
//! Runs on a dedicated 96 KB thread (libgit2's init→push chain nests ~67 KB of
|
||||
//! `GIT_PATH_MAX` stack buffers — see git_push.rs / postmortem #3). Config is
|
||||
//! baked at build time (`TW_*`, ADR-007: v0.1 device config is compiled in).
|
||||
|
||||
use std::cell::RefCell;
|
||||
use std::path::Path;
|
||||
use std::collections::BTreeSet;
|
||||
use std::fs;
|
||||
use std::rc::Rc;
|
||||
use std::sync::mpsc::{Receiver, Sender};
|
||||
use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH};
|
||||
@@ -39,8 +47,8 @@ use esp_idf_svc::sntp::{EspSntp, SyncStatus};
|
||||
use esp_idf_svc::sys;
|
||||
use esp_idf_svc::wifi::{BlockingWifi, EspWifi};
|
||||
use git2::{
|
||||
CertificateCheckStatus, Commit, Cred, CredentialType, FetchOptions, IndexAddOption,
|
||||
PushOptions, RemoteCallbacks, Repository, Signature,
|
||||
CertificateCheckStatus, Commit, Cred, CredentialType, FetchOptions, ObjectType, Oid,
|
||||
PushOptions, RemoteCallbacks, Repository, Signature, Tree,
|
||||
};
|
||||
|
||||
use crate::net::connect_wifi;
|
||||
@@ -73,10 +81,15 @@ const SNTP_TIMEOUT: Duration = Duration::from_secs(20);
|
||||
/// now also runs here, but it's shallow next to libgit2's path-buffer nesting.
|
||||
pub const GIT_STACK: usize = 96 * 1024;
|
||||
|
||||
/// A request to publish. The note is already saved to `/sd/repo/notes.md` by the
|
||||
/// UI task before this is sent, so the request carries no payload (a future
|
||||
/// multi-file publish can grow one).
|
||||
pub struct PublishRequest;
|
||||
/// A request to publish. The UI task has already saved every dirty buffer to
|
||||
/// the card before sending this; `paths` is `Storage::take_dirty`'s snapshot —
|
||||
/// the repo-relative paths saved or `:delete`d since the last confirmed
|
||||
/// publish. The working tree stays the source of truth: at commit time a path
|
||||
/// that exists on the card is spliced into the tree from disk, a missing one
|
||||
/// is spliced out. An unchanged path is a no-op, so over-reporting is safe.
|
||||
pub struct PublishRequest {
|
||||
pub paths: BTreeSet<String>,
|
||||
}
|
||||
|
||||
/// Result of a publish attempt, sent back to the UI task for the snackbar. The
|
||||
/// detailed error always goes to the serial log; the panel gets a short line.
|
||||
@@ -103,6 +116,22 @@ pub fn run_git_service(
|
||||
rx: Receiver<PublishRequest>,
|
||||
tx: Sender<PublishOutcome>,
|
||||
) {
|
||||
// Process-global libgit2 tuning, once, before any repo work. The 32-bit
|
||||
// defaults (32 MB window / 256 MB mapped budget, mwindow.c) would
|
||||
// git__malloc past PSRAM on the first pack access of the real 570 MB-pack
|
||||
// clone; these are the bench-proven values (git_bench), and ~1.9 MB of
|
||||
// windows stays live during git ops — the p_mmap emulation (esp_map.c)
|
||||
// relies on this mapped limit being real.
|
||||
// SAFETY: set on the git thread before any Repository is opened.
|
||||
unsafe {
|
||||
if let Err(e) = git2::opts::set_mwindow_size(256 * 1024) {
|
||||
log::error!("set_mwindow_size failed ({e}); first pack access may OOM");
|
||||
}
|
||||
if let Err(e) = git2::opts::set_mwindow_mapped_limit(4 * 1024 * 1024) {
|
||||
log::error!("set_mwindow_mapped_limit failed ({e}); first pack access may OOM");
|
||||
}
|
||||
}
|
||||
|
||||
// Lazily initialised on the first request, then reused across publishes.
|
||||
let mut wifi: Option<BlockingWifi<EspWifi<'static>>> = None;
|
||||
let mut modem = Some(modem);
|
||||
@@ -110,7 +139,7 @@ pub fn run_git_service(
|
||||
let mut clock_synced = false;
|
||||
let mut tls_ready = false;
|
||||
|
||||
while rx.recv().is_ok() {
|
||||
while let Ok(req) = rx.recv() {
|
||||
let outcome = publish_cycle(
|
||||
&sys_loop,
|
||||
&mut wifi,
|
||||
@@ -118,6 +147,7 @@ pub fn run_git_service(
|
||||
&mut nvs,
|
||||
&mut clock_synced,
|
||||
&mut tls_ready,
|
||||
&req.paths,
|
||||
);
|
||||
let msg = match outcome {
|
||||
Ok(o) => o,
|
||||
@@ -143,11 +173,22 @@ fn publish_cycle(
|
||||
nvs: &mut Option<EspDefaultNvsPartition>,
|
||||
clock_synced: &mut bool,
|
||||
tls_ready: &mut bool,
|
||||
paths: &BTreeSet<String>,
|
||||
) -> Result<PublishOutcome> {
|
||||
if REMOTE_URL.is_empty() || GH_USER.is_empty() || PAT.is_empty() || WIFI_SSID.is_empty() {
|
||||
bail!("git config missing — set TW_WIFI_SSID / TW_REMOTE_URL / TW_GH_USER / TW_PAT in firmware/.env and rebuild");
|
||||
}
|
||||
|
||||
// Nothing recorded dirty and origin's tracking ref already has HEAD: this
|
||||
// `:sync` has nothing to do — say so without touching the radio (~150 ms
|
||||
// instead of a Wi-Fi + TLS round). A stranded local commit (committed but
|
||||
// never pushed, e.g. a push that failed mid-air) makes the check false and
|
||||
// takes the full path below, where publish_once pushes it.
|
||||
if paths.is_empty() && remote_current().unwrap_or(false) {
|
||||
log::info!(":sync — no dirty paths and origin has HEAD; up to date, radio untouched");
|
||||
return Ok(PublishOutcome::UpToDate);
|
||||
}
|
||||
|
||||
// Phases are timed so a cold :sync reports where the seconds go. Wi-Fi, clock
|
||||
// and TLS run only on the first sync of a session; a warm sync skips them, so
|
||||
// they read 0 ms and the total collapses to just publish(fetch+commit+push).
|
||||
@@ -186,7 +227,7 @@ fn publish_cycle(
|
||||
}
|
||||
|
||||
let t_publish = Instant::now();
|
||||
let outcome = publish_once()?;
|
||||
let outcome = publish_once(paths)?;
|
||||
log::info!(
|
||||
":sync timing — wifi {wifi_ms}ms, clock {clock_ms}ms, tls {tls_ms}ms, publish(commit+push) {}ms, total {}ms",
|
||||
t_publish.elapsed().as_millis(),
|
||||
@@ -205,15 +246,17 @@ fn publish_cycle(
|
||||
///
|
||||
/// Never clones or wipes: a `/sd/repo` that isn't a valid repo is a provisioning
|
||||
/// error, surfaced as such.
|
||||
fn publish_once() -> Result<PublishOutcome> {
|
||||
log::info!("publish started — free heap {}", free_heap());
|
||||
fn publish_once(paths: &BTreeSet<String>) -> Result<PublishOutcome> {
|
||||
log::info!(
|
||||
"publish started — {} dirty path(s), free heap {} ({} internal)",
|
||||
paths.len(),
|
||||
free_heap(),
|
||||
internal_free_heap()
|
||||
);
|
||||
let repo = Repository::open(REPO_DIR).with_context(|| {
|
||||
format!("opening git repo at {REPO_DIR} — provision the card with a clone (just init) whose origin is your remote")
|
||||
})?;
|
||||
|
||||
let Some(mut oid) = stage_and_commit(&repo)? else {
|
||||
return Ok(PublishOutcome::UpToDate);
|
||||
};
|
||||
let branch = repo
|
||||
.head()?
|
||||
.shorthand()
|
||||
@@ -221,17 +264,46 @@ fn publish_once() -> Result<PublishOutcome> {
|
||||
.to_string();
|
||||
let refspec = format!("refs/heads/{branch}:refs/heads/{branch}");
|
||||
|
||||
// Optimistic push. A non-fast-forward rejection means the remote moved under
|
||||
// us: reconcile onto origin and replay the note on the new tip, then retry
|
||||
// once. reconcile_onto_origin mixed-resets, so the just-saved note survives in
|
||||
// the working tree and stage_and_commit lands it on top of origin.
|
||||
if let Err(first) = try_push(&repo, &refspec) {
|
||||
log::warn!("push rejected ({first}); reconciling onto origin and replaying the note");
|
||||
let mut oid = match stage_and_commit(&repo, paths)? {
|
||||
Some(oid) => oid,
|
||||
None => {
|
||||
// Nothing new to commit. Usually genuinely up to date — but a
|
||||
// previous cycle may have committed and then failed to push,
|
||||
// stranding a local-only commit (the old add_all path silently
|
||||
// never retried those). Push whenever origin's tracking ref
|
||||
// doesn't already have HEAD.
|
||||
let head = repo.head()?.peel_to_commit()?.id();
|
||||
if tracking_tip(&repo, &branch) == Some(head) {
|
||||
return Ok(PublishOutcome::UpToDate);
|
||||
}
|
||||
log::info!(
|
||||
"tree unchanged but origin/{branch} lacks HEAD {} — pushing the stranded commit",
|
||||
short(head)
|
||||
);
|
||||
head
|
||||
}
|
||||
};
|
||||
|
||||
// Optimistic push. A non-fast-forward *rejection* means the remote moved
|
||||
// under us: reconcile onto origin and replay the dirty paths on the new
|
||||
// tip, then retry once (reconcile_onto_origin soft-resets — ref move only —
|
||||
// so the notes stay on the card and stage_and_commit splices them on top of
|
||||
// origin). A transport-level failure is surfaced as-is: its fetch would die
|
||||
// the same way, and the commit is safe locally — the stranded-commit check
|
||||
// above pushes it once the transport works again.
|
||||
if let Err(failure) = try_push(&repo, &refspec) {
|
||||
let rejection = match failure {
|
||||
PushFailure::Rejected(msg) => msg,
|
||||
PushFailure::Other(e) => return Err(e),
|
||||
};
|
||||
log::warn!("push rejected ({rejection}); reconciling onto origin and replaying the note");
|
||||
reconcile_onto_origin(&repo, &branch).context("reconciling after a rejected push")?;
|
||||
match stage_and_commit(&repo)? {
|
||||
match stage_and_commit(&repo, paths)? {
|
||||
Some(replayed) => {
|
||||
oid = replayed;
|
||||
try_push(&repo, &refspec).context("push after reconcile")?;
|
||||
try_push(&repo, &refspec)
|
||||
.map_err(PushFailure::into_error)
|
||||
.context("push after reconcile")?;
|
||||
}
|
||||
// The note was already on origin (nothing to replay) — treat as done.
|
||||
None => {
|
||||
@@ -242,41 +314,67 @@ fn publish_once() -> Result<PublishOutcome> {
|
||||
}
|
||||
|
||||
log::info!(
|
||||
"push done — free heap {}, min-ever {}",
|
||||
"push done — free heap {} ({} internal), min-ever {}",
|
||||
free_heap(),
|
||||
internal_free_heap(),
|
||||
min_free_heap()
|
||||
);
|
||||
Ok(PublishOutcome::Pushed(short(oid)))
|
||||
}
|
||||
|
||||
/// Stage the working tree and commit it on top of the current branch tip.
|
||||
/// Returns the new commit id, or `None` when the tree already matches the parent
|
||||
/// (nothing to publish). Called on the first attempt and again to replay the note
|
||||
/// after a reconcile.
|
||||
/// Build the commit for `paths` as an O(depth) TreeBuilder splice onto HEAD's
|
||||
/// tree and return the new commit id — or `None` when the result matches the
|
||||
/// parent (nothing to publish). Called on the first attempt and again to
|
||||
/// replay the dirty paths after a reconcile.
|
||||
///
|
||||
/// `add_all` runs a per-path filter that drops macOS AppleDouble sidecars
|
||||
/// (`._name`) and `.DS_Store` that Finder/Spotlight sprinkle onto the FAT card
|
||||
/// whenever it's mounted on a Mac — without it, a blind add --all sweeps them into
|
||||
/// the commit (it did once: 07d87772 shipped `._.git`, `._README.md`,
|
||||
/// `._notes.md`). Filtering here fixes it for *every* repo at the device level, so
|
||||
/// no per-repo `.gitignore` is needed. (add --all also stages deletions, for a
|
||||
/// future note-delete.)
|
||||
fn stage_and_commit(repo: &Repository) -> Result<Option<git2::Oid>> {
|
||||
let mut index = repo.index().context("opening index")?;
|
||||
let mut skip_macos_cruft = |path: &Path, _matched: &[u8]| -> i32 {
|
||||
match path.file_name().and_then(|n| n.to_str()) {
|
||||
Some(name) if name.starts_with("._") || name == ".DS_Store" => 1, // skip
|
||||
_ => 0, // add
|
||||
}
|
||||
};
|
||||
index
|
||||
.add_all(["*"], IndexAddOption::DEFAULT, Some(&mut skip_macos_cruft))
|
||||
.context("staging (add --all)")?;
|
||||
index.write().context("writing index")?;
|
||||
let tree = repo.find_tree(index.write_tree().context("writing tree")?)?;
|
||||
|
||||
// Commit on top of the current branch tip (None on an empty/unborn remote).
|
||||
/// This replaces the index pipeline (`add_all` → `index.write` → `write_tree`),
|
||||
/// which is O(N_tree) and cannot run on the real 1179-file / 570 MB-pack clone:
|
||||
/// `index.write`'s racy-clean pass re-hashes ~every entry on FAT's 2 s mtimes
|
||||
/// (measured up to **611 s**), and even the index-free `read_tree` walk was
|
||||
/// 77 s. The splice reads and writes only the dirty paths' ancestor chains —
|
||||
/// O(depth × dirty), flat in repo size, **~2–2.8 s measured on the real
|
||||
/// clone** — and carries every untouched entry (including the ~150 MB of
|
||||
/// images) forward by OID without ever opening it. Trail + bench numbers:
|
||||
/// docs/tradeoff-curves/sync-commit-staging.md.
|
||||
///
|
||||
/// The working tree is the source of truth: a recorded path that exists on the
|
||||
/// card is spliced in from disk, a missing one is spliced out (a `:delete`).
|
||||
/// Unrecorded paths are never visited — so Finder cruft (`._*`, `.DS_Store`)
|
||||
/// on the FAT card can no longer ride into a commit the way it once did with
|
||||
/// `add_all` (07d87772), and the old cruft filter is gone with the walk.
|
||||
fn stage_and_commit(repo: &Repository, paths: &BTreeSet<String>) -> Result<Option<Oid>> {
|
||||
// Commit on top of the current branch tip (None on an empty/unborn remote,
|
||||
// where the splice starts from an empty base and makes a parentless commit).
|
||||
let parent = repo.head().ok().and_then(|h| h.peel_to_commit().ok());
|
||||
let base = match &parent {
|
||||
Some(c) => Some(c.tree().context("loading HEAD tree")?),
|
||||
None => None,
|
||||
};
|
||||
|
||||
let t_splice = Instant::now();
|
||||
let mut tree = base;
|
||||
for path in paths {
|
||||
let parts: Vec<&str> = path.split('/').filter(|p| !p.is_empty()).collect();
|
||||
if parts.is_empty() {
|
||||
continue;
|
||||
}
|
||||
let blob = match fs::read(format!("{REPO_DIR}/{path}")) {
|
||||
Ok(bytes) => Some(
|
||||
repo.blob(&bytes)
|
||||
.with_context(|| format!("writing blob for {path}"))?,
|
||||
),
|
||||
Err(e) if e.kind() == std::io::ErrorKind::NotFound => None, // deleted → splice out
|
||||
Err(e) => return Err(e).with_context(|| format!("reading {path}")),
|
||||
};
|
||||
let spliced = splice(repo, tree.as_ref(), &parts, blob)
|
||||
.with_context(|| format!("splicing {path}"))?;
|
||||
tree = Some(repo.find_tree(spliced).context("loading spliced tree")?);
|
||||
}
|
||||
let Some(tree) = tree else {
|
||||
return Ok(None); // unborn branch and nothing dirty — nothing to commit
|
||||
};
|
||||
let splice_ms = t_splice.elapsed().as_millis();
|
||||
|
||||
if let Some(p) = &parent {
|
||||
if p.tree_id() == tree.id() {
|
||||
log::info!("nothing to publish — tree unchanged @ {}", short(p.id()));
|
||||
@@ -287,18 +385,116 @@ fn stage_and_commit(repo: &Repository) -> Result<Option<git2::Oid>> {
|
||||
let sig = Signature::now(AUTHOR_NAME, AUTHOR_EMAIL).context("building signature")?;
|
||||
let message = format!("Typoena publish — unix {}", now_unix());
|
||||
let parents: Vec<&Commit> = parent.iter().collect();
|
||||
let t_commit = Instant::now();
|
||||
let oid = repo
|
||||
.commit(Some("HEAD"), &sig, &sig, &message, &tree, &parents)
|
||||
.context("creating commit")?;
|
||||
log::info!("committed {} — free heap {}", short(oid), free_heap());
|
||||
log::info!(
|
||||
"commit split — splice {splice_ms}ms ({} path(s)), commit-obj {}ms; committed {} — free heap {} ({} internal)",
|
||||
paths.len(),
|
||||
t_commit.elapsed().as_millis(),
|
||||
short(oid),
|
||||
free_heap(),
|
||||
internal_free_heap()
|
||||
);
|
||||
Ok(Some(oid))
|
||||
}
|
||||
|
||||
/// Return a new tree = `base` with `path` set to `blob` (`Some` inserts or
|
||||
/// replaces, `None` removes). Recurses down the path's subtree chain: reads
|
||||
/// ~depth tree objects and writes ~depth new ones, leaving every sibling entry
|
||||
/// untouched (carried by OID — never opened). A missing intermediate directory
|
||||
/// is synthesized on the way down; a directory emptied by a remove is pruned
|
||||
/// on the way up rather than left behind as an empty tree entry.
|
||||
fn splice(repo: &Repository, base: Option<&Tree>, path: &[&str], blob: Option<Oid>) -> Result<Oid> {
|
||||
let (head, rest) = path.split_first().context("splice: empty path")?;
|
||||
let mut tb = repo.treebuilder(base).context("treebuilder")?;
|
||||
if rest.is_empty() {
|
||||
match blob {
|
||||
Some(oid) => {
|
||||
tb.insert(*head, oid, 0o100644)
|
||||
.context("inserting blob entry")?;
|
||||
}
|
||||
// Removing a never-committed path is a no-op, not an error (a note
|
||||
// created and deleted between two syncs).
|
||||
None => {
|
||||
let _ = tb.remove(*head);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
let sub = match base.and_then(|b| b.get_name(head)) {
|
||||
Some(e) if e.kind() == Some(ObjectType::Tree) => {
|
||||
Some(repo.find_tree(e.id()).context("loading subtree")?)
|
||||
}
|
||||
// Absent (a new directory) or a non-tree shadowing the name —
|
||||
// build the subtree from scratch either way.
|
||||
_ => None,
|
||||
};
|
||||
let new_sub = splice(repo, sub.as_ref(), rest, blob)?;
|
||||
if repo.find_tree(new_sub)?.len() == 0 {
|
||||
let _ = tb.remove(*head); // the remove emptied this directory — prune it
|
||||
} else {
|
||||
tb.insert(*head, new_sub, 0o040000)
|
||||
.context("inserting subtree entry")?;
|
||||
}
|
||||
}
|
||||
tb.write().context("writing spliced tree")
|
||||
}
|
||||
|
||||
/// Origin's remote-tracking tip for `branch`, if the ref exists. libgit2
|
||||
/// updates it after a successful push/fetch, so it is "the newest commit we
|
||||
/// know origin has" — without touching the network.
|
||||
fn tracking_tip(repo: &Repository, branch: &str) -> Option<Oid> {
|
||||
repo.find_reference(&format!("refs/remotes/origin/{branch}"))
|
||||
.ok()?
|
||||
.peel_to_commit()
|
||||
.ok()
|
||||
.map(|c| c.id())
|
||||
}
|
||||
|
||||
/// Whether origin is known to already have HEAD (local refs only, no network).
|
||||
/// Errors read as "not current", so the caller falls through to the full
|
||||
/// publish path where the real failure surfaces with context.
|
||||
fn remote_current() -> Result<bool> {
|
||||
let repo = Repository::open(REPO_DIR)?;
|
||||
let head = repo.head()?.peel_to_commit()?.id();
|
||||
let branch = repo
|
||||
.head()?
|
||||
.shorthand()
|
||||
.context("HEAD has no branch shorthand")?
|
||||
.to_string();
|
||||
Ok(tracking_tip(&repo, &branch) == Some(head))
|
||||
}
|
||||
|
||||
/// How a push attempt failed — this decides whether reconciling can help.
|
||||
enum PushFailure {
|
||||
/// The server processed the push but refused the ref update (arrives via
|
||||
/// the `push_update_reference` callback — e.g. non-fast-forward): the
|
||||
/// remote moved under us, and reconcile + replay is the right response.
|
||||
Rejected(String),
|
||||
/// Transport / TLS / auth / URL — the push never reached a ref decision,
|
||||
/// so a reconcile (whose fetch needs the same transport) cannot help.
|
||||
/// Surfaced directly; the 2026-07-13 on-device run burned a doomed
|
||||
/// reconcile on an "unsupported URL protocol" because this wasn't split.
|
||||
Other(anyhow::Error),
|
||||
}
|
||||
|
||||
impl PushFailure {
|
||||
fn into_error(self) -> anyhow::Error {
|
||||
match self {
|
||||
Self::Rejected(msg) => anyhow::anyhow!("remote rejected ref: {msg}"),
|
||||
Self::Other(e) => e,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// One push attempt over HTTPS. Binds the PAT credential + the cert-verify
|
||||
/// callback, and surfaces a server-side ref rejection (e.g. non-fast-forward) as
|
||||
/// an error (it arrives via `push_update_reference`, not as a `push()` error).
|
||||
fn try_push(repo: &Repository, refspec: &str) -> Result<()> {
|
||||
let mut remote = repo.find_remote("origin")?;
|
||||
/// callback, and separates a server-side ref rejection (reconcilable) from a
|
||||
/// transport-level failure (not).
|
||||
fn try_push(repo: &Repository, refspec: &str) -> Result<(), PushFailure> {
|
||||
let mut remote = repo
|
||||
.find_remote("origin")
|
||||
.map_err(|e| PushFailure::Other(anyhow::Error::new(e).context("finding remote origin")))?;
|
||||
let rejection: Rc<RefCell<Option<String>>> = Rc::new(RefCell::new(None));
|
||||
|
||||
let mut cbs = auth_callbacks();
|
||||
@@ -316,27 +512,30 @@ fn try_push(repo: &Repository, refspec: &str) -> Result<()> {
|
||||
opts.remote_callbacks(cbs);
|
||||
remote
|
||||
.push(&[refspec], Some(&mut opts))
|
||||
.context("push transport")?;
|
||||
.map_err(|e| PushFailure::Other(anyhow::Error::new(e).context("push transport")))?;
|
||||
|
||||
if let Some(msg) = rejection.borrow().clone() {
|
||||
bail!("remote rejected ref: {msg}");
|
||||
return Err(PushFailure::Rejected(msg));
|
||||
}
|
||||
log::info!("push accepted by remote");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Fetch origin and mixed-reset the local branch onto it, so our just-made commit
|
||||
/// can be replayed on the current tip. Only runs after a non-fast-forward push
|
||||
/// Fetch origin and *soft*-reset the local branch onto it, so our changes can
|
||||
/// be replayed on the current tip. Only runs after a non-fast-forward push
|
||||
/// rejection — i.e. the remote moved under us.
|
||||
///
|
||||
/// **MIXED**, deliberately not a force checkout: the note we're publishing lives
|
||||
/// in the working tree, and a force checkout would clobber it. Mixed moves the
|
||||
/// branch ref + index onto origin but leaves the working tree, so the note
|
||||
/// survives and `stage_and_commit` replays it on top. For a single-writer
|
||||
/// appliance this resolves last-writer-wins — a concurrent remote *edit* to the
|
||||
/// same note loses to ours, and a remote-only *added* file the card doesn't have
|
||||
/// is dropped by the replay's add --all. Both need a real merge (increment B) and
|
||||
/// don't arise from this device's own use.
|
||||
/// **SOFT**, deliberately: it moves only the branch ref. The previous Mixed
|
||||
/// reset also rewrote the index — pure waste now that the splice commit never
|
||||
/// reads the index, and on the real repo an index write is exactly the
|
||||
/// racy-clean wall the splice exists to avoid. Neither flavor touches the
|
||||
/// working tree, so the notes being published survive on the card and the
|
||||
/// replay splices them onto the new tip. For a single-writer appliance this
|
||||
/// resolves last-writer-wins: a concurrent remote *edit* to a note we're
|
||||
/// publishing loses to ours, while a remote-only added/changed file is simply
|
||||
/// carried forward — origin's tree is now the splice base, so the replay
|
||||
/// keeps it (an improvement over the old `add --all` replay, which dropped
|
||||
/// files the card didn't have). A real merge stays increment-B work.
|
||||
fn reconcile_onto_origin(repo: &Repository, branch: &str) -> Result<()> {
|
||||
let mut remote = repo.find_remote("origin")?;
|
||||
let mut fo = FetchOptions::new();
|
||||
@@ -350,12 +549,12 @@ fn reconcile_onto_origin(repo: &Repository, branch: &str) -> Result<()> {
|
||||
.context("no FETCH_HEAD after fetch")?;
|
||||
let theirs = repo.reference_to_annotated_commit(&fetch_head)?;
|
||||
log::info!(
|
||||
"reconcile: resetting local {branch} onto origin @ {} (mixed, keeps the note)",
|
||||
"reconcile: resetting local {branch} onto origin @ {} (soft — ref move only, notes stay on the card)",
|
||||
short(theirs.id())
|
||||
);
|
||||
let their_obj = repo.find_object(theirs.id(), None)?;
|
||||
repo.reset(&their_obj, git2::ResetType::Mixed, None)
|
||||
.context("mixed reset onto origin")?;
|
||||
repo.reset(&their_obj, git2::ResetType::Soft, None)
|
||||
.context("soft reset onto origin")?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -440,6 +639,14 @@ fn free_heap() -> u32 {
|
||||
unsafe { sys::esp_get_free_heap_size() }
|
||||
}
|
||||
|
||||
/// Free INTERNAL RAM (DRAM), excluding PSRAM. `free_heap` is dominated by the
|
||||
/// 8 MB PSRAM pool and masks internal exhaustion — which is what actually
|
||||
/// killed the first real-repo push (mbedTLS's ssl_setup could not get its
|
||||
/// ~33 KB while Wi-Fi + USB + editor + libgit2 were resident).
|
||||
fn internal_free_heap() -> u32 {
|
||||
unsafe { sys::heap_caps_get_free_size(sys::MALLOC_CAP_INTERNAL) as u32 }
|
||||
}
|
||||
|
||||
fn min_free_heap() -> u32 {
|
||||
unsafe { sys::esp_get_minimum_free_heap_size() }
|
||||
}
|
||||
|
||||
@@ -10,7 +10,10 @@ use esp_idf_svc::hal::spi::{Dma, SpiBusDriver, SpiDriver};
|
||||
use esp_idf_svc::hal::units::FromValueType;
|
||||
|
||||
use display::Frame;
|
||||
use editor::{Editor, Effect, Mode, CH};
|
||||
use editor::{
|
||||
Editor, Effect, Mode, Prefs, Scope, Snippets, CH, LOCAL_DIR, PREFS_PATH, REPO_DIR,
|
||||
SNIPPETS_PATH,
|
||||
};
|
||||
use firmware::epd::{self, Epd};
|
||||
use firmware::persistence::{Storage, NOTES};
|
||||
|
||||
@@ -26,6 +29,12 @@ const FULL_REFRESH_EVERY: u32 = 64;
|
||||
/// reappears once you settle. Normal/View draw their own caret every action.
|
||||
const CURSOR_DEBOUNCE_MS: u128 = 750;
|
||||
|
||||
/// How long input must pause before `save_on_idle` persists a dirty buffer.
|
||||
/// Longer than the caret debounce so autosave settles after typing, not during
|
||||
/// a mid-sentence pause. The save is silent (no snackbar, no forced e-ink
|
||||
/// flash) — a safety net against power loss, not a user action.
|
||||
const IDLE_SAVE_MS: u128 = 1500;
|
||||
|
||||
fn main() -> anyhow::Result<()> {
|
||||
// Required once before any esp-idf-svc call; some runtime patches
|
||||
// only link if this symbol is referenced. See esp-idf-template#71.
|
||||
@@ -101,7 +110,9 @@ fn main() -> anyhow::Result<()> {
|
||||
|
||||
// Seed the editor from the saved note. Boots in Normal mode with the caret
|
||||
// on the last character (the resume point) — press `i`/`a`/`o` to write.
|
||||
let mut ed = Editor::with_text(saved);
|
||||
// The boot note is Tracked (`/sd/repo/notes.md`); `:e` / the palette (v0.5)
|
||||
// open others, Tracked or Local.
|
||||
let mut ed = Editor::with_file(NOTES.to_string(), Scope::Tracked, saved);
|
||||
// Confirm the boot-load on the panel (no serial console in normal use):
|
||||
// "loaded <name>" using the note's filename without its suffix (notes.md ->
|
||||
// notes). Cleared by the first keystroke, like any snackbar.
|
||||
@@ -110,9 +121,55 @@ fn main() -> anyhow::Result<()> {
|
||||
.and_then(|s| s.to_str())
|
||||
.unwrap_or("notes");
|
||||
ed.set_notice(format!("loaded {name}"));
|
||||
// Feed the file palette (Ctrl-P). Enumerated once at boot — the v0.5 slices
|
||||
// that create/delete files (`:enew`, delete) will re-feed it then.
|
||||
// Bracketed with internal-DRAM readings: each path is a small String, kept
|
||||
// internal by the SPIRAM malloc threshold (16 KB), so the list competes
|
||||
// with Wi-Fi/TLS for DRAM. Estimate to confirm: ~60-70 KB at 1098 files —
|
||||
// this number decides whether interning the paths into one shared buffer
|
||||
// (a single >16 KB alloc, which lands in PSRAM) is worth the refactor.
|
||||
let dram_before = internal_free_heap();
|
||||
ed.set_file_list(enumerate_files());
|
||||
let dram_after = internal_free_heap();
|
||||
log::info!(
|
||||
"file list: internal heap {dram_before} -> {dram_after} ({} KB consumed)",
|
||||
dram_before.saturating_sub(dram_after) / 1024
|
||||
);
|
||||
// Editor preferences (.typoena.toml, git-tracked). Read before the first
|
||||
// render so `line_numbers` shapes the opening frame. A missing / unreadable /
|
||||
// partial file falls back to defaults, so a fresh card just works.
|
||||
let prefs = match storage.load_path(PREFS_PATH) {
|
||||
Ok(src) => Prefs::parse(&src),
|
||||
Err(_) => Prefs::default(),
|
||||
};
|
||||
log::info!("prefs: {prefs:?}");
|
||||
ed.set_prefs(prefs);
|
||||
// Snippet library (.typoena.snippets.json, git-tracked). Parsed with
|
||||
// serde_json in the editor crate; a missing / unreadable / malformed file is
|
||||
// non-fatal — the editor simply has no snippets and runs unchanged.
|
||||
let snippets = match storage.load_path(SNIPPETS_PATH) {
|
||||
Ok(src) => match Snippets::parse(&src) {
|
||||
Ok(s) => s,
|
||||
Err(e) => {
|
||||
log::warn!("snippets parse FAILED ({e}); none loaded");
|
||||
Snippets::default()
|
||||
}
|
||||
},
|
||||
Err(_) => Snippets::default(),
|
||||
};
|
||||
log::info!("snippets: {} loaded", snippets.0.len());
|
||||
ed.set_snippets(snippets);
|
||||
let mut updates: u32 = 0;
|
||||
let mut cursor_shown = true; // the initial render includes the caret
|
||||
let mut last_activity = Instant::now();
|
||||
// Whether `save_on_idle` already persisted the current idle window, so it
|
||||
// fires once per typing burst (and doesn't retry-storm if a save fails).
|
||||
// Reset on the next activity.
|
||||
let mut idle_saved = false;
|
||||
// Set when a paint fails (see the refresh block below): the next paint then
|
||||
// does a full refresh to re-establish both RAM banks, since a partial that
|
||||
// died mid-transfer may have left them inconsistent.
|
||||
let mut force_full = false;
|
||||
|
||||
// Keyboard attach/detach state drives the panel's disconnect flag; seed it
|
||||
// (and the word-count snapshot) before the first render.
|
||||
@@ -145,41 +202,68 @@ fn main() -> anyhow::Result<()> {
|
||||
// Drain all queued keystrokes (type-ahead absorbed during a refresh),
|
||||
// apply them, then do a single refresh for the batch.
|
||||
let mut keys = 0;
|
||||
let mut effect = Effect::None;
|
||||
while let Some(k) = usb_kbd::next_key() {
|
||||
// A `:` command (only) yields an Effect; keep the last one in the batch.
|
||||
match ed.handle(k) {
|
||||
Effect::None => {}
|
||||
e => effect = e,
|
||||
}
|
||||
ed.handle(k);
|
||||
keys += 1;
|
||||
}
|
||||
|
||||
// Carry out any host-side effect a `:` command asked for. Save is inline
|
||||
// (fast). `:sync` persists, then hands off to the git thread — the push
|
||||
// is behind the `git` feature, so a light build carries no libgit2/git2.
|
||||
match effect {
|
||||
Effect::None => {}
|
||||
Effect::Save => save_note(&storage, &mut ed),
|
||||
Effect::Publish => {
|
||||
// Publishing an unsaved buffer is meaningless, so persist first.
|
||||
save_note(&storage, &mut ed);
|
||||
// Then signal the git thread — non-blocking, so the ~10 s push
|
||||
// never stalls the editor. The outcome returns on `git_rx` and
|
||||
// updates the snackbar (see the idle branch below).
|
||||
#[cfg(feature = "git")]
|
||||
match git_tx.send(firmware::git_sync::PublishRequest) {
|
||||
Ok(()) => ed.set_notice("syncing..."),
|
||||
Err(_) => ed.set_notice("sync: git thread down"),
|
||||
}
|
||||
#[cfg(not(feature = "git"))]
|
||||
log::info!(":sync — saved; light build (no `git` feature) — push skipped");
|
||||
// Service the host-side effects the batch queued, in order. A file open
|
||||
// queues a Save of the outgoing dirty buffer *then* a Load of the target;
|
||||
// `:sync` queues a Save of the current buffer *then* Publish. Save/Load
|
||||
// are inline (fast SD IO); Publish hands off to the git thread — behind
|
||||
// the `git` feature, so a light build carries no libgit2/git2.
|
||||
//
|
||||
// Drain to empty rather than once: servicing a Load can itself queue an
|
||||
// eviction Save (when the swap pushes a dirty parked buffer out of the
|
||||
// ≤3 window), and that must be persisted now, not deferred to the next
|
||||
// keystroke where a power-off could lose it. The queue strictly shrinks
|
||||
// (a Save/Publish/Pull queues nothing; a Load queues at most one Save),
|
||||
// so this terminates.
|
||||
loop {
|
||||
let effects = ed.take_effects();
|
||||
if effects.is_empty() {
|
||||
break;
|
||||
}
|
||||
Effect::Pull => {
|
||||
// `:gl` — fetch + fast-forward from the remote. The on-device
|
||||
// fetch/fast-forward on the git thread is v0.7 work (git_sync
|
||||
// only exposes push today), so acknowledge and no-op for now.
|
||||
ed.set_notice("pull: not wired yet (v0.7)");
|
||||
for effect in effects {
|
||||
match effect {
|
||||
Effect::Save { path, contents, .. } => {
|
||||
save_buffer(&storage, &mut ed, &path, &contents)
|
||||
}
|
||||
Effect::Load { path, scope } => open_buffer(&storage, &mut ed, path, scope),
|
||||
Effect::Publish => {
|
||||
// Non-blocking, so the ~10 s push never stalls the editor.
|
||||
// The outcome returns on `git_rx` and updates the snackbar
|
||||
// (see the idle branch below). The Save that preceded this
|
||||
// in the batch already persisted the buffer, so this is a
|
||||
// pure git publish of the recorded dirty paths — the
|
||||
// outcome decides whether the snapshot is forgotten
|
||||
// (publish_succeeded) or retried (publish_failed).
|
||||
#[cfg(feature = "git")]
|
||||
{
|
||||
let paths = storage.take_dirty();
|
||||
match git_tx.send(firmware::git_sync::PublishRequest { paths }) {
|
||||
Ok(()) => ed.set_notice("syncing..."),
|
||||
Err(_) => {
|
||||
// Thread gone — nothing will report back, so
|
||||
// return the snapshot to pending ourselves.
|
||||
storage.publish_failed();
|
||||
ed.set_notice("sync: git thread down");
|
||||
}
|
||||
}
|
||||
}
|
||||
#[cfg(not(feature = "git"))]
|
||||
log::info!(":sync — saved; light build (no `git` feature) — push skipped");
|
||||
}
|
||||
Effect::Pull => {
|
||||
// `:gl` — fetch + fast-forward from the remote. The
|
||||
// on-device fetch/fast-forward on the git thread is v0.7
|
||||
// work (git_sync only exposes push today), so acknowledge
|
||||
// and no-op for now.
|
||||
ed.set_notice("pull: not wired yet (v0.7)");
|
||||
}
|
||||
Effect::Delete { path, scope } => delete_buffer(&storage, &mut ed, path, scope),
|
||||
Effect::SavePrefs { contents } => save_prefs(&storage, &mut ed, &contents),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -196,13 +280,24 @@ fn main() -> anyhow::Result<()> {
|
||||
#[cfg(feature = "git")]
|
||||
if let Ok(outcome) = git_rx.try_recv() {
|
||||
use firmware::git_sync::PublishOutcome::*;
|
||||
// Settle the dirty snapshot this publish took: confirmed
|
||||
// published (or up to date) → forget it; failed → back to
|
||||
// pending so the next :sync retries the same paths.
|
||||
match &outcome {
|
||||
Pushed(_) | UpToDate => storage.publish_succeeded(),
|
||||
Failed(_) => storage.publish_failed(),
|
||||
}
|
||||
ed.set_notice(match outcome {
|
||||
Pushed(oid) => format!("synced {oid}"),
|
||||
UpToDate => "up to date".to_string(),
|
||||
Failed(reason) => reason,
|
||||
});
|
||||
let f = ed.draw(true);
|
||||
epd.display_frame_partial_window(f.bytes(), 0, epd::HEIGHT)?;
|
||||
if let Err(e) = epd.display_frame_partial_window(f.bytes(), 0, epd::HEIGHT) {
|
||||
log::warn!("sync-notice repaint FAILED ({e}); full refresh next");
|
||||
force_full = true;
|
||||
continue;
|
||||
}
|
||||
shown = f;
|
||||
cursor_shown = true;
|
||||
continue;
|
||||
@@ -211,12 +306,40 @@ fn main() -> anyhow::Result<()> {
|
||||
// no keystroke will arrive to trigger it otherwise.
|
||||
if kbd_changed {
|
||||
let f = ed.draw(true);
|
||||
epd.display_frame_partial_window(f.bytes(), 0, epd::HEIGHT)?;
|
||||
if let Err(e) = epd.display_frame_partial_window(f.bytes(), 0, epd::HEIGHT) {
|
||||
log::warn!("kbd-flag repaint FAILED ({e}); full refresh next");
|
||||
force_full = true;
|
||||
continue;
|
||||
}
|
||||
shown = f;
|
||||
cursor_shown = true;
|
||||
log::info!("keyboard {}", if kbd { "connected" } else { "disconnected" });
|
||||
continue;
|
||||
}
|
||||
// save_on_idle: once input has paused, quietly persist a dirty named
|
||||
// buffer so a power pull can't cost more than the last couple seconds.
|
||||
// Silent — no snackbar and no forced e-ink flash (a safety net, not an
|
||||
// action; `:w` is the loud save). Unformatted: fmt only runs on an
|
||||
// explicit `:w`/`:sync`, never reflowing text mid-session. Fires once
|
||||
// per idle window (`idle_saved`), so a failing save can't busy-loop.
|
||||
if !idle_saved
|
||||
&& ed.prefs().save_on_idle
|
||||
&& ed.dirty()
|
||||
&& !ed.path().is_empty()
|
||||
&& last_activity.elapsed().as_millis() >= IDLE_SAVE_MS
|
||||
{
|
||||
idle_saved = true;
|
||||
let path = ed.path().to_string();
|
||||
match storage.save_path(&path, ed.text()) {
|
||||
Ok(()) => {
|
||||
log::info!("idle-save: {} bytes to {path}", ed.text().len());
|
||||
ed.mark_saved(&path);
|
||||
}
|
||||
Err(e) => log::warn!("idle-save FAILED ({e:#}); buffer kept in RAM"),
|
||||
}
|
||||
// No repaint: `dirty` clearing has no visible effect, and a flash
|
||||
// here would defeat the point. Fall through to the caret/idle path.
|
||||
}
|
||||
// Debounced caret, Insert mode only: once typing pauses, bring the
|
||||
// bar caret back and refresh the panel word count with a silent
|
||||
// full-area partial (no flash). Normal/View draw their caret on action.
|
||||
@@ -226,10 +349,14 @@ fn main() -> anyhow::Result<()> {
|
||||
{
|
||||
ed.refresh_stats();
|
||||
let f = ed.draw(true);
|
||||
epd.display_frame_partial_window(f.bytes(), 0, epd::HEIGHT)?;
|
||||
shown = f;
|
||||
cursor_shown = true;
|
||||
log::info!("caret shown");
|
||||
if let Err(e) = epd.display_frame_partial_window(f.bytes(), 0, epd::HEIGHT) {
|
||||
log::warn!("caret repaint FAILED ({e}); full refresh next");
|
||||
force_full = true;
|
||||
} else {
|
||||
shown = f;
|
||||
cursor_shown = true;
|
||||
log::info!("caret shown");
|
||||
}
|
||||
} else {
|
||||
FreeRtos::delay_ms(8);
|
||||
}
|
||||
@@ -237,6 +364,7 @@ fn main() -> anyhow::Result<()> {
|
||||
}
|
||||
|
||||
last_activity = Instant::now();
|
||||
idle_saved = false; // fresh activity reopens the save_on_idle window
|
||||
// Non-Insert actions (Normal edits, mode switches) aren't rapid typing,
|
||||
// so the panel word count can refresh immediately; in Insert the snapshot
|
||||
// stays frozen until the typing-pause path above refreshes it.
|
||||
@@ -273,17 +401,29 @@ fn main() -> anyhow::Result<()> {
|
||||
&& only_adds_ink(shown.bytes(), frame.bytes(), y0, y1);
|
||||
|
||||
let t0 = Instant::now();
|
||||
let refresh = if periodic {
|
||||
epd.display_frame(frame.bytes())?;
|
||||
"FULL"
|
||||
// `force_full` promotes to a full refresh after a failed paint: it
|
||||
// rewrites both RAM banks, recovering from a partial that may have died
|
||||
// mid-transfer and desynced them.
|
||||
let (result, refresh) = if periodic || force_full {
|
||||
(epd.display_frame(frame.bytes()), "FULL")
|
||||
} else if additive {
|
||||
epd.display_frame_partial_window(frame.bytes(), y0, y1 - y0 + 1)?;
|
||||
"windowed"
|
||||
(epd.display_frame_partial_window(frame.bytes(), y0, y1 - y0 + 1), "windowed")
|
||||
} else {
|
||||
epd.display_frame_partial_window(frame.bytes(), 0, epd::HEIGHT)?;
|
||||
"full-area"
|
||||
(epd.display_frame_partial_window(frame.bytes(), 0, epd::HEIGHT), "full-area")
|
||||
};
|
||||
let ms = t0.elapsed().as_millis();
|
||||
if let Err(e) = result {
|
||||
// Never fatal — the buffer is the source of truth and safe in RAM,
|
||||
// exactly like a failed `save_buffer`. Drop this frame, leave `shown`
|
||||
// untouched so the next paint repaints the same diff, and force a
|
||||
// clean full refresh then. Typical cause: internal DMA-capable RAM
|
||||
// briefly starved by Wi-Fi/TLS during a background `:sync`; it frees
|
||||
// the moment the push finishes.
|
||||
log::warn!("{refresh} refresh #{updates} FAILED ({e}); frame dropped, full refresh next");
|
||||
force_full = true;
|
||||
continue;
|
||||
}
|
||||
force_full = false;
|
||||
log::info!(
|
||||
"{refresh} refresh #{updates} [{:?}]: {ms} ms (rows {y0}..={y1}, {keys} key(s))",
|
||||
ed.mode()
|
||||
@@ -298,7 +438,15 @@ fn main() -> anyhow::Result<()> {
|
||||
/// `main`): the note is the whole point of the appliance, so we refuse to run
|
||||
/// in a state where the next save could destroy it.
|
||||
fn boot_storage(epd: &mut Epd) -> (Storage, String) {
|
||||
let storage = match Storage::mount() {
|
||||
// A git build shares this mount with the git thread, and libgit2 keeps the
|
||||
// pack + idx descriptors open across a publish — that overruns the
|
||||
// editor's tight 4-FD budget, so mount with the 16-FD one (persistence.rs,
|
||||
// MAX_FILES_GIT). The light build keeps the editor's own budget.
|
||||
#[cfg(feature = "git")]
|
||||
let mounted = Storage::mount_for_git();
|
||||
#[cfg(not(feature = "git"))]
|
||||
let mounted = Storage::mount();
|
||||
let storage = match mounted {
|
||||
Ok(s) => s,
|
||||
Err(e) => boot_halt(epd, "SD card not ready", &format!("{e:#}")),
|
||||
};
|
||||
@@ -337,23 +485,159 @@ fn show_message(epd: &mut Epd, msg: &str) -> anyhow::Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Persist the buffer to SD. Errors are logged, never propagated: the in-RAM
|
||||
/// buffer is the source of truth and must survive a failed write (e.g. a card
|
||||
/// pulled mid-session) so the user can fix the card and retry `:w`.
|
||||
fn save_note(storage: &Storage, ed: &mut Editor) {
|
||||
let n = ed.text().len();
|
||||
match storage.save(ed.text()) {
|
||||
/// Persist a buffer to SD at `path`. Errors are logged, never propagated: the
|
||||
/// in-RAM buffer is the source of truth and must survive a failed write (e.g. a
|
||||
/// card pulled mid-session) so the user can fix the card and retry `:w`. On
|
||||
/// success the editor's dirty flag for that path is cleared.
|
||||
fn save_buffer(storage: &Storage, ed: &mut Editor, path: &str, contents: &str) {
|
||||
match storage.save_path(path, contents) {
|
||||
Ok(()) => {
|
||||
log::info!(":w — saved {n} bytes to {NOTES}");
|
||||
log::info!(":w — saved {} bytes to {path}", contents.len());
|
||||
ed.mark_saved(path);
|
||||
ed.set_notice("saved");
|
||||
}
|
||||
Err(e) => {
|
||||
log::error!(":w — save FAILED ({e:#}); buffer kept in RAM, retry :w");
|
||||
log::error!("save FAILED ({e:#}); buffer kept in RAM, retry :w");
|
||||
ed.set_notice("save FAILED - retry :w");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Persist the preferences file after a palette `>` command changed a pref
|
||||
/// (`Effect::SavePrefs`). The editor already applied the change live and
|
||||
/// serialized it; this is a plain atomic write to the fixed `.typoena.toml`
|
||||
/// path. Under `/sd/repo`, so it rides the next `:sync` to other devices.
|
||||
fn save_prefs(storage: &Storage, ed: &mut Editor, contents: &str) {
|
||||
match storage.save_path(PREFS_PATH, contents) {
|
||||
Ok(()) => log::info!("prefs saved to {PREFS_PATH}"),
|
||||
Err(e) => {
|
||||
log::error!("prefs save FAILED ({e:#})");
|
||||
ed.set_notice("prefs save FAILED");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Read `path` from SD and install it as the active buffer (the multi-file open
|
||||
/// path, from `:e` / the palette). A read failure keeps the current buffer and
|
||||
/// surfaces the reason on the snackbar rather than swapping to an empty screen.
|
||||
fn open_buffer(storage: &Storage, ed: &mut Editor, path: String, scope: Scope) {
|
||||
match storage.load_path(&path) {
|
||||
Ok(text) => {
|
||||
log::info!("opened {path} ({} bytes, {scope:?})", text.len());
|
||||
let name = file_stem(&path);
|
||||
ed.set_notice(format!("loaded {name}"));
|
||||
ed.install_loaded(path, scope, text);
|
||||
}
|
||||
Err(e) => {
|
||||
log::error!("open {path} FAILED ({e:#})");
|
||||
ed.set_notice(format!("can't open {}", file_stem(&path)));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Unlink a file from the card (`:delete`). The editor has already dropped it
|
||||
/// from its model and switched away, so this is pure IO plus the snackbar. For a
|
||||
/// Tracked file the removal is left in the git working copy — the next `:sync`'s
|
||||
/// `add --all` stages the deletion — so nothing git-specific happens here. A
|
||||
/// failure keeps the file on disk and says so; the buffer has still switched, so
|
||||
/// the file is recoverable by re-opening it.
|
||||
fn delete_buffer(storage: &Storage, ed: &mut Editor, path: String, scope: Scope) {
|
||||
// Scope-qualified label (`repo/notes.md`), so the snackbar names exactly which
|
||||
// file left the card — and, for a Tracked file, that the removal is only local
|
||||
// until the next `:sync` publishes it (deleting from the card alone never
|
||||
// touches the remote — that mirrors how a Save is local until Publish).
|
||||
let label = path.strip_prefix("/sd/").unwrap_or(&path);
|
||||
match storage.delete_path(&path) {
|
||||
Ok(()) => {
|
||||
log::info!("deleted {path} ({scope:?})");
|
||||
ed.set_notice(match scope {
|
||||
Scope::Tracked => format!("deleted {label} - :sync to publish"),
|
||||
Scope::Local => format!("deleted {label}"),
|
||||
});
|
||||
}
|
||||
Err(e) => {
|
||||
log::error!("delete {path} FAILED ({e:#})");
|
||||
ed.set_notice(format!("delete FAILED: {label}"));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Enumerate the palette's openable files: the regular files under `/sd/repo`
|
||||
/// and `/sd/local`, recursively, as absolute paths. Skips dot entries at every
|
||||
/// level (so `.git` and its thousands of object files, `.typoena.toml`, and the
|
||||
/// like never show or get walked). Best-effort: an unreadable directory (e.g.
|
||||
/// no `/sd/local` yet) contributes nothing rather than failing. The editor
|
||||
/// sorts and dedupes. Runs once at boot, so the walk time is logged — on a big
|
||||
/// repo the FAT directory IO is the cost to watch.
|
||||
fn enumerate_files() -> Vec<String> {
|
||||
let start = std::time::Instant::now();
|
||||
let mut out = Vec::new();
|
||||
for dir in [REPO_DIR, LOCAL_DIR] {
|
||||
walk_files(std::path::Path::new(dir), 0, &mut out);
|
||||
}
|
||||
log::info!("file walk: {} files in {}ms", out.len(), start.elapsed().as_millis());
|
||||
out
|
||||
}
|
||||
|
||||
/// Depth bound for [`walk_files`] — belt-and-braces against pathological
|
||||
/// nesting on a hand-edited card; notes trees are a couple of levels deep.
|
||||
const WALK_MAX_DEPTH: usize = 8;
|
||||
|
||||
/// Recursive helper for [`enumerate_files`]: push `dir`'s files onto `out`,
|
||||
/// then descend into its subdirectories. Reads each directory fully before
|
||||
/// recursing (the `remove_dir_recursive` pattern in `git_sync`), so only one
|
||||
/// FatFS directory handle is open at a time regardless of depth — relevant on
|
||||
/// the FD-bounded SD mount.
|
||||
fn walk_files(dir: &std::path::Path, depth: usize, out: &mut Vec<String>) {
|
||||
if depth > WALK_MAX_DEPTH {
|
||||
log::warn!("file walk: {} exceeds depth {WALK_MAX_DEPTH}, skipped", dir.display());
|
||||
return;
|
||||
}
|
||||
let Ok(entries) = std::fs::read_dir(dir) else {
|
||||
return;
|
||||
};
|
||||
// Keep the dirent's own file type: esp-idf's FAT VFS always fills d_type
|
||||
// (DT_DIR/DT_REG, straight from the FILINFO readdir already holds), so
|
||||
// `file_type()` is free. A per-entry `metadata()` stat instead re-walks
|
||||
// the directory by path every time — measured at ~32ms/file on the SD
|
||||
// card, it turned a 1098-file walk into 35s.
|
||||
let children: Vec<_> = entries
|
||||
.flatten()
|
||||
.filter_map(|e| e.file_type().ok().map(|t| (e.path(), t)))
|
||||
.collect();
|
||||
for (path, ftype) in children {
|
||||
let Some(name) = path.file_name().and_then(|n| n.to_str()) else {
|
||||
continue;
|
||||
};
|
||||
if name.starts_with('.') {
|
||||
continue;
|
||||
}
|
||||
if ftype.is_file() {
|
||||
if let Some(p) = path.to_str() {
|
||||
out.push(p.to_string());
|
||||
}
|
||||
} else if ftype.is_dir() {
|
||||
walk_files(&path, depth + 1, out);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Free internal DRAM (excludes the 8 MB PSRAM pool, which dominates the total
|
||||
/// free-heap number and masks DRAM exhaustion). Same reading `git_sync` logs.
|
||||
fn internal_free_heap() -> u32 {
|
||||
use esp_idf_svc::sys;
|
||||
unsafe { sys::heap_caps_get_free_size(sys::MALLOC_CAP_INTERNAL) as u32 }
|
||||
}
|
||||
|
||||
/// A file's display name — its basename without extension (`/sd/repo/notes.md`
|
||||
/// → `notes`), for the snackbar. Falls back to the raw path if it has no stem.
|
||||
fn file_stem(path: &str) -> &str {
|
||||
std::path::Path::new(path)
|
||||
.file_stem()
|
||||
.and_then(|s| s.to_str())
|
||||
.unwrap_or(path)
|
||||
}
|
||||
|
||||
/// First and last (inclusive) framebuffer rows that differ between two frames,
|
||||
/// or `None` if identical. Lets the partial refresh target just the band a
|
||||
/// keystroke touched instead of all 272 rows.
|
||||
|
||||
@@ -34,6 +34,8 @@
|
||||
//! sits in the tmp. [`Storage::recover`] closes the loop at boot — see its docs
|
||||
//! for the exact case analysis, which is subtler than "promote the tmp."
|
||||
|
||||
use std::cell::RefCell;
|
||||
use std::collections::BTreeSet;
|
||||
use std::fs;
|
||||
use std::io::Write as _;
|
||||
use std::mem::MaybeUninit;
|
||||
@@ -79,6 +81,23 @@ pub const MAX_FILE_BYTES: u64 = 256 * 1024;
|
||||
/// The C mount point (`/sd\0`) for the esp-idf FFI calls.
|
||||
const MOUNT_C: &std::ffi::CStr = c"/sd";
|
||||
|
||||
/// Dirty-path journal — one repo-relative path per line, mirroring the in-RAM
|
||||
/// dirty set (see [`Storage::take_dirty`]). At the card root, *outside*
|
||||
/// `/sd/repo`, so it can never itself be committed. Without it a power pull
|
||||
/// would strand every file saved-but-not-yet-published in that session: the
|
||||
/// splice commit only visits recorded paths (nothing walks the tree anymore),
|
||||
/// so an unrecorded change would never reach the remote.
|
||||
const DIRTY_JOURNAL: &str = "/sd/.typoena-dirty";
|
||||
|
||||
/// VFS open-file budget for the editor path: it opens only a note and its
|
||||
/// `*.tmp`, so a tight budget keeps FatFS's per-file buffers off the heap.
|
||||
const MAX_FILES_EDITOR: i32 = 4;
|
||||
/// VFS open-file budget for the git tooling. libgit2 keeps the pack + `.idx`
|
||||
/// (and commit-graph) descriptors open for the repo's lifetime and opens loose
|
||||
/// objects on top, so a `read_tree` walk overruns [`MAX_FILES_EDITOR`] with a
|
||||
/// "no free file descriptors" error. Matches the flash-FAT git binaries' 16.
|
||||
const MAX_FILES_GIT: i32 = 16;
|
||||
|
||||
/// A mounted SD card. Holds the live card handle for its lifetime; v0.1 never
|
||||
/// unmounts (the card stays up for the whole power session). Not `Send` — the
|
||||
/// handle lives on the task that mounted it (the ui/main task). The git thread
|
||||
@@ -86,6 +105,23 @@ const MOUNT_C: &std::ffi::CStr = c"/sd";
|
||||
/// lock serialises the two, so no extra mutex is needed here.
|
||||
pub struct Storage {
|
||||
card: *mut sys::sdmmc_card_t,
|
||||
/// Repo-relative paths saved or `:delete`d since the last confirmed
|
||||
/// publish — the editor-side half of the O(depth) splice commit
|
||||
/// (`git_sync::stage_and_commit` visits exactly these paths and nothing
|
||||
/// else). Mirrored to [`DIRTY_JOURNAL`] whenever it changes, so the record
|
||||
/// survives a power pull. `RefCell` because recording happens inside
|
||||
/// `&self` save/delete calls; `Storage` already lives on one task only.
|
||||
dirty: RefCell<Dirty>,
|
||||
}
|
||||
|
||||
/// The two halves of the dirty record: `pending` accumulates between syncs;
|
||||
/// `take_dirty` moves it to `in_flight` for the duration of a publish so a
|
||||
/// failure can put it back (and a save landing *during* the publish re-enters
|
||||
/// `pending`, riding the next one). The journal always carries the union.
|
||||
#[derive(Default)]
|
||||
struct Dirty {
|
||||
pending: BTreeSet<String>,
|
||||
in_flight: BTreeSet<String>,
|
||||
}
|
||||
|
||||
/// What [`Storage::recover`] did with a leftover `*.tmp` at boot.
|
||||
@@ -113,6 +149,17 @@ impl Storage {
|
||||
/// (The Spike 3 bench binary sets it true for convenience on blank cards;
|
||||
/// this path must not.)
|
||||
pub fn mount() -> Result<Self> {
|
||||
Self::mount_with_max_files(MAX_FILES_EDITOR)
|
||||
}
|
||||
|
||||
/// Like [`Storage::mount`], but with the larger [`MAX_FILES_GIT`] open-file
|
||||
/// budget the git tooling (bench / sync) needs — libgit2 holds several
|
||||
/// descriptors open at once, which the editor's default budget can't cover.
|
||||
pub fn mount_for_git() -> Result<Self> {
|
||||
Self::mount_with_max_files(MAX_FILES_GIT)
|
||||
}
|
||||
|
||||
fn mount_with_max_files(max_files: i32) -> Result<Self> {
|
||||
// 1) SPI3 with the SD's four lines. Dedicated bus (ADR-012) — no EPD
|
||||
// deselect needed: the panel is on SPI2 and can't contend here.
|
||||
// SAFETY: a zeroed spi_bus_config_t is valid (all pins default 0); we
|
||||
@@ -177,7 +224,7 @@ impl Storage {
|
||||
// 4) Mount config. format_if_mount_failed = FALSE — see method docs.
|
||||
let mount = sys::esp_vfs_fat_mount_config_t {
|
||||
format_if_mount_failed: false,
|
||||
max_files: 4,
|
||||
max_files,
|
||||
allocation_unit_size: 16 * 1024,
|
||||
disk_status_check_enable: false,
|
||||
use_one_fat: false,
|
||||
@@ -203,7 +250,10 @@ impl Storage {
|
||||
}
|
||||
esp!(rc).context("esp_vfs_fat_sdspi_mount (card present? inserted? FAT-formatted?)")?;
|
||||
|
||||
let storage = Storage { card };
|
||||
let storage = Storage {
|
||||
card,
|
||||
dirty: RefCell::new(Dirty::default()),
|
||||
};
|
||||
let (max_khz, real_khz) = storage.negotiated_khz();
|
||||
log::info!("SD mounted at {MOUNT} — max {max_khz} kHz, negotiated {real_khz} kHz");
|
||||
|
||||
@@ -218,6 +268,13 @@ impl Storage {
|
||||
newest complete copy)"
|
||||
),
|
||||
}
|
||||
let carried = storage.load_dirty_journal();
|
||||
if carried > 0 {
|
||||
log::info!(
|
||||
"dirty journal: {carried} unpublished path(s) carried over from a previous \
|
||||
session — the next :sync will commit them"
|
||||
);
|
||||
}
|
||||
Ok(storage)
|
||||
}
|
||||
|
||||
@@ -250,45 +307,189 @@ impl Storage {
|
||||
Path::new(REPO_DIR).is_dir()
|
||||
}
|
||||
|
||||
/// Read `notes.md` into a `String`. Returns an empty string if the file
|
||||
/// doesn't exist yet (fresh, but provisioned, repo). Refuses a file larger
|
||||
/// than [`MAX_FILE_BYTES`] rather than loading it.
|
||||
/// Read `notes.md` into a `String` — the boot default note. Thin wrapper over
|
||||
/// [`Storage::load_path`].
|
||||
pub fn load(&self) -> Result<String> {
|
||||
match fs::metadata(NOTES) {
|
||||
self.load_path(NOTES)
|
||||
}
|
||||
|
||||
/// Read an arbitrary file under `/sd` into a `String`. Returns an empty string
|
||||
/// if the file doesn't exist yet (a `:e` of a not-yet-created name, or a fresh
|
||||
/// repo). Refuses a file larger than [`MAX_FILE_BYTES`] rather than loading it.
|
||||
///
|
||||
/// The multi-file (v0.5) load path: the editor names the file, the host reads
|
||||
/// it here and hands the text back through `Editor::install_loaded`.
|
||||
pub fn load_path(&self, path: &str) -> Result<String> {
|
||||
match fs::metadata(path) {
|
||||
Ok(m) if m.len() > MAX_FILE_BYTES => bail!(
|
||||
"{NOTES} is {} KiB — over the {} KiB v0.1 limit; open it on a computer to split it",
|
||||
"{path} is {} KiB — over the {} KiB limit; open it on a computer to split it",
|
||||
m.len() / 1024,
|
||||
MAX_FILE_BYTES / 1024
|
||||
),
|
||||
Ok(_) => fs::read_to_string(NOTES).with_context(|| format!("reading {NOTES}")),
|
||||
// Read the file verbatim. The editor's `rows = #\n + 1` model renders a
|
||||
// trailing '\n' as an empty last line, and we *want* that: a note ends
|
||||
// with a visible blank line that reflects its POSIX terminator. Since
|
||||
// `save_path` guarantees that terminator, this load and that save form an
|
||||
// identity round-trip for any device-written file (which always ends in
|
||||
// '\n') — no strip needed, and none wanted.
|
||||
Ok(_) => fs::read_to_string(path).with_context(|| format!("reading {path}")),
|
||||
Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(String::new()),
|
||||
Err(e) => Err(e).with_context(|| format!("stat {NOTES}")),
|
||||
Err(e) => Err(e).with_context(|| format!("stat {path}")),
|
||||
}
|
||||
}
|
||||
|
||||
/// Atomically persist `contents` to `notes.md`: write the tmp, fsync,
|
||||
/// unlink the target, rename over it. See the module docs for why the unlink
|
||||
/// is mandatory on FAT and [`Storage::recover`] for the crash window it opens.
|
||||
/// Atomically persist `contents` to `notes.md`. Thin wrapper over
|
||||
/// [`Storage::save_path`].
|
||||
pub fn save(&self, contents: &str) -> Result<()> {
|
||||
self.save_path(NOTES, contents)
|
||||
}
|
||||
|
||||
/// Atomically persist `contents` to an arbitrary file under `/sd`: write the
|
||||
/// tmp, fsync, unlink the target, rename over it. See the module docs for why
|
||||
/// the unlink is mandatory on FAT. Boot recovery ([`Storage::recover`]) still
|
||||
/// only covers the default `notes.md`; per-file recovery for the other v0.5
|
||||
/// buffers is deferred to the v0.9 crash-safety work — the atomic swap here
|
||||
/// already protects each individual save.
|
||||
pub fn save_path(&self, path: &str, contents: &str) -> Result<()> {
|
||||
// Record BEFORE writing: a crash in between leaves an over-approximate
|
||||
// journal (the splice of an unchanged path is a no-op), whereas the
|
||||
// reverse order could leave a changed file no record ever points at.
|
||||
self.record_dirty(path);
|
||||
Self::atomic_write(path, contents)
|
||||
}
|
||||
|
||||
/// The atomic write primitive behind [`Storage::save_path`] and the dirty
|
||||
/// journal: write `{path}.tmp`, fsync, unlink the target, rename over it.
|
||||
fn atomic_write(path: &str, contents: &str) -> Result<()> {
|
||||
let tmp = format!("{path}.tmp");
|
||||
{
|
||||
let mut f = fs::File::create(NOTES_TMP)
|
||||
.with_context(|| format!("create {NOTES_TMP} (is {REPO_DIR} present?)"))?;
|
||||
let mut f = fs::File::create(&tmp)
|
||||
.with_context(|| format!("create {tmp} (does its directory exist?)"))?;
|
||||
f.write_all(contents.as_bytes())
|
||||
.with_context(|| format!("write {NOTES_TMP}"))?;
|
||||
.with_context(|| format!("write {tmp}"))?;
|
||||
// Insert a final newline only if the buffer lacks one (POSIX text
|
||||
// convention; keeps git from flagging "No newline at end of file").
|
||||
// `load_path` reads verbatim, so this is the sole place the terminator is
|
||||
// guaranteed — and because it's guarded, the file mirrors the buffer's
|
||||
// trailing newlines exactly: one visible trailing blank line stays one,
|
||||
// never doubled. A buffer that already ends in '\n' passes through as-is.
|
||||
if !contents.ends_with('\n') {
|
||||
f.write_all(b"\n")
|
||||
.with_context(|| format!("write final newline to {tmp}"))?;
|
||||
}
|
||||
// FatFS f_sync — flush the tmp fully before it can replace the target.
|
||||
f.sync_all().with_context(|| format!("fsync {NOTES_TMP}"))?;
|
||||
f.sync_all().with_context(|| format!("fsync {tmp}"))?;
|
||||
}
|
||||
// FatFS f_rename won't overwrite, so unlink the target first (tolerate a
|
||||
// missing target: the first-ever save has nothing to remove).
|
||||
match fs::remove_file(NOTES) {
|
||||
match fs::remove_file(path) {
|
||||
Ok(()) => {}
|
||||
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {}
|
||||
Err(e) => return Err(e).with_context(|| format!("unlink {NOTES} before rename")),
|
||||
Err(e) => return Err(e).with_context(|| format!("unlink {path} before rename")),
|
||||
}
|
||||
fs::rename(NOTES_TMP, NOTES).with_context(|| format!("rename {NOTES_TMP} -> {NOTES}"))?;
|
||||
fs::rename(&tmp, path).with_context(|| format!("rename {tmp} -> {path}"))?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Unlink a file under `/sd` (`:delete`). Tolerates a missing target — an
|
||||
/// already-gone file is a success, so the call is idempotent. Also clears a
|
||||
/// stray `{path}.tmp` best-effort, so a crash-interrupted save can't leave the
|
||||
/// file half-present after a delete. For a Tracked file this leaves the
|
||||
/// working copy short one file; the next publish's `add --all` stages it.
|
||||
pub fn delete_path(&self, path: &str) -> Result<()> {
|
||||
// Same record-first rule as `save_path`: the splice treats a recorded
|
||||
// path with no file behind it as "remove from the tree".
|
||||
self.record_dirty(path);
|
||||
let _ = fs::remove_file(format!("{path}.tmp"));
|
||||
match fs::remove_file(path) {
|
||||
Ok(()) => Ok(()),
|
||||
Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(()),
|
||||
Err(e) => Err(e).with_context(|| format!("unlink {path}")),
|
||||
}
|
||||
}
|
||||
|
||||
/// Note a working-copy file as (possibly) differing from HEAD. Paths
|
||||
/// outside `/sd/repo` (`/sd/local`, `/sd/ca.pem`, the journal itself) are
|
||||
/// not git's business and are skipped. The journal is rewritten only when
|
||||
/// the set actually grows, so re-saving the same note between syncs costs
|
||||
/// no extra card I/O.
|
||||
fn record_dirty(&self, abs_path: &str) {
|
||||
let Some(rel) = abs_path
|
||||
.strip_prefix(REPO_DIR)
|
||||
.and_then(|r| r.strip_prefix('/'))
|
||||
else {
|
||||
return;
|
||||
};
|
||||
if rel.is_empty() {
|
||||
return;
|
||||
}
|
||||
let grew = self.dirty.borrow_mut().pending.insert(rel.to_string());
|
||||
if grew {
|
||||
self.persist_dirty();
|
||||
}
|
||||
}
|
||||
|
||||
/// Snapshot the dirty paths for a publish (repo-relative). The snapshot
|
||||
/// moves to `in_flight` — the journal keeps carrying it — until the UI
|
||||
/// task reports the outcome: [`Storage::publish_succeeded`] forgets it,
|
||||
/// [`Storage::publish_failed`] returns it to pending for the next `:sync`.
|
||||
pub fn take_dirty(&self) -> BTreeSet<String> {
|
||||
let mut d = self.dirty.borrow_mut();
|
||||
let taken = std::mem::take(&mut d.pending);
|
||||
d.in_flight.extend(taken.iter().cloned());
|
||||
taken
|
||||
}
|
||||
|
||||
/// The publish that took the last snapshot committed (or confirmed
|
||||
/// up-to-date): drop its paths and shrink the journal. Anything saved
|
||||
/// while it ran is still in `pending` and rides the next sync.
|
||||
pub fn publish_succeeded(&self) {
|
||||
self.dirty.borrow_mut().in_flight.clear();
|
||||
self.persist_dirty();
|
||||
}
|
||||
|
||||
/// The publish failed: return its snapshot to pending so the next `:sync`
|
||||
/// retries it (the splice is idempotent, so a retry of an already-clean
|
||||
/// path is free). The journal already carries these paths — no rewrite.
|
||||
pub fn publish_failed(&self) {
|
||||
let mut d = self.dirty.borrow_mut();
|
||||
let inflight = std::mem::take(&mut d.in_flight);
|
||||
d.pending.extend(inflight);
|
||||
}
|
||||
|
||||
/// Mirror `pending ∪ in_flight` to [`DIRTY_JOURNAL`], atomically.
|
||||
/// Best-effort: a failed journal write must not fail the save that
|
||||
/// triggered it — the set stays correct in RAM and the journal heals on
|
||||
/// the next change.
|
||||
fn persist_dirty(&self) {
|
||||
let contents = {
|
||||
let d = self.dirty.borrow();
|
||||
let mut out = String::new();
|
||||
for p in d.pending.union(&d.in_flight) {
|
||||
out.push_str(p);
|
||||
out.push('\n');
|
||||
}
|
||||
out
|
||||
};
|
||||
if let Err(e) = Self::atomic_write(DIRTY_JOURNAL, &contents) {
|
||||
log::warn!("dirty journal write FAILED ({e:#}); set kept in RAM only");
|
||||
}
|
||||
}
|
||||
|
||||
/// Seed the dirty set from the journal at mount — the paths a previous
|
||||
/// session saved but never got confirmed as published (power pull, failed
|
||||
/// sync, or simply no `:sync` before shutdown). Returns how many.
|
||||
fn load_dirty_journal(&self) -> usize {
|
||||
let Ok(text) = fs::read_to_string(DIRTY_JOURNAL) else {
|
||||
return 0; // no journal yet — nothing carried over
|
||||
};
|
||||
let mut d = self.dirty.borrow_mut();
|
||||
for line in text.lines().map(str::trim).filter(|l| !l.is_empty()) {
|
||||
d.pending.insert(line.to_string());
|
||||
}
|
||||
d.pending.len()
|
||||
}
|
||||
|
||||
/// Reconcile a leftover `notes.md.tmp` at boot. The save sequence is
|
||||
/// write-tmp → fsync → unlink-target → rename, so a lingering tmp means the
|
||||
/// last save was interrupted. Which way to recover depends on whether the
|
||||
|
||||
@@ -37,6 +37,16 @@ pub enum Key {
|
||||
/// Ctrl+R — redo (vim `Ctrl-r`); the inverse of `u`. Meaningful in Normal;
|
||||
/// ignored elsewhere.
|
||||
Redo,
|
||||
/// Cmd+P — open the file palette (fuzzy open, v0.5), VS Code "Go to File"
|
||||
/// style. A Normal-mode gesture; **inside** the palette the same chord closes
|
||||
/// it (toggle). Esc also closes. Ignored in Insert.
|
||||
Palette,
|
||||
/// Ctrl+N — move down: one line in Normal/View (vim `CTRL-N` ≡ `j`), or one
|
||||
/// row in the file palette. Ignored in Insert.
|
||||
Down,
|
||||
/// Ctrl+P — move up: one line in Normal/View (vim `CTRL-P` ≡ `k`), or one row
|
||||
/// in the file palette. Ignored in Insert.
|
||||
Up,
|
||||
/// Caps Lock tapped on its own. A no-op for now; groundwork for a future
|
||||
/// vim-style normal mode.
|
||||
Escape,
|
||||
@@ -150,6 +160,9 @@ fn translate(usage: u8, shift: bool, ctrl: bool, cmd: bool) -> Option<Key> {
|
||||
0x07 if ctrl => return Some(Key::HalfPageDown), // Ctrl+D, half-page down
|
||||
0x18 if ctrl => return Some(Key::HalfPageUp), // Ctrl+U, half-page up
|
||||
0x15 if ctrl => return Some(Key::Redo), // Ctrl+R, redo
|
||||
0x13 if ctrl => return Some(Key::Up), // Ctrl+P, move up (vim CTRL-P)
|
||||
0x13 if cmd => return Some(Key::Palette), // Cmd+P, file palette
|
||||
0x11 if ctrl => return Some(Key::Down), // Ctrl+N, move down (vim CTRL-N)
|
||||
_ => {}
|
||||
}
|
||||
|
||||
@@ -394,8 +407,14 @@ mod tests {
|
||||
assert_eq!(translate(0x07, false, true, false), Some(Key::HalfPageDown)); // Ctrl+D
|
||||
assert_eq!(translate(0x18, false, true, false), Some(Key::HalfPageUp)); // Ctrl+U
|
||||
assert_eq!(translate(0x15, false, true, false), Some(Key::Redo)); // Ctrl+R
|
||||
// Without Ctrl these are ordinary letters, not intents.
|
||||
assert_eq!(translate(0x13, false, true, false), Some(Key::Up)); // Ctrl+P, up
|
||||
assert_eq!(translate(0x13, false, false, true), Some(Key::Palette)); // Cmd+P, palette
|
||||
assert_eq!(translate(0x11, false, true, false), Some(Key::Down)); // Ctrl+N, down
|
||||
assert_eq!(translate(0x11, false, false, true), None); // Cmd+N reserved (:enew, v0.5)
|
||||
// Without a modifier these are ordinary letters, not intents.
|
||||
assert_eq!(translate(0x15, false, false, false), Some(Key::Char('r')));
|
||||
assert_eq!(translate(0x13, false, false, false), Some(Key::Char('p')));
|
||||
assert_eq!(translate(0x11, false, false, false), Some(Key::Char('n')));
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
Reference in New Issue
Block a user