Compare commits
3 Commits
76fae36178
...
2f2f1227ec
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2f2f1227ec | ||
|
|
d397b31ca4 | ||
|
|
15955a13f4 |
@@ -217,8 +217,13 @@ up.
|
||||
provisioning / from secure storage instead of `env!()` into the image.
|
||||
- [ ] Fold the push into the editor's `git` module (persistent clone +
|
||||
fast-forward, not a fresh per-boot branch) over the HTTPS+PAT remote.
|
||||
- [ ] Optional tidy: move git to a dedicated large-stack task so the shared
|
||||
main-task stack (and the editor build) can drop back to ~16 KB.
|
||||
- [x] Move git to a dedicated large-stack task so the shared main-task stack (and
|
||||
the editor build) can drop back — **DONE + hardware-verified 2026-07-06**.
|
||||
`git_publish` now runs on its own `std::thread` (`GIT_STACK = 96 KB` via
|
||||
`Builder::stack_size`; main joins it), and `CONFIG_ESP_MAIN_TASK_STACK_SIZE`
|
||||
dropped 98304 → **12288** (the Spike-6 value proven with the editor +
|
||||
TLS-on-main). On-device push succeeded off-main — no panic/overflow, no
|
||||
ENOMEM on the spawn — retiring the "time()-only-works-on-main" misdiagnosis.
|
||||
|
||||
## Path 2 result — libgit2 compiles and links on xtensa (Gate A + Gate B)
|
||||
|
||||
@@ -362,9 +367,10 @@ microcommits through `a15789a`).
|
||||
only works on the main task, not a std::thread" conclusion from the first
|
||||
on-device attempt was wrong — that thread had the *default 4 KB* stack, so the
|
||||
same deep chain just overflowed sooner. It was always stack depth, not
|
||||
thread-vs-main. (Caveat: `sdkconfig.defaults` is shared with the editor build,
|
||||
which now over-reserves this stack; a dedicated large-stack git task would let
|
||||
it drop back to ~16 KB.)
|
||||
thread-vs-main. (This stack has since moved: `sdkconfig.defaults` is shared
|
||||
with the editor build, so git was later given its OWN 96 KB `std::thread` and
|
||||
the main-task stack dropped back to 12 KB — see the follow-ups. The misdiagnosis
|
||||
is now doubly retired: the push runs fine off the main task.)
|
||||
|
||||
2. **`p_rename` = remove-then-rename** (`esp_stubs.c`). FATFS `f_rename` fails
|
||||
`EEXIST` if the target exists and FAT has no hardlinks, so libgit2's own
|
||||
|
||||
@@ -48,7 +48,7 @@ blocked by I/O.
|
||||
| `wifi_task` | 0 | 8 KB | On-demand station: off by default, brought up by `Ctrl-G` |
|
||||
| `ui_task` | 1 | 16 KB | Consume `KeyEvent`s, mutate editor state, enqueue render |
|
||||
| `render_task` | 1 | 12 KB | Drain render queue, do partial/full refresh on EPD |
|
||||
| `git_task` | 1 | 32 KB | Triggered by `Ctrl-G`; runs gitoxide commit + push |
|
||||
| `git_task` | 1 | 96 KB | Triggered by `Ctrl-G`; runs libgit2 commit + push |
|
||||
|
||||
All inter-task communication is via `crossbeam-channel` or `std::sync::mpsc`
|
||||
bounded queues. The editor state is `Arc<Mutex<EditorState>>`; the lock is
|
||||
@@ -97,8 +97,10 @@ spike 4 is the gate for
|
||||
refresh per character, measure end-to-end latency.
|
||||
6. **Spike 6 — Wi-Fi + TLS.** Connect to home Wi-Fi, do an HTTPS GET to
|
||||
`api.github.com`, validate cert chain.
|
||||
7. **Spike 7 — gitoxide push.** Smoke test: from desktop-Rust first, then on
|
||||
device, push a single commit to a test repo over HTTPS+PAT.
|
||||
7. **Spike 7 — git push.** Smoke test: from desktop-Rust first, then on
|
||||
device, push a single commit to a test repo over HTTPS+PAT. **Done** — the
|
||||
gitoxide fall-back to **libgit2** (`git2`) fired here; push is proven on
|
||||
hardware ([postmortem](postmortems/2026-07-05-spike7-gix-https-push.md)).
|
||||
|
||||
Only after spike 7 do we start integration. Any spike that fails forces a
|
||||
stack decision (e.g. fall back to libgit2; switch to a separate SD SPI bus).
|
||||
@@ -196,26 +198,46 @@ to v0.9. v0.1 reads all config from the binary at build time — see
|
||||
|
||||
### `git` — commit + push
|
||||
|
||||
`gitoxide` choice and kill-switch:
|
||||
[ADR-004](adr.md#adr-004-git-implementation--gitoxide-gix).
|
||||
Auth model: [ADR-005](adr.md#adr-005-auth--https--github-personal-access-token).
|
||||
Library + kill-switch:
|
||||
[ADR-004](adr.md#adr-004-git-implementation--gitoxide-gix) — the gitoxide bet was
|
||||
**retired in spike 7** (gix has no HTTP(S) push at all), so the module is built on
|
||||
**libgit2 via the `git2` crate**.
|
||||
Auth + transport are settled: **HTTPS + PAT over esp-idf's mbedTLS**
|
||||
([ADR-005](adr.md#adr-005-auth--https--github-personal-access-token)) — no custom
|
||||
transport and no `reqwest`/`rustls` layer are needed; libgit2's own smart-HTTP
|
||||
client speaks TLS through the mbedTLS stream it is compiled against. The full
|
||||
`init → commit → push` path is **proven on hardware** — see the
|
||||
[spike 7 postmortem](postmortems/2026-07-05-spike7-gix-https-push.md).
|
||||
PSRAM heap during push is a top-3 watched metric — see
|
||||
[qfd.md §6](qfd.md#6-critical-performance-budget).
|
||||
|
||||
- `gix` with the smart-HTTP transport backed by `esp-idf` mbedtls (via a
|
||||
custom transport impl, or `gix-transport` with `reqwest`+`rustls-mbedtls`
|
||||
if that path is cleaner — decided in spike 7).
|
||||
- libgit2 **1.9.4** is vendored as an esp-idf component
|
||||
(`firmware/components/libgit2`) built with `GIT_HTTPS + GIT_MBEDTLS`; `git2`
|
||||
binds it in system mode. A short `esp_stubs.c` supplies the POSIX calls
|
||||
picolibc/FATFS lack (`utimes` existence-gate, `p_rename` replace, uid/symlink).
|
||||
- The server cert chain is verified against an embedded GitHub root bundle
|
||||
(`GIT_OPT_SET_SSL_CERT_LOCATIONS`); the `certificate_check` callback returns
|
||||
PASSTHROUGH so libgit2's own verdict stands → **fail-closed** on an untrusted /
|
||||
MITM cert. A product should refresh those roots (or reuse esp-idf's bundle) at
|
||||
provisioning.
|
||||
- Runs on a **dedicated ~96 KB thread**, not the shared UI/main stack: libgit2's
|
||||
init→push chain nests ~10 `GIT_PATH_MAX` (4 KB) buffers (~67 KB measured).
|
||||
- Operations needed in v0.1 (the [`gct` shell function](../CONTEXT.md#user-facing-actions) is the reference):
|
||||
- `gix::open` the existing working copy at `/sd/repo`
|
||||
- stage everything under the working copy (`git add .` equivalent)
|
||||
- open a **persistent clone** at `/sd/repo` (spike 7 proved the flow on a
|
||||
fresh per-boot init pushing a throwaway `device/<unix>` branch, to isolate
|
||||
the transport from history reconciliation; the product keeps one clone and
|
||||
fast-forwards — storage is [ADR-007](adr.md), still SD-vs-flash-FAT while SD
|
||||
is blocked)
|
||||
- stage everything under the working copy (`git add --all`, incl. deletions)
|
||||
- short-circuit return if nothing is staged — status: "nothing to publish"
|
||||
- commit with author from config, message `"<ISO-8601 timestamp>"` (no `wip`
|
||||
prefix; the timestamp _is_ the message)
|
||||
- push HEAD to `origin/<current branch>`
|
||||
- push HEAD to `origin/<current branch>` (normally a clean fast-forward)
|
||||
- on push failure: `git pull --no-edit` (merge), then retry the push once.
|
||||
Only surface failure to the side panel's publish-state field if the pull
|
||||
conflicts or the second push also fails.
|
||||
- The PAT is loaded into the Authorization header per request; never logged.
|
||||
- The PAT is handed to libgit2's credential callback (`USER_PASS_PLAINTEXT`);
|
||||
never logged, never persisted to the working copy.
|
||||
- The whole sequence is atomic from the user's view — see
|
||||
[`CONTEXT.md` → Publish](../CONTEXT.md#user-facing-actions).
|
||||
|
||||
@@ -254,7 +276,7 @@ ESP32-S3-N16R8: 512 KB SRAM + 8 MB PSRAM. We budget conservatively.
|
||||
| Internal SRAM | ~80 KB | mbedtls runtime (TLS handshake working set) |
|
||||
| PSRAM | ~128 KB | EPD framebuffer (792×272×1 ≈ 27 KB) + shadow + glyph cache |
|
||||
| PSRAM | ~512 KB | rope buffer + edit history headroom |
|
||||
| PSRAM | ~1.5 MB | gitoxide working set during push (pack delta, etc.) |
|
||||
| PSRAM | ~1.5 MB | libgit2 working set during push (pack delta, etc.) |
|
||||
| PSRAM | rest | heap headroom |
|
||||
|
||||
PSRAM is the default for `Box::new` via a custom allocator wrapper; DMA-able
|
||||
|
||||
@@ -1,20 +1,21 @@
|
||||
# Rust often needs a bit of an extra main task stack size compared to C (the default is 3K)
|
||||
# You might have to increase this further if you allocate large stack variables in the main task.
|
||||
# Bumped for the TLS handshake (Spike 6): mbedtls runs on the calling task and
|
||||
# wants several KB of stack on top of the app's own usage.
|
||||
# Bumped hard for Spike 7: libgit2's call chain is stack-hungry. Nearly every
|
||||
# function puts a `char path[GIT_PATH_MAX]` (4 KB) buffer on the stack, and the
|
||||
# repository_init → config-write → FATFS → wear-leveling chain nests ~10 of them
|
||||
# deep — a trivial config write measured ~67 KB of stack on hardware, overflowing
|
||||
# the previous 48 KB and smashing an adjacent newlib lock handle (LoadProhibited
|
||||
# in xQueueGenericSend). The push (smart-HTTP + pack + mbedTLS) is deeper still.
|
||||
# NOTE: this earlier looked like "time() only works on the main task" — but that
|
||||
# iteration ran git on a std::thread with the default 4 KB stack; the same chain
|
||||
# just overflowed sooner. It's stack depth, not thread-vs-main.
|
||||
# CAVEAT: this sdkconfig is shared with the editor build, so the editor also
|
||||
# reserves this stack for nothing. Once the push is proven, git should move to a
|
||||
# dedicated large-stack task and this can drop back to ~16 KB.
|
||||
CONFIG_ESP_MAIN_TASK_STACK_SIZE=98304
|
||||
# wants several KB of stack on top of the app's own usage. 12 KB ran the editor
|
||||
# AND a full TLS-on-main handshake on hardware — the last-known-good value for
|
||||
# this shared stack.
|
||||
#
|
||||
# Spike 7 briefly bumped this to 96 KB because libgit2's push chain is far deeper
|
||||
# (~67 KB measured for a trivial config write: nearly every function puts a 4 KB
|
||||
# `char path[GIT_PATH_MAX]` on the stack and init→config→FATFS→wear-leveling
|
||||
# nests ~10 deep). But this sdkconfig is SHARED with the editor build, so that
|
||||
# forced the editor to over-reserve ~80 KB for nothing. Git now runs on its own
|
||||
# large-stack thread instead (git_push.rs — GIT_STACK via
|
||||
# std::thread::Builder::stack_size), so this drops back to the Spike-6 value.
|
||||
# (The "time() only works on the main task" theory was a misdiagnosis: that
|
||||
# iteration ran git on a std::thread with the DEFAULT 4 KB stack and just
|
||||
# overflowed sooner — it was always stack depth, never thread-vs-main.)
|
||||
CONFIG_ESP_MAIN_TASK_STACK_SIZE=12288
|
||||
|
||||
# Increase a bit these stack sizes as they are also a bit too small by default
|
||||
CONFIG_ESP_SYSTEM_EVENT_TASK_STACK_SIZE=4096
|
||||
|
||||
@@ -93,6 +93,13 @@ const CA_BUNDLE_PATH: &str = "/spiflash/ca.pem";
|
||||
/// SNTP first-sync budget (same as Spike 6).
|
||||
const SNTP_TIMEOUT: Duration = Duration::from_secs(20);
|
||||
|
||||
/// Stack for the dedicated git thread (see run()). libgit2's init→push chain
|
||||
/// measured ~67 KB on hardware and the push is deeper; 96 KB is the value proven
|
||||
/// on the main task before git moved off it. Sizing the thread — not the shared
|
||||
/// main-task stack — is the whole point of this task: the editor build no longer
|
||||
/// reserves it. Allocated from internal DRAM only while a push is running.
|
||||
const GIT_STACK: usize = 96 * 1024;
|
||||
|
||||
fn main() -> Result<()> {
|
||||
// Required once before any esp-idf-svc call; some runtime patches only link
|
||||
// if this symbol is referenced. See esp-idf-template#71.
|
||||
@@ -143,15 +150,30 @@ fn run() -> Result<()> {
|
||||
mount_fat().context("mounting flash-FAT")?;
|
||||
install_tls_trust_store().context("installing TLS trust store")?;
|
||||
|
||||
// Git work runs on the MAIN task, not a spawned thread. libgit2 (via mbedTLS
|
||||
// cert validation and FATFS timestamping) calls time()/gettimeofday, whose
|
||||
// newlib lock asserts when taken from a Rust std::thread but works on main
|
||||
// (Spike 6 ran TLS on main fine). The main stack is sized for it in
|
||||
// sdkconfig.defaults (CONFIG_ESP_MAIN_TASK_STACK_SIZE). Errors are LOGGED,
|
||||
// not propagated, so the radio stays up and the monitor shows the result.
|
||||
match git_publish() {
|
||||
Ok(summary) => log::info!("✅ Spike 7 complete — {summary}"),
|
||||
Err(e) => log::error!("❌ git_publish failed: {e:?}"),
|
||||
// Git runs on a DEDICATED large-stack thread, not the main task. libgit2's
|
||||
// call chain is deeply stack-hungry (see GIT_STACK), and sizing the *shared*
|
||||
// main-task stack for it made the editor build over-reserve ~80 KB. Its own
|
||||
// thread lets the main task stay small (sdkconfig.defaults, back to 12 KB).
|
||||
//
|
||||
// NB: an earlier iteration ran git on a std::thread and appeared to fail,
|
||||
// which we wrongly blamed on a "newlib time() lock that only works on main".
|
||||
// The real cause was the default 4 KB pthread stack overflowing — the same
|
||||
// chain just smashed the stack sooner. An explicit stack_size fixes it; there
|
||||
// is no thread-vs-main limitation. (Verified: the push below runs mbedTLS +
|
||||
// FATFS timestamping off-main.)
|
||||
//
|
||||
// Errors are LOGGED, not propagated, so the radio stays up (_wifi is held on
|
||||
// this task) and the monitor shows the result. join() blocks main until git
|
||||
// finishes; a panic almost certainly means GIT_STACK is too small.
|
||||
let git = std::thread::Builder::new()
|
||||
.name("git".into())
|
||||
.stack_size(GIT_STACK)
|
||||
.spawn(git_publish)
|
||||
.context("spawning git thread")?;
|
||||
match git.join() {
|
||||
Ok(Ok(summary)) => log::info!("✅ Spike 7 complete — {summary}"),
|
||||
Ok(Err(e)) => log::error!("❌ git_publish failed: {e:?}"),
|
||||
Err(_) => log::error!("❌ git thread panicked — likely stack overflow, raise GIT_STACK"),
|
||||
}
|
||||
|
||||
log::info!("idling with Wi-Fi up — press reset to re-run");
|
||||
@@ -251,8 +273,9 @@ fn install_tls_trust_store() -> Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// The whole publish (on the main task): init a fresh working copy, write a
|
||||
/// file, commit, and push to a fresh remote branch. Returns a one-line summary.
|
||||
/// The whole publish (runs on the dedicated git thread — see run()): init a fresh
|
||||
/// working copy, write a file, commit, and push to a fresh remote branch. Returns
|
||||
/// a one-line summary.
|
||||
fn git_publish() -> Result<String> {
|
||||
log::info!("git_publish started — free heap {}", free_heap());
|
||||
let unix = now_unix();
|
||||
|
||||
Reference in New Issue
Block a user