The last E2 story — completes the E2 epic (F→O auto-sync, "safe but silent").
A "Catch up now" button forces an immediate shallow + deep sweep out of cadence,
so the DM doesn't wait for the next jittered tick before trusting the vault is
current. Gated by foundryPoll (default off).
- src/foundry-poll.ts catchUpNow(): cancels pending shallow + deep timers, runs
an immediate shallowPoll, then on completion immediately triggers a deepPoll
(no cadence wait). Debounced (returns {skipped: true} if either is in flight).
Reuses the same shallowPoll/deepPoll methods (no parallel code path — same
mapPool, same lock, same routing). On completion: activity panel summary
{shallow: {new, renamed, missing}, deep: {pulled, skipped, conflicts},
durationMs}, lastPoll updated, regular cadence resumes from now. Persistent
errors halt; transient errors logged.
- src/server.ts POST /api/foundry-poll/catchup (gated by foundryPoll; 404 when
off; 400 when poll not enabled): calls catchUpNow(), returns the summary.
- src/dashboard.html: "Catch up now" button in the live-new-entries panel header.
catchUpNow() POSTs + toasts the result (or "catch-up already running").
- tests/e2-6-catchup.test.ts: 5 tests — catchUpNow returns a summary with
durationMs; debounced (skipped if in flight); endpoint 200 with summary; 400
when poll not enabled; 404 when foundryPoll off.
tsc clean; 271 passing project-wide (18 pre-existing fixture-missing unchanged).
E2 epic COMPLETE (all 6 stories E2.1–E2.6).
Co-Authored-By: Claude <noreply@anthropic.com>
New Foundry entries from the shallow poll appear in a separate "Live new entries
(from Foundry)" panel with a one-click "Import as new refined note" action. Never
auto-import — each entry sits until the user clicks. Gated by foundryPoll.
- src/server.ts POST /api/foundry-poll/import {uuid}: fetches relay.getEntry(uuid),
checks name collision against the index (matched + refinedOnly) → 409 if
collided, builds a refined note via importRow (batch.ts → entryToObsidian +
foundry block), writes under refined/imported/<subfolder>/, removes from
liveNewEntries, returns {ok, uuid, name, filename, subfolder}. Gated by
foundryPoll (404 when off). No live entry → 404.
- src/dashboard.html: "Live new entries (from Foundry)" panel (details/summary,
purple border) rendered from GET /api/foundry-poll's liveNewEntries array on
the 2s poll (refreshLiveNewEntries). Each entry has an "Import as new refined
note" button → POST /api/foundry-poll/import with a confirm dialog. On success
→ toast + refreshIndex. Hidden when foundryPoll is off (GET /api/foundry-poll
404 → panel hidden).
- tests/e2-5-import.test.ts: 5 tests — import succeeds (file written +
liveNewEntries cleared); name collision → 409 (NOT cleared); no live entry →
404; foundryPoll off → 404; GET /api/foundry-poll includes liveNewEntries.
tsc clean; 265 passing project-wide (19 pre-existing fixture-missing unchanged).
Co-Authored-By: Claude <noreply@anthropic.com>
When both sides changed since last sync (both-diverged) OR Foundry's entry is
missing while the vault note exists (vault-newer), the F→O pull is SKIPPED and a
pending conflict row is recorded for E3 to render. No auto-pick, no auto-write.
- src/server.ts PendingConflictRow type: {uuid, name, state:
"both-diverged"|"vault-newer", detectedAt, lastFHash, lastOHash}.
AutoSyncController.recordPendingConflict(uuid, name, state, lastFHash,
lastOHash): records in sync-state.json.pendingConflicts (deduped by uuid),
removes from fPending, updates parity, saves atomically.
pullFChanged: the "conflict" return now records a both-diverged pending conflict
row (with lastFHash=ccHash(liveEntry), lastOHash=contentHash(body)) before
returning — awaited so the save is durable.
- src/foundry-poll.ts shallowPoll: "missing" entries now route to
recordPendingConflict("vault-newer") via state.autosync (guarded — falls back
to fPending-only if autosync is unavailable). GET /api/foundry-poll includes
pendingConflicts from sync-state.json in the response.
- tests/e2-4-conflict.test.ts: 3 tests — both-diverged → pendingConflicts row
with correct hashes + removed from fPending; vault-newer (shallow poll missing)
→ pendingConflicts row; recordPendingConflict accessible.
tsc clean; 261 passing project-wide (18 pre-existing fixture-missing unchanged).
Co-Authored-By: Claude <noreply@anthropic.com>
When the deep poll detects a F-changed note AND the Obsidian side is unchanged,
the note is pulled from Foundry into the vault: /get → entryToObsidian (via
rePullRow) → writeWithBackup → dual re-baseline (contentHash + ccHash). Removes
from fPending. If O-side also changed → "conflict" (left in fPending for E2.4).
- src/server.ts AutoSyncController.pullFChanged(uuid, liveEntry): finds the
matching row, checks O-side unchanged (contentHash(body) === foundry.contentHash),
acquires the per-uuid "pull" lock, calls rePullRow (batch.ts — converts the live
entry to refined markdown + injects the foundry block), writes via
writeWithBackup (dev: mirror; apply: real vault + .bak), re-baselines both
contentHash + ccHash(liveEntry) via baselineNote, removes from fPending +
updates parity, logs "pulled ← <uuid> · baselined (content+cc)". Returns
"pulled" | "conflict" | "skipped".
- src/foundry-poll.ts deepPoll: for each F-changed note, calls
pullFChanged(uuid, liveEntry). "pulled" → removed from fPending (no change
recorded). "conflict"/"skipped" → recorded in fPending for E2.4 / retry.
Guard: if autosync isn't available (E2.2 tests), falls back to recording only.
- tests/e2-3-pull.test.ts: 5 tests — F-changed + O-unchanged → pulled +
re-baselined + fPending cleared; F-changed + O-changed → "conflict" (note
untouched); no matching row → "skipped"; note missing → "skipped"; unseeded →
"skipped".
tsc clean; 258 passing project-wide (18 pre-existing fixture-missing unchanged).
Co-Authored-By: Claude <noreply@anthropic.com>
The F→O auto-sync track begins. A FoundryPollController periodically snapshots
the relay /search (minified: {uuid,id,name,img,documentType} — no folder/
content) and diffs against the previous snapshot to detect structural changes.
"Safe but silent" — not user-visible until E2+E4+E3 all land. Gated by
foundryPoll (default off).
- src/foundry-poll.ts FoundryPollController: ~10s cadence with ±20% jitter
(FOUNDRY_POLL_CADENCE_MS env). Diffs the /search snapshot keyed by uuid:
rename (name change on known uuid), new (uuid absent from prior snapshot +
not in linked index → liveNewEntries), missing (uuid in linked index but
absent from /search). Records changes to sync-state.json.fPending (the
F-pending badge's data from E4.3) + updates parity.fPending count + lastPollAt.
Overlap guard (skip if a round is in flight, increment skipCounter). Transient
retry per E1b.7 (inline backoff); persistent errors (404 "No connected Foundry
clients found") halt the timer. liveNewEntries deduped by uuid; removed when a
uuid appears in the linked index (after a manual refresh/import).
- src/server.ts: FoundryPollController added to State. features.foundryPoll flag
(env FOUNDRY_POLL=1, default off). GET /api/foundry-poll (status) + POST
/api/foundry-poll {enabled} (toggle), both gated by foundryPoll (404 when off).
Created in startServer alongside AutoSyncController.
- tests/e2-1-shallow-poll.test.ts: 7 tests — rename detection; new entry →
liveNewEntries; missing entry detection; parity.fPending + lastPollAt updated;
overlap guard (skipCounter); persistent error halts; status().
tsc clean; 247 passing project-wide (18 pre-existing fixture-missing unchanged).
Co-Authored-By: Claude <noreply@anthropic.com>
The last E4 story — completes the E4 epic (sync status & parity). A maintained
status note in the vault mirrors the dashboard (on/off, mode, last-sync, parity,
recent activity), carrying a foundry.sync_status:"true" sentinel. The exclusion
is airtight + rename-safe: skip by BOTH path (dotfile) AND sentinel.
- src/server.ts writeStatusNote(state): renders .sync-status.md (frontmatter
with foundry.sync_status:"true" + a body with on/off, mode, lastSync, parity,
last 10 activity events) → writeWithBackup to refinedDir/.sync-status.md
(apply mode backs up the previous). Called from log() (after appendActivity),
refreshParity, and the mode handler — fire-and-forget.
- src/server.ts onChange: dotfile skip (rel.split("/").pop()?.startsWith("."))
gated by features.syncStatus. A renamed status note (non-dot) is caught by the
sentinel in runPushAttempt.
- src/server.ts runPushAttempt: sentinel check — if foundry.sync_status ===
"true" → skip "sync status note (sentinel)", before the cc_uuid/contentHash
checks. Gated by features.syncStatus. Rename-safe (sentinel survives a rename).
- src/batch.ts walkMd: exclude dotfiles from the index (!ent.name.startsWith("."))
so .sync-status.md never enters state.index.matched/refinedOnly (never a row,
never a push candidate).
- tests/e4-5-statusnote.test.ts: 5 tests — note written with sentinel + content;
onChange skips .sync-status.md (no timer); sentinel check skips a renamed copy;
sentinel absent on a normal note; index excludes .sync-status.md (only Roland
indexed).
tsc clean; 240 passing project-wide (18 pre-existing fixture-missing unchanged).
Co-Authored-By: Claude <noreply@anthropic.com>
The dashboard shows at-a-glance sync state: ON/OFF, PREP/RUN mode, watched dir,
parity (in-parity/O-pending/F-pending/conflict/unsynced-linked), last-sync. The
F-pending badge is the Slice 1 demo hook — clickable to expand an "Incoming F→O
changes" list (latent until E2's poll populates fPending). Gated by
features.syncStatus (default off → existing /api/autosync unchanged).
- src/server.ts GET /api/sync-state (gated; 404 when off): returns the full
SyncState. refreshParity(state) — derives oPending (sync-cc count),
unsyncedLinked (seed count), conflict (conflict count) from the index rows'
recommendation on each /api/index fetch; fPending=0/lastPollAt=null (E2 not
yet); lastSyncAt from the newest push/pull activity event; parity status with
precedence (conflict > O-pending > F-pending > unsynced-linked > in-parity);
writes to sync-state.json atomically.
- src/dashboard.html: #syncHeader strip (ON/OFF + mode + watchedDir + lastSync)
+ #parityIndicator badge (color-coded by status) + #fPendingPanel (the
Incoming F→O changes list, hidden until fPending>0). refreshSyncState() fetches
/api/sync-state on init + the 2s poll (gated by featuresSyncStatus from
/api/status). Falls back to existing /api/autosync + /api/status when off.
- tests/e4-3-parity.test.ts: 5 tests — /api/sync-state 200 (features on) / 404
(off); empty index → counts 0, in-parity, lastSyncAt null; lastSyncAt from a
push activity event; watchedDir set from cfg.refinedDir.
tsc clean; 235 passing project-wide (18 pre-existing fixture-missing unchanged).
Co-Authored-By: Claude <noreply@anthropic.com>
In PREP, auto-sync is blocked (the DM is still curating/seeding/linking — must
not push half-curated notes to live Foundry). In RUN-THE-MATCH, auto-sync is
unblocked. The toggle is the only writer of `mode` in sync-state.json. Gated by
features.syncStatus (default off → existing behavior unchanged).
- src/server.ts setEnabled: PREP gate — if features.syncStatus + mode===PREP →
throw "auto-sync is blocked in PREP mode — switch to RUN-THE-MATCH first".
Stacks on top of the E1b.5 apply-mode gate (both must pass).
- src/server.ts POST /api/sync-state/mode (gated by features.syncStatus; 404 when
off): flips mode in sync-state.json; switching to PREP while auto-sync is ON
tears down the watcher (stop()) + flips autoSyncOn=false first. /api/status
exposes syncMode + featuresSyncStatus.
- src/server.ts boot reconcile: if mode===PREP + autoSyncOn=true → autoSyncOn
flipped to false + "PREP mode auto-sync disabled on boot" event (doesn't call
setEnabled, which would throw the PREP message).
- src/dashboard.html: PREP⇄RUN toggle button in the header (gated by
featuresSyncStatus from /api/status); autoSyncBtn disabled in PREP with a
tooltip "Switch to RUN-THE-MATCH mode first"; toggleSyncMode() POSTs the flip.
- tests/e4-2-mode.test.ts: 8 tests — setEnabled in PREP throws / in RUN proceeds
/ flag-off no gate; POST mode flip; switching to PREP while ON → stop + off;
boot reconcile PREP + autoSyncOn → off + event; invalid mode → 400; features
off → 404.
tsc clean; 230 passing project-wide (18 pre-existing fixture-missing unchanged).
Co-Authored-By: Claude <noreply@anthropic.com>
The first Slice 1 story. A durable aggregate (on/off, mode, parity, activity,
last-sync) that survives restarts — the single status source E2/E3/E4 read
from. E4.1 only delivers load/save + the boot reconcile; the dashboard migration
to read sync-state.json is E4.2-E4.6 (gated behind features.syncStatus, defined
here, default off).
- src/sync-state.ts: loadSyncState/saveSyncState (atomic: tmp+rename, no
truncation on crash), the shape {syncStateSchemaVersion, mode, autoSyncOn,
lastSyncAt, parity, watchedDir, activity, updatedAt, conflict:null} (conflict
reserved for E3 — E4 forces it null), appendActivity (newest-first, trimmed to
200), defaultSyncState (fresh install: autoSyncOn=false, mode=PREP). Schema
mismatch (≠ SYNC_STATE_SCHEMA_VERSION from E0.3) → back up to
sync-state.json.bak-<stamp> + fresh defaults + an error event. (Reconciliation:
E0.3 froze SYNC_STATE_SCHEMA_VERSION = "sync-state/v1" — E4.1's AC "= 1" is
superseded; E4.1 persists autoSyncOn, superseding E1b.5's "no persistence until
E3" — a fresh install still defaults OFF; a user who toggled ON is restored ON.)
- src/server.ts: features.syncStatus flag (env OFS_SYNC_STATUS=1, default off)
on ServerConfig/State. startServer loads sync-state.json + reconciles
AutoSyncController.enabled with autoSyncOn — if true, setEnabled(true); on
throw (dev mode apply-gate / no relay) → autoSyncOn=false + persist + an
"auto-sync could not resume" error event. State.syncState holds the aggregate.
- tests/e4-1-syncstate.test.ts: 10 tests — load creates defaults; atomic save
(no .tmp left); schema-mismatch backup + fresh + error event; restart survival
(autoSyncOn + 17 activity preserved); activity trimmed to 200; conflict forced
null; boot reconcile (autoSyncOn=true + apply + relay → enabled restored; no
relay → flipped off + error; dev mode → flipped off; fresh → stays off).
tsc clean; 222 passing project-wide (18 pre-existing fixture-missing unchanged).
Co-Authored-By: Claude <noreply@anthropic.com>
The last E7 story — completes Slice 0 (E1b ∥ E7). Every requireCSRF route (the
POST mutations) is now defended against cross-site forgery: same-origin check
+ a per-session CSRF token. Ships dark (flag default off → no-op).
- src/server.ts checkCSRF(req,res,route): flag off → no-op; requireCSRF routes
(POST mutations) check (1) same-origin — Origin (or Referer fallback) host must
equal the request's Host header host; both missing → 403 "origin required"
(curl/scripts send an explicit Origin); mismatch → 403 "cross-origin
forbidden"; (2) X-CSRF-Token header === csrf_token cookie (constant-time);
absent/mismatch → 403 "missing or invalid csrf token". GET/HEAD/OPTIONS exempt
(requireCSRF is only on POST). Auth (E7.1) runs BEFORE CSRF — 401 fires first.
- src/server.ts issueCSRF + GET /api/auth/csrf (open): issues a random per-session
token in an HttpOnly SameSite=Strict cookie + returns {csrfToken} (the JS-readable
mirror; a cross-site can't read the body via CORS nor the HttpOnly cookie, so it
can't forge X-CSRF-Token). Dispatch runs checkCSRF after authenticate.
- src/dashboard.html: apiFetch attaches X-CSRF-Token on non-GET requests (from
localStorage, fetched via /api/auth/csrf on init). The browser auto-sends the
HttpOnly csrf cookie on same-origin POSTs.
- tests: e7-1-dispatch + e7-3-nosecret valid-Bearer POST tests updated to send
Origin + X-CSRF-Token + cookie (getCSRF helper). e7-4-csrf.test.ts (10 tests):
csrf issuance (HttpOnly SameSite cookie + token); same-origin+valid → 200;
cross-origin → 403; no-origin → 403 origin required; missing/invalid token →
403; missing cookie → 403; auth-before-CSRF (no Bearer → 401); GET exempt; flag
off → no-op.
tsc clean; 212 passing project-wide (18 pre-existing fixture-missing unchanged).
Co-Authored-By: Claude <noreply@anthropic.com>
The dashboard never leaks secrets or sensitive data to the browser, even on a
public bind. /api/auth/status is the only unauthenticated data endpoint in
public mode.
- src/server.ts ROUTES: gate the data-leaking GET routes (/api/index, /api/status,
/api/file, /api/entries, /api/autosync, /api/autosync/conflicts,
/api/autosync/last-push) → requireAuth:true. Only / (login page) + /api/auth/*
stay open in public mode. /api/status leaks dir paths, /api/file leaks file
contents, /api/index the full index — all gated. (On 127.0.0.1, authenticate
doesn't enforce, so localhost stays open — the gate only bites a 0.0.0.0 bind.)
- src/dashboard.html: checkAuth reordered — fetch /api/auth/status (open) first;
only fetch /api/status (gated) if not (authRequired && no token), avoiding a
pre-login 401-throws. Masked presence chip ("Relay ✓/✗ · Foundry ✓/✗") driven
by /api/auth/status booleans (relayConfigured/foundryConfigured), never env
values.
- tests/e7-1-dispatch.test.ts: flag-on read tests updated — /api/auth/status open
(200 without token); /api/status + /api/autosync gated (401 without token in
public mode).
- tests/e7-3-nosecret.test.ts: regression guard — server with distinctive secrets
(relay API key + dashboard token), hits endpoints incl. error paths
(/api/refresh relay-failure 500, login 401, gated 401, the login page HTML) and
asserts JSON.stringify of NO response contains either secret substring. Errors
name env vars, never values; the relay key is a header, never serialized.
tsc clean; 202 passing project-wide (18 pre-existing fixture-missing unchanged).
Co-Authored-By: Claude <noreply@anthropic.com>
Safe-by-default dashboard exposure: localhost-only unless you explicitly opt in
to a public bind with a token.
- src/cli.ts: host default 0.0.0.0 → 127.0.0.1 (localhost-only; --host 0.0.0.0 to
expose, which then requires a token when the auth flag is on).
- src/server.ts startServer guards (before listen): SELF-LOCKOUT (flag on + no
token → throw — you'd otherwise brick the dashboard with no recovery short of
editing .env) + PUBLIC-EXPOSURE gate (flag on + 0.0.0.0 + no token → throw).
Both use __authState (mutable seam) so they're testable. __authState.bound set
from cfg.host. When the flag is off, the guards are skipped (back-compat
escape hatch — the bind still defaults to 127.0.0.1).
- src/server.ts authenticate: E7.2 bind-gating — enforce only on a PUBLIC bind
(0.0.0.0); 127.0.0.1 is localhost-trusted (requireAuth routes stay open even
with the flag on). The refuse-to-start guard ensures a 0.0.0.0 bind has a token.
- src/server.ts routes: GET /api/auth/status (open; {authRequired, bound,
relayConfigured, foundryConfigured} — booleans only, no secret values),
POST /api/auth/login (open; validates token constant-time, sets an HttpOnly
SameSite=Strict cookie; 401 invalid credentials on mismatch, no leak; empty
token = unset; no token configured → 401), POST /api/auth/logout (clears
cookie, Max-Age=0).
- src/dashboard.html: first-run login card (token input, shown when authRequired
&& no stored token), a shared apiFetch wrapper (attaches the stored token as a
Bearer header, on 401 → show login), checkAuth gating init; bare fetch('/api/')
calls migrated to apiFetch (auth endpoints stay plain fetch).
- .env.example: documents DASHBOARD_AUTH_TOKEN + ENABLE_AUTH_MIDDLEWARE.
- tests: e7-1-auth/dispatch updated for bind-gating (enforcement tests set
bound=0.0.0.0; + a 127.0.0.1 no-enforcement test). e7-2-auth.test.ts (13
tests): auth-status (off/localhost/public/no-secret-leak), login
(valid-cookie/invalid-401/empty/unset), logout (clears), refuse-to-start
(self-lockout / 0.0.0.0-no-token / token-set-passes / flag-off-no-guard).
tsc clean; 194 passing project-wide (18 pre-existing fixture-missing unchanged).
Co-Authored-By: Claude <noreply@anthropic.com>
The other half of Slice 0. A single auth contract + a route table so every
endpoint declares its auth/CSRF requirement once — neighbor epics (E2/E3/E4)
register routes via ROUTES["<METHOD> /api/<x>"] = {method, requireAuth,
requireCSRF, handler} with no new auth code. Ships dark (flag default off →
byte-identical); the flag flips at the launch gate.
- src/server.ts auth infra: ENABLE_AUTH_MIDDLEWARE flag (env, default false),
DASHBOARD_AUTH_TOKEN env, mutable __authState test seam. authenticate(req,res,
route): flag off → no-op pass-through; flag on → requireAuth routes check a
Bearer/cookie token (constant-time compare via timingSafeEqual, length-padded).
Malformed Authorization → 401 bad auth header; missing/wrong token → 401
unauthorized; token unset → 500. requireCSRF is declared (enforced in E7.4).
Error responses reuse send() (x-content-type-options + cache-control).
- src/server.ts ROUTES table: every existing endpoint migrated — GET read routes
(/, /api/index, /api/status, /api/file, /api/entries, /api/autosync,
/api/autosync/conflicts, /api/autosync/last-push) → requireAuth:false; POST
mutation routes (/api/action, /api/push, /api/push-all, /api/link,
/api/refresh, /api/autosync, /api/autosync/revert) → requireAuth:true,
requireCSRF:true (now gating the previously-unguarded E1b mutation endpoints
when the flag flips). The dispatch loop consults the table before calling the
handler. Top-of-file comment documents the ROUTES[...] contract.
- tests/e7-1-auth.test.ts: 8 unit tests (flag off no-op; flag on read-passes /
no-token-401 / valid-bearer / wrong-401 / malformed-401 / cookie / unset-500).
- tests/e7-1-dispatch.test.ts: 11 integration tests via real startServer (empty
temp LevelDB) — flag off byte-identical (read+mutation pass, favicon 204, 404);
flag on (read no-token 200; mutation no-token/wrong/malformed 401; valid Bearer
→ handler 200/400; apply-gate after auth).
tsc clean; 180 passing project-wide (18 pre-existing fixture-missing unchanged).
Co-Authored-By: Claude <noreply@anthropic.com>
The last E1b story. Every auto-sync decision is durably logged (NFR-5), and old
seeded notes are stamped with the current flags schema version at startup
before auto-sync engages (NFR-11) — so the controller can reason about which
notes carry the dual (contentHash + ccHash) baseline.
- src/server.ts persistent log: log() writes a JSON-line
{time,level,name,status,message} to <outDir>/logs/sync-<YYYY-MM-DD>.log via a
WriteStream, alongside the in-memory events (the dashboard's fast poll). Daily
rotation (date change → close+open) + size rotation (10MB → rename to
sync-<date>.<n>.log + reopen). pruneOldLogs at startup deletes logs/sync-*.log
older than AUTOSYNC_LOG_RETAIN_DAYS (14). Gated by foundryGuardEnabled (off →
in-memory only). startServer prunes + opens the log.
- src/server.ts flagsSchemaVersion: baselineFoundryBlock/baselineNote stamp
foundry.flagsSchemaVersion (FLAGS_SCHEMA_VERSION from E0.3) on every baseline.
stampFlagsSchemaVersion() (used by the migration) touches ONLY flagsSchemaVersion
— no contentHash/ccHash change, so it doesn't falsely trigger a push.
- src/server.ts migrateFlagsSchemaVersion(): startup pass walks index.matched,
stamps notes whose foundry.flagsSchemaVersion is absent/older (idempotent —
already-current notes skipped), in-flight-safe (mtime changes between read and
write → skip + log "in-flight edit"), logged to the persistent log. Runs BEFORE
auto-sync is allowed to start. status() exposes migrationCount/migrationRan/
schemaVersion.
- src/dashboard.html: "Migrated N notes to flagsSchemaVersion <v>" banner
(dismissible, shown once on first poll when migrationRan && migrationCount>0).
- tests/e1b8-log-migration.test.ts: 7 tests — log() writes a JSON-line (info +
error levels); pruneOldLogs runs without error; a clean push stamps
flagsSchemaVersion; migration stamps absent notes idempotently with NO
contentHash/ccHash change; already-current notes left alone; status exposes
migrationCount + schemaVersion.
tsc clean; 161 passing project-wide (18 pre-existing fixture-missing unchanged).
Co-Authored-By: Claude <noreply@anthropic.com>
Transient relay blips retry with backoff; persistent failures surface immediately
without burning Foundry.
- src/server.ts classifyRelayError(err) → "transient" | "persistent". The relay
STATUS dominates: 408/504/500 → transient; 400/401/403/404 (and any other relay
status) → persistent. No-status network failures (ECONNRESET/ETIMEDOUT/
ENOTFOUND/fetch failed/Request timed out) → transient. Non-relay/unknown →
persistent (safer, no retry). Exported for unit testing.
- src/server.ts PushRetryError (kinds: transient-exhausted/persistent/lock-busy)
+ sleepJitter (±20%). pushWithRetry: 3 attempts when foundryGuardEnabled (1
when off — no retry); per attempt acquires the per-uuid lock (skip → throw
lock-busy), runs runPushAttempt (pushNote + TOCTOU + baseline), releases in
finally. On a transient pushNote failure: log "transient (attempt n/3) —
retrying", add the uuid to a `retrying` set (so a concurrent save during the
backoff is dropped, not duplicated), sleep the backoff (500/1500/4500ms ±20%,
a field so tests shrink it), then retry. On persistent/exhaustion: throw
PushRetryError. The lock is released between attempts so a concurrent F→O
pull isn't blocked for the full backoff. A retry that succeeds still runs the
full TOCTOU + baseline path (runPushAttempt) — no shortcut around the guard.
- runPushBody → runPushAttempt (pushNote failures propagate to pushWithRetry for
classification; TOCTOU/baseline failures stay internal with their own handling,
not retried). process flag-on: retrying-set check → pushWithRetry → catch
PushRetryError → log (lock-busy = skipped; persistent/exhausted = error).
- tests/e1b7-retry.test.ts: 6 tests — classifyRelayError (transient 408/504/500/
network; persistent 401/404/no-cc_uuid/unknown); 504×2→200 succeeds on attempt
3 (one PUT, one baseline, TOCTOU re-verify, transient-retrying logs); 401 → zero
retries, immediate "persistent" error, no PUT; lock-busy → skipped; retrying-set
drops a concurrent same-uuid save.
tsc clean; 154 passing project-wide (18 pre-existing fixture-missing unchanged).
Co-Authored-By: Claude <noreply@anthropic.com>
The watcher fallback, rescanSubs, debounce (700ms), and concurrency (3) already
existed; this story makes them env-configurable with bounds and adds the burst
validation harness + the recursive-fallback exercise.
- src/server.ts: AUTOSYNC_CONCURRENCY (default 3, clamped 1-8) +
AUTOSYNC_DEBOUNCE_MS (default 700, clamped 100-5000) via a clampInt helper
(replaces the hardcoded concurrency=3 / debounceMs=700).
- tests/e1b6-burst.test.ts: 50-note burst validation harness (drives queue/drain
directly with a mocked 100ms pushNote + concurrency counter; guard off so no
relay /get). Asserts: 50 distinct-uuid notes → 50 completions, max concurrent
pushNote ≤3, no uuid pushed twice, bounded wall time (<30s); a same-uuid pair
(rename) → the second is skip-dropped, no duplicate PUT ("lock busy" log); a
push that throws releases its slot in finally and the queue drains the rest
(one failure does not stall the burst — pushed N-1, error 1, queue empty).
- tests/e1b6-watch.test.ts: recursive-watch fallback — rescanSubs attaches
watchers to subdirs (excl .obsidian/dotfiles) and picks up a subdir created
mid-burst; debounce — two rapid onChange for the same rel collapse to one
timer, different rels arm independent timers.
tsc clean; 148 passing project-wide (18 pre-existing fixture-missing unchanged).
Co-Authored-By: Claude <noreply@anthropic.com>
Auto-sync refuses to go live unless the server is in --apply mode, starts OFF
every restart (opt-in per session until E3), and the watcher never pushes the
status-note / wiki-structure paths.
- src/server.ts setEnabled: throws "auto-sync requires --apply mode..." when
enabled in dev mode (defense in depth). POST /api/autosync handler pre-checks
mode → 400 with that message (leaves enabled=false). Apply-mode gating is a
HARD safety floor (not feature-flagged): in dev mode the refined dir is the
--out mirror, so enabling would push mirror edits to live Foundry.
- OFF-by-default: constructor sets enabled=false; no startup path auto-enables
(verified — the only setEnabled call is the POST toggle). Per-session: a
restart always returns to OFF (no persisted-on state until E3).
- src/server.ts watcher: STATUS_NOTE_PATHS = ["_meta/", "wiki/", ".raw/"]
(FR-4.3 status-note exclusion); onChange skips rel under these with a logged
"status-note path" reason, no debounce timer. (Unlinked/unseeded frontmatter
skips stay in process — already logged there; a hot-path read in onChange
conflicts with the single-read design and yields no net savings — deferred.)
- src/dashboard.html: dev-mode banner (yellow strip above the auto-sync panel
when /api/status mode==="dev"); "Auto-sync is opt-in per session — resets to
OFF on restart" note near the toggle when enabled in apply mode.
- tests/e1b5-applymode.test.ts: 6 tests — constructor enabled=false; dev-mode
setEnabled throws + stays off; apply-mode setEnabled starts/stops; onChange
skips _meta/ + wiki/ + .raw/ with logged reason + no timer; a normal note arms
the debounce.
tsc clean; 141 passing project-wide (18 pre-existing fixture-missing unchanged).
Co-Authored-By: Claude <noreply@anthropic.com>
Before any auto/manual push, the pre-push Foundry entry is cached so a wrong
push is one-click undoable from the dashboard (no shell, no hunting backups).
- src/push.ts: optional backupPath in PushDeps (writes there with mkdir -p;
else the default flat <outDir>/bak/<name>.<stamp>.json for manual push).
PushOutcome.backupPath returns the actual path.
- src/server.ts runPushBody: auto-sync passes the per-uuid
<outDir>/foundry-backups/<uuid>/<iso>.json layout (so retention is per-entry,
distinct from manual push's flat bak/). After a successful push: records a
last-push {uuid,name,backupPath,time,relPath} in an in-memory map + as
controller.lastPush (drives the dashboard button); prunes the per-uuid backup
dir to the last N (AUTOSYNC_BACKUP_RETAIN default 10, by mtime, awaited so the
post-push state is deterministic). GET /api/autosync/last-push?uuid=… returns
the record or 404; status() exposes lastPush + conflictCount.
- src/server.ts AutoSyncController.revert(uuid): acquires the per-uuid lock in
the "pull" direction (queues behind an in-flight push, 5s max); reads the
backup JSON (400 if no last-push, 409 if the file is missing/corrupt); calls
relay.updateEntry(uuid, fullBackupDoc) — a FULL /update (not a diff — the one
place a full PUT is correct, to restore _id/pages/ownership/flags exactly);
re-baselines the note (contentHash=body, ccHash=ccHash(backupDoc)) via the
E1b.2 path + records self-write suppression; clears any TOCTOU conflict for
the uuid. POST /api/autosync/revert endpoint (unguarded — E7 not landed).
Gated by AUTOSYNC_FOUNDRY_GUARD (404 when off).
- src/dashboard.html: "Revert last push" button in the auto-sync panel (shown
when a recent push exists + guard on), wired to POST /api/autosync/revert
with a confirm stating the note keeps the edit while Foundry reverts.
- tests/e1b4-revert.test.ts: 6 tests — per-uuid backup written + last-push
recorded; retention prunes to N; revert full-PUTs the backup + re-baselines;
400 (no last-push); 409 (missing backup); 404 (guard off).
tsc clean; 135 passing project-wide (18 pre-existing fixture-missing unchanged).
Co-Authored-By: Claude <noreply@anthropic.com>
Closes the TOCTOU window E1b.1/E1b.2 left open: Foundry can be edited by
another client between the pre-push /get and the PUT response.
- src/server.ts runPushBody: after a successful push, issue a SECOND relay.getEntry
(the one extra /get the epic permits), compare post-push ccHash to
ccHash(pushedEntry) (what we just wrote). Match → baseline per E1b.2. Mismatch →
TOCTOU conflict: record a ConflictRow {uuid,name,obsidianHash,foundryPreHash,
foundryPostHash,time,relPath} in a bounded (50, newest-first) in-memory list,
log "skipped" "TOCTOU conflict — Foundry edited during push, NOT baselined; use
Sync / Re-pull", leave baselines + in-memory ccHash untouched so the next save
re-surfaces the divergence. Re-verify /get failure → error log, no baseline,
push stays live (no rollback). A successful push clears any prior conflict for
that uuid. (Documented deviation: the AC said compare to the "pre-PUT captured
entry", but our own push changes Foundry so that would always differ — the
correct comparison is post-push /get vs ccHash(pushedEntry), the expected state.)
Gated by foundryGuardEnabled (flag off → no TOCTOU check, baseline directly).
- src/server.ts: ConflictRow type exported; status() adds conflictCount + the
conflicts list; GET /api/autosync/conflicts endpoint (registered unguarded —
E7's auth middleware not landed yet; E7 will gate it).
- src/dashboard.html: conflict badge on the Auto-sync button (⚠ N conflicts) +
a loud note when non-zero, driven by the existing /api/autosync poll.
- tests: new e1b3-toctou.test.ts (5 tests — clean, TOCTOU mismatch, re-verify
failure, conflict cleared on next success, status exposes conflicts). Updated
e1b1/e1b2 mocks to behave like Foundry (/get returns current state, /update
applies the diff) so the re-/get returns the pushed state (no false conflict);
e1b1 clean test now expects 2 /gets (push + re-verify).
tsc clean; 129 passing project-wide (18 pre-existing fixture-missing unchanged).
Co-Authored-By: Claude <noreply@anthropic.com>
After a successful push, re-baseline BOTH foundry.contentHash (body) and
foundry.ccHash so the next save is correctly gated against both sides, and
suppress the watcher's own baseline write so it doesn't re-trigger a push.
- src/push.ts: PushOutcome.pushedEntry (the buildPushPayload `full` — the
post-push Foundry content). Exposed so the caller can baseline ccHash with no
extra /get.
- src/server.ts baselineFoundryBlock/baselineNote: extended to write foundry.ccHash
(insert the line if absent). runPushBody re-baselines both: ccHash =
ccHash(pushedEntry) — the post-push state. (Documented deviation: the AC said
"pre-PUT captured entry", but a push changes Foundry so pre-push would
false-abort the next save; post-push = ccHash(pushedEntry) is correct, no /get.)
In-memory ccHashBaselines map (uuid→ccHash) is set BEFORE the disk write and
preferred by the guard, so a disk-write failure still leaves the next save's
guard correct (frontmatter is the durable fallback on restart). Log
"baselined (content+cc)".
- src/server.ts self-write suppression: recentlyBaselined map (relPath→{mtime,
expires}) with TTL AUTOSYNC_BASELINE_SUPPRESS_MS (default 2000). onChange checks
it BEFORE arming the debounce — if the change's mtime equals the recorded
baseline mtime, drop with "self-write (baseline)" log and NO timer. A user
edit within the TTL has a different mtime → processed. stop() clears the map.
- tests/e1b2-baseline.test.ts: 5 tests — dual re-baseline (contentHash + ccHash =
ccHash(pushedEntry)); drift-abort leaves baselines untouched; same-mtime
self-write dropped (no re-push); user edit (different mtime) processed; TTL
expiry re-enables processing.
tsc clean; 124 passing project-wide (18 pre-existing fixture-missing unchanged).
Co-Authored-By: Claude <noreply@anthropic.com>
First story of the E1b controller-hardening epic. Wires the E1b-alt HTML
ccHash into AutoSyncController.process as a Foundry-side divergence guard:
before the PUT, compare ccHash(liveEntry) to the note's foundry.ccHash
baseline; if Foundry drifted since the last sync, ABORT the PUT (fail-safe —
never overwrite a Foundry-side edit).
- src/push.ts: prePushGuard?(liveEntry)=>{abort,reason?} on PushDeps, called
right after relay.getEntry (the SAME /get pushNote already makes — no extra
round-trip), before any side effect (image upload/backup/PUT). On abort,
returns an aborted PushOutcome (no PUT, no backup). PushOutcome gains
aborted/abortReason/liveEntry (liveEntry exposed so E1b.2 can re-baseline
ccHash with no second /get).
- src/server.ts: runPushBody passes a prePushGuard that computes ccHash on
the reused live entry and compares to foundry.ccHash. Absent ccHash (legacy)
proceeds (one-time migration; E1b.2 writes the post-push baseline).
Unreadable Foundry side (CcHashError) aborts fail-safe. Feature flag
AUTOSYNC_FOUNDRY_GUARD (env, default true; controller field for test override;
off → body-only, documented unsafe).
- tests/e1b1-noclobber.test.ts: 7 tests with a mock relay (real pushNote) —
clean push PUTs; Foundry-side drift → NO PUT; legacy note proceeds; unreadable
Foundry side → NO PUT; guard works with sync-state.json ABSENT (no E4 dep);
flag on/off. Live SM-2 verification stays gated on the operator's headless
session.
tsc clean; 119 passing project-wide (18 pre-existing fixture-missing unchanged).
Co-Authored-By: Claude <noreply@anthropic.com>
E1a.1 go/no-go gate. Built the linkedom htmlToMarkdown inverse
(src/fromFoundry.ts) and a round-trip hash-stability fixture suite
(tests/e1a-hash-spike.test.ts) against the real forward transform.
Verdict: NO-GO. 7 of 12 fixtures round-trip; 5 fail for 5 distinct reasons:
1. wikilinks/@UUID — forward converts [[ ]]→@UUID via resolver; the
(html)=>string seam has no resolver.
2. tables — markdownToHtml has no table branch (forward gap).
3. ## Secrets order — forward moves Secrets to data.notes (always last).
4. ## Secrets heading case — canonicalize doesn't normalize case.
5. bold-leading bodies — parseBody tagline regex matches inside **bold**.
The markdown-hash divergence guard is not viable. E1b adopts the E1b-alt fork
(canonicalize Foundry HTML directly, hash the HTML, never hash markdown).
Findings + recommendation in e1a-spike-findings.md. The spike ships unwired
(fromFoundry.ts not consumed by the push path or ccHash); the forward push
path and canonicalize/contentHash signatures are unchanged. The suite is green
with the 5 unstable fixtures asserting toBe(false) to pin the evidence.
E0.2's markdown ccHash contract is superseded — E1b-alt re-baselines it.
Co-Authored-By: Claude <noreply@anthropic.com>
Foundation for the live-sync epic plan (E0). Frozen-contract primitives that
gate E1a/E1b/E2, grounded in the real code shapes.
- synclock.ts: per-UUID bidirectional lock (push/pull/baseline) replacing the
per-relPath inflight Set. acquire/release/withLock/isHeld/heldOps, skip|queue
policy, reentrant-NO, relPath: fallback. Queue path retries until acquired or
throws LockAcquireTimeout (no silent drop); fairness — acquire defers to
queued waiters. SYNC_LOCK_ENABLED flag (default on; off = byte-identical
legacy inflight).
- cchash.ts: ccHash(entry, inverse) + ccHashFromGet, frozen CC_HASH_CONTRACT.
Contract corrected from epic prose: the body spans data.description +
data.notes (CcData object, not a string), with ## Secrets re-inserted.
Throws CcHashError on missing description (not coerced to ""). Three E1a-gate
assumptions documented (sidebar exclusion, Secrets heading/case, section order).
- schema-version.ts: FLAGS_SCHEMA_VERSION / SYNC_STATE_SCHEMA_VERSION branded
constants + parseSchemaVersion (branded discriminated union).
- server.ts: AutoSyncController refactored to use the lock (single read; uuid
resolved from the same read used to gate — no stale-uuid gap; uuidCache powers
the debounce pre-check without a file read). AutoSyncController/State exported,
lock public for E2's poll path.
- 43 tests across 4 files (synclock, cchash, schema-version, server-lock
integration incl. cross-direction block + flag-off byte-identical). tsc clean.
Co-Authored-By: Claude <noreply@anthropic.com>
Portrait upload was broken: it wrote to Data/worlds/<world>/uploads (per-world,
missing the Data/ segment) and did not URL-encode. Fixed to the verified
convention -- copy to the GLOBAL Data/uploads/<world>/ dir (unencoded on disk),
flag stored URL-encoded as uploads/<world>/<encodeURIComponent(file)>. Extracted
a reusable uploadImageAsset helper.
Body ![[image]] embeds were dropped by mdToHtml. Added push-layer pre-processing
(processBodyImages in push.ts): upload co-located files, rewrite the embed to
, drop non-local refs gracefully. mdToHtml renders 
-> <img>. parseBody now returns a preface (preamble content besides the tagline)
so a loose image between the tagline and the first ## section is no longer
dropped before conversion.
scripts/ helpers for the live-relay path (no offline CC export needed):
- seed-by-name: link refined notes to Foundry by name -> uuid, inject foundry block
- import-missing: create Foundry entries with no vault note as new pushable notes
- gen-cc-dir: generate a cc dir from a journal snapshot so the dashboard ui can run
- resync: push every body-changed note and baseline its foundry.contentHash so
re-runs only catch new edits (idempotent, re-runnable)
Verified end-to-end against the dev Foundry world: portraits + body images
served by Foundry, bidirectional round-trip faithful, push is no-clobber
(diff keys only name + flags.campaign-codex).
Co-Authored-By: Claude <noreply@anthropic.com>
