feat: refine workflow, take first steps towards thiking about other peoples preferences
This commit is contained in:
157
.claude/skills/fobsidian-sync-setup/SKILL.md
Normal file
157
.claude/skills/fobsidian-sync-setup/SKILL.md
Normal file
@@ -0,0 +1,157 @@
|
||||
---
|
||||
name: fobsidian-sync-setup
|
||||
description: Onboard a fresh clone — bring up the relay, wire it to the user's external Foundry, launch the headless session, and start the sync dashboard. Run when the user says "setup", "/setup", "get me running", or "onboarding".
|
||||
allowed-tools: Bash, Read, Write, Edit, AskUserQuestion, WebFetch
|
||||
---
|
||||
|
||||
# Setup — get a fresh clone running
|
||||
|
||||
You bring up the pieces this project needs and wire them to the user's
|
||||
**external** Foundry. You run every command yourself — do NOT hand the user a shell
|
||||
command to type. The only things the user types are values you can't know (their
|
||||
Foundry URL, the API key the relay shows them) and those are collected through
|
||||
`AskUserQuestion`.
|
||||
|
||||
The stack:
|
||||
- **relay** (container) — ThreeHats rest-api relay. Foundry's rest-api module
|
||||
connects OUT to it over WebSocket.
|
||||
- **dashboard** (host-run, `./sync.sh ui`) — the sync tool on :7788. The user edits
|
||||
notes in their own Obsidian desktop app pointed at `$VAULT`; the dashboard
|
||||
reads/writes the same files on disk.
|
||||
|
||||
## Before you start
|
||||
|
||||
Read `.env.example` so you know every var. The sync tool reads env directly
|
||||
(`src/config.ts`); `sync.sh` loads `.env` and injects `--journal`. Pushing edits back
|
||||
to Foundry happens through the dashboard's HTTP API (`POST /api/push`,
|
||||
`/api/push-all`) or its **Auto-sync mode** (watch the vault + poll Foundry).
|
||||
|
||||
If `.env` already exists and `RELAY_API_KEY` + `FOUNDRY_URL` are set, skip to the
|
||||
"Bring up the relay" step (the user may be re-running you).
|
||||
|
||||
## 1. Create `.env` from the template
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
```
|
||||
|
||||
## 2. Collect the user's Foundry details
|
||||
|
||||
Use `AskUserQuestion` to gather:
|
||||
|
||||
- `FOUNDRY_URL` — their Foundry server URL (e.g. `https://foundry.example.com`).
|
||||
- `FOUNDRY_WORLD` — the world id/title to launch.
|
||||
- `RELAY_USER` / `RELAY_PASSWORD` — a Foundry player account the headless session
|
||||
logs in as.
|
||||
- `VAULT` — absolute host path to their Obsidian vault root (the folder with
|
||||
`.obsidian`); they edit this in their own Obsidian desktop app. `REFINED` defaults
|
||||
to `${VAULT}/Refined`.
|
||||
|
||||
Write each into `.env` with `Edit`. Leave `CC`, `JOURNAL`, and the
|
||||
`FOUNDRY_CONTAINER`/`FOUNDRY_DATA_DIR` block commented/blank unless the user has them.
|
||||
|
||||
## 3. Bring up the relay
|
||||
|
||||
```bash
|
||||
docker compose up -d relay
|
||||
```
|
||||
|
||||
Poll until it answers (it does a first-run migration on `./relay-data`):
|
||||
|
||||
```bash
|
||||
# Retry for ~60s:
|
||||
curl -sf http://localhost:${RELAY_PORT:-3010}/session
|
||||
```
|
||||
|
||||
Print the relay UI URL to the user: `http://localhost:${RELAY_PORT:-3010}`.
|
||||
|
||||
## 4. Create the relay API key (human-in-browser)
|
||||
|
||||
This genuinely can't be scripted — the relay creates keys through its web signup:
|
||||
|
||||
1. Tell the user: "Open `http://localhost:${RELAY_PORT}` in a browser, click Sign Up,
|
||||
create an account, then copy the API key shown on your dashboard."
|
||||
2. Use `AskUserQuestion` to collect the pasted key.
|
||||
3. `Edit` it into `.env` as `RELAY_API_KEY=`.
|
||||
|
||||
## 5. Connect Foundry's rest-api module (human-in-Foundry)
|
||||
|
||||
Foundry's module connects OUT to the relay, so the relay must be reachable FROM the
|
||||
Foundry host. Tell the user clearly:
|
||||
|
||||
> In Foundry: install the "rest-api" module, enable it, and in its settings set the
|
||||
> relay URL to `ws(s)://<host-Foundry-can-reach>:<RELAY_PORT>`. If Foundry is on a
|
||||
> different machine than this compose, expose `RELAY_PORT` (port-forward / tailnet /
|
||||
> public domain) first — the module can't reach `localhost` from another host.
|
||||
|
||||
Then verify a client connected:
|
||||
|
||||
```bash
|
||||
curl -s -H "x-api-key: ${RELAY_API_KEY}" http://localhost:${RELAY_PORT:-3010}/session
|
||||
```
|
||||
|
||||
The response includes `activeClients` (or similar). If zero, stop and tell the user
|
||||
the module isn't connected yet — do not proceed to the headless session.
|
||||
|
||||
## 6. Launch the headless Foundry session
|
||||
|
||||
```bash
|
||||
node scripts/start-relay-session.js
|
||||
```
|
||||
|
||||
This posts `/session-handshake` then `/start-session` (RSA-encrypts the password) to
|
||||
spin up a headless browser logged into their Foundry. It can take up to 2 min. Poll
|
||||
until `activeSessions` is non-empty:
|
||||
|
||||
```bash
|
||||
curl -s -H "x-api-key: ${RELAY_API_KEY}" http://localhost:${RELAY_PORT:-3010}/session
|
||||
```
|
||||
|
||||
If it fails, the script prints which var is wrong (FOUNDRY_URL / RELAY_USER /
|
||||
RELAY_PASSWORD / FOUNDRY_WORLD). Re-collect the bad value via `AskUserQuestion`,
|
||||
`Edit` `.env`, and retry.
|
||||
|
||||
## 7. Build the name↔uuid map + start the dashboard
|
||||
|
||||
```bash
|
||||
./sync.sh refresh --out "${OUT:-./out}"
|
||||
```
|
||||
|
||||
This lists journal entries via the relay `/search` (zero Foundry downtime) and caches
|
||||
`name-uuid.json`.
|
||||
|
||||
Then launch the dashboard (omit `--cc` if `CC` is blank — the CLI auto-builds cc stubs
|
||||
from the journal, or runs without if `JOURNAL` is also blank):
|
||||
|
||||
```bash
|
||||
./sync.sh ui --vault "${REFINED}" --out "${OUT:-./out}" --host 0.0.0.0
|
||||
```
|
||||
|
||||
Run it in the background (`run_in_background: true`) so it stays up. Print to the
|
||||
user:
|
||||
|
||||
- Dashboard: `http://<this-host-ip>:7788`
|
||||
- It opens in dev mode with dry-run ON by default — actions preview without writing.
|
||||
Uncheck dry-run to write into `--out`; restart with `--apply` to write back to the
|
||||
real refined/cc dirs (with `.bak-<iso>` backups).
|
||||
- Open `$VAULT` in your Obsidian desktop app to edit notes; the dashboard reads/writes
|
||||
the same files.
|
||||
|
||||
## 8. Tell the user how syncing works
|
||||
|
||||
The project's hard rule: if it's not in the dashboard UI, it doesn't exist. From the
|
||||
dashboard the user can:
|
||||
|
||||
- **Push** one note (`POST /api/push {name}`) or **Push all** edited notes
|
||||
(`POST /api/push-all`).
|
||||
- Flip on **Auto-sync mode** — the dashboard watches the refined vault and pushes
|
||||
Obsidian→Foundry automatically the moment a linked, seeded note is saved (guarded by
|
||||
the note's `foundry.contentHash` baseline, so no-op saves don't re-push). Needs the
|
||||
relay + a live headless session. Foundry→Obsidian is NOT automatic (the relay has no
|
||||
change-push) — use the Sync / Re-pull buttons for that direction.
|
||||
|
||||
## If something is already running
|
||||
|
||||
- Relay already up + key already set → skip to step 5/6.
|
||||
- Headless session already active (`activeSessions` non-empty) → skip to step 7.
|
||||
- Dashboard already on :7788 → don't start a second one; just print its URL.
|
||||
66
.env.example
Normal file
66
.env.example
Normal file
@@ -0,0 +1,66 @@
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# foundry-obsidian-sync — environment template.
|
||||
# Copy to .env and fill in: cp .env.example .env
|
||||
# (Or just run the /setup skill, which fills most of this in for you.)
|
||||
# .env is gitignored — never commit real keys or passwords.
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
|
||||
# === Relay (ThreeHats foundryvtt-rest-api-relay) ===
|
||||
# Host port the relay publishes (its web UI + REST API + WS endpoint live here).
|
||||
RELAY_PORT=3010
|
||||
RELAY_CONTAINER=foundry-rest-api-relay
|
||||
# URL the relay advertises to itself (email links, etc.). Usually http://localhost:<RELAY_PORT>.
|
||||
RELAY_FRONTEND_URL=http://localhost:3010
|
||||
# URL the sync TOOL calls the relay at. For a local compose this is
|
||||
# http://localhost:<RELAY_PORT>. If Foundry is remote and you front the relay with
|
||||
# a public/tailnet domain, set this to that URL instead.
|
||||
RELAY_URL=http://localhost:3010
|
||||
# x-api-key for /get, /update, /search. Created via the relay's web UI on first run
|
||||
# (sign up at http://localhost:<RELAY_PORT>, copy the key from the dashboard).
|
||||
RELAY_API_KEY=
|
||||
# Optional: pin a specific connected Foundry client (leave blank to auto-resolve
|
||||
# when exactly one client is connected to the key).
|
||||
RELAY_CLIENT_ID=
|
||||
|
||||
# === Headless Foundry session (the relay drives a headless browser at Foundry) ===
|
||||
# Your Foundry server URL. The relay logs a headless user into THIS Foundry.
|
||||
FOUNDRY_URL=https://your-foundry.example.com
|
||||
# Foundry player credentials the headless session uses to log in.
|
||||
RELAY_USER=
|
||||
RELAY_PASSWORD=
|
||||
# Foundry world id/title to launch in the headless session.
|
||||
FOUNDRY_WORLD=
|
||||
# Seconds before an idle headless session is reaped (relay env).
|
||||
HEADLESS_SESSION_TIMEOUT=3600
|
||||
|
||||
# IMPORTANT — networking: Foundry's rest-api module connects OUT to the relay over
|
||||
# WebSocket. So the relay must be REACHABLE FROM your Foundry host. If Foundry runs
|
||||
# elsewhere, expose RELAY_PORT (port-forward / tailnet / public domain) and point the
|
||||
# module's relay URL at ws(s)://<that-reachable-host>:<RELAY_PORT>.
|
||||
|
||||
# === Vault (the sync tool / dashboard reads & writes this directly on the host) ===
|
||||
# Absolute path on the HOST to your Obsidian vault root (the folder containing
|
||||
# .obsidian). You edit these notes in your own Obsidian desktop app, pointed at this
|
||||
# vault; the host-run dashboard reads/writes the same files.
|
||||
# QUOTE any path with spaces — sync.sh sources this file in bash, so an unquoted
|
||||
# "VAULT=/home/me/My Vault" tries to run "Vault" as a command and VAULT stays unset.
|
||||
VAULT="/home/me/My Vault"
|
||||
# The refined-notes subdirectory the dashboard's --vault points at. Defaults to
|
||||
# ${VAULT}/Refined if you leave this blank.
|
||||
REFINED="${VAULT}/Refined"
|
||||
# Optional Campaign Codex export dir (omit to let the dashboard auto-build stubs).
|
||||
CC=
|
||||
|
||||
# === Sync tool ===
|
||||
# Optional: a Foundry journal LevelDB snapshot, for offline dashboard indexing
|
||||
# (to-foundry / ui). Leave blank if you only ever push live via the relay.
|
||||
JOURNAL=
|
||||
# Sandbox output dir for the tool (name-uuid.json, index.json, converted notes).
|
||||
OUT=./out
|
||||
|
||||
# === Foundry host control (OPTIONAL — only for `refresh --full-index`) ===
|
||||
# Only needed if you run `./sync.sh refresh --full-index` to stop/start a LOCAL
|
||||
# Foundry Docker container and read its live journal LevelDB. Live `push` and plain
|
||||
# `refresh` go through the relay and never touch these.
|
||||
FOUNDRY_CONTAINER=
|
||||
FOUNDRY_DATA_DIR=
|
||||
14
.gitignore
vendored
14
.gitignore
vendored
@@ -2,8 +2,20 @@ node_modules/
|
||||
# Local live-stack secrets (relay API key, Foundry container/world paths).
|
||||
.env
|
||||
.env.local
|
||||
# Compose runtime state (relay sqlite DB + headless-chrome profile).
|
||||
relay-data/
|
||||
# Tool sandbox output (name-uuid.json, index.json, converted notes).
|
||||
out/
|
||||
# Copied world-data fixtures (your private Foundry snapshot) — never commit.
|
||||
tests/fixtures/
|
||||
# Apply-mode backups written by --apply.
|
||||
*.bak-*
|
||||
*.foundry.json
|
||||
*.foundry.json
|
||||
|
||||
# Wiki (Mode B): .raw/ holds local source dumps (private/large) — keep the folder via .gitkeep.
|
||||
.raw/*
|
||||
!.raw/.gitkeep
|
||||
# Obsidian app config is machine-specific — but share the CSS snippet.
|
||||
.obsidian/*
|
||||
!.obsidian/snippets/
|
||||
!.obsidian/snippets/*
|
||||
38
.obsidian/snippets/vault-colors.css
vendored
Normal file
38
.obsidian/snippets/vault-colors.css
vendored
Normal file
@@ -0,0 +1,38 @@
|
||||
/* vault-colors.css — Mode B (GitHub/Repository) folder colors + custom callouts.
|
||||
Enable: Settings > Appearance > CSS Snippets > open folder > (file is here) >
|
||||
refresh icon > toggle on. */
|
||||
:root {
|
||||
--wiki-1: #4fc1ff; /* modules */
|
||||
--wiki-2: #c586c0; /* components */
|
||||
--wiki-3: #dcdcaa; /* decisions */
|
||||
--wiki-4: #ce9178; /* dependencies */
|
||||
--wiki-5: #6a9955; /* flows */
|
||||
--wiki-7: #569cd6; /* _meta */
|
||||
}
|
||||
|
||||
/* Folder colors in the file explorer */
|
||||
.nav-folder-title[data-path^="wiki/modules"] { color: var(--wiki-1); }
|
||||
.nav-folder-title[data-path^="wiki/components"] { color: var(--wiki-2); }
|
||||
.nav-folder-title[data-path^="wiki/decisions"] { color: var(--wiki-3); }
|
||||
.nav-folder-title[data-path^="wiki/dependencies"] { color: var(--wiki-4); }
|
||||
.nav-folder-title[data-path^="wiki/flows"] { color: var(--wiki-5); }
|
||||
.nav-folder-title[data-path^="_meta"] { color: var(--wiki-7); }
|
||||
.nav-folder-title[data-path=".raw"] { color: #808080; opacity: 0.6; }
|
||||
|
||||
/* Custom callouts */
|
||||
.callout[data-callout='contradiction'] {
|
||||
--callout-color: 209, 105, 105;
|
||||
--callout-icon: lucide-alert-triangle;
|
||||
}
|
||||
.callout[data-callout='gap'] {
|
||||
--callout-color: 220, 220, 170;
|
||||
--callout-icon: lucide-help-circle;
|
||||
}
|
||||
.callout[data-callout='key-insight'] {
|
||||
--callout-color: 79, 193, 255;
|
||||
--callout-icon: lucide-lightbulb;
|
||||
}
|
||||
.callout[data-callout='stale'] {
|
||||
--callout-color: 128, 128, 128;
|
||||
--callout-icon: lucide-clock;
|
||||
}
|
||||
0
.raw/.gitkeep
Normal file
0
.raw/.gitkeep
Normal file
55
CLAUDE.md
Normal file
55
CLAUDE.md
Normal file
@@ -0,0 +1,55 @@
|
||||
# obsidian-foundry-sync: LLM Wiki
|
||||
|
||||
Mode: B (GitHub / Repository)
|
||||
Purpose: Map the architecture of the obsidian-foundry-sync tool — modules, data flows, and the design decisions behind them.
|
||||
Owner: Kaysser Kayyali
|
||||
Created: 2026-06-22
|
||||
|
||||
## Structure
|
||||
|
||||
```
|
||||
vault root (= this repo)
|
||||
├── .raw/ # immutable source documents (README, code dumps, issue exports)
|
||||
├── wiki/
|
||||
│ ├── modules/ # one note per major module / package (server, push, batch, autosync, …)
|
||||
│ ├── components/ # reusable pieces (content-hash guard, foundry block, name↔uuid resolver)
|
||||
│ ├── decisions/ # Architecture Decision Records (ADRs)
|
||||
│ ├── dependencies/ # external deps + services (relay, classic-level, linkedom, rest-api module)
|
||||
│ └── flows/ # data flows / request paths (push, refresh, auto-sync watch loop, index)
|
||||
├── _meta/
|
||||
│ ├── index.md # master catalog of all wiki pages
|
||||
│ ├── log.md # append-only operation log (newest at top)
|
||||
│ ├── overview.md # executive summary of the whole wiki
|
||||
│ └── hot.md # hot cache: ~500-word recent-context summary
|
||||
├── _templates/ # one template per note type
|
||||
└── CLAUDE.md # this file
|
||||
```
|
||||
|
||||
This repo is ALSO the obsidian-foundry-sync code project. The wiki layer (above) documents
|
||||
it; the code itself, build/test/usage, and the live-stack wiring live in `README.md`,
|
||||
`src/`, `scripts/`, `docker-compose.yml`, and `.env.example`. When doing code work, prefer
|
||||
those; when doing knowledge work, prefer the wiki.
|
||||
|
||||
## Conventions
|
||||
|
||||
- All notes use YAML frontmatter: `type`, `status`, `created`, `updated`, `tags` (minimum).
|
||||
Module/flow/decision/dependency notes add the fields in their `_templates/` file.
|
||||
- Wikilinks use `[[Note Name]]` format: filenames are unique, no paths needed.
|
||||
- `.raw/` contains source documents: never modify them.
|
||||
- `_meta/index.md` is the master catalog: update on every ingest/page creation.
|
||||
- `_meta/log.md` is append-only: never edit past entries; new entries go at the TOP.
|
||||
- `_meta/hot.md` is a cache, not a journal: overwrite it completely each time (keep <500 words).
|
||||
|
||||
## Operations
|
||||
|
||||
- **Ingest**: drop a source in `.raw/`, say "ingest [filename]" (→ `wiki-ingest`).
|
||||
- **Query**: ask any question; Claude reads `_meta/index.md` first, then drills in (→ `wiki-query`).
|
||||
- **Lint**: say "lint the wiki" to run a health check (→ `wiki-lint`).
|
||||
- **Save**: say "save this" / "file this" (→ `save`).
|
||||
- **Archive**: move cold sources to `.archive/` to keep `.raw/` clean.
|
||||
|
||||
## Cross-project referencing
|
||||
|
||||
Other Claude Code projects can point at this vault without duplicating context. In that
|
||||
project's CLAUDE.md, add a `## Wiki Knowledge Base` block with `Path: <this repo>`, and the
|
||||
reading order: `_meta/hot.md` → `_meta/index.md` → `wiki/<domain>/_index.md` → individual pages.
|
||||
26
README.md
26
README.md
@@ -18,6 +18,32 @@ Foundry is the source of truth. Each Obsidian note carries a `foundry:` identity
|
||||
block (`cc_uuid`, `cc_type`, `folder_path`, `contentHash`, `syncedAt`) so the two
|
||||
sides stay linked across syncs and renames.
|
||||
|
||||
## Quick start (Docker Compose)
|
||||
|
||||
Clone, then bring up the relay wired to your own **external** Foundry. You need: a
|
||||
running Foundry VTT (license + the
|
||||
[rest-api](https://github.com/ThreeHats/foundryvtt-rest-api-relay) module) and an
|
||||
Obsidian vault on disk that you edit in your own Obsidian desktop app.
|
||||
|
||||
1. `cp .env.example .env` and fill in `FOUNDRY_URL`, `FOUNDRY_WORLD`, `RELAY_USER`/
|
||||
`RELAY_PASSWORD`, and `VAULT` (or let the **`/setup`** skill do all of this for
|
||||
you — it runs the commands itself; you only paste values it can't know).
|
||||
2. `docker compose up -d relay` — relay on `:3010`.
|
||||
3. Open `http://localhost:3010`, sign up, and copy the API key into `.env` as
|
||||
`RELAY_API_KEY`. In Foundry, point the rest-api module at
|
||||
`ws(s)://<host-Foundry-can-reach>:3010`.
|
||||
4. `node scripts/start-relay-session.js` — launches a headless Foundry session the
|
||||
relay drives.
|
||||
5. `./sync.sh refresh --out ./out` then `./sync.sh ui --vault "$VAULT/Refined" \
|
||||
--out ./out --host 0.0.0.0` — opens the dashboard on `:7788`.
|
||||
|
||||
The sync tool stays host-run (it reads the journal LevelDB and shells out to
|
||||
`docker`); the compose provides only the relay. You edit notes in your own Obsidian
|
||||
desktop app pointed at `$VAULT`; the dashboard reads/writes the same files on disk.
|
||||
Push edits back to Foundry from the **dashboard** (`/api/push`, `/api/push-all`), or
|
||||
flip on **Auto-sync mode** to push Obsidian→Foundry automatically the moment you save
|
||||
a note (Foundry→Obsidian stays manual — use Sync / Re-pull for that direction).
|
||||
|
||||
## How it works
|
||||
|
||||
1. `db.ts` opens the journal LevelDB **read-only** with `classic-level`, indexes every
|
||||
|
||||
24
_meta/hot.md
Normal file
24
_meta/hot.md
Normal file
@@ -0,0 +1,24 @@
|
||||
---
|
||||
type: meta
|
||||
title: "Hot Cache"
|
||||
updated: 2026-06-22T03:30:00
|
||||
---
|
||||
|
||||
# Recent Context
|
||||
|
||||
## Last Updated
|
||||
2026-06-22. Scaffolded the Mode B architecture wiki inside the obsidian-foundry-sync repo and seeded it from the codebase.
|
||||
|
||||
## Key Recent Facts
|
||||
- The compose now bundles ONLY the relay (browser Obsidian + plugin were removed 2026-06-21). See [[004-dropped-browser-obsidian]].
|
||||
- Auto-sync mode ([[autosync]]) is Obsidian→Foundry instant only, via `fs.watch` + a `foundry.contentHash` baseline guard. Foundry→Obsidian stays manual — the relay has no change-push and `/search` is minified with no content hash ([[005-autosync-o-to-f-only]]).
|
||||
- `pushNote` always PUTs (no internal idempotency); the guard + `baselineNote` are what prevent loops/redundant pushes ([[push-flow]], [[autosync-watch-loop]]).
|
||||
- The dashboard runs dev/dry-run by default; auto-sync baselines land in the `--out` mirror in dev, the real vault (with `.bak`) in apply.
|
||||
|
||||
## Recent Changes
|
||||
- Created: [[server]], [[autosync]], [[push]], [[relay-client]], [[batch]], [[content-hash]], [[foundry-block]], [[name-uuid-resolver]], [[push-flow]], [[refresh-flow]], [[autosync-watch-loop]], [[index-recommend]], ADRs [[001-external-foundry]]–[[005-autosync-o-to-f-only]], dependency pages.
|
||||
- Wiki decisions recorded for: external Foundry, host-run tool, relay-over-LevelDB, dropped browser Obsidian, O→F-only auto-sync.
|
||||
|
||||
## Active Threads
|
||||
- Verify a real auto-sync push end-to-end once the dev headless session is back up (`activeClients > 0`); the scratch-note test proved the path but the session was down.
|
||||
- MCP server for the wiki is NOT configured (deferred).
|
||||
47
_meta/index.md
Normal file
47
_meta/index.md
Normal file
@@ -0,0 +1,47 @@
|
||||
---
|
||||
type: meta
|
||||
title: "Wiki Index"
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# Wiki Index
|
||||
|
||||
Master catalog of every page in the wiki. Update on every ingest / page creation.
|
||||
|
||||
## modules
|
||||
- [[server]] — HTTP dashboard + JSON API (dev/apply modes, mirror-aware reads, push/refresh/autosync)
|
||||
- [[autosync]] — AutoSyncController: vault watcher → guard → live push → baseline
|
||||
- [[push]] — pushNote: one refined note → live Foundry via relay (no Docker stop)
|
||||
- [[relay-client]] — RelayClient over the ThreeHats rest-api relay (/get /update /create /search)
|
||||
- [[batch]] — indexAll + recommend(); seed/sync/rePull/import row builders
|
||||
- [[cli]] — cmdUi / refresh / push commands (env-driven)
|
||||
- [[config]] — env-driven relay + Foundry host config
|
||||
|
||||
## components
|
||||
- [[content-hash]] — body contentHash idempotency guard
|
||||
- [[foundry-block]] — the `foundry:` frontmatter block + read/baseline helpers
|
||||
- [[name-uuid-resolver]] — name↔uuid map for link resolution
|
||||
|
||||
## flows
|
||||
- [[push-flow]] — Obsidian → live Foundry (relay /get → diff → /update)
|
||||
- [[refresh-flow]] — name↔uuid map via relay /search (zero downtime)
|
||||
- [[autosync-watch-loop]] — fs.watch → debounce → guard → pushNote → baselineNote
|
||||
- [[index-recommend]] — indexAll + recommend() drift detection (local baselines + snapshot)
|
||||
|
||||
## decisions (ADRs)
|
||||
- [[001-external-foundry]] — cloner supplies FOUNDRY_URL; no Foundry in the compose
|
||||
- [[002-tool-host-run]] — sync tool runs on the host, not in a container
|
||||
- [[003-relay-no-journal-db-no-docker-stop]] — live push/refresh via relay; LevelDB only for full index
|
||||
- [[004-dropped-browser-obsidian]] — removed containerized Obsidian + plugin; dashboard is the UI
|
||||
- [[005-autosync-o-to-f-only]] — auto-sync is Obsidian→Foundry instant; Foundry→Obsidian stays manual
|
||||
|
||||
## dependencies
|
||||
- [[threehats-relay]] — foundryvtt-rest-api-relay (Go + headless Chrome)
|
||||
- [[foundryvtt-rest-api-module]] — Foundry module that connects OUT to the relay
|
||||
- [[classic-level]] — journal LevelDB reader
|
||||
- [[linkedom]] — HTML parsing for the Obsidian→Foundry conversion
|
||||
|
||||
## meta
|
||||
- [[overview]] — executive summary
|
||||
- [[hot]] — hot cache (~500 words)
|
||||
- [[log]] — append-only operation log
|
||||
20
_meta/log.md
Normal file
20
_meta/log.md
Normal file
@@ -0,0 +1,20 @@
|
||||
---
|
||||
type: meta
|
||||
title: "Operation Log"
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# Operation Log
|
||||
|
||||
Append-only. Newest entry at the TOP. Never edit past entries.
|
||||
|
||||
---
|
||||
|
||||
## 2026-06-22 — SCAFFOLD (Mode B)
|
||||
|
||||
- Created vault structure inside the obsidian-foundry-sync repo: `wiki/{modules,components,decisions,dependencies,flows}`, `_meta/`, `_templates/`, `.raw/`.
|
||||
- Created `_meta/index.md`, `_meta/log.md`, `_meta/overview.md`, `_meta/hot.md`; vault `CLAUDE.md`.
|
||||
- Created `_templates/` for module, component, decision, dependency, flow.
|
||||
- Added `.obsidian/snippets/vault-colors.css` (Mode B folder colors + custom callouts).
|
||||
- Seeded real pages from the codebase: modules [[server]] [[autosync]] [[push]] [[relay-client]] [[batch]] [[cli]] [[config]]; components [[content-hash]] [[foundry-block]] [[name-uuid-resolver]]; flows [[push-flow]] [[refresh-flow]] [[autosync-watch-loop]] [[index-recommend]]; decisions [[001-external-foundry]]–[[005-autosync-o-to-f-only]]; dependencies [[threehats-relay]] [[foundryvtt-rest-api-module]] [[classic-level]] [[linkedom]].
|
||||
- Choices: vault inside the repo; no MCP server (yet); Obsidian app not installed (headless host).
|
||||
44
_meta/overview.md
Normal file
44
_meta/overview.md
Normal file
@@ -0,0 +1,44 @@
|
||||
---
|
||||
type: meta
|
||||
title: "Wiki Overview"
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# obsidian-foundry-sync — Architecture Wiki
|
||||
|
||||
`obsidian-foundry-sync` is a host-run Node/TypeScript tool that bridges **Foundry VTT**
|
||||
Campaign Codex journal data and an **Obsidian vault** bidirectionally. It reads the
|
||||
Foundry journal LevelDB, converts between Obsidian markdown and Foundry's
|
||||
`flags.campaign-codex` JSON, and pushes/pulls live through a ThreeHats rest-api relay
|
||||
(Foundry keeps running — no Docker stop, no LevelDB write lock).
|
||||
|
||||
## The three pieces
|
||||
|
||||
1. **Relay** (container, `docker compose up -d relay`) — ThreeHats `foundryvtt-rest-api-relay`.
|
||||
Foundry's rest-api module connects OUT to it over WebSocket. API keys come from its web
|
||||
signup; a headless Foundry session drives it.
|
||||
2. **Dashboard** (host-run, `./sync.sh ui`, :7788) — the sync tool. JSON API + UI for
|
||||
seed/sync/import/link/refresh/push, and an **Auto-sync** toggle.
|
||||
3. **Vault** (host path `$VAULT`, edited in your own Obsidian desktop app) — the
|
||||
`Refined` subdir is the tool's `--vault`.
|
||||
|
||||
## Key flows
|
||||
|
||||
- **Push** ([[push-flow]]): refined note → relay `/get` live entry → build minimal diff
|
||||
(`name` + dot-path `flags.campaign-codex`) → `/update`. Reversible (live-entry backup).
|
||||
- **Refresh** ([[refresh-flow]]): relay `/search` (minified) → `name-uuid.json`. Zero downtime.
|
||||
- **Auto-sync** ([[autosync-watch-loop]]): `fs.watch` the vault → guard on the note's
|
||||
`foundry.contentHash` baseline → `pushNote` → `baselineNote`. Obsidian→Foundry instant.
|
||||
Foundry→Obsidian is manual (Sync / Re-pull) — see [[005-autosync-o-to-f-only]].
|
||||
|
||||
## Key design decisions
|
||||
|
||||
See [[decisions]]. Headlines: external Foundry ([[001-external-foundry]]); tool stays
|
||||
host-run ([[002-tool-host-run]]); relay for live ops, LevelDB only for full index
|
||||
([[003-relay-no-journal-db-no-docker-stop]]); browser Obsidian + plugin were dropped — the
|
||||
dashboard is the UI ([[004-dropped-browser-obsidian]]).
|
||||
|
||||
## Hard rule
|
||||
|
||||
UI-only: if it's not in the dashboard UI, it doesn't exist. The agent runs commands; the
|
||||
human only does UI steps (plus the unavoidable relay-signup + Foundry-module-connect).
|
||||
23
_templates/component.md
Normal file
23
_templates/component.md
Normal file
@@ -0,0 +1,23 @@
|
||||
---
|
||||
type: component
|
||||
path: "src/<file>.ts"
|
||||
status: active # active | deprecated | experimental
|
||||
purpose: ""
|
||||
depends_on: []
|
||||
used_by: []
|
||||
tags: [component]
|
||||
created: YYYY-MM-DD
|
||||
updated: YYYY-MM-DD
|
||||
---
|
||||
|
||||
# <Component Name>
|
||||
|
||||
## Purpose
|
||||
|
||||
## API
|
||||
|
||||
## How it works
|
||||
|
||||
## Used by
|
||||
|
||||
## Notes / gotchas
|
||||
22
_templates/decision.md
Normal file
22
_templates/decision.md
Normal file
@@ -0,0 +1,22 @@
|
||||
---
|
||||
type: decision
|
||||
status: active # active | superseded | deprecated
|
||||
date: YYYY-MM-DD
|
||||
context: ""
|
||||
decision: ""
|
||||
consequences: ""
|
||||
tags: [decision, adr]
|
||||
created: YYYY-MM-DD
|
||||
updated: YYYY-MM-DD
|
||||
---
|
||||
|
||||
# ADR <NNN> — <Title>
|
||||
|
||||
## Context
|
||||
|
||||
## Decision
|
||||
|
||||
## Consequences
|
||||
|
||||
## Related
|
||||
- [[<other ADR>]]
|
||||
25
_templates/dependency.md
Normal file
25
_templates/dependency.md
Normal file
@@ -0,0 +1,25 @@
|
||||
---
|
||||
type: dependency
|
||||
name: ""
|
||||
version: ""
|
||||
kind: service # service | library | module | image
|
||||
status: active # active | evaluating | deprecated
|
||||
purpose: ""
|
||||
risk: low # low | medium | high
|
||||
tags: [dependency]
|
||||
created: YYYY-MM-DD
|
||||
updated: YYYY-MM-DD
|
||||
---
|
||||
|
||||
# <Dependency Name>
|
||||
|
||||
## What it is
|
||||
|
||||
## Why we depend on it
|
||||
|
||||
## How we use it
|
||||
|
||||
## Risk / lock-in
|
||||
|
||||
## Related
|
||||
- [[<module that uses it>]]
|
||||
26
_templates/flow.md
Normal file
26
_templates/flow.md
Normal file
@@ -0,0 +1,26 @@
|
||||
---
|
||||
type: flow
|
||||
status: active # active | draft | deprecated
|
||||
purpose: ""
|
||||
actors: [] # modules / components / services involved
|
||||
steps: []
|
||||
tags: [flow]
|
||||
created: YYYY-MM-DD
|
||||
updated: YYYY-MM-DD
|
||||
---
|
||||
|
||||
# <Flow Name>
|
||||
|
||||
## Purpose
|
||||
|
||||
## Steps
|
||||
1.
|
||||
2.
|
||||
|
||||
## Actors
|
||||
- [[<module>]]
|
||||
|
||||
## Edge cases / failure modes
|
||||
|
||||
## Related
|
||||
- [[<other flow>]]
|
||||
30
_templates/module.md
Normal file
30
_templates/module.md
Normal file
@@ -0,0 +1,30 @@
|
||||
---
|
||||
type: module
|
||||
path: "src/<file>.ts"
|
||||
status: active # active | deprecated | experimental | planned
|
||||
language: typescript
|
||||
purpose: ""
|
||||
maintainer: ""
|
||||
last_updated: YYYY-MM-DD
|
||||
linked_issues: []
|
||||
depends_on: [] # [[other module]] / [[component]] / [[dependency]]
|
||||
used_by: []
|
||||
tags: [module]
|
||||
created: YYYY-MM-DD
|
||||
updated: YYYY-MM-DD
|
||||
---
|
||||
|
||||
# <Module Name>
|
||||
|
||||
## Purpose
|
||||
|
||||
## Public surface
|
||||
- `export fn …` —
|
||||
|
||||
## How it works
|
||||
|
||||
## Depends on
|
||||
|
||||
## Used by
|
||||
|
||||
## Notes / gotchas
|
||||
59
docker-compose.yml
Normal file
59
docker-compose.yml
Normal file
@@ -0,0 +1,59 @@
|
||||
# Shareable stack: ThreeHats rest-api relay + browser-served Obsidian.
|
||||
#
|
||||
# Brings up the two containers the sync tool needs alongside an EXTERNAL Foundry
|
||||
# instance (you supply FOUNDRY_URL — Foundry itself is not bundled; it needs a
|
||||
# license + the rest-api module installed). The sync tool itself stays host-run
|
||||
# (./sync.sh ui) so it can read the journal LevelDB and shell out to `docker`.
|
||||
#
|
||||
# Copy .env.example -> .env and fill it in, or run the /setup skill which does it
|
||||
# for you. Then: docker compose up -d
|
||||
#
|
||||
# NOTE on networking: Foundry's rest-api module connects OUT to the relay over
|
||||
# WebSocket. If your Foundry is on a different host than this compose, the relay's
|
||||
# port (RELAY_PORT) must be reachable from Foundry (port-forward / tailnet / public
|
||||
# domain). RELAY_URL below is what the *tool* calls (usually localhost).
|
||||
|
||||
services:
|
||||
relay:
|
||||
# Pin to a release tag (e.g. :2.1.0) for production stability.
|
||||
image: threehats/foundryvtt-rest-api-relay:latest
|
||||
container_name: ${RELAY_CONTAINER:-foundry-rest-api-relay}
|
||||
environment:
|
||||
- DB_TYPE=sqlite
|
||||
- APP_ENV=production
|
||||
- ALLOW_HEADLESS=true
|
||||
- PORT=3010
|
||||
- HEADLESS_SESSION_TIMEOUT=${HEADLESS_SESSION_TIMEOUT:-3600}
|
||||
- FRONTEND_URL=${RELAY_FRONTEND_URL:-http://localhost:3010}
|
||||
volumes:
|
||||
- ./relay-data:/app/data
|
||||
ports:
|
||||
- "${RELAY_PORT:-3010}:3010"
|
||||
# Headless Chrome (Puppeteer) needs a sizable /dev/shm to avoid crashes.
|
||||
shm_size: "1g"
|
||||
restart: unless-stopped
|
||||
|
||||
# Optional GPU-accelerated relay for hosts with a /dev/dri device. Software
|
||||
# rendering (the default `relay` service above) works without it, just slower.
|
||||
# Enable with: docker compose --profile gpu up -d relay-gpu
|
||||
relay-gpu:
|
||||
image: threehats/foundryvtt-rest-api-relay:latest
|
||||
container_name: ${RELAY_CONTAINER:-foundry-rest-api-relay}-gpu
|
||||
profiles: ["gpu"]
|
||||
environment:
|
||||
- DB_TYPE=sqlite
|
||||
- APP_ENV=production
|
||||
- ALLOW_HEADLESS=true
|
||||
- PORT=3010
|
||||
- HEADLESS_SESSION_TIMEOUT=${HEADLESS_SESSION_TIMEOUT:-3600}
|
||||
- FRONTEND_URL=${RELAY_FRONTEND_URL:-http://localhost:3010}
|
||||
- CHROME_GPU_MODE=gpu
|
||||
- CAPTURE_BROWSER_CONSOLE=error
|
||||
volumes:
|
||||
- ./relay-data:/app/data
|
||||
ports:
|
||||
- "${RELAY_PORT:-3010}:3010"
|
||||
devices:
|
||||
- /dev/dri:/dev/dri
|
||||
shm_size: "1g"
|
||||
restart: unless-stopped
|
||||
@@ -0,0 +1,79 @@
|
||||
# Decision Log — Live Relay Sync PRD
|
||||
|
||||
Canonical memory + audit trail for this PRD run. Newest at top. Every decision,
|
||||
change, and override lands here as the conversation unfolds.
|
||||
|
||||
---
|
||||
|
||||
## 2026-06-22 — Divergence / conflict posture (the no-clobber decisions)
|
||||
|
||||
- **Foundry-side baseline hash scope = content + name + folder_path.** Catches
|
||||
Foundry-side renames and folder moves as divergence (surfaces them instead of
|
||||
silently breaking name↔uuid links). Stored alongside the existing Obsidian
|
||||
`foundry.contentHash` (new field, e.g. `foundry.ccHash`).
|
||||
- **Both-sides-diverged default = surface conflict, DM resolves.** Never
|
||||
auto-overwrite. Conflict shows as a dashboard row; DM picks a side or merges
|
||||
via manual buttons. Strictest "never clobber work" posture.
|
||||
- **Fail-safe when relay /get can't read Foundry = skip push + surface error.**
|
||||
Do not fall back to Obsidian-side-only check (that reintroduces clobber risk).
|
||||
DM sees a "couldn't verify Foundry side — not pushed" row.
|
||||
- **Implies a hard NFR:** no auto-sync operation may overwrite a side that has
|
||||
changed since the last sync. The current O→F auto-sync code violates this
|
||||
(checks Obsidian side only) → must be fixed before/within delivery A.
|
||||
|
||||
## 2026-06-22 — Vision follow-ups (source of truth, divergence, status)
|
||||
|
||||
- **Source of truth = newest version of a doc** — not a fixed side. But **never
|
||||
clobber work**: both-sides-diverged → reconcile, not overwrite. Manual buttons
|
||||
stay as override/repair for unsynced-on-either-side or both-deviate cases.
|
||||
- **Foundry = equal-origin editing surface during prep.** "Foundry is source of
|
||||
truth" demoted to both-diverged tie-breaker rule (newest-wins vs Foundry-wins
|
||||
vs manual merge — undecided).
|
||||
- **"Not syncing" indicator location → DEFERRED.** No easy answer; Foundry UI not
|
||||
ours; dashboard is the only surface we control now. Custom Foundry module is
|
||||
future, not this PRD.
|
||||
- **Sync-status-when-ON → dashboard, for now.** User proposed a maintained
|
||||
**status note inside the vault** (`Sync Status.md`) as a lightweight parity
|
||||
indicator — candidate feature, to confirm.
|
||||
- **Divergence-detection question raised by user** → answered in conversation:
|
||||
yes, guaranteeable, but only with a Foundry-side baseline hash + relay /get
|
||||
before any auto push/pull. Current code lacks this (only checks Obsidian side)
|
||||
→ can silently clobber Foundry edits today. Gap for delivery A to close.
|
||||
|
||||
## 2026-06-22 — Vision input (prep / run-the-match / status)
|
||||
|
||||
- **Prep is the heart.** DM edits many notes at once, draws links, summarizes/
|
||||
closes prior journal notes, adds images/lore, creates objects. Data originates
|
||||
from whichever tool is easier — Obsidian **or** Foundry. Files must stay synced
|
||||
the whole time, bidirectionally, no button-babysitting.
|
||||
- **Run-the-match = sync OFF, Foundry-centric.** DM generates many notes, mostly
|
||||
in Foundry. System must make "not syncing" legible (Foundry should show it).
|
||||
- **Status always visible** — on or off, the DM should see sync status at a
|
||||
glance: when to expect parity and when not.
|
||||
- **Open (probed next):**
|
||||
- Confirm Foundry is an equal-origin editing surface during prep (challenges
|
||||
ADR-005 / README "Foundry is source of truth, O→F hot path"). Reconcile.
|
||||
- Where does the "not syncing" indicator live? Foundry UI is rendered by the
|
||||
rest-api module (not ours). Candidate: a journal entry/flag the sync tool
|
||||
writes that the module surfaces; or dashboard "SYNC PAUSED" state; or both.
|
||||
- Sync-status-when-ON: is the existing autosync activity panel enough, or want
|
||||
a persistent parity indicator ("in parity / N pending / last sync 12s ago")?
|
||||
- Manual buttons (Sync/Re-pull/Push-all) — repair/override tools (stay) vs retire.
|
||||
|
||||
## 2026-06-22 — Run opened
|
||||
|
||||
- **Scope chosen:** A + B + C — full live-sync surface. A = ship & verify O→F
|
||||
auto-sync (uncommitted controller/UI + empty `RELAY_CLIENT_ID` gap + end-to-end
|
||||
live verification). B = Foundry→Obsidian auto direction (no relay push channel;
|
||||
needs polling/snapshot diff). C = operational hardening (watch fallback,
|
||||
concurrency/debounce, error surfacing, retry).
|
||||
- **Stakes chosen:** public/launchable → full NFRs, error contracts, onboarding
|
||||
rigor.
|
||||
- **Working mode chosen:** coaching path, Vision + Features entry (capability-first;
|
||||
single operator role = GM/world-builder).
|
||||
- **Workspace bound:** `docs/prds/prd-foundry-obsidian-sync-2026-06-22/`.
|
||||
- **No `_bmad/` project config** — running on neutral defaults. `planning_artifacts`
|
||||
resolved to `docs/`.
|
||||
- **External session blocker noted:** relay has no connected Foundry client right
|
||||
now and `RELAY_CLIENT_ID` is empty; user will bring the headless session up
|
||||
themselves. Live end-to-end verification (A) gated on that.
|
||||
304
docs/prds/prd-foundry-obsidian-sync-2026-06-22/prd.md
Normal file
304
docs/prds/prd-foundry-obsidian-sync-2026-06-22/prd.md
Normal file
@@ -0,0 +1,304 @@
|
||||
---
|
||||
title: "Live Relay Sync — Auto-Sync & Bidirectional Hardening"
|
||||
status: draft
|
||||
created: 2026-06-22
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# Live Relay Sync — Auto-Sync & Bidirectional Hardening
|
||||
|
||||
> Status: **draft** — being authored via bmad-prd coaching path.
|
||||
> Scope: full live-sync surface over the ThreeHats relay — (A) ship & verify
|
||||
> Obsidian→Foundry instant auto-sync, (B) add Foundry→Obsidian auto direction,
|
||||
> (C) operational hardening. Stakes: public/launchable.
|
||||
|
||||
## 1. Vision
|
||||
|
||||
A DM's work has two phases, and live sync serves them differently.
|
||||
|
||||
**Prep** is the heart of the tool. The DM is thinking through situations, lore,
|
||||
and NPCs — editing many notes at once, drawing new links between items,
|
||||
summarizing or closing out previous journal notes, dropping in images and lore,
|
||||
creating new objects to build out the world. Data is coming from **whichever
|
||||
tool is easier in the moment** — Obsidian or Foundry — and files must stay in
|
||||
sync the whole time, across both directions, without the DM babysitting a sync
|
||||
button. The feeling to deliver: *always in lockstep while I work*, bidirectionally,
|
||||
across a flurry of simultaneous edits.
|
||||
|
||||
**Running the match** is the live session. The DM is generating a lot of notes,
|
||||
almost entirely inside Foundry, and the sync service is almost certainly **off**
|
||||
in this phase. The system should make that state legible — Foundry should show
|
||||
"not syncing" somewhere, so the DM never wonders whether prep edits are
|
||||
propagating when they aren't. Sync is deliberately paused, not silently absent.
|
||||
|
||||
**Status, always.** Whether sync is on or off, the DM should be able to see the
|
||||
sync status at a glance — when to expect parity and when not to. No guessing
|
||||
whether the vault and Foundry agree.
|
||||
|
||||
_[CONFIRMED] Source of truth = the newest version of a document — not a fixed
|
||||
side. But the system must **never clobber work**: when both sides have diverged
|
||||
since the last sync, it must route to reconciliation instead of overwriting
|
||||
either side. The manual buttons (Sync / Re-pull / Push-all) stay precisely for
|
||||
this — quick overrides when a file is unsynced on any side or both deviate._
|
||||
|
||||
_[CONFIRMED] Foundry is an equal-origin editing surface during prep — data
|
||||
originates in whichever tool is easier. "Foundry is source of truth" is demoted
|
||||
to a tie-breaker rule for the both-diverged conflict case (to be decided:
|
||||
newest-wins, Foundry-wins, or manual merge)._
|
||||
|
||||
_[DEFERRED] Where the "not syncing" indicator lives during run-the-match — no
|
||||
easy answer yet; decision deferred. Foundry UI is rendered by the rest-api
|
||||
module (not ours); the dashboard is the only realistic surface we control now.
|
||||
A custom Foundry module (giving us what the relay does, plus indicators) is a
|
||||
future exploration, not this PRD._
|
||||
|
||||
_[CONFIRMED] Sync-status-when-ON lives in the dashboard for now. Candidate
|
||||
addition: a single **status note** the sync tool maintains inside the vault
|
||||
(e.g. `Sync Status.md`) showing last-sync time/state — a lightweight,
|
||||
in-our-control way for the user to see parity at a glance without a custom
|
||||
module. (Proposed by user; to be confirmed as a feature.)_
|
||||
|
||||
## 2. Problem & Context
|
||||
|
||||
**Where the tool is today.** obsidian-foundry-sync is an offline bidirectional
|
||||
converter between a Foundry VTT Campaign Codex LevelDB snapshot and an Obsidian
|
||||
vault, plus a dashboard that does **manual** live relay operations — push one,
|
||||
push-all, refresh the name↔uuid index, sync, re-pull. An Obsidian→Foundry
|
||||
instant auto-sync (`AutoSyncController` + dashboard toggle) is **code-complete
|
||||
but uncommitted**, and it only checks the Obsidian side before pushing (it
|
||||
compares the note's body hash to its `foundry.contentHash` baseline and pushes
|
||||
on any difference). There is **no Foundry→Obsidian auto direction** — the relay
|
||||
has no change-push channel and `/search` is minified with no content hash, so
|
||||
F→O stays manual (Sync / Re-pull buttons).
|
||||
|
||||
**The prep workflow doesn't fit manual sync.** A DM prepping a session edits
|
||||
many notes at once, draws new links, summarizes/closes prior journal notes,
|
||||
drops in images and lore, and creates new objects — with data originating in
|
||||
**whichever tool is easier in the moment**, Obsidian or Foundry. Pressing
|
||||
Sync/Re-pull per file doesn't scale to that churn; the value proposition of
|
||||
auto-sync is exactly this phase. But auto-sync that only watches one direction
|
||||
leaves half the prep edits unpropagated, and auto-sync that pushes without
|
||||
checking the other side **can silently clobber Foundry edits** — the current
|
||||
O→F code has this risk.
|
||||
|
||||
**The run-the-match phase is Foundry-centric with sync off.** During the live
|
||||
session the DM generates many notes, mostly inside Foundry, and the sync
|
||||
service is expected to be off. Today "off" is silent — the DM can't tell
|
||||
whether prep edits are still propagating. Status legibility (on/off, parity,
|
||||
last sync) is missing on both sides.
|
||||
|
||||
**Config/onboarding gap.** Live relay sync needs a connected Foundry client and
|
||||
a valid `clientId`. `RELAY_CLIENT_ID` is currently empty in `.env`, and the
|
||||
headless Foundry session that the relay drives is not always up. A launchable
|
||||
tool needs this onboarding path to be discoverable and self-correcting.
|
||||
|
||||
**Audience.** Today a single operator (the DM/world-builder). Intended to be
|
||||
launchable — other DMs running it against their own Foundry worlds — so the
|
||||
no-clobber, error-surfacing, and onboarding rigor must hold for non-author
|
||||
users, not just for the one who wrote it.
|
||||
|
||||
_[ASSUMPTION] The relay remains the live transport for this PRD; the "custom
|
||||
Foundry module that gives us what the relay does" is explicitly future work,
|
||||
not a dependency here._
|
||||
|
||||
## 3. Goals & Non-Goals
|
||||
|
||||
### Goals
|
||||
|
||||
- **G1 — No-clobber bidirectional auto-sync for prep.** Both directions sync
|
||||
automatically; neither side is ever overwritten if it changed since last sync;
|
||||
both-diverged conflicts surface for the DM to resolve.
|
||||
- **G2 — Ship & verify the O→F auto-sync safely.** Commit the existing
|
||||
controller/UI **with the divergence guard added** (Foundry-side baseline hash
|
||||
+ relay `/get` before acting), and verify one end-to-end live push.
|
||||
- **G3 — Foundry→Obsidian auto direction.** A working F→O auto path despite the
|
||||
relay having no push channel (polling / snapshot-diff design).
|
||||
- **G4 — Legible sync status at all times.** The DM can see, at a glance,
|
||||
whether sync is on or off, whether the vault and Foundry are in parity, and
|
||||
when the last sync landed — in the dashboard, plus a maintained status note
|
||||
inside the vault.
|
||||
- **G5 — Operational hardening.** Recursive-watch fallback, tuned
|
||||
concurrency/debounce, retry on transient relay 404/timeout, and visible
|
||||
error rows in the dashboard.
|
||||
- **G6 — Closed onboarding/config.** `RELAY_CLIENT_ID` and the connected-Foundry-
|
||||
client requirement are discoverable and self-correcting from the dashboard.
|
||||
|
||||
### Non-Goals
|
||||
|
||||
- **Custom Foundry module** (indicators inside Foundry UI, relay replacement) —
|
||||
future exploration, out of scope.
|
||||
- **"Not syncing" indicator rendered inside Foundry** — deferred; Foundry UI is
|
||||
not a surface we control. Surfaced via dashboard + vault status note instead.
|
||||
- **Syncing during run-the-match.** Sync stays off by design in that phase; the
|
||||
goal is legibility of the off state, not automation during the session.
|
||||
- **Automatic semantic/3-way content merge.** Both-diverged = pick a side or
|
||||
merge manually via the buttons; no auto content-merge engine.
|
||||
- **Auto-sync of unlinked or unseeded notes.** Seed/link first remains a manual
|
||||
prerequisite (unchanged from current behavior).
|
||||
- **Full LevelDB / docker-stop index in the dashboard.** Remains CLI-only
|
||||
(unchanged).
|
||||
|
||||
## 4. Features & Functional Requirements
|
||||
|
||||
FR IDs are stable. Grouped F1–F6; IDs are `FR-<group>.<n>`.
|
||||
|
||||
### F1 — Obsidian→Foundry auto-sync (safe)
|
||||
|
||||
- **FR-1.1** Watch the refined vault dir for `.md` saves using recursive
|
||||
`fs.watch`, with a per-subdir fallback (re-scanning on subdir create/rename)
|
||||
for platforms/Node versions without recursive watch. Skip `.obsidian` and
|
||||
dotfiles.
|
||||
- **FR-1.2** On a save, read the note and skip it if it has no
|
||||
`foundry.cc_uuid` (unlinked) or no `foundry.contentHash` baseline (unseeded)
|
||||
— seed/link remain manual prerequisites.
|
||||
- **FR-1.3** Compute the current Obsidian body hash; if it equals the
|
||||
`foundry.contentHash` baseline, skip (covers no-op saves and the watcher's
|
||||
own post-push baseline write — no feedback loop).
|
||||
- **FR-1.4** **Before pushing**, `relay /get` the live Foundry entry and
|
||||
compute its Foundry-side hash over **content + name + folder_path**; compare
|
||||
to the stored Foundry-side baseline (`foundry.ccHash`, new field).
|
||||
- **FR-1.5** Route: Obsidian-changed **and** Foundry-unchanged (F-hash equals
|
||||
`ccHash` baseline) → push O→F via the same `pushNote` path the manual push
|
||||
button uses; re-baseline both sides on success.
|
||||
- **FR-1.6** If the Foundry side is unreadable (`/get` 404 / timeout / session
|
||||
down), **skip the push and surface an error row** — do not fall back to an
|
||||
Obsidian-side-only check (that reintroduces clobber risk).
|
||||
- **FR-1.7** After a successful push, re-baseline **both** `foundry.contentHash`
|
||||
(Obsidian body) **and** `foundry.ccHash` (Foundry-side) to the new values.
|
||||
Dev mode baselines land in the `--out` mirror; apply mode in the real vault
|
||||
with a `.bak`.
|
||||
- **FR-1.8** Auto-sync always applies live (dry-run not honored) — unchanged
|
||||
from current behavior; the whole point is hands-off live push.
|
||||
|
||||
### F2 — Foundry→Obsidian auto-sync
|
||||
|
||||
- **FR-2.1** While auto-sync is ON, poll `relay /search`
|
||||
(`documentType:JournalEntry`, minified) on a configurable cadence; build a
|
||||
current `{uuid → name/img/folder}` snapshot.
|
||||
- **FR-2.2** Diff the current snapshot against the last snapshot and the
|
||||
vault's known linked notes (via `foundry.cc_uuid`) to detect Foundry-side
|
||||
changes: renamed (name change, same uuid), moved (folder change),
|
||||
content-changed (detected via `/get` hash compare), missing, or new.
|
||||
- **FR-2.3** For each Foundry-changed **linked** note where the Obsidian side
|
||||
is unchanged: `/get` the live entry, convert to refined markdown, write into
|
||||
the vault, and re-baseline both sides.
|
||||
- **FR-2.4** Never clobber an Obsidian-side change: vault-newer or
|
||||
both-diverged notes route to F3 conflict handling, not auto-pull.
|
||||
- **FR-2.5** New (cc-only) Foundry entries surface as **import candidates**
|
||||
(existing import row) — do not auto-import.
|
||||
- **FR-2.6** Poll cadence is configurable with a prep-tuned default (seconds);
|
||||
a manual "catch up now" trigger is available alongside the background poll.
|
||||
|
||||
### F3 — Divergence detection & conflict routing
|
||||
|
||||
- **FR-3.1** Every sync tick (O→F or F→O) computes both-side hashes and routes
|
||||
per the 2×2: parity / O-changed / F-changed / both-changed.
|
||||
- **FR-3.2** both-changed → **do not auto-overwrite**; create a conflict row
|
||||
in the dashboard summarizing both versions and highlighting the diff.
|
||||
- **FR-3.3** The conflict row offers manual resolution: "push vault →
|
||||
Foundry", "pull Foundry → vault", "mark resolved (no change)".
|
||||
- **FR-3.4** Conflict state persists until the DM resolves it; a re-save on
|
||||
either side does not auto-clear a known conflict.
|
||||
- **FR-3.5** Foundry-side renames and folder moves (caught via name + folder in
|
||||
the hash) surface as changes/conflicts, not silently absorbed.
|
||||
|
||||
### F4 — Sync status & parity
|
||||
|
||||
- **FR-4.1** Dashboard shows a persistent sync-status header: ON/OFF, mode
|
||||
(dev/apply), watched dir.
|
||||
- **FR-4.2** Dashboard shows a parity indicator: counts of in-parity /
|
||||
O-pending / F-pending / conflict / unsynced-linked notes, plus a last-sync
|
||||
timestamp.
|
||||
- **FR-4.3** The sync tool maintains a `Sync Status.md` note in the vault (on a
|
||||
path excluded from the watcher, never synced to Foundry) showing on/off, last
|
||||
sync time, parity counts, and recent events — updated each tick.
|
||||
- **FR-4.4** When sync is OFF, the dashboard shows a loud "SYNC PAUSED" state,
|
||||
not a silent absence.
|
||||
- **FR-4.5** Dashboard parity and the vault status note reflect one underlying
|
||||
state (single source of truth for status).
|
||||
|
||||
### F5 — Operational hardening
|
||||
|
||||
- **FR-5.1** Recursive-watch fallback (FR-1.1) verified on the host kernel,
|
||||
including re-scan on subdir create/rename so new folders get watched.
|
||||
- **FR-5.2** Debounce window and max concurrency configurable; defaults tuned
|
||||
for prep so a burst of simultaneous saves doesn't thrash or drop events.
|
||||
- **FR-5.3** Transient relay errors (404 invalid client, 408/504 timeout, 5xx)
|
||||
retried with bounded backoff; persistent failures surface as error rows.
|
||||
- **FR-5.4** Every auto-sync op (push/skip/error/conflict) logged to the
|
||||
activity panel with time, note, status, message; panel capped and scrollable.
|
||||
- **FR-5.5** Inflight dedup + queue drain verified under burst — no dropped
|
||||
events, no duplicate pushes.
|
||||
|
||||
### F6 — Onboarding & config
|
||||
|
||||
- **FR-6.1** If `RELAY_CLIENT_ID` is unset/empty, the dashboard surfaces a
|
||||
clear "no clientId configured" state with guidance — not a silent 404 at push
|
||||
time.
|
||||
- **FR-6.2** If no Foundry client is connected to the relay (`activeClients =
|
||||
0`), the dashboard surfaces "Foundry not connected" and disables
|
||||
enable-auto-sync until resolved; re-checked on a cadence.
|
||||
- **FR-6.3** The dashboard can list connected relay clients (relay `/search`
|
||||
with no clientId returns the client list on >1, or "No connected clients" on
|
||||
0) so the DM can pick/copy a valid `clientId` from the UI — no shell command.
|
||||
- **FR-6.4** The existing `/setup` skill covers env wiring; the dashboard
|
||||
reflects setup state and links into it.
|
||||
|
||||
## 5. Non-Functional Requirements
|
||||
|
||||
- **NFR-1 — No-clobber safety.** No auto-sync operation may overwrite a side
|
||||
that has changed since the last sync. Both-diverged → conflict, never
|
||||
auto-overwrite. (The hard requirement; the current O→F code violates it.)
|
||||
- **NFR-2 — Fail-safe.** If the relay cannot read the Foundry side, the
|
||||
operation is skipped and surfaced — never a blind push on Obsidian-side-only
|
||||
evidence.
|
||||
- **NFR-3 — Performance.** Debounce + bounded concurrency handle a ~50-note
|
||||
prep burst without dropped events or relay thrash; F→O poll cadence does not
|
||||
overload the relay or the host.
|
||||
- **NFR-4 — Reliability.** Transient relay errors retried with backoff;
|
||||
persistent errors surfaced within one tick.
|
||||
- **NFR-5 — Observability.** Every operation is visible in the dashboard
|
||||
activity panel and the vault status note; no silent skips or silent
|
||||
overwrites.
|
||||
- **NFR-6 — Onboardability.** A non-author user can get from clone → live sync
|
||||
using the dashboard + `/setup`, without editing shell commands (per the
|
||||
UI-only convention).
|
||||
- **NFR-7 — Configurability.** Poll cadence, debounce, concurrency, and
|
||||
status-note path are env/config-driven with safe defaults.
|
||||
- **NFR-8 — Backward compatibility.** Existing manual buttons, seed/sync/
|
||||
rePull/import rows, dev/apply modes, and the CLI-only full LevelDB index all
|
||||
keep working unchanged.
|
||||
|
||||
## 6. Open Questions
|
||||
|
||||
- **OQ-1** Conflict row quick-actions: beyond "push vault / pull Foundry / mark
|
||||
resolved," do we also offer a one-click "newest mtime wins" convenience, or
|
||||
keep every conflict fully manual? (Default posture is manual; this is a
|
||||
convenience question.)
|
||||
- **OQ-2** Where the "not syncing" indicator lives during run-the-match —
|
||||
**deferred**. Foundry UI is not a surface we control; dashboard + vault status
|
||||
note cover the legible case for now. Reopens when a custom Foundry module is
|
||||
explored.
|
||||
- **OQ-3** F→O poll cadence default (seconds) — pick during delivery B with
|
||||
relay-load testing.
|
||||
- **OQ-4** `Sync Status.md` path/name and its exclusion from both the O→F
|
||||
watcher and the F→O pull — confirm during delivery.
|
||||
- **OQ-5** New (cc-only) Foundry entries created during prep: stay as import
|
||||
candidates (FR-2.5) or auto-import? Current spec: candidates only.
|
||||
- **OQ-6** Conflict-row diff format — full unified diff vs. a condensed
|
||||
summary — design during delivery.
|
||||
|
||||
## 7. Out of Scope / Future
|
||||
|
||||
Mirrors §3 Non-Goals, plus:
|
||||
|
||||
- **Custom Foundry module** — a module that gives us what the relay does plus
|
||||
in-Foundry indicators (sync status, parity). Future exploration; would reopen
|
||||
OQ-2.
|
||||
- **Syncing during run-the-match** — sync stays off by design in the live
|
||||
session; legibility of the off state is in scope, automation during the
|
||||
session is not.
|
||||
- **Automatic semantic / 3-way content merge** — both-diverged is manual
|
||||
resolution via buttons; no auto content-merge engine.
|
||||
- **Auto-sync of unlinked or unseeded notes** — seed/link first stays manual.
|
||||
- **Full LevelDB / docker-stop index in the dashboard** — stays CLI-only.
|
||||
@@ -0,0 +1,297 @@
|
||||
---
|
||||
title: "Engineering-feasibility review — Live Relay Sync PRD"
|
||||
reviewer: Claude (engineering-feasibility pass)
|
||||
prd: docs/prds/prd-foundry-obsidian-sync-2026-06-22/prd.md
|
||||
date: 2026-06-22
|
||||
scope: "Stress-test PRD technical claims against the actual codebase + relay constraints."
|
||||
---
|
||||
|
||||
# Engineering-feasibility review
|
||||
|
||||
Verdict at the end. Findings tiered **BLOCKER / HIGH / MEDIUM / LOW** with file:line
|
||||
citations and a recommended fix for each.
|
||||
|
||||
## What the code actually does today (baseline facts)
|
||||
|
||||
- `contentHash(text)` hashes **only the canonicalized body** — wikilink-canonicalize +
|
||||
whitespace-canonicalize, then SHA-256. It does **not** include the note name, the
|
||||
folder path, or any frontmatter. (`src/normalize.ts:24`, `canonicalize` at
|
||||
`src/normalize.ts:19`.)
|
||||
- The `foundry:` block has no `ccHash` field. `readFoundryBlock` returns whatever keys
|
||||
are in the nested map (`src/frontmatter.ts:9`); `baselineFoundryBlock`
|
||||
(`src/server.ts:289`) rewrites **only** `contentHash:` and `syncedAt:` lines. So the
|
||||
Foundry-side baseline the PRD depends on does not exist yet — it is genuinely new.
|
||||
- `AutoSyncController.process()` (`src/server.ts:582`) reads the note, compares
|
||||
`contentHash(body)` to `fb.contentHash`, and on a difference calls `pushNote` **with
|
||||
no relay `/get` of its own**. `pushNote` does call `relay.getEntry(id)`
|
||||
(`src/push.ts:142`) but **only to build the diff** (preserve ownership/pages/
|
||||
existing image) — it never compares a Foundry-side hash to a baseline. So if the
|
||||
Foundry side changed since the last sync, `pushNote` still PUTs `{ name,
|
||||
"flags.campaign-codex": cc }` and **overwrites the Foundry-side edit silently**.
|
||||
Clobber risk is **real, not theoretical**.
|
||||
- Relay `/get` returns the full doc: `{ data: { name, type, _id, uuid, folder, pages,
|
||||
ownership, flags, ... } }` (`docs/relay-api.md:31-33`). `folder` is present on
|
||||
`/get`.
|
||||
- Relay `/search` minified returns **only** `{ uuid, id, name, img, documentType }`
|
||||
(`docs/relay-api.md:55-59`, mirrored in `src/relay/client.ts:19-25`,
|
||||
`searchJournalEntries` at `src/relay/client.ts:97`). **No `folder`. No content. No
|
||||
content hash. No mtime.** This is the load-bearing constraint for F2.
|
||||
- ADR 005 (`wiki/decisions/005-autosync-o-to-f-only.md`) already analyzed F→O polling
|
||||
and rejected it on cost grounds: ~800 `/get` calls/min for a ~200-entry world at a
|
||||
15s cadence. The PRD does not cite or reconcile with this ADR.
|
||||
|
||||
---
|
||||
|
||||
## BLOCKER
|
||||
|
||||
### B1 — FR-2.1 / FR-2.2: `/search` minified has NO `folder`. Folder-move detection from the poll snapshot is impossible.
|
||||
|
||||
**Claim (PRD FR-2.1):** "poll `relay /search` … build a current `{uuid →
|
||||
name/img/folder}` snapshot."
|
||||
|
||||
**Claim (PRD FR-2.2):** "moved (folder change)" is detected by diffing the current
|
||||
snapshot against the last snapshot.
|
||||
|
||||
**Reality:** `relay /search?minified=true` returns `{ uuid, id, name, img,
|
||||
documentType }` only (`docs/relay-api.md:55-59`, `src/relay/client.ts:19-25`,
|
||||
`src/relay/client.ts:97-105`). There is **no `folder` field** in the minified result.
|
||||
The PRD's own §2 admits "/search is minified with no content hash" but then FR-2.1
|
||||
quietly assumes `folder` is present — it is not. You cannot build a
|
||||
`{uuid → folder}` snapshot from `/search`, and you cannot detect folder moves by
|
||||
snapshot-diffing alone.
|
||||
|
||||
**Impact:** FR-2.2's "moved (folder change)" detection and FR-3.5's "folder moves
|
||||
surface as changes/conflicts" are **not implementable as specified**. Folder moves
|
||||
would be invisible to the F→O poll entirely.
|
||||
|
||||
**Fix:** Pick one and write it into the PRD:
|
||||
1. **Per-note `/get` on every poll** to read `folder` (and content). This is the only
|
||||
way to detect moves from the relay at all, and it collapses F2 into "every poll =
|
||||
`/get` per linked note" (see B2). Accept the cost explicitly and override ADR 005.
|
||||
2. **Drop folder-move detection from F2.** Foundry-side renames are detectable
|
||||
(name is in `/search`); folder moves are not. State this as a known gap and route
|
||||
moves to the next manual Sync / Re-pull. This is the cheaper path and is honest
|
||||
about the relay's surface.
|
||||
3. Propose a relay change (non-goal per PRD §3) — not available for this PRD.
|
||||
|
||||
Recommended: **option 2** for this PRD, with option 1 as an explicit opt-in
|
||||
"deep poll" mode. Either way FR-2.1 and FR-2.2 must be rewritten to drop `folder`
|
||||
from the `/search`-derived snapshot.
|
||||
|
||||
---
|
||||
|
||||
### B2 — FR-2.2: content-change detection requires a `/get` per linked note per poll. The PRD does not account for the load, and this directly contradicts ADR 005.
|
||||
|
||||
**Claim (PRD FR-2.2):** "content-changed (detected via `/get` hash compare)."
|
||||
|
||||
**Reality:** `/search` minified has no content and no content hash
|
||||
(`docs/relay-api.md:55-59`). The only way to detect a Foundry-side content edit is to
|
||||
`/get` the full entry and hash it. So "detect content changes" = " `/get` every
|
||||
linked note every poll cycle." For the ~200-entry world in ADR 005 at the PRD's
|
||||
"seconds" default cadence (FR-2.6, OQ-3), this is **thousands of `/get` calls/min**,
|
||||
all funneled through one headless Foundry WebSocket. ADR 005 rejected exactly this
|
||||
design at 15s/800-calls-min as "heavy and fragile."
|
||||
|
||||
**Impact:** NFR-3 ("does not overload the relay or the host") is **unsatisfiable** at
|
||||
the cadence the PRD hints at, with the design the PRD specifies. The PRD cannot
|
||||
both (a) detect Foundry content changes automatically and (b) avoid per-note `/get`
|
||||
load. One has to give.
|
||||
|
||||
**Fix:** Make the F→O design explicit and costed in the PRD:
|
||||
- **Default: shallow poll.** `/search` only → detect **renames + new + missing**
|
||||
(all detectable from minified `/search`). Content changes and folder moves are
|
||||
**not** auto-detected; they surface on the next manual Sync / Re-pull or on a
|
||||
user-triggered "catch up now" deep sweep (FR-2.6 already has the trigger — promote
|
||||
it to the only content-change path).
|
||||
- **Opt-in deep poll:** a `/get`-per-linked-note sweep, cadence in **minutes**, not
|
||||
seconds, with a concurrency cap (the existing `mapPool` at `src/server.ts:317` is
|
||||
the right primitive). State the call-rate budget in NFR-3.
|
||||
- Add a line to §6 acknowledging this supersedes ADR 005's "F→O stays manual"
|
||||
conclusion **only for rename/new/missing**, not for content.
|
||||
|
||||
Without this, F2 is not shippable as written.
|
||||
|
||||
---
|
||||
|
||||
## HIGH
|
||||
|
||||
### H1 — FR-1.4 / NFR-1: the "Foundry-side hash over content + name + folder" is a brand-new function, not derivable from `contentHash`. The PRD under-specifies it.
|
||||
|
||||
**Claim (PRD FR-1.4):** compute a Foundry-side hash over "content + name +
|
||||
folder_path" from the `/get` response and compare to a new `foundry.ccHash` baseline.
|
||||
|
||||
**Reality:**
|
||||
- `contentHash` (`src/normalize.ts:24`) hashes **canonicalized Obsidian markdown
|
||||
body only**. It cannot be reused for the Foundry side — the Foundry `/get` response
|
||||
stores content inside `flags["campaign-codex"].data` (HTML per
|
||||
`obsidianToFoundryJsonLive`), not as refined markdown. Hashing the raw HTML is
|
||||
fragile (Foundry may round-trip HTML attributes/whitespace); hashing requires
|
||||
either (a) a stable HTML canonicalizer or (b) converting `cc.data` HTML back to
|
||||
refined markdown and reusing `contentHash` — which is a non-trivial round-trip
|
||||
through `entryToObsidian`-style logic that the PRD does not mention.
|
||||
- `name` and `folder` are present on `/get` (`docs/relay-api.md:31-33`) and are
|
||||
stable scalars — hashing those is trivial.
|
||||
- The hard part is **content**. The PRD says "content + name + folder_path" as if
|
||||
it's obvious; it is not. The PRD needs to specify the exact Foundry-side hash
|
||||
input: is it `cc.data` raw, canonicalized HTML, or re-converted markdown? Each
|
||||
has different drift properties.
|
||||
|
||||
**Impact:** If the Foundry-side hash is unstable (e.g. Foundry re-serializes HTML
|
||||
between pushes), the divergence guard false-positives every tick → every save becomes
|
||||
a conflict → the tool is unusable. This is the single most likely delivery failure.
|
||||
|
||||
**Fix:** Add a subsection to FR-1.4 specifying the hash:
|
||||
- Input: `canonicalize(htmlToMarkdown(flags["campaign-codex"].data)) + "\n" + name +
|
||||
"\n" + folder` — i.e. reuse the existing `contentHash` pipeline by converting the
|
||||
Foundry HTML body back to refined markdown first (the inverse of
|
||||
`obsidianToFoundryJsonLive`). This makes the Foundry-side hash and the Obsidian
|
||||
body hash **directly comparable** (same canonical form), which also makes the 2×2
|
||||
routing in F3 well-defined.
|
||||
- Specify the HTML→markdown conversion (linkedom is already a dep per the wiki;
|
||||
confirm it round-trips stably). Add a unit test for hash stability across a
|
||||
push→/get round-trip before shipping FR-1.4.
|
||||
|
||||
Also: `baselineFoundryBlock` (`src/server.ts:289`) and `baselineNote`
|
||||
(`src/server.ts:307`) currently touch only `contentHash`/`syncedAt`. FR-1.7 requires
|
||||
baseline of **both** `contentHash` and `ccHash` — both functions need a `ccHash:`
|
||||
line added to the rewrite, and `frontmatter.ts`'s `readFoundryBlock` consumers need
|
||||
to read it. Call this out as a concrete code change in the PRD.
|
||||
|
||||
---
|
||||
|
||||
### H2 — FR-1.6 fail-safe: the PRD correctly identifies the gap, but the fix has a subtle hole — the `/get`-then-`/update` race window.
|
||||
|
||||
**Confirmed:** current `process()` (`src/server.ts:582-617`) does **not** `/get` to
|
||||
compare a Foundry-side hash before pushing. It pushes on Obsidian-body-hash-diff
|
||||
only. `pushNote`'s internal `/get` (`src/push.ts:142`) builds the diff but does not
|
||||
gate the push. So NFR-1 is **violated today** — the PRD is right to call this out.
|
||||
|
||||
**Subtle hole in the proposed fix:** even with FR-1.4's "`/get` then compare ccHash
|
||||
then push" sequence, there is a TOCTOU window between the `/get` (read F-side
|
||||
baseline) and the `/update` (write). If a Foundry edit lands in that window, the
|
||||
push still clobbers it. The window is small (one relay round-trip) but nonzero.
|
||||
|
||||
**Impact:** Low probability, but it's the same class of bug the PRD is trying to
|
||||
eliminate. Worth a defensive measure.
|
||||
|
||||
**Fix:** After `pushNote`'s `relay.updateEntry` succeeds, re-`/get` and verify the
|
||||
entry's content hash matches what we just wrote; if it diverges (concurrent edit
|
||||
landed mid-flight), surface a conflict row instead of baselining. Cheaper
|
||||
alternative: document the window as accepted residual risk in NFR-1. Either is
|
||||
fine; the PRD should pick one explicitly.
|
||||
|
||||
---
|
||||
|
||||
### H3 — F3 routing: the O→F `inflight` set and the F→O poll do not share a lock. Concurrent O-save + F-poll on the same note can double-write.
|
||||
|
||||
**Reality:** `AutoSyncController.inflight` (`src/server.ts:463`, used at
|
||||
`src/server.ts:583`) is keyed by **relPath** and only guards the O→F watcher path.
|
||||
The proposed F→O poll is a separate loop with no shared lock. If an O-save and an
|
||||
F-poll fire on the same uuid in the same tick:
|
||||
- O→F sees O-changed, F-unchanged → pushes O→F.
|
||||
- F→O sees F-changed (from the just-pushed O content? or a real F edit?), O-unchanged
|
||||
(pre-save snapshot) → pulls F→O.
|
||||
Result depends on timing and can oscillate or clobber.
|
||||
|
||||
**Impact:** NFR-1 ("never clobber") can be violated under concurrent cross-direction
|
||||
activity on the same note. Prep bursts are exactly this pattern.
|
||||
|
||||
**Fix:** Add a **per-uuid lock** shared between the watcher path and the poll path
|
||||
(not per-relPath). While a sync op is in flight for a uuid, the other direction
|
||||
queues/skips. Add to FR-3.1 / FR-5.5. Also: the debounce window
|
||||
(`src/server.ts:567`, 700ms) is O-side only; an F edit arriving during the debounce
|
||||
is invisible to the watcher. The `/get` at push time (FR-1.4) is what catches it —
|
||||
confirm FR-1.4 is evaluated **after** debounce drains, not on the raw save event.
|
||||
|
||||
---
|
||||
|
||||
## MEDIUM
|
||||
|
||||
### M1 — FR-1.4: one extra `/get` per changed note is acceptable, but the PRD should state the cost and cap it.
|
||||
|
||||
One additional `/get` per push (on top of the `/get` `pushNote` already does at
|
||||
`src/push.ts:142`) doubles relay reads per O→F op. At the existing concurrency=3
|
||||
(`src/server.ts:466`) and debounce=700ms (`src/server.ts:467`) this is fine for a
|
||||
prep burst. But **reuse the `/get` `pushNote` already makes** — don't do two `/get`s.
|
||||
The cleanest implementation is to have `pushNote` (or a new wrapper) compute the F
|
||||
hash from the `liveEntry` it already fetched, compare to `ccHash` baseline, and
|
||||
gate the `/update` on that. The PRD reads as if FR-1.4 adds a **separate** `/get`
|
||||
before the push — that would be wasteful. Clarify: "the `/get` that `pushNote`
|
||||
already performs is reused for the F-side hash compare; no extra round-trip."
|
||||
|
||||
### M2 — FR-2.5 new-entry detection: feasible, but `img` is the only extra signal beyond name+uuid.
|
||||
|
||||
`/search` minified gives `{uuid, id, name, img, documentType}`. For new Foundry
|
||||
JournalEntries, `name + uuid` is enough to surface as import candidates (the
|
||||
existing import row at `src/server.ts:182-187` and `importRow` already works off
|
||||
`row.entry`). **Feasible.** One caveat: the dashboard's `ccOnly` pool
|
||||
(`src/server.ts:154`) is built from the **LevelDB journal snapshot** via `indexAll`,
|
||||
not from live `/search`. So "new entries surface as import candidates" requires
|
||||
either (a) merging live `/search` results into the index's `ccOnly` pool, or (b) a
|
||||
separate "live new entries" list in the dashboard. The PRD doesn't specify which.
|
||||
Recommend (b) to avoid conflating the static LevelDB index with live relay state.
|
||||
Add to FR-2.5.
|
||||
|
||||
### M3 — FR-1.7 dev-mode baseline landing: `baselineNote` writes via `targetPath` (mirror in dev, real vault in apply) — correct, but the F→O pull writes the **vault** file and must use the same path resolution.
|
||||
|
||||
`resolveRefined` (`src/server.ts:85`) prefers the dev mirror if it's newer, else the
|
||||
real vault. The F→O pull (FR-2.3) writes into the vault; in dev mode it must write
|
||||
to the mirror (`targetPath(state, "refined", rel)`), not the real vault, or dev
|
||||
mode stops being a safe preview. The PRD says "Dev mode baselines land in the
|
||||
`--out` mirror" (FR-1.7) — good — but FR-2.3 doesn't repeat the constraint for the
|
||||
**pulled content**. Add it.
|
||||
|
||||
### M4 — FR-4.3 `Sync Status.md` watcher exclusion: the watcher's exclusion list today is `.obsidian` + dotfiles only (`src/server.ts:561`, `src/server.ts:533`).
|
||||
|
||||
`onChange` skips `.obsidian` and dotfiles (`src/server.ts:561`). A `Sync Status.md`
|
||||
at the vault root is **not** a dotfile and **not** in `.obsidian`, so the watcher
|
||||
will fire on every status-note write → auto-sync will try to push it (it has no
|
||||
`foundry.cc_uuid` → skipped at `src/server.ts:591`, but still generates noise and a
|
||||
`/get`-less skip). The PRD (OQ-4) flags the path as TBD but doesn't note that the
|
||||
exclusion must be **added to `onChange`** explicitly. Add a concrete code-change
|
||||
note to FR-4.3: extend the skip predicate at `src/server.ts:561` to include the
|
||||
status-note path.
|
||||
|
||||
---
|
||||
|
||||
## LOW
|
||||
|
||||
### L1 — FR-6.3 client-list discovery: the relay returns the client list on `>1` connected clients with no clientId, and "No connected clients" on 0 (`docs/relay-api.md:13-14`). With exactly 1, the relay auto-resolves and returns the doc — **no client list is surfaced**. So FR-6.3's "list connected relay clients" works only when there are 0 or >1 clients. The single-client case (the most common during prep) returns no list. The PRD should note this: when exactly one client is connected, the dashboard should treat that as "clientId auto-resolved, no pick needed" rather than showing an empty list.
|
||||
|
||||
### L2 — FR-5.3 retry on 404: a 404 from the relay is **not always** transient. `404 {"error":"No connected Foundry clients found"}` (`docs/relay-api.md:13`) is a **persistent** condition (Foundry down) — retrying with backoff just delays the error. The PRD's FR-5.3 lists "404 invalid client" as transient-retryable; distinguish "invalid clientId" (retryable after re-fetching the client list) from "no connected clients" (persistent, surface immediately). Update FR-5.3.
|
||||
|
||||
### L3 — FR-1.2 / FR-1.3 skip semantics: `process()` skips on no `cc_uuid` or no `contentHash` (`src/server.ts:591-592`) and on body-hash-equal (`src/server.ts:594`). All three are correct and already implemented. The PRD's FR-1.2/1.3 matches the code. No change needed; noted for traceability.
|
||||
|
||||
### L4 — `foundry.ccHash` naming: the existing `foundry:` block uses snake_case-ish `cc_uuid`, `cc_type`, `folder_path`, `contentHash` (mixed). `ccHash` is fine but consider `cc_content_hash` or `foundryHash` for grep-ability and consistency with `cc_uuid`/`cc_type`. Cosmetic; decide during delivery.
|
||||
|
||||
---
|
||||
|
||||
## Overall verdict
|
||||
|
||||
**Conditional-go.** The PRD's O→F hardening (F1, FR-1.4–1.7) is feasible and the
|
||||
clobber-risk diagnosis is accurate, but the Foundry-side hash (H1) is under-specified
|
||||
and is the most likely thing to derail delivery. The F→O direction (F2) has **two
|
||||
blockers**: the PRD assumes `/search` returns `folder` (it does not — B1) and assumes
|
||||
content-change detection is cheap (it requires a `/get` per linked note per poll,
|
||||
which ADR 005 already rejected on cost grounds — B2). F2 as written is not
|
||||
shippable; it needs to be scoped down to "shallow poll: renames + new + missing" with
|
||||
content/folder changes routed to the manual trigger, or it needs an explicit,
|
||||
costed deep-poll mode in minutes not seconds.
|
||||
|
||||
The two-baseline 2×2 routing (F3) is sound in principle but has a real cross-direction
|
||||
lock gap (H3) and a TOCTOU window (H2) that the PRD should address.
|
||||
|
||||
Recommended pre-delivery edits to the PRD:
|
||||
1. Rewrite FR-2.1/2.2 to drop `folder` from the `/search` snapshot (B1) and split
|
||||
shallow vs deep poll (B2).
|
||||
2. Specify the exact Foundry-side hash input and the HTML→markdown reuse (H1).
|
||||
3. Add a shared per-uuid lock to F3/FR-5.5 (H3) and decide on the TOCTOU fix (H2).
|
||||
4. Clarify that FR-1.4 **reuses** the `/get` `pushNote` already makes (M1).
|
||||
5. Note the watcher-exclusion code change for `Sync Status.md` (M4) and the
|
||||
single-client blind spot in FR-6.3 (L1).
|
||||
|
||||
Top findings: **B1** (folder not in `/search`), **B2** (content detection = per-note
|
||||
`/get` every poll, contradicts ADR 005), **H1** (Foundry-side hash under-specified),
|
||||
**H2** (TOCTOU after `/get`), **H3** (no shared cross-direction lock).
|
||||
|
||||
File: `/home/kaykayyali/docker/obsidian-foundry-sync/docs/prds/prd-foundry-obsidian-sync-2026-06-22/review-engineering.md`
|
||||
@@ -0,0 +1,396 @@
|
||||
---
|
||||
title: "Launchability & Onboarding Adversarial Review"
|
||||
review_of: prd-foundry-obsidian-sync-2026-06-22/prd.md
|
||||
reviewer_role: adversarial reviewer (launchability / onboarding)
|
||||
stakes: public/launchable
|
||||
created: 2026-06-22
|
||||
---
|
||||
|
||||
# Launchability & Onboarding Adversarial Review
|
||||
|
||||
**Verdict: NOT launchable as specified.** The PRD sets stakes = "public/launchable" and
|
||||
NFR-6 promises "clone → live sync using the dashboard + `/setup`, without editing shell
|
||||
commands," but the F6 feature set only closes two of roughly seven onboarding gates, and
|
||||
several launchable-grade concerns (dashboard auth, Foundry-side data integrity, error
|
||||
contracts, status-note feedback-loop safety, conflict UX comprehension) are unspecified,
|
||||
hand-waved to "design during delivery," or silently dropped. The PRD reads as a strong
|
||||
*author-hardening* PRD wearing a *launchable* label. Two blockers and four high-severity
|
||||
findings stand between this draft and a non-author DM actually succeeding.
|
||||
|
||||
Cross-references in this review: PRD = `prd.md` in this dir; README =
|
||||
`/home/kaykayyali/docker/obsidian-foundry-sync/README.md`; env =
|
||||
`/home/kaykayyali/docker/obsidian-foundry-sync/.env.example`.
|
||||
|
||||
---
|
||||
|
||||
## BLOCKER
|
||||
|
||||
### B1 — `/setup` is not a product surface; NFR-6 is unmet as written
|
||||
|
||||
**Finding.** FR-6.4 ("The existing `/setup` skill covers env wiring; the dashboard
|
||||
reflects setup state and links into it") and NFR-6 ("A non-author user can get from
|
||||
clone → live sync using the dashboard + `/setup`, without editing shell commands")
|
||||
treat `/setup` as a product feature. It is not. `/setup` is a Claude Code skill — an
|
||||
authoring-time convenience available only to someone running Claude Code against this
|
||||
repo. A non-author DM who clones the repo has no `/setup`. README §"Quick start"
|
||||
confirms the real onboarding path is five shell steps: `cp .env.example .env`,
|
||||
`docker compose up -d relay`, browser signup to copy `RELAY_API_KEY`,
|
||||
`node scripts/start-relay-session.js`, and `./sync.sh refresh` + `./sync.sh ui`.
|
||||
|
||||
The F6 FRs only cover *detection* of missing config (FR-6.1 clientId empty, FR-6.2 no
|
||||
connected client, FR-6.3 list clients). They do not cover *acquisition* of:
|
||||
|
||||
1. The `RELAY_API_KEY` (requires bringing up the relay container + browser signup —
|
||||
shell + browser, not dashboard).
|
||||
2. Starting the headless Foundry session (`scripts/start-relay-session.js` — shell).
|
||||
3. Pointing Foundry's rest-api module at the relay WebSocket URL (Foundry-side admin
|
||||
config — cannot be driven from the dashboard at all).
|
||||
4. Bringing up the relay container itself (`docker compose up -d relay` — shell).
|
||||
5. Installing deps / starting the dashboard (`npm install`, `./sync.sh ui` — shell).
|
||||
|
||||
FR-6.2 detects "Foundry not connected" but the dashboard cannot *start* the headless
|
||||
session that would connect it. So the DM is stuck at a red banner with no in-UI remedy.
|
||||
|
||||
**Why it matters for launchable.** This is the single promise that defines "launchable"
|
||||
for this product, and it is the one the PRD most overstates. A non-author DM following
|
||||
the PRD literally cannot reach live sync without shell, and the PRD does not acknowledge
|
||||
the gap — it papers over it by referencing a tool the end user does not have.
|
||||
|
||||
**Recommended fix.** Pick one of two honest paths and state it in §3 and F6:
|
||||
|
||||
- **(a) Downscope NFR-6** to "clone → live sync using the dashboard, *given a running
|
||||
relay + headless session + REST-API module wired by the operator*," and add a §2
|
||||
"Operator prerequisites" block that enumerates the shell/Foundry steps the DM (or
|
||||
their operator) must complete once. This matches reality and is still a real
|
||||
improvement over today.
|
||||
- **(b) Actually build the in-UI onboarding** for the five gates above: a dashboard
|
||||
"first run" wizard that brings up the relay container via a host-side helper, guides
|
||||
API-key creation (deep-link to the relay's signup page, paste key into a form),
|
||||
launches the headless session via a server endpoint (not a shell command the user
|
||||
types), and shows step-by-step instructions for the Foundry-side rest-api module
|
||||
wiring with a connectivity check. Then NFR-6 stands. This is substantially more
|
||||
delivery work than F6 currently implies and must be sized as such.
|
||||
|
||||
Either way, drop `/setup` from the FRs and NFR-6 — it is an author tool, not a product
|
||||
surface. If (b) is chosen, add FR-6.5..6.8 for the four remaining gates.
|
||||
|
||||
---
|
||||
|
||||
### B2 — Dashboard binds `0.0.0.0:7788` with no auth; security is absent from the PRD
|
||||
|
||||
**Finding.** README §"Batch dashboard" confirms the dashboard "binds 0.0.0.0 by default
|
||||
so it's reachable on your tailnet at this VM's IP, port 7788; pass `--host 127.0.0.1` to
|
||||
restrict to localhost." The PRD has **no security section**. There is no FR or NFR for
|
||||
authentication, authorization, transport security, or access control on the dashboard.
|
||||
The dashboard can push to a live Foundry world (`/api/push`, `/api/push-all`), read and
|
||||
write the vault on disk, and presumably surfaces the relay API key in some form. With
|
||||
auto-sync (F1) the dashboard also drives automatic live writes to Foundry. An
|
||||
unauthenticated `0.0.0.0` listener that can overwrite a live Foundry world is a
|
||||
launchable-grade security hole.
|
||||
|
||||
The PRD also does not discuss: storage of `RELAY_API_KEY` and `RELAY_PASSWORD` (plaintext
|
||||
in `.env` per env.example), whether the dashboard ever reflects secrets to the browser,
|
||||
CSRF on POST endpoints, or what happens when the dashboard is exposed on a tailnet/shared
|
||||
network.
|
||||
|
||||
**Why it matters for launchable.** A launchable product that can clobber a live Foundry
|
||||
world over an open port with no auth is irresponsible to ship. "Bind localhost" is a
|
||||
mitigation the README mentions but the PRD never requires; the default is `0.0.0.0`.
|
||||
|
||||
**Recommended fix.** Add a §5 NFR-9 — Security, and a F7 — Security & access control
|
||||
feature group, minimum:
|
||||
|
||||
- **FR-7.1** Dashboard authenticates by default (token or password set via env or
|
||||
first-run prompt); unauthenticated requests get 401, not the UI.
|
||||
- **FR-7.2** Default bind changes to `127.0.0.1`; `0.0.0.0` requires explicit opt-in
|
||||
*and* an auth token set (refuse to start otherwise).
|
||||
- **FR-7.3** Secrets (`RELAY_API_KEY`, `RELAY_PASSWORD`) are never rendered to the
|
||||
browser; the dashboard shows only masked presence/absence.
|
||||
- **FR-7.4** POST mutation endpoints require a CSRF token or same-origin check.
|
||||
- **NFR-9** No unauthenticated mutation path; no secret egress to the client; TLS
|
||||
recommended when bound beyond localhost.
|
||||
|
||||
---
|
||||
|
||||
## HIGH
|
||||
|
||||
### H1 — Error contracts are unspecified; "surface an error row" is not actionable for a non-author DM
|
||||
|
||||
**Finding.** FR-1.6, FR-5.3, NFR-2, NFR-4 all reference "error rows" / "surfaced" but
|
||||
none specify the error *vocabulary*, *messages*, or *remediation guidance*. The relay
|
||||
returns at least four distinct failure modes that a non-author DM cannot distinguish
|
||||
from a raw status code: relay down (network), session down (relay up but no Foundry
|
||||
client), clientId invalid (relay up, client up, wrong/empty id), hash-mismatch (conflict,
|
||||
not an error), and transient 404/408/504 (retryable). FR-5.3 collapses these to
|
||||
"transient relay errors (404 invalid client, 408/504 timeout, 5xx)" — but "404 invalid
|
||||
client" is *not* transient (a wrong clientId will 404 forever), and "session down" is
|
||||
not in the retry list at all. The mapping from relay response → user-facing message →
|
||||
recommended action is absent.
|
||||
|
||||
**Why it matters for launchable.** NFR-5 promises "no silent skips," but a red row that
|
||||
says `Relay 404` is silent in the sense that matters: the DM does not know whether to
|
||||
fix `.env`, restart the headless session, or pick a different clientId. The author can
|
||||
debug from a status code; a non-author DM cannot.
|
||||
|
||||
**Recommended fix.** Add a §5a "Error contracts" table mapping each failure mode to:
|
||||
(a) detection signal, (b) retry-or-not, (c) the exact user-facing message, (d) the
|
||||
remediation hint shown. Minimum rows: relay unreachable, relay 401 (bad API key),
|
||||
clientId invalid/empty, no connected Foundry client, session idle-reaped, hash mismatch
|
||||
(route to F3, not error),Foundry-side /get 404 on a known uuid (deleted on Foundry),
|
||||
and persistent 5xx after backoff. Specify that error rows include a one-line "what to
|
||||
do" string, not just a status. Split FR-5.3's retry list into *transient* (timeout, 5xx,
|
||||
session-temporarily-unavailable) vs *persistent* (404 invalid client, 401 bad key) with
|
||||
no retry on persistent.
|
||||
|
||||
### H2 — `Sync Status.md` exclusion is not airtight; feedback-loop risk is unaddressed
|
||||
|
||||
**Finding.** FR-4.3 maintains `Sync Status.md` in the vault, "on a path excluded from
|
||||
the watcher, never synced to Foundry, updated each tick." OQ-4 defers the path/exclusion
|
||||
mechanism to delivery. The vault is what the watcher watches (FR-1.1). If the exclusion
|
||||
is path/filename-based and the DM renames or moves the note, or the tool's path config
|
||||
drifts, the watcher fires on every tick's status-note write → auto-sync attempts to push
|
||||
a `foundry:`-less note (skipped by FR-1.2, but it still churns the activity panel and
|
||||
wakes the pipeline) or, worse, a future version seeds it and a feedback loop starts.
|
||||
The PRD does not specify: the exclusion *mechanism* (glob? exact path? frontmatter flag?
|
||||
a dedicated subdir?), what guarantees it across renames, whether the F→O poll also skips
|
||||
it (FR-2.x doesn't mention it), and what happens on a name collision with a real note.
|
||||
|
||||
**Why it matters for launchable.** A status note that triggers sync is a self-inflicted
|
||||
loop that a non-author DM will not diagnose. "Excluded" stated as a requirement without
|
||||
a mechanism is a bug waiting to ship.
|
||||
|
||||
**Recommended fix.** Promote OQ-4 to a specification: define the exclusion as a
|
||||
*dedicated, reserved path* (e.g. `${VAULT}/.sync-status.md` or a dot-prefixed file that
|
||||
FR-1.1's "skip dotfiles" rule already covers), AND a frontmatter sentinel (e.g.
|
||||
`foundry.sync_status: true`) that both the O→F watcher and the F→O poll check and skip
|
||||
on, so the exclusion holds even if the path changes. Add FR-4.6: "The status note is
|
||||
excluded from both O→F watch and F→O poll by *both* a path rule and a content sentinel;
|
||||
a status note that loses its sentinel is treated as user error and surfaced, not
|
||||
synced." Add an explicit test to NFR-5: status-note writes never produce a sync op.
|
||||
|
||||
### H3 — Conflict UX is unspecified; "mark resolved (no change)" is a data-loss footgun for a non-author DM
|
||||
|
||||
**Finding.** F3 routes both-diverged to a conflict row with "push vault → Foundry",
|
||||
"pull Foundry → vault", "mark resolved (no change)" (FR-3.3). OQ-6 defers diff format
|
||||
to delivery. Two problems:
|
||||
|
||||
1. **Diff format is unspecified.** "Summarizing both versions and highlighting the diff"
|
||||
(FR-3.2) could be a unified diff, a side-by-side, or a condensed summary — three very
|
||||
different UX outcomes for a non-author DM staring at a conflict.
|
||||
2. **"Mark resolved (no change)" is genuinely ambiguous.** Its actual semantic (per
|
||||
FR-3.1 + the no-clobber rule) is presumably "accept the current diverged state as the
|
||||
new parity baseline without transferring content in either direction" — i.e. *both*
|
||||
sides keep their diverged content and both hashes re-baseline to current values. A
|
||||
DM reading "mark resolved (no change)" will reasonably think "do nothing, come back
|
||||
later" or "keep both" — and may click it expecting safety, then discover later that
|
||||
the two sides now permanently disagree and neither has the other's edits. There is no
|
||||
FR specifying what "mark resolved" actually does to the baselines, whether the
|
||||
divergence is preserved or collapsed, or what the DM is told just happened.
|
||||
|
||||
**Why it matters for launchable.** This is the one screen where the DM makes an
|
||||
irreversible content decision. For the author it is obvious; for a non-author DM it is
|
||||
the screen most likely to lose work. "Design during delivery" is not a launchable
|
||||
specification for the highest-stakes UX in the product.
|
||||
|
||||
**Recommended fix.** Resolve OQ-6 in the PRD, not in delivery. Specify:
|
||||
|
||||
- **FR-3.6** Conflict diff format: side-by-side with a one-line plain-language summary
|
||||
("Vault adds 3 paragraphs about X; Foundry renamed to Y and changed folder to Z"),
|
||||
not a raw unified diff.
|
||||
- **FR-3.7** "Mark resolved" is renamed to an explicit label — e.g. "Accept both as-is
|
||||
(keep divergence)" — with a confirmation dialog that states in plain text: "The vault
|
||||
and Foundry will keep their current versions. They will be treated as in-sync from
|
||||
now on. Neither side's changes will be copied to the other."
|
||||
- **FR-3.8** Each conflict action states, before commit, what it will do to each side
|
||||
and to the baselines (one-line preview). No irreversible action without a confirm.
|
||||
- **FR-3.9** A resolved conflict produces a visible activity-panel entry stating which
|
||||
side won and that the other side's edits were *not* transferred.
|
||||
|
||||
### H4 — Foundry-side data integrity has no backup/recovery story; auto-sync writes live with no Foundry-side undo
|
||||
|
||||
**Finding.** FR-1.8 states "Auto-sync always applies live (dry-run not honored)." Manual
|
||||
`--apply` mode writes a timestamped `.bak-<iso>` of vault files it overwrites (README
|
||||
§"Modes"). But push to Foundry goes through the relay `/update`, which overwrites the
|
||||
live Foundry entry — there is no Foundry-side `.bak`, no undo, no rollback specified.
|
||||
For a non-author DM who auto-syncs a bad edit (or hits a bug, or a hash race), the
|
||||
Foundry world is the side with no recovery path. The PRD has no §"Data integrity" or
|
||||
backups section. The vault side has `.bak`; the Foundry side has nothing.
|
||||
|
||||
**Why it matters for launchable.** A launchable sync tool that can overwrite a live
|
||||
Foundry world automatically, with no specified recovery path on the overwritten side,
|
||||
is a data-loss hazard. The author can restore from a world backup; a non-author DM may
|
||||
not have one scheduled.
|
||||
|
||||
**Recommended fix.** Add a §5 NFR-10 — Data integrity, minimum:
|
||||
|
||||
- **FR-5.6** Before any auto or manual push to Foundry, the prior Foundry-side content
|
||||
is fetched and cached locally (a `foundry-backups/<uuid>/<iso>.json` dir) so a bad
|
||||
push can be reversed from the dashboard. Specify retention (e.g. last N per uuid).
|
||||
- **FR-5.7** A "Revert last push" action exists for the most recent push per note.
|
||||
- **NFR-10** Foundry-side overwrites are always preceded by a local backup of the
|
||||
pre-push Foundry state; the dashboard exposes a restore path.
|
||||
- Document in §2 that Foundry-side world backups (Foundry's own backup feature) remain
|
||||
the DM's responsibility and are recommended before enabling auto-sync.
|
||||
|
||||
---
|
||||
|
||||
## MEDIUM
|
||||
|
||||
### M1 — Performance NFR is sized to the author's vault, not real-world vaults
|
||||
|
||||
**Finding.** NFR-3 bounds the burst target at "~50-note prep burst" with no stated basis.
|
||||
FR-2.1 polls `/search` for all `JournalEntry` minified on every tick — on a large world
|
||||
(hundreds or thousands of JournalEntries) this is a full scan per poll. No vault-size
|
||||
assumption is stated, no relay-load ceiling, no concurrency default numbers (FR-5.2
|
||||
"defaults tuned for prep" — what numbers?). The `RELAY_URL=http://localhost:3010`
|
||||
topology in `.env.example` reflects the author's all-on-one-host setup; the PRD does
|
||||
not acknowledge that a DM with remote Foundry has higher latency and a different
|
||||
retry/backoff profile.
|
||||
|
||||
**Recommended fix.** State the operating envelope in NFR-3: tested vault size range,
|
||||
JournalEntry count range the poll is validated against, default poll cadence (resolve
|
||||
OQ-3 with a number + rationale), default debounce and max-concurrency numbers, and the
|
||||
relay-load ceiling (concurrent `/update` calls). Add NFR-3a: "F→O poll and O→F push
|
||||
paths are validated against a vault of at least N notes and M JournalEntries," with N
|
||||
and M chosen above the author's own size.
|
||||
|
||||
### M2 — Auto-sync in dev vs apply mode is contradictory
|
||||
|
||||
**Finding.** FR-1.7 says "Dev mode baselines land in the `--out` mirror; apply mode in
|
||||
the real vault with a `.bak`." FR-1.8 says "Auto-sync always applies live (dry-run not
|
||||
honored)." These interact unclearly: can auto-sync be enabled in dev mode at all? If
|
||||
yes, "applies live" to what — the `--out` mirror or the real vault? If auto-sync only
|
||||
makes sense in apply mode, say so and gate the toggle on `--apply`. If it can run in dev
|
||||
mode, define what "live" means there. A non-author DM who starts the dashboard in the
|
||||
default dev mode and flips on auto-sync will not know what is being written where.
|
||||
|
||||
**Recommended fix.** Add FR-1.9: "Auto-sync requires apply mode; enabling it in dev mode
|
||||
is either blocked with an explanatory banner or explicitly writes to `--out` only (pick
|
||||
one and state it)." Reconcile FR-1.7/1.8 accordingly.
|
||||
|
||||
### M3 — Status state single-source-of-truth has no persistence story across restart
|
||||
|
||||
**Finding.** FR-4.5 says dashboard parity and the vault status note reflect "one
|
||||
underlying state (single source of truth for status)." The PRD doesn't say where that
|
||||
state lives. If it is in-memory in the server, a restart loses parity counts, conflict
|
||||
state (FR-3.4 "persists until resolved"), and last-sync time — but the vault status note
|
||||
persists. On restart, does the dashboard re-derive parity from a full re-scan, or does
|
||||
it show stale/empty until the next tick? Conflict persistence (FR-3.4) is especially
|
||||
load-bearing: a restart that drops conflict state could let a re-save auto-clear a known
|
||||
conflict, violating FR-3.4.
|
||||
|
||||
**Recommended fix.** Specify the status store: either (a) a small persisted state file
|
||||
on disk (`sync-state.json`) that survives restart and is the single source the dashboard
|
||||
and status note both read from, or (b) re-derive on restart with an explicit "recomputing
|
||||
parity…" state. Add FR-3.4a: "Conflict state persists across server restarts, not just
|
||||
across ticks."
|
||||
|
||||
### M4 — Versioning, updates, and frontmatter schema migration are unaddressed
|
||||
|
||||
**Finding.** The PRD introduces a new frontmatter field `foundry.ccHash` (FR-1.4) and
|
||||
relies on existing `foundry.contentHash`/`cc_uuid`/`folder_path`. There is no schema
|
||||
version field on the `foundry:` block and no migration story. A non-author DM who
|
||||
updates the tool to a later version that changes hash algorithm or adds fields has no
|
||||
specified upgrade path — old baselines will mismatch and every note may route to
|
||||
conflict on first sync after upgrade. No §"Versioning & updates" section.
|
||||
|
||||
**Recommended fix.** Add `foundry.schema_version` (or similar) to the identity block and
|
||||
an FR in F1/F5: on startup, notes with a missing/older schema version are migrated
|
||||
(re-hash, re-baseline) in a single pass before auto-sync is allowed to engage, with a
|
||||
dashboard migration banner. Add a §5 NFR-11 — Upgrades: "A version bump that changes
|
||||
hashing or identity fields ships with an idempotent migration; auto-sync stays disabled
|
||||
until migration completes."
|
||||
|
||||
### M5 — Supportability: no persistent log / diagnostics export
|
||||
|
||||
**Finding.** NFR-5 mandates observability via the activity panel and status note, but
|
||||
FR-5.4 says the panel is "capped and scrollable" — i.e. not persistent. A non-author DM
|
||||
who hits a problem and asks for help has no log file to share. There is no FR for a
|
||||
persistent log, a diagnostics export, or a "copy debug info" action. For a launchable
|
||||
product this is the difference between "I can help you" and "I can't see what happened."
|
||||
|
||||
**Recommended fix.** Add FR-5.7 (renumber from H4's FR-5.7): "All auto-sync ops
|
||||
additionally append to a persistent, rotated log file on disk
|
||||
(`logs/sync-<date>.log`)." Add FR-5.8: "A 'Copy diagnostics' dashboard action bundles
|
||||
recent log tail, current config (secrets redacted), parity counts, and relay/clientId
|
||||
status into a single redacted blob for support." NFR-5a: "Operation history is
|
||||
persistent across restarts, not just visible in-session."
|
||||
|
||||
---
|
||||
|
||||
## LOW
|
||||
|
||||
### L1 — Polling load on shared relays is a courtesy concern
|
||||
|
||||
**Finding.** FR-2.1 polls `/search` on a cadence for every DM running the tool. If the
|
||||
relay is a shared instance (multiple DMs), uncoordinated polling multiplies. The PRD
|
||||
doesn't note this as a consideration or suggest a jittered/backoff poll.
|
||||
|
||||
**Recommended fix.** Add jitter to the poll cadence and a note in §2 that on shared
|
||||
relays the cadence should be raised. Non-blocking.
|
||||
|
||||
### L2 — New (cc-only) Foundry entries during prep: candidate vs auto-import is an open question with launchability impact
|
||||
|
||||
**Finding.** OQ-5 leaves "auto-import new Foundry entries" open with current spec =
|
||||
candidates only. For a non-author DM, "import candidates" with no further guidance may
|
||||
mean a growing pile of unimported rows they don't know what to do with.
|
||||
|
||||
**Recommended fix.** Resolve OQ-5 in the PRD: keep candidates-only (safe) but specify
|
||||
that the candidate row has a one-click "Import as new refined note" action with a clear
|
||||
explanation of what import does. The current import row exists per README; confirm it is
|
||||
comprehensible to a non-author DM.
|
||||
|
||||
### L3 — "Foundry-wins" tie-breaker for both-diverged is deferred but conflict UX depends on it
|
||||
|
||||
**Finding.** §1 confirms "Foundry is source of truth" is demoted to a tie-breaker for
|
||||
both-diverged, "to be decided: newest-wins, Foundry-wins, or manual merge." F3 then
|
||||
specifies manual resolution via buttons, which makes the tie-breaker moot *if* the DM
|
||||
always resolves manually. But the conflict row needs a *default ordering* (which side is
|
||||
shown on the left, which action is highlighted) and that ordering embeds a tie-breaker
|
||||
opinion. Leaving it undecided leaks into UX.
|
||||
|
||||
**Recommended fix.** Pick a neutral default (e.g. vault on left, Foundry on right, no
|
||||
pre-highlighted action) so the undecided tie-breaker doesn't bias the DM. State it in F3.
|
||||
|
||||
### L4 — README still describes Foundry as "source of truth"; PRD demotes it
|
||||
|
||||
**Finding.** README L17 says "Foundry is the source of truth." The PRD §1 demotes that
|
||||
to a tie-breaker. For a launchable product the README is the onboarding doc and the PRD
|
||||
is the spec; a non-author DM reading both will be confused about which side wins.
|
||||
|
||||
**Recommended fix.** Note in §6 / §"Out of Scope" that README wording will be updated to
|
||||
match the demoted-source-of-truth model before launch. Track as a doc task.
|
||||
|
||||
---
|
||||
|
||||
## Summary table
|
||||
|
||||
| ID | Tier | Theme | One-line |
|
||||
|-----|---------|------------------------|--------------------------------------------------------------------------|
|
||||
| B1 | BLOCKER | Onboarding | `/setup` is not a product surface; NFR-6 unmet as written |
|
||||
| B2 | BLOCKER | Security | `0.0.0.0:7788` dashboard, no auth, no security section |
|
||||
| H1 | HIGH | Error contracts | Error vocabulary/messages/remediation unspecified |
|
||||
| H2 | HIGH | Status-note loop | `Sync Status.md` exclusion mechanism not airtight |
|
||||
| H3 | HIGH | Conflict UX | Diff format + "mark resolved" semantics unspecified; data-loss footgun |
|
||||
| H4 | HIGH | Data integrity | Auto-sync writes Foundry live with no Foundry-side backup/undo |
|
||||
| M1 | MEDIUM | Perf sizing | "~50-note burst" not sized to real vaults; no relay-load ceiling |
|
||||
| M2 | MEDIUM | Dev/apply vs auto-sync | FR-1.7/1.8 interaction contradictory |
|
||||
| M3 | MEDIUM | Status persistence | Single-source-of-truth state has no restart story; conflict state at risk|
|
||||
| M4 | MEDIUM | Versioning | No schema version / migration story for `foundry:` block |
|
||||
| M5 | MEDIUM | Supportability | No persistent log or diagnostics export |
|
||||
| L1 | LOW | Poll courtesy | Jittered poll on shared relays |
|
||||
| L2 | LOW | Import candidates UX | OQ-5 candidate UX needs comprehensibility check |
|
||||
| L3 | LOW | Tie-breaker default | Undecided tie-breaker leaks into conflict-row ordering |
|
||||
| L4 | LOW | Doc drift | README "source of truth" contradicts PRD demotion |
|
||||
|
||||
---
|
||||
|
||||
## Overall recommendation
|
||||
|
||||
**Do not mark this PRD ready for launchable delivery** until B1 and B2 are resolved
|
||||
(either by honest downscoping of NFR-6 or by committing to build the in-UI onboarding
|
||||
and auth) and H1–H4 are specified concretely enough to implement against. As a
|
||||
*hardening* PRD for the single author it is solid; as a *launchable* PRD it currently
|
||||
treats onboarding, security, error contracts, status-loop safety, conflict UX, and
|
||||
Foundry-side data integrity as "design during delivery" — exactly the surfaces where a
|
||||
non-author DM falls off.
|
||||
192
docs/prds/prd-foundry-obsidian-sync-2026-06-22/review-rubric.md
Normal file
192
docs/prds/prd-foundry-obsidian-sync-2026-06-22/review-rubric.md
Normal file
@@ -0,0 +1,192 @@
|
||||
# PRD Quality Review — Live Relay Sync — Auto-Sync & Bidirectional Hardening
|
||||
|
||||
## Overall verdict
|
||||
|
||||
This is a tightly-argued, capability-spec PRD that earns most of its length: the
|
||||
no-clobber thesis is crisp, the trade-offs are named with what was given up (not
|
||||
just what was chosen), and Non-Goals do real work. What's at risk is downstream
|
||||
extractability and thesis-validation: there is no Glossary and no Success Metrics
|
||||
section, so a story author will re-derive domain terms and the PRD has no
|
||||
machine-readable way to confirm the thesis landed. A stale "to be decided" in the
|
||||
Vision also drifts from a decision the FRs already made. None of this blocks a
|
||||
green-light; all of it is one editing pass away from closed.
|
||||
|
||||
## Decision-readiness — strong
|
||||
|
||||
Decisions are stated as decisions, not buried as considerations. The `[CONFIRMED]` /
|
||||
`[DEFERRED]` tags in §1 are honest — e.g. _"Foundry is source of truth" is demoted
|
||||
to a tie-breaker rule for the both-diverged conflict case"_ names both the choice
|
||||
and the demotion. Trade-offs name what was given up: FR-1.6 explicitly says _"do
|
||||
not fall back to an Obsidian-side-only check (that reintroduces clobber risk)"_ —
|
||||
the rejected path is on the page, not hidden. The Open Questions are actually open:
|
||||
OQ-1 carries a default posture (_"Default posture is manual"_), OQ-3/OQ-4/OQ-6 are
|
||||
genuinely delivery-time (_"pick during delivery B with relay-load testing"_), and
|
||||
OQ-2 is a real deferral with the reopening condition named (_"Reopens when a
|
||||
custom Foundry module is explored"_). The decision-log shows real tensions
|
||||
surfaced (the clobber gap in the current O→F code) rather than smoothed to neutral.
|
||||
|
||||
### Findings
|
||||
- **medium** Stale "to be decided" in Vision contradicts the FRs (§1, _"to be
|
||||
decided: newest-wins, Foundry-wins, or manual merge"_). FR-3.2 and FR-3.3
|
||||
already specify manual resolution with three explicit actions (_"push vault →
|
||||
Foundry", "pull Foundry → vault", "mark resolved"_); OQ-1 only asks whether to
|
||||
add a newest-mtime convenience. The §1 phrasing reads as unresolved when the
|
||||
PRD has in fact decided. *Fix:* rewrite the §1 parenthetical to
|
||||
_"default = manual resolution (FR-3.2); a newest-mtime convenience is open
|
||||
(OQ-1)"_.
|
||||
|
||||
## Substance over theater — strong
|
||||
|
||||
No personas, no User Journeys, no differentiation section — and that is correct
|
||||
for a single-operator brownfield capability spec (see Shape fit). The Vision is
|
||||
product-specific, not swappable: the prep / run-the-match duality, the
|
||||
_"always in lockstep while I work"_ feeling, and the _"sync deliberately paused,
|
||||
not silently absent"_ framing would not transfer to another tool. NFRs carry
|
||||
product-specific thresholds, not boilerplate: NFR-3 names a _"~50-note prep
|
||||
burst"_, NFR-4 _"within one tick"_, NFR-6 names the _"UI-only convention"_. No
|
||||
innovation theater, no persona theater. Nothing here reads like furniture.
|
||||
|
||||
### Findings
|
||||
- (none)
|
||||
|
||||
## Strategic coherence — adequate
|
||||
|
||||
The thesis is stated and the features serve it. G1 (no-clobber bidirectional
|
||||
auto-sync for prep) is the thesis; F1+F2+F3 implement it directly. G4 (legible
|
||||
status) is the second arc; F4 serves it. G5/G6 (hardening/onboarding) are the
|
||||
launchable-stakes rigor that follows from the chosen stakes, not a bolt-on.
|
||||
Prioritization follows the thesis: delivery A (ship O→F *with the divergence
|
||||
guard*) is gated on the safety fix, not on "what's easy first."
|
||||
|
||||
The gap is that there is **no Success Metrics section**. The rubric asks whether
|
||||
SMs validate the thesis and whether counter-metrics are named. G2 embeds one
|
||||
criterion (_"verify one end-to-end live push"_), but there is no consolidated set
|
||||
of operational metrics that would let a reviewer confirm the thesis — e.g.
|
||||
"zero silent clobbers across a 50-note prep burst", "F→O poll detects a Foundry
|
||||
rename within N ticks", "non-author user reaches live sync via dashboard + /setup
|
||||
without shell". For a public/launchable PRD, the absence is a coherence tell: the
|
||||
PRD argues the thesis but doesn't state how it would know it landed.
|
||||
|
||||
### Findings
|
||||
- **medium** No Success Metrics section (whole PRD). The thesis is clear but
|
||||
unmeasured; activity metrics are absent too. *Fix:* add an SM block with 3–5
|
||||
operational metrics tied to G1/G2/G4/G6 (e.g. _SM-1: zero auto-overwrite
|
||||
events on a side that changed since last sync, across a 50-note burst_;
|
||||
_SM-2: one verified end-to-end live O→F push_; _SM-3: F→O detects a Foundry
|
||||
rename/move within ≤2 poll ticks_; _SM-6: a non-author user clones, runs
|
||||
/setup, and reaches a connected live sync with no shell command_). Name a
|
||||
counter-metric for SM-1 (e.g. _conflict-rows-surfaced > 0 when both sides
|
||||
diverged_).
|
||||
|
||||
## Done-ness clarity — adequate
|
||||
|
||||
Most FRs carry testable consequences. FR-1.4's hash-over-_content + name +
|
||||
folder_path_ is concrete and verifiable. FR-1.5's 2×2 routing is a truth table.
|
||||
FR-3.1 makes the routing explicit. FR-2.2 enumerates the change kinds
|
||||
(_renamed / moved / content-changed / missing / new_). G2's _"verify one
|
||||
end-to-end live push"_ is an acceptance gate for delivery A. NFR-1 is the hard
|
||||
contract (_"No auto-sync operation may overwrite a side that has changed since
|
||||
the last sync"_).
|
||||
|
||||
The unforgiving part: a few FRs use adjectives where bounds belong. FR-5.2 says
|
||||
_"defaults tuned for prep so a burst of simultaneous saves doesn't thrash or drop
|
||||
events"_ — "thrash" is undefined and "tuned for prep" is an adjective, not a
|
||||
bound; NFR-3 supplies the ~50-note number but FR-5.2 doesn't reference it.
|
||||
FR-5.4's _"panel capped and scrollable"_ leaves the cap unspecified. These are
|
||||
the spots where an engineer writing stories will have to go back and ask "how
|
||||
many?"
|
||||
|
||||
### Findings
|
||||
- **low** FR-5.2 "thrash" / "tuned for prep" — bound-less adjectives (§F5).
|
||||
*Fix:* bind FR-5.2 to NFR-3's ~50-note burst and name concrete defaults
|
||||
ranges (debounce window, max concurrency) or defer the numbers to a config
|
||||
table with placeholders.
|
||||
- **low** FR-5.4 "panel capped" — cap value missing (§F5). *Fix:* state the cap
|
||||
(e.g. _"last 200 events, scrollable for older"_).
|
||||
|
||||
## Scope honesty — strong
|
||||
|
||||
Non-Goals do real work and name what could be silently assumed: _"Not syncing"
|
||||
indicator rendered inside Foundry_ is explicitly deferred with the reason
|
||||
(_"Foundry UI is not a surface we control"_); _"Syncing during run-the-match"_
|
||||
is called out as a design decision, not an omission; _"Automatic semantic / 3-way
|
||||
content merge"_ is excluded so the conflict UI doesn't get over-built. The
|
||||
`[ASSUMPTION]` tag in §2 (_"The relay remains the live transport for this PRD;
|
||||
the custom Foundry module … is explicitly future work, not a dependency here"_)
|
||||
flags the key inference. Open-items density is moderate (6 OQs + 1 ASSUMPTION)
|
||||
against launchable stakes; every OQ carries a default or a named
|
||||
delivery-trigger, so the density is honest rather than a blocker.
|
||||
|
||||
### Findings
|
||||
- **low** Implicit relay-load assumption not flagged (§F2 / NFR-3). FR-2.2
|
||||
detects content-changed entries _"via /get hash compare"_, which implies one
|
||||
/get per changed linked note per poll tick — a load assumption that
|
||||
interacts with OQ-3's cadence. This is an inference the user didn't directly
|
||||
confirm and should be tagged `[ASSUMPTION]`. *Fix:* add
|
||||
_`[ASSUMPTION]` one relay /get per changed linked note per poll tick is
|
||||
acceptable relay load at the chosen cadence (OQ-3)_ to §F2, and index it.
|
||||
|
||||
## Downstream usability — adequate
|
||||
|
||||
FR IDs are contiguous (FR-1.1 → FR-6.4), unique, and cross-references resolve
|
||||
(FR-5.1 → FR-1.1 ✓; OQ-5 → FR-2.5 ✓; NFR-1 → the divergence guard ✓). Brownfield
|
||||
references are accurate against the decision log and code memory
|
||||
(_AutoSyncController_, _pushNote_, _foundry.cc_uuid_, _foundry.contentHash_,
|
||||
_rest-api module_, _relay /search_ minified, _relay /get_). No floating UJs (no
|
||||
UJs at all — appropriate). Each section reads sensibly pulled out alone.
|
||||
|
||||
The gap is the **missing Glossary**. The PRD uses a thick domain vocabulary —
|
||||
_cc_uuid_, _contentHash_, _ccHash_ (new field, introduced in FR-1.4), _linked
|
||||
note_, _seeded_, _refined vault dir_, _refined markdown_, _import candidate_,
|
||||
_dev/apply mode_, _activity panel_, _parity_, _cc-only entry_ — and these are
|
||||
defined only inline and inconsistently (FR-1.2 glosses _foundry.cc_uuid_ as
|
||||
"unlinked"; FR-1.4 defines _ccHash_ in passing). A story author source-extracting
|
||||
from this PRD will have to reassemble the term set, and drift risk is real
|
||||
(_ccHash_ vs _contentHash_ vs _cc_uuid_ are easy to confuse). For a PRD that
|
||||
feeds architecture and stories, this is the dimension that needs the most work.
|
||||
|
||||
### Findings
|
||||
- **medium** No Glossary section (whole PRD). Domain nouns are inline-defined
|
||||
and not consolidated; _ccHash_ is introduced mid-FR without a term-of-art
|
||||
definition. *Fix:* add a Glossary section defining at minimum: _cc_uuid_,
|
||||
_contentHash_, _ccHash_, _linked note_, _seeded_, _refined vault dir_,
|
||||
_refined markdown_, _import candidate_, _cc-only entry_, _dev/apply mode_,
|
||||
_activity panel_, _parity_, _tick_. Reference Glossary terms from the FRs
|
||||
instead of re-defining inline.
|
||||
|
||||
## Shape fit — strong
|
||||
|
||||
The shape matches the product. This is a brownfield, single-operator (DM/world-
|
||||
builder) capability spec intended to go launchable — exactly the case the rubric
|
||||
flags as _"Internal tool, single-operator role → capability spec shape; UJs may
|
||||
be overhead; SMs may be operational."_ No personas, no UJs: correct, not a gap.
|
||||
Operational SMs would fit the shape but their absence (see Strategic coherence)
|
||||
is a coherence gap, not a shape mismatch. The PRD is neither over-formalized (no
|
||||
UJ theater for a one-operator tool) nor under-formalized (NFRs and error
|
||||
contracts are present at launchable rigor). Existing-code references are
|
||||
distinguished from new behavior (_"code-complete but uncommitted"_, _"new field,
|
||||
e.g. foundry.ccHash"_), which is what brownfield shape demands.
|
||||
|
||||
### Findings
|
||||
- **low** Operational SMs would fit this shape and are absent (§5 / §SM). *Fix:*
|
||||
see the Strategic-coherence finding — operational metrics (zero silent
|
||||
clobbers, ticks-to-detect, onboard-without-shell) are the right SM shape for
|
||||
this product and would close both dimensions at once.
|
||||
|
||||
## Mechanical notes
|
||||
|
||||
- **Glossary drift:** No glossary; _ccHash_ vs _contentHash_ vs _cc_uuid_ are
|
||||
used close together and differ only in case/short suffix — high confusion
|
||||
risk. _"cc-only entry"_ (FR-2.5) and _"import candidate"_ (FR-2.5) are used
|
||||
as synonyms without being linked. _"refined vault dir"_ / _"refined markdown"_
|
||||
appear without definition (FR-1.1, FR-2.3).
|
||||
- **ID continuity:** FR IDs contiguous and unique; cross-refs (FR-5.1→FR-1.1,
|
||||
OQ-5→FR-2.5, NFR-1→the divergence guard) all resolve. No duplicates, no gaps.
|
||||
- **Assumptions Index roundtrip:** broken. One inline `[ASSUMPTION]` exists
|
||||
(§2, relay-as-transport); no Assumptions Index at the end of the PRD. The
|
||||
implicit relay-load assumption (Scope honesty finding) is not tagged at all.
|
||||
- **UJ persona linkage:** N/A — no UJs, no personas (appropriate for shape).
|
||||
- **Required sections for stakes:** Vision ✓, Problem & Context ✓, Goals &
|
||||
Non-Goals ✓, FRs ✓, NFRs ✓, Open Questions ✓, Out of Scope ✓. **Missing:
|
||||
Success Metrics, Glossary.** For public/launchable stakes, SMs are expected;
|
||||
Glossary is expected for a PRD that feeds architecture + stories.
|
||||
141
scripts/start-relay-session.js
Normal file
141
scripts/start-relay-session.js
Normal file
@@ -0,0 +1,141 @@
|
||||
// Starts a headless Foundry session on a ThreeHats foundryvtt-rest-api-relay.
|
||||
//
|
||||
// Generalized version: every value comes from the environment (no hardcoded
|
||||
// host). Reads this repo's .env first, then process env. Required: RELAY_API_KEY,
|
||||
// RELAY_URL, FOUNDRY_URL, RELAY_USER, RELAY_PASSWORD, FOUNDRY_WORLD.
|
||||
//
|
||||
// Flow (per docs/relay-api.md):
|
||||
// 1. POST /session-handshake (x-api-key, x-foundry-url, x-username) -> {nonce, publicKey, token}
|
||||
// 2. RSA-OAEP encrypt {password, nonce} with the relay's public key.
|
||||
// 3. POST /start-session {handshakeToken, encryptedPassword, world} -> headless launch.
|
||||
//
|
||||
// Run from the repo root: node scripts/start-relay-session.js
|
||||
// (The /setup skill does this for you after the Foundry rest-api module is connected.)
|
||||
|
||||
const crypto = require('crypto');
|
||||
const http = require('http');
|
||||
const https = require('https');
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const { URL } = require('url');
|
||||
|
||||
// --- Load .env (simple KEY=value; real process.env wins) ---
|
||||
(function loadDotEnv() {
|
||||
const p = path.join(__dirname, '..', '.env');
|
||||
let txt;
|
||||
try { txt = fs.readFileSync(p, 'utf8'); } catch { return; } // no .env is fine
|
||||
for (const line of txt.split(/\r?\n/)) {
|
||||
const m = line.match(/^\s*([A-Za-z_][A-Za-z0-9_]*)\s*=\s*(.*)\s*$/);
|
||||
if (!m) continue;
|
||||
const [, k, vRaw] = m;
|
||||
if (process.env[k] !== undefined) continue; // real env wins over .env
|
||||
const v = vRaw.replace(/^['"]|['"]$/g, '');
|
||||
if (v !== '') process.env[k] = v;
|
||||
}
|
||||
})();
|
||||
|
||||
const env = (k) => (process.env[k] !== undefined && process.env[k] !== '' ? process.env[k] : '');
|
||||
|
||||
const API_KEY = env('RELAY_API_KEY');
|
||||
const RELAY_URL = env('RELAY_URL');
|
||||
const FOUNDRY_URL = env('FOUNDRY_URL');
|
||||
const USERNAME = env('RELAY_USER');
|
||||
const PASSWORD = env('RELAY_PASSWORD');
|
||||
const WORLD = env('FOUNDRY_WORLD');
|
||||
|
||||
const missing = [
|
||||
['RELAY_API_KEY', API_KEY],
|
||||
['RELAY_URL', RELAY_URL],
|
||||
['FOUNDRY_URL', FOUNDRY_URL],
|
||||
['RELAY_USER', USERNAME],
|
||||
['RELAY_PASSWORD', PASSWORD],
|
||||
['FOUNDRY_WORLD', WORLD],
|
||||
].filter(([, v]) => !v).map(([k]) => k);
|
||||
if (missing.length) {
|
||||
console.error(`Missing required env var(s): ${missing.join(', ')}.\n` +
|
||||
`Copy .env.example to .env and fill them in (or run the /setup skill).`);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
let relay;
|
||||
try {
|
||||
relay = new URL(RELAY_URL);
|
||||
} catch {
|
||||
console.error(`RELAY_URL is not a valid URL: ${RELAY_URL}`);
|
||||
process.exit(1);
|
||||
}
|
||||
const transport = relay.protocol === 'https:' ? https : http;
|
||||
|
||||
function post(pathname, headers, body) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const data = JSON.stringify(body || {});
|
||||
const req = transport.request(
|
||||
{
|
||||
protocol: relay.protocol,
|
||||
hostname: relay.hostname,
|
||||
port: relay.port || (relay.protocol === 'https:' ? 443 : 80),
|
||||
path: pathname,
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Content-Length': Buffer.byteLength(data),
|
||||
'x-api-key': API_KEY,
|
||||
...headers,
|
||||
},
|
||||
},
|
||||
(res) => {
|
||||
let d = '';
|
||||
res.on('data', (c) => (d += c));
|
||||
res.on('end', () => {
|
||||
if (res.statusCode && res.statusCode >= 400) {
|
||||
reject(new Error(`relay ${res.statusCode} POST ${pathname}: ${d.slice(0, 300)}`));
|
||||
return;
|
||||
}
|
||||
try { resolve(JSON.parse(d)); } catch { resolve(d); }
|
||||
});
|
||||
}
|
||||
);
|
||||
req.on('error', reject);
|
||||
req.write(data);
|
||||
req.end();
|
||||
});
|
||||
}
|
||||
|
||||
(async () => {
|
||||
console.log(`Starting handshake on ${RELAY_URL.href} (world: ${WORLD}, foundry: ${FOUNDRY_URL}, user: ${USERNAME})...`);
|
||||
let hs;
|
||||
try {
|
||||
hs = await post('/session-handshake', {
|
||||
'x-foundry-url': FOUNDRY_URL,
|
||||
'x-username': USERNAME,
|
||||
}, {});
|
||||
} catch (e) {
|
||||
console.error(`Handshake failed: ${e.message}\n` +
|
||||
`Is the relay up (docker compose up -d relay) and is the Foundry rest-api module connected to it?`);
|
||||
process.exit(1);
|
||||
}
|
||||
if (!hs || !hs.nonce || !hs.publicKey || !hs.token) {
|
||||
console.error('Handshake returned an unexpected response:', JSON.stringify(hs));
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const payload = JSON.stringify({ password: PASSWORD, nonce: hs.nonce });
|
||||
const encryptedPassword = crypto.publicEncrypt(
|
||||
{ key: hs.publicKey, padding: crypto.constants.RSA_PKCS1_OAEP_PADDING, oaepHash: 'sha256' },
|
||||
Buffer.from(payload)
|
||||
).toString('base64');
|
||||
|
||||
console.log('Launching headless session (may take up to 2 min)...');
|
||||
try {
|
||||
const result = await post('/start-session', {}, {
|
||||
handshakeToken: hs.token,
|
||||
encryptedPassword,
|
||||
world: WORLD,
|
||||
});
|
||||
console.log('Result:', JSON.stringify(result, null, 2));
|
||||
} catch (e) {
|
||||
console.error(`Start-session failed: ${e.message}\n` +
|
||||
`Check FOUNDRY_URL/RELAY_USER/RELAY_PASSWORD/FOUNDRY_WORLD are correct and the world exists.`);
|
||||
process.exit(1);
|
||||
}
|
||||
})();
|
||||
@@ -89,7 +89,12 @@
|
||||
<button onclick="act('importAll')" title="Pull every cc-only Foundry entry (not yet in your vault) into a new refined note under refined/imported/<folder>. Un-curated staging for review.">Import all</button>
|
||||
<button onclick="pushAll()" title="Push every vault-newer (sync→cc) note into the LIVE Foundry world via the relay in one run. dry-run (default) lists what would be pushed; uncheck dry-run to apply — each live entry is backed up to <out>/bak first and the note's foundry.contentHash is baselined so a re-run only catches new edits. Replaces scripts/resync.ts.">Push all changed</button>
|
||||
<button onclick="refreshLive()" title="Rebuild the cached name↔uuid map (for link resolution in pushes) via the relay /search — zero Foundry downtime. The heavy docker-stop full index is CLI-only: npx tsx src/cli.ts refresh --full-index.">Refresh live index</button>
|
||||
<button id="autoSyncBtn" onclick="toggleAutosync()" title="Toggle Obsidian→Foundry auto-sync. ON = saving a linked, seeded note in your vault pushes it into LIVE Foundry instantly (guarded by the note's foundry.contentHash baseline, so no-op saves and the post-push baseline write don't re-push; unlinked/unseeded notes are skipped). Foundry→Obsidian is NOT auto — use Sync / Re-pull for that. Needs the relay (RELAY_API_KEY). Dev mode baselines land in the --out mirror; apply mode in the real vault.">Auto-sync: off</button>
|
||||
</header>
|
||||
<details id="autoSyncPanel" class="autosync-panel" open style="display:none;margin:0 12px;border-top:1px solid #ddd;padding:6px 12px;background:#fafafa">
|
||||
<summary style="cursor:pointer">Auto-sync activity <span id="autoSyncCounts" class="meta"></span><span id="autoSyncNote" class="meta" style="margin-left:8px"></span></summary>
|
||||
<pre id="autoSyncLog" class="autosync-log" style="max-height:180px;overflow:auto;margin:6px 0 8px;font-size:12px;background:#fff;border:1px solid #eee;padding:6px">(no activity yet)</pre>
|
||||
</details>
|
||||
<main>
|
||||
<section class="list">
|
||||
<div class="toolbar">
|
||||
@@ -118,7 +123,7 @@
|
||||
<section class="detail" id="detail">Select a row to inspect.</section>
|
||||
</main>
|
||||
<script>
|
||||
let INDEX = null, STATUS = null, SEL = null, REC_FILTER = null;
|
||||
let INDEX = null, STATUS = null, SEL = null, REC_FILTER = null, AUTO = null, autoPoll = null;
|
||||
const dryEl = () => document.getElementById('dryRun');
|
||||
|
||||
// Recommendation -> display label, badge class, bulk op, and one-line guidance.
|
||||
@@ -147,6 +152,7 @@ async function init() {
|
||||
`matched <b>${c.matched}</b> · cc-only <b>${c.ccOnly}</b> · refined-only <b>${c.refinedOnly}</b> · unlinked <b>${c.unlinked}</b>`;
|
||||
renderRecPanel();
|
||||
render();
|
||||
refreshAutosync();
|
||||
}
|
||||
function esc(s){ return (s==null?'':String(s)).replace(/[&<>]/g,c=>({'&':'&','<':'<','>':'>'}[c])); }
|
||||
function attr(s){ return (s==null?'':String(s)).replace(/[&"]/g,c=>({'&':'&','"':'"'}[c])); }
|
||||
@@ -318,6 +324,38 @@ async function refreshLive(){
|
||||
if (r.error){ toast('error: ' + r.error); return; }
|
||||
toast(`live index refreshed: ${r.pairs} name↔uuid pairs cached`);
|
||||
}
|
||||
// Auto-sync (Obsidian→Foundry, instant): the server watches the vault and pushes
|
||||
// saved notes into live Foundry. Toggle here; poll for the activity log while on.
|
||||
async function refreshAutosync(){
|
||||
const r = await fetch('/api/autosync').then(r=>r.json()).catch(()=>null);
|
||||
if (!r) return;
|
||||
AUTO = r;
|
||||
const btn = document.getElementById('autoSyncBtn');
|
||||
btn.textContent = `Auto-sync: ${r.enabled ? 'on' : 'off'}`;
|
||||
btn.classList.toggle('primary', r.enabled);
|
||||
const panel = document.getElementById('autoSyncPanel');
|
||||
panel.style.display = r.enabled ? '' : 'none';
|
||||
document.getElementById('autoSyncCounts').textContent =
|
||||
r.enabled ? `pushed ${r.pushed} · skipped ${r.skipped} · errors ${r.errors}` : '';
|
||||
document.getElementById('autoSyncNote').textContent =
|
||||
r.enabled && STATUS && STATUS.mode !== 'apply'
|
||||
? '(dev mode — baselines land in the --out mirror, not the real vault; run --apply to baseline the real vault)'
|
||||
: '';
|
||||
const log = document.getElementById('autoSyncLog');
|
||||
log.textContent = r.events && r.events.length
|
||||
? r.events.map(e => `${e.time.replace('T',' ').slice(5,19)} ${e.status.padEnd(7)} ${e.name} — ${e.message}`).join('\n')
|
||||
: '(no activity yet — save a linked, seeded note in your vault to trigger a push)';
|
||||
if (r.enabled && !autoPoll) autoPoll = setInterval(refreshAutosync, 2000);
|
||||
if (!r.enabled && autoPoll) { clearInterval(autoPoll); autoPoll = null; }
|
||||
}
|
||||
async function toggleAutosync(){
|
||||
const want = !(AUTO && AUTO.enabled);
|
||||
toast(`turning auto-sync ${want ? 'on' : 'off'}…`);
|
||||
const r = await fetch('/api/autosync', {method:'POST', headers:{'content-type':'application/json'}, body: JSON.stringify({enabled: want})}).then(r=>r.json());
|
||||
if (r.error){ toast('error: ' + r.error); return; }
|
||||
toast(`auto-sync ${r.enabled ? 'ON — saving a note pushes it to live Foundry' : 'off'}`);
|
||||
refreshAutosync();
|
||||
}
|
||||
// Push every vault-newer (sync-cc) note into live Foundry in one run. dry-run (default)
|
||||
// lists what would be pushed; apply pushes each, backs up the live entry, and baselines
|
||||
// the note so a re-run only catches new edits. Replaces scripts/resync.ts.
|
||||
|
||||
211
src/server.ts
211
src/server.ts
@@ -10,14 +10,15 @@
|
||||
|
||||
import { createServer, type IncomingMessage, type ServerResponse, type Server } from "node:http";
|
||||
import { readFile, writeFile, mkdir, copyFile, access, stat } from "node:fs/promises";
|
||||
import { join, dirname } from "node:path";
|
||||
import { watch, readdirSync, statSync, type FSWatcher } from "node:fs";
|
||||
import { join, dirname, relative, basename, extname } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { JournalDb } from "./db.js";
|
||||
import { indexAll, seedRow, syncRow, rePullRow, importRow, buildBlock, seedBlockContent, type FileRow, type IndexResult } from "./batch.js";
|
||||
import { RelayClient } from "./relay/client.js";
|
||||
import { pushNote } from "./push.js";
|
||||
import { nameUuidIndexFromEntries, saveNameUuidIndex, loadNameUuidIndex, MapNameResolver, type NameResolver } from "./resolver.js";
|
||||
import { splitFrontmatter } from "./frontmatter.js";
|
||||
import { splitFrontmatter, readFoundryBlock } from "./frontmatter.js";
|
||||
import { contentHash } from "./normalize.js";
|
||||
import { backupStamp } from "./write.js";
|
||||
import type { RelayConfig, FoundryHostConfig } from "./config.js";
|
||||
@@ -53,6 +54,7 @@ interface State {
|
||||
db: JournalDb;
|
||||
cfg: ServerConfig;
|
||||
index: IndexResult | null;
|
||||
autosync: AutoSyncController;
|
||||
}
|
||||
|
||||
function send(res: ServerResponse, code: number, body: unknown): void {
|
||||
@@ -427,9 +429,198 @@ async function handleLink(state: State, req: IncomingMessage, res: ServerRespons
|
||||
}
|
||||
}
|
||||
|
||||
export interface AutoSyncEvent {
|
||||
time: string;
|
||||
name: string;
|
||||
status: "pushed" | "skipped" | "error";
|
||||
message: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Auto-sync (Obsidian→Foundry, instant). Watches the refined vault dir for .md saves
|
||||
* and pushes each changed, linked, seeded note into live Foundry via the relay — the
|
||||
* same `pushNote` the manual "push" button uses. Foundry keeps running; no LevelDB,
|
||||
* no Docker stop.
|
||||
*
|
||||
* Loop/idempotency guard: a save only pushes if the note's body contentHash differs
|
||||
* from its `foundry.contentHash` baseline. After a push, `baselineNote` rewrites that
|
||||
* baseline to the new body hash, so the watcher's own baseline write (and any no-op
|
||||
* save that didn't change the body) is skipped — no feedback loop, no redundant push.
|
||||
* Unlinked (no `foundry.cc_uuid`) or unseeded (no `foundry.contentHash`) notes are
|
||||
* skipped; seed/link them manually first. Foundry→Obsidian is NOT auto (the relay has
|
||||
* no change-push; use the dashboard's Sync / Re-pull buttons for that direction).
|
||||
*
|
||||
* In dev mode the baseline lands in the `--out` mirror (consistent with dev semantics);
|
||||
* in apply mode it lands in the real vault with a `.bak`. The push itself goes live via
|
||||
* the relay regardless of mode.
|
||||
*/
|
||||
class AutoSyncController {
|
||||
enabled = false;
|
||||
private recursive = false;
|
||||
private root: FSWatcher | null = null;
|
||||
private subs: { w: FSWatcher; prefix: string }[] = [];
|
||||
private timers = new Map<string, NodeJS.Timeout>();
|
||||
private inflight = new Set<string>();
|
||||
private queue: string[] = [];
|
||||
private running = 0;
|
||||
private readonly concurrency = 3;
|
||||
private readonly debounceMs = 700;
|
||||
events: AutoSyncEvent[] = [];
|
||||
private readonly maxEvents = 100;
|
||||
pushed = 0;
|
||||
skipped = 0;
|
||||
errors = 0;
|
||||
|
||||
constructor(private state: State) {}
|
||||
|
||||
status() {
|
||||
return {
|
||||
enabled: this.enabled,
|
||||
mode: this.state.cfg.mode,
|
||||
watching: this.state.cfg.refinedDir,
|
||||
pushed: this.pushed,
|
||||
skipped: this.skipped,
|
||||
errors: this.errors,
|
||||
events: this.events,
|
||||
};
|
||||
}
|
||||
|
||||
private log(name: string, status: AutoSyncEvent["status"], message: string): void {
|
||||
this.events.unshift({ time: new Date().toISOString(), name, status, message });
|
||||
if (this.events.length > this.maxEvents) this.events.length = this.maxEvents;
|
||||
if (status === "pushed") this.pushed++;
|
||||
else if (status === "error") this.errors++;
|
||||
else this.skipped++;
|
||||
}
|
||||
|
||||
async setEnabled(on: boolean): Promise<void> {
|
||||
if (on === this.enabled) return;
|
||||
if (on) await this.start();
|
||||
else this.stop();
|
||||
}
|
||||
|
||||
private async start(): Promise<void> {
|
||||
if (!this.state.cfg.relayCfg) throw new Error("relay not configured — auto-sync needs RELAY_API_KEY / --relay-api-key");
|
||||
const dir = this.state.cfg.refinedDir;
|
||||
try {
|
||||
this.root = watch(dir, { recursive: true }, (evt, fn) => this.onChange(evt, fn, ""));
|
||||
this.recursive = true;
|
||||
} catch {
|
||||
// Older Node / platform without recursive fs.watch: watch the root + every
|
||||
// subdir individually, and re-scan on renames so new folders get watched too.
|
||||
this.root = watch(dir, (evt, fn) => this.onChange(evt, fn, ""));
|
||||
this.recursive = false;
|
||||
}
|
||||
if (!this.recursive) this.rescanSubs(dir);
|
||||
this.enabled = true;
|
||||
this.log("(watcher)", "skipped", `auto-sync ON — watching ${dir}${this.recursive ? " (recursive)" : " (per-dir fallback)"}`);
|
||||
}
|
||||
|
||||
/** Attach non-recursive watchers to every subdir (skipping .obsidian). Fallback path. */
|
||||
private rescanSubs(dir: string): void {
|
||||
const root = this.state.cfg.refinedDir;
|
||||
const seen = new Set(this.subs.map((s) => s.prefix));
|
||||
const stack = [dir];
|
||||
while (stack.length) {
|
||||
const d = stack.pop()!;
|
||||
let entries: string[];
|
||||
try { entries = readdirSync(d); } catch { continue; }
|
||||
for (const e of entries) {
|
||||
const full = join(d, e);
|
||||
let isDir = false;
|
||||
try { isDir = statSync(full).isDirectory(); } catch { continue; }
|
||||
if (!isDir) continue;
|
||||
if (e === ".obsidian" || e.startsWith(".")) continue;
|
||||
const prefix = relative(root, full).replace(/\\/g, "/");
|
||||
if (seen.has(prefix)) { stack.push(full); continue; }
|
||||
seen.add(prefix);
|
||||
try {
|
||||
const w = watch(full, (evt, fn) => this.onChange(evt, fn, prefix));
|
||||
this.subs.push({ w, prefix });
|
||||
} catch { /* unreadable dir — skip */ }
|
||||
stack.push(full);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
stop(): void {
|
||||
this.root?.close(); this.root = null;
|
||||
for (const s of this.subs) s.w.close();
|
||||
this.subs = [];
|
||||
for (const t of this.timers.values()) clearTimeout(t);
|
||||
this.timers.clear();
|
||||
this.queue.length = 0;
|
||||
this.enabled = false;
|
||||
this.log("(watcher)", "skipped", "auto-sync OFF");
|
||||
}
|
||||
|
||||
private onChange(_evt: string, filename: string | null, prefix: string): void {
|
||||
if (!filename) return;
|
||||
const rel = (prefix ? `${prefix}/${filename}` : filename).replace(/\\/g, "/");
|
||||
if (!rel.endsWith(".md")) return;
|
||||
if (rel.split("/").includes(".obsidian")) return;
|
||||
// Fallback path: a rename may have created a new subdir — re-scan to watch it.
|
||||
if (!this.recursive) this.rescanSubs(this.state.cfg.refinedDir);
|
||||
const prev = this.timers.get(rel);
|
||||
if (prev) clearTimeout(prev);
|
||||
this.timers.set(rel, setTimeout(() => {
|
||||
this.timers.delete(rel);
|
||||
if (this.inflight.has(rel)) return;
|
||||
this.queue.push(rel);
|
||||
this.drain();
|
||||
}, this.debounceMs));
|
||||
}
|
||||
|
||||
private drain(): void {
|
||||
while (this.running < this.concurrency && this.queue.length) {
|
||||
const rel = this.queue.shift()!;
|
||||
this.running++;
|
||||
this.process(rel).catch(() => {}).finally(() => { this.running--; this.drain(); });
|
||||
}
|
||||
}
|
||||
|
||||
private async process(relPath: string): Promise<void> {
|
||||
this.inflight.add(relPath);
|
||||
const name = basename(relPath, extname(relPath));
|
||||
try {
|
||||
const abs = await resolveRefined(this.state, relPath);
|
||||
let md: string;
|
||||
try { md = await readFile(abs, "utf8"); } catch { this.log(name, "skipped", "file removed"); return; }
|
||||
const { fm, body } = splitFrontmatter(md);
|
||||
const fb = readFoundryBlock(fm);
|
||||
if (!fb?.cc_uuid) { this.log(name, "skipped", "not linked — no foundry.cc_uuid (seed/link first)"); return; }
|
||||
if (!fb.contentHash) { this.log(name, "skipped", "not seeded — no foundry.contentHash baseline"); return; }
|
||||
const bodyHash = contentHash(body);
|
||||
if (bodyHash === fb.contentHash) { this.log(name, "skipped", "unchanged since last push"); return; }
|
||||
|
||||
const relay = relayClient(this.state);
|
||||
const outcome = await pushNote({
|
||||
notePath: abs,
|
||||
noteName: name,
|
||||
outDir: this.state.cfg.outDir,
|
||||
relay,
|
||||
foundryDataDir: this.state.cfg.foundryCfg?.dataDir ?? "",
|
||||
world: this.state.cfg.foundryCfg?.world ?? "",
|
||||
dryRun: false, // auto-sync always applies — the whole point is hands-off live push
|
||||
log: () => {},
|
||||
});
|
||||
// Baseline foundry.contentHash to the new body hash so a re-save with no further
|
||||
// edit (and the watcher's own baseline write) is a no-op. Idempotency lives here,
|
||||
// not in pushNote (which always PUTs).
|
||||
const baselined = await baselineNote(this.state, relPath, abs);
|
||||
this.log(name, "pushed", `→ ${outcome.ccUuid}${outcome.updatedName ? ` ("${outcome.updatedName}")` : ""}${baselined ? " · baselined" : ""}`);
|
||||
} catch (e) {
|
||||
this.log(name, "error", (e as Error).message);
|
||||
} finally {
|
||||
this.inflight.delete(relPath);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
export async function startServer(cfg: ServerConfig): Promise<{ server: Server; state: State }> {
|
||||
const db = await JournalDb.open(cfg.journal);
|
||||
const state: State = { db, cfg, index: null };
|
||||
const state = { db, cfg, index: null } as State;
|
||||
state.autosync = new AutoSyncController(state);
|
||||
// Build the index once at startup. In dev mode overlay the <out> mirror so dev
|
||||
// writes are reflected (the corpora are otherwise static while the server runs).
|
||||
const ov = overlayDirs(state);
|
||||
@@ -489,6 +680,20 @@ export async function startServer(cfg: ServerConfig): Promise<{ server: Server;
|
||||
if (req.method === "POST" && url.pathname === "/api/refresh") {
|
||||
return handleRefresh(state, req, res);
|
||||
}
|
||||
if (req.method === "GET" && url.pathname === "/api/autosync") {
|
||||
return send(res, 200, state.autosync.status());
|
||||
}
|
||||
if (req.method === "POST" && url.pathname === "/api/autosync") {
|
||||
const body = await readJsonBody(req);
|
||||
if (body === null) return send(res, 400, { error: "bad json" });
|
||||
try {
|
||||
await state.autosync.setEnabled(body.enabled === true);
|
||||
send(res, 200, state.autosync.status());
|
||||
} catch (e) {
|
||||
send(res, 500, { error: (e as Error).message });
|
||||
}
|
||||
return;
|
||||
}
|
||||
send(res, 404, { error: "not found" });
|
||||
} catch (e) {
|
||||
send(res, 500, { error: (e as Error).message });
|
||||
|
||||
13
wiki/components/_index.md
Normal file
13
wiki/components/_index.md
Normal file
@@ -0,0 +1,13 @@
|
||||
---
|
||||
type: meta
|
||||
title: "Components Index"
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# Components
|
||||
|
||||
Reusable pieces shared across modules. See `_templates/component.md`.
|
||||
|
||||
- [[content-hash]] — body contentHash idempotency guard
|
||||
- [[foundry-block]] — the `foundry:` frontmatter block + read/baseline helpers
|
||||
- [[name-uuid-resolver]] — name↔uuid map for link resolution
|
||||
35
wiki/components/content-hash.md
Normal file
35
wiki/components/content-hash.md
Normal file
@@ -0,0 +1,35 @@
|
||||
---
|
||||
type: component
|
||||
path: "src/normalize.ts"
|
||||
status: active
|
||||
purpose: "Stable body hash used as the sync baseline / idempotency guard."
|
||||
depends_on: []
|
||||
used_by: [[batch]], [[server]], [[autosync]], [[push]]
|
||||
tags: [component]
|
||||
created: 2026-06-22
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# content-hash
|
||||
|
||||
`contentHash(body)` in `src/normalize.ts`. Hashes a note's BODY (frontmatter excluded) to a
|
||||
stable string, stored in the note's `foundry.contentHash` and the cc file's `cc_sync_hash`.
|
||||
|
||||
## How it's used
|
||||
- **Drift detection** ([[index-recommend]]): `refinedChanged` = `foundry.contentHash` !=
|
||||
`contentHash(body)` (vault newer than last baseline). `ccChanged` = `cc_sync_hash` !=
|
||||
cc body hash.
|
||||
- **Idempotency guard** ([[autosync]]): a save only pushes if `contentHash(body) !=
|
||||
foundry.contentHash`. After a push, `baselineNote` sets `foundry.contentHash =
|
||||
contentHash(body)` → subsequent no-op saves skip.
|
||||
- **push-all**: only rows with `recommendation === "sync-cc"` (i.e. body hash drifted from
|
||||
baseline) are pushed; each is baselined after.
|
||||
|
||||
## Why it matters
|
||||
`pushNote` always PUTs. The hash is the ONLY thing preventing redundant pushes and the
|
||||
auto-sync feedback loop. It's the project's core sync primitive.
|
||||
|
||||
## Notes / gotchas
|
||||
- It hashes the BODY only, so a frontmatter-only edit (e.g. the post-push baseline rewrite)
|
||||
does NOT change the body hash → that's exactly why the baseline write is a no-op for the
|
||||
watcher.
|
||||
36
wiki/components/foundry-block.md
Normal file
36
wiki/components/foundry-block.md
Normal file
@@ -0,0 +1,36 @@
|
||||
---
|
||||
type: component
|
||||
path: "src/frontmatter.ts"
|
||||
status: active
|
||||
purpose: "Parse / read / rewrite the `foundry:` identity block in a refined note's frontmatter."
|
||||
depends_on: []
|
||||
used_by: [[batch]], [[push]], [[server]], [[autosync]]
|
||||
tags: [component]
|
||||
created: 2026-06-22
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# foundry-block
|
||||
|
||||
The `foundry:` nested mapping in a refined note's YAML frontmatter ties the note to its
|
||||
Foundry journal entry. Fields: `cc_uuid` (e.g. `JournalEntry.<id>`), `cc_type`, `folder_path`,
|
||||
`contentHash`, `syncedAt`.
|
||||
|
||||
## Helpers (src/frontmatter.ts)
|
||||
- `splitFrontmatter(md)` → `{fm, body}`.
|
||||
- `readFoundryBlock(fm)` → the `foundry:` mapping as `Record<string,string> | undefined`
|
||||
(so `fb.cc_uuid`, `fb.contentHash` are available).
|
||||
- `buildBlock(entry, body, stamp)` → the block content for seeding (used by `seedRow`/`handleLink`).
|
||||
- `seedBlockContent(md, block)` → inject/refresh the block in a note.
|
||||
|
||||
## In server.ts
|
||||
- `baselineFoundryBlock(md, newHash, newSyncedAt)` — text surgery that rewrites ONLY the
|
||||
`contentHash` + `syncedAt` lines inside the `foundry:` block, leaving `cc_uuid`/`cc_type`/
|
||||
`folder_path` and all curation untouched. Mirrors `scripts/resync.ts`.
|
||||
- `baselineNote(state, relPath, absPath)` — reads the note, computes `contentHash(body)`,
|
||||
calls `baselineFoundryBlock`, writes via the mirror-aware `targetPath`. Returns true if it
|
||||
wrote. Used by push-all and [[autosync]].
|
||||
|
||||
## Notes / gotchas
|
||||
- Baseline text-surgery is line-based: it scans for `foundry:` then the indented ` ` lines
|
||||
beneath it. A note without a `foundry:` block is returned unchanged (no baseline).
|
||||
28
wiki/components/name-uuid-resolver.md
Normal file
28
wiki/components/name-uuid-resolver.md
Normal file
@@ -0,0 +1,28 @@
|
||||
---
|
||||
type: component
|
||||
path: "src/resolver.ts"
|
||||
status: active
|
||||
purpose: "name↔uuid map for resolving wikilinks/mentions in notes to Foundry journal UUIDs during push."
|
||||
depends_on: [[relay-client]]
|
||||
used_by: [[push]], [[server]], [[cli]]
|
||||
tags: [component]
|
||||
created: 2026-06-22
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# name-uuid-resolver
|
||||
|
||||
`src/resolver.ts` — the `NameResolver` interface + `MapNameResolver` impl, plus
|
||||
`nameUuidIndexFromEntries`, `saveNameUuidIndex`, `loadNameUuidIndex`.
|
||||
|
||||
## How it's built / cached
|
||||
- `refresh` ([[refresh-flow]]) calls `relay.searchJournalEntries()` (minified) →
|
||||
`nameUuidIndexFromEntries([{name,uuid}…])` → saves `<out>/name-uuid.json`.
|
||||
- `pushNote` resolves links via: a preloaded `resolver` arg → else `loadNameUuidIndex(<out>/
|
||||
name-uuid.json)` → else build via relay `/search` (and cache it).
|
||||
|
||||
## Notes / gotchas
|
||||
- The map is keyed by entry NAME. Curated notes whose filename differs from the Foundry entry
|
||||
name (e.g. "Angro Harn" vs "Angro Harn - Journal") still resolve because the cc payload
|
||||
carries explicit UUID references, and the index's uuid-fallback matching handles filenames.
|
||||
- `name-uuid.json` is gitignored output (lives under `--out`).
|
||||
32
wiki/decisions/001-external-foundry.md
Normal file
32
wiki/decisions/001-external-foundry.md
Normal file
@@ -0,0 +1,32 @@
|
||||
---
|
||||
type: decision
|
||||
status: active
|
||||
date: 2026-06-20
|
||||
context: "The shareable compose bundle must let a cloner get running against their own Foundry."
|
||||
decision: "Foundry is external — the cloner supplies FOUNDRY_URL. No Foundry service in the compose."
|
||||
consequences: "Compose bundles only the relay (+ the host-run dashboard). The relay must be reachable FROM the Foundry host (the rest-api module connects OUT to it)."
|
||||
tags: [decision, adr]
|
||||
created: 2026-06-22
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# ADR 001 — External Foundry
|
||||
|
||||
## Context
|
||||
The live stack on the original host runs Foundry in its own compose (`~/docker/foundryvtt`,
|
||||
Traefik-fronted). For a shareable bundle, baking Foundry in is impractical (license, world
|
||||
data, secrets). The cloner already has Foundry somewhere.
|
||||
|
||||
## Decision
|
||||
Foundry stays external. The cloner supplies `FOUNDRY_URL` (+ world, user, password). The
|
||||
compose provides only the [[threehats-relay | relay]] and the (host-run) dashboard.
|
||||
|
||||
## Consequences
|
||||
- One fewer service to bundle/license.
|
||||
- The relay must be reachable from the Foundry host (the rest-api module connects OUT over
|
||||
WebSocket). If Foundry is remote, expose the relay port (port-forward / tailnet / public domain).
|
||||
- The `/setup` skill collects `FOUNDRY_URL`/`FOUNDRY_WORLD`/`RELAY_USER`/`RELAY_PASSWORD` from
|
||||
the user and points their rest-api module at the relay.
|
||||
|
||||
## Related
|
||||
- [[002-tool-host-run]], [[003-relay-no-journal-db-no-docker-stop]]
|
||||
30
wiki/decisions/002-tool-host-run.md
Normal file
30
wiki/decisions/002-tool-host-run.md
Normal file
@@ -0,0 +1,30 @@
|
||||
---
|
||||
type: decision
|
||||
status: active
|
||||
date: 2026-06-20
|
||||
context: "The sync tool reads the Foundry journal LevelDB and shells out to `docker` for refresh --full-index."
|
||||
decision: "The sync tool runs on the HOST, not in a container. The compose never includes it."
|
||||
consequences: "The tool needs host FS access (LevelDB) + the docker socket. The dashboard (:7788) is host-run via ./sync.sh ui."
|
||||
tags: [decision, adr]
|
||||
created: 2026-06-22
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# ADR 002 — Tool stays host-run
|
||||
|
||||
## Context
|
||||
The tool opens the Foundry journal LevelDB (read-only) for the dashboard's full index, and
|
||||
`refresh --full-index` stops/starts a local Foundry container via `docker`. Both need host
|
||||
access. Containerizing it would require bind-mounting the LevelDB + docker socket — fragile.
|
||||
|
||||
## Decision
|
||||
The sync tool (dashboard, CLI) runs on the host. The compose bundles only the relay. The
|
||||
dashboard binds `0.0.0.0:7788` (reachable on the tailnet) via `./sync.sh ui`.
|
||||
|
||||
## Consequences
|
||||
- Cloner needs Node on the host (the tool runs via `npx tsx`).
|
||||
- `.env` carries host-specific paths (`VAULT`, `JOURNAL`, `FOUNDRY_DATA_DIR`) — gitignored.
|
||||
- The `/setup` skill starts the dashboard on the host, not via compose.
|
||||
|
||||
## Related
|
||||
- [[001-external-foundry]], [[003-relay-no-journal-db-no-docker-stop]]
|
||||
35
wiki/decisions/003-relay-no-journal-db-no-docker-stop.md
Normal file
35
wiki/decisions/003-relay-no-journal-db-no-docker-stop.md
Normal file
@@ -0,0 +1,35 @@
|
||||
---
|
||||
type: decision
|
||||
status: active
|
||||
date: 2026-06-20
|
||||
context: "Live push/refresh must not stop Foundry or fight the LevelDB write lock."
|
||||
decision: "Live ops (push, push-all, refresh) go through the relay. The LevelDB snapshot is only for the full dashboard index (CLI, docker-stop)."
|
||||
consequences: "Zero-downtime live sync. The index's pull ops reflect the SNAPSHOT, not live Foundry — a known limitation."
|
||||
tags: [decision, adr]
|
||||
created: 2026-06-22
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# ADR 003 — Relay for live ops; LevelDB only for full index
|
||||
|
||||
## Context
|
||||
Foundry holds the journal LevelDB open with a write lock. Reading it live requires stopping
|
||||
Foundry (the docker-stop path). That's downtime — unacceptable for routine push/refresh.
|
||||
|
||||
## Decision
|
||||
- **Push / push-all / refresh** use the [[relay-client | relay]] (`/get`, `/update`, `/search`).
|
||||
Foundry keeps running; no LevelDB open, no Docker stop. `cp -r` of the live journal dir is
|
||||
used for the dev dashboard's snapshot (works while locked — only opening it is blocked).
|
||||
- The **full LevelDB read** (full entry docs, not minified) is CLI-only: `refresh --full-index`
|
||||
stops/starts a local Foundry container. The dashboard's `indexAll` uses the snapshot copy.
|
||||
|
||||
## Consequences
|
||||
- Live sync is zero-downtime.
|
||||
- The dashboard index's `entry` is the SNAPSHOT (`db.byId`), so pull ops (`rePullRow`) reflect
|
||||
the snapshot, not live Foundry edits. Live edits need a fresh snapshot or a future live-pull
|
||||
path. See [[index-recommend]].
|
||||
- `/search` is minified (no content hash) → can't detect live content changes cheaply
|
||||
([[005-autosync-o-to-f-only]]).
|
||||
|
||||
## Related
|
||||
- [[002-tool-host-run]], [[push-flow]], [[refresh-flow]]
|
||||
37
wiki/decisions/004-dropped-browser-obsidian.md
Normal file
37
wiki/decisions/004-dropped-browser-obsidian.md
Normal file
@@ -0,0 +1,37 @@
|
||||
---
|
||||
type: decision
|
||||
status: active
|
||||
date: 2026-06-21
|
||||
context: "The containerized browser-Obsidian (noVNC) + auto-installed plugin were clunky and had restricted-mode friction."
|
||||
decision: "Removed the obsidian + obsidian-init compose services and the plugin/. Edit in your own Obsidian desktop app; the dashboard is the only UI."
|
||||
consequences: "Compose bundles only the relay. The foundry-sync Obsidian plugin no longer exists. Dashboard push/pull/sync + auto-sync replace it."
|
||||
tags: [decision, adr]
|
||||
created: 2026-06-22
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# ADR 004 — Dropped browser Obsidian + plugin
|
||||
|
||||
## Context
|
||||
Phase 1 of the shareable bundle added `ghcr.io/sytone/obsidian-remote` (browser Obsidian via
|
||||
noVNC) bind-mounted to the vault, plus a `foundry-sync` Obsidian plugin auto-installed by an
|
||||
`obsidian-init` one-shot. In practice: the noVNC desktop was heavy/laggy, plugin loading hit
|
||||
Obsidian's restricted-mode trust prompt (uncertain whether a preseed would be honored), and the
|
||||
user already edits a real desktop-Obsidian vault on this host (`Land of Mardonar`).
|
||||
|
||||
## Decision
|
||||
Removed entirely (2026-06-21):
|
||||
- `obsidian` + `obsidian-init` services deleted from `docker-compose.yml`.
|
||||
- `plugin/` directory and `scripts/obsidian-init.js` deleted.
|
||||
- `.env` / `.env.example` obsidian vars (`OBSIDIAN_*`, `DASHBOARD_URL`, `PUID/PGID/TZ`) removed.
|
||||
- README + `/setup` skill rewritten: relay + dashboard only; edit notes in your own Obsidian
|
||||
desktop app pointed at `$VAULT`.
|
||||
|
||||
## Consequences
|
||||
- Compose now bundles ONLY the relay (`docker compose config` → `relay` + `relay-gpu` profile).
|
||||
- The dashboard (:7788) is the sole UI — consistent with the user's hard rule: "if it's not in
|
||||
the dashboard UI, it doesn't exist." Push/pull/sync + auto-sync cover what the plugin did.
|
||||
- The user edits the vault in their own Obsidian app; the dashboard reads/writes the same files.
|
||||
|
||||
## Related
|
||||
- [[005-autosync-o-to-f-only]] — the dashboard feature that replaced the plugin's push button.
|
||||
40
wiki/decisions/005-autosync-o-to-f-only.md
Normal file
40
wiki/decisions/005-autosync-o-to-f-only.md
Normal file
@@ -0,0 +1,40 @@
|
||||
---
|
||||
type: decision
|
||||
status: active
|
||||
date: 2026-06-21
|
||||
context: "User asked for an auto-sync button so 'any file syncs from either direction instantly.'"
|
||||
decision: "Auto-sync is Obsidian→Foundry instant only (inotify + contentHash guard). Foundry→Obsidian stays manual (Sync / Re-pull)."
|
||||
consequences: "The common pain (manual push after editing a note) is solved. Foundry→Obsidian remains a dashboard button click — the relay can't push change events."
|
||||
tags: [decision, adr]
|
||||
created: 2026-06-22
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# ADR 005 — Auto-sync is Obsidian→Foundry only
|
||||
|
||||
## Context
|
||||
The user wanted hands-off sync both ways. Two facts shaped feasibility:
|
||||
1. The relay has **no change-push / events stream**. `/search` is minified (`{uuid,id,name,img}`)
|
||||
with **no content hash** — so a live Foundry content edit can't be detected from `/search`.
|
||||
2. Detecting a live Foundry edit needs a `/get` per matched entry per cycle and a diff. For a
|
||||
~200-entry world, a fast poll (~15s) is ~800 `/get` calls/min through the headless Foundry —
|
||||
heavy and fragile. The index's drift detection compares LOCAL baselines + a stale snapshot,
|
||||
not live Foundry ([[index-recommend]]).
|
||||
|
||||
So "instantly from either direction" is impossible: Obsidian→Foundry can be instant (inotify on
|
||||
the host vault); Foundry→Obsidian cannot.
|
||||
|
||||
## Decision
|
||||
Auto-sync ([[autosync]]) is **Obsidian→Foundry instant only**. Foundry→Obsidian stays manual
|
||||
via the dashboard's Sync / Re-pull buttons. The user was offered three scopes (both + 60s poll,
|
||||
O→F only, both + 15s poll) and chose O→F only.
|
||||
|
||||
## Consequences
|
||||
- Solves the actual pain: saving a linked, seeded note pushes it to live Foundry within ~1s.
|
||||
- Foundry→Obsidian requires a manual button click. A future live-pull path (relay `/get` per
|
||||
matched entry + `entryToObsidian` on a poll) is feasible but not built — cost scales with
|
||||
matched-entry count.
|
||||
- The guard + `baselineNote` are what keep it loop-free ([[autosync-watch-loop]]).
|
||||
|
||||
## Related
|
||||
- [[autosync]], [[autosync-watch-loop]], [[003-relay-no-journal-db-no-docker-stop]]
|
||||
15
wiki/decisions/_index.md
Normal file
15
wiki/decisions/_index.md
Normal file
@@ -0,0 +1,15 @@
|
||||
---
|
||||
type: meta
|
||||
title: "Decisions Index (ADRs)"
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# Decisions (ADRs)
|
||||
|
||||
Architecture Decision Records. Newest context at the bottom of the file; highest number = most recent. See `_templates/decision.md`.
|
||||
|
||||
- [[001-external-foundry]] — external Foundry (cloner supplies FOUNDRY_URL)
|
||||
- [[002-tool-host-run]] — sync tool runs on the host, not in a container
|
||||
- [[003-relay-no-journal-db-no-docker-stop]] — live ops via relay; LevelDB only for full index
|
||||
- [[004-dropped-browser-obsidian]] — removed containerized Obsidian + plugin
|
||||
- [[005-autosync-o-to-f-only]] — auto-sync is Obsidian→Foundry instant only
|
||||
14
wiki/dependencies/_index.md
Normal file
14
wiki/dependencies/_index.md
Normal file
@@ -0,0 +1,14 @@
|
||||
---
|
||||
type: meta
|
||||
title: "Dependencies Index"
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# Dependencies
|
||||
|
||||
External services, libraries, and modules. See `_templates/dependency.md`.
|
||||
|
||||
- [[threehats-relay]] — foundryvtt-rest-api-relay (Go + headless Chrome)
|
||||
- [[foundryvtt-rest-api-module]] — Foundry module that connects OUT to the relay
|
||||
- [[classic-level]] — journal LevelDB reader (classic-level npm)
|
||||
- [[linkedom]] — HTML parsing for Obsidian→Foundry conversion
|
||||
34
wiki/dependencies/classic-level.md
Normal file
34
wiki/dependencies/classic-level.md
Normal file
@@ -0,0 +1,34 @@
|
||||
---
|
||||
type: dependency
|
||||
name: "classic-level"
|
||||
version: "^3.0.0"
|
||||
kind: library
|
||||
status: active
|
||||
purpose: "Read the Foundry journal LevelDB (read-only) for the dashboard's full index."
|
||||
risk: low
|
||||
tags: [dependency]
|
||||
created: 2026-06-22
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# classic-level
|
||||
|
||||
## What it is
|
||||
The `classic-level` npm package — a Node binding for LevelDB. Used by `src/db.ts`
|
||||
(`JournalDb.open`, `all()`, `byId()`).
|
||||
|
||||
## Why we depend on it
|
||||
The dashboard's full index (`indexAll`) needs full journal entry docs (not the minified
|
||||
`/search` list). Those live in Foundry's journal LevelDB.
|
||||
|
||||
## How we use it
|
||||
- `JournalDb` opens the LevelDB read-only (a `cp -r` snapshot in dev, since Foundry holds the
|
||||
live DB with a write lock — `cp` works while locked, opening doesn't).
|
||||
- `db.byId(ccId)` returns the snapshot entry used by [[batch]] row builders + `handleEntries`/`handleLink`.
|
||||
|
||||
## Risk / lock-in
|
||||
- Low: stable, narrow use (read-only). Only the full-index path needs the live DB (docker-stop,
|
||||
CLI-only — [[003-relay-no-journal-db-no-docker-stop]]).
|
||||
|
||||
## Related
|
||||
- [[batch]], [[index-recommend]], [[003-relay-no-journal-db-no-docker-stop]]
|
||||
36
wiki/dependencies/foundryvtt-rest-api-module.md
Normal file
36
wiki/dependencies/foundryvtt-rest-api-module.md
Normal file
@@ -0,0 +1,36 @@
|
||||
---
|
||||
type: dependency
|
||||
name: "Foundry rest-api module"
|
||||
version: ""
|
||||
kind: module
|
||||
status: active
|
||||
purpose: "The Foundry-side module that exposes the relay's endpoints by connecting OUT to it."
|
||||
risk: medium
|
||||
tags: [dependency]
|
||||
created: 2026-06-22
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# Foundry rest-api module
|
||||
|
||||
## What it is
|
||||
A Foundry VTT module (`github.com/ThreeHats/foundryvtt-rest-api-relay` ecosystem) installed in
|
||||
the user's Foundry world. It connects OUT to the relay over WebSocket and services the
|
||||
relay's forwarded requests (CRUD on journal entries, search).
|
||||
|
||||
## Why we depend on it
|
||||
Without it, the relay has nothing to forward to — `activeClients` is 0 and every
|
||||
`/get`/`/update`/`/search` 404s ("No connected Foundry clients found" / "Invalid client ID").
|
||||
|
||||
## How we use it
|
||||
- The `/setup` skill tells the user: install the module in Foundry, enable it, and set its
|
||||
relay URL to `ws(s)://<host-Foundry-can-reach>:3010`. The relay must be reachable FROM the
|
||||
Foundry host ([[001-external-foundry]]).
|
||||
- With >1 connected client, requests need a `clientId` query param ([[relay-client]]).
|
||||
|
||||
## Risk / lock-in
|
||||
- Medium: Foundry-version compatibility; module defines the response envelopes.
|
||||
- Must be reconnected after Foundry/server restarts.
|
||||
|
||||
## Related
|
||||
- [[threehats-relay]], [[relay-client]], [[001-external-foundry]]
|
||||
32
wiki/dependencies/linkedom.md
Normal file
32
wiki/dependencies/linkedom.md
Normal file
@@ -0,0 +1,32 @@
|
||||
---
|
||||
type: dependency
|
||||
name: "linkedom"
|
||||
version: "^0.18.12"
|
||||
kind: library
|
||||
status: active
|
||||
purpose: "Parse HTML in the Obsidian→Foundry conversion (markdown→HTML→Campaign Codex JSON)."
|
||||
risk: low
|
||||
tags: [dependency]
|
||||
created: 2026-06-22
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# linkedom
|
||||
|
||||
## What it is
|
||||
A DOM-like HTML parser (`linkedom` npm). Used in the Obsidian→Foundry conversion path
|
||||
(`src/toFoundry.ts` / `obsidianToFoundryJsonLive`).
|
||||
|
||||
## Why we depend on it
|
||||
The note body is markdown → HTML → the Campaign Codex `flags.campaign-codex.data` structure.
|
||||
linkedom parses that HTML without a browser.
|
||||
|
||||
## How we use it
|
||||
- `obsidianToFoundryJsonLive` (called by `buildPushPayload` in [[push]]) builds the full
|
||||
JournalEntry + the minimal `/update` diff.
|
||||
|
||||
## Risk / lock-in
|
||||
- Low: pure parsing, no network. Only the conversion path depends on it.
|
||||
|
||||
## Related
|
||||
- [[push]], [[push-flow]]
|
||||
38
wiki/dependencies/threehats-relay.md
Normal file
38
wiki/dependencies/threehats-relay.md
Normal file
@@ -0,0 +1,38 @@
|
||||
---
|
||||
type: dependency
|
||||
name: "ThreeHats foundryvtt-rest-api-relay"
|
||||
version: latest
|
||||
kind: service
|
||||
status: active
|
||||
purpose: "REST/WebSocket bridge to a running Foundry world for live push/refresh without downtime."
|
||||
risk: medium
|
||||
tags: [dependency]
|
||||
created: 2026-06-22
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# ThreeHats foundryvtt-rest-api-relay
|
||||
|
||||
## What it is
|
||||
A Go relay (`github.com/ThreeHats/foundryvtt-rest-api-relay`) that runs a headless Chrome
|
||||
session logged into Foundry. Foundry's rest-api module connects OUT to it over WebSocket; API
|
||||
clients call the relay's HTTP endpoints (`/get`, `/update`, `/create`, `/search`, `/session`,
|
||||
`/session-handshake`, `/start-session`), which it forwards to the module.
|
||||
|
||||
## Why we depend on it
|
||||
Live push/refresh without stopping Foundry or touching the LevelDB write lock ([[003-relay-no-journal-db-no-docker-stop]]).
|
||||
|
||||
## How we use it
|
||||
- Bundled in `docker-compose.yml` as the `relay` service (port 3010, sqlite, `ALLOW_HEADLESS`,
|
||||
`shm_size 1g`; `relay-gpu` profile for `/dev/dri` passthrough).
|
||||
- API keys created via the relay's web signup (no env/CLI) → `RELAY_API_KEY`.
|
||||
- Headless session launched by `scripts/start-relay-session.js` (handshake → RSA-OAEP encrypt
|
||||
password → start-session). On the original host, `relay-keepalive.sh` keeps it alive.
|
||||
- [[relay-client]] wraps the endpoints we use.
|
||||
|
||||
## Risk / lock-in
|
||||
- Medium: external project; envelope shapes are module-defined and per-endpoint inconsistent.
|
||||
- No change-push/events — `/search` is minified with no content hash ([[005-autosync-o-to-f-only]]).
|
||||
|
||||
## Related
|
||||
- [[relay-client]], [[foundryvtt-rest-api-module]], [[refresh-flow]], [[push-flow]]
|
||||
14
wiki/flows/_index.md
Normal file
14
wiki/flows/_index.md
Normal file
@@ -0,0 +1,14 @@
|
||||
---
|
||||
type: meta
|
||||
title: "Flows Index"
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# Flows
|
||||
|
||||
Data flows / request paths. See `_templates/flow.md`.
|
||||
|
||||
- [[push-flow]] — Obsidian → live Foundry (relay /get → diff → /update)
|
||||
- [[refresh-flow]] — name↔uuid map via relay /search (zero downtime)
|
||||
- [[autosync-watch-loop]] — fs.watch → debounce → guard → push → baseline
|
||||
- [[index-recommend]] — indexAll + recommend() drift detection
|
||||
41
wiki/flows/autosync-watch-loop.md
Normal file
41
wiki/flows/autosync-watch-loop.md
Normal file
@@ -0,0 +1,41 @@
|
||||
---
|
||||
type: flow
|
||||
status: active
|
||||
purpose: "On a vault save, push the changed note to live Foundry instantly (Obsidian→Foundry only)."
|
||||
actors: [[autosync]], [[push]], [[foundry-block]], [[content-hash]], [[relay-client]]
|
||||
steps: [fs.watch event, filter .md, debounce, guard, pushNote, baselineNote, log event]
|
||||
tags: [flow]
|
||||
created: 2026-06-22
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# autosync-watch-loop
|
||||
|
||||
Driven by [[autosync]] `AutoSyncController`, toggled from the dashboard's "Auto-sync" button.
|
||||
|
||||
## Steps
|
||||
1. `fs.watch(refinedDir, {recursive:true})` (per-dir fallback) fires on a `.md` change.
|
||||
2. Filter: must end `.md`, not be a `.bak`, not under `.obsidian/`.
|
||||
3. Debounce 700ms per file (coalesce rapid saves).
|
||||
4. `process(relPath)`:
|
||||
- read via `resolveRefined` (mirror-aware);
|
||||
- **guard**: skip unless `foundry.cc_uuid` (linked) AND `foundry.contentHash` (seeded) AND
|
||||
`contentHash(body) !== foundry.contentHash` (actually changed);
|
||||
- else `pushNote({dryRun:false})` ([[push-flow]]) → `baselineNote`.
|
||||
5. Log to the 100-event ring buffer (`pushed` / `skipped` / `error`); UI polls `GET /api/autosync`.
|
||||
|
||||
## Why no loop
|
||||
After a push, `baselineNote` sets `foundry.contentHash = contentHash(body)`. The watcher then
|
||||
fires on that baseline write too, but the guard reads `body hash == contentHash` → `skipped:
|
||||
unchanged`. So the baseline write is a no-op for the watcher. No feedback loop, no double-push.
|
||||
|
||||
## Edge cases / failure modes
|
||||
- Unlinked note (no `cc_uuid`) → `skipped: not linked` (seed/link first).
|
||||
- Unseeded note (no `contentHash` baseline) → `skipped: not seeded`.
|
||||
- Relay down / no headless session → `pushNote` → `relay.getEntry` 404 → `error` logged (no
|
||||
crash, no Foundry write); the note is NOT baselined, so it retries on the next real save.
|
||||
- New note created from scratch → no `foundry:` block → skipped (seed/link it manually first).
|
||||
|
||||
## Related
|
||||
- [[005-autosync-o-to-f-only]] — why Foundry→Obsidian isn't in this loop.
|
||||
- [[push-flow]] — the per-note push this loop invokes.
|
||||
41
wiki/flows/index-recommend.md
Normal file
41
wiki/flows/index-recommend.md
Normal file
@@ -0,0 +1,41 @@
|
||||
---
|
||||
type: flow
|
||||
status: active
|
||||
purpose: "Build the file↔entry index and decide a per-row recommendation from local baselines."
|
||||
actors: [[batch]], [[content-hash]], [[foundry-block]], db.ts
|
||||
steps: [walk refined+cc, read meta, basename match + uuid fallback, look up snapshot entry, recommend]
|
||||
tags: [flow]
|
||||
created: 2026-06-22
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# index-recommend
|
||||
|
||||
`indexAll` + `recommend` in [[batch]]. Rebuilt on every `GET /api/index` (hash-only, cheap) and
|
||||
after mutating ops. Powers the dashboard's recommendation badges + bulk actions.
|
||||
|
||||
## Steps
|
||||
1. Walk refined + cc dirs (with dev-mode `<out>` overlays).
|
||||
2. Read each file's meta: refined → `foundry.contentHash` (storedHash), body hash, mtime;
|
||||
cc → `cc_id`, `cc_sync_hash`, body hash, mtime.
|
||||
3. Match by **basename** (canonical); **uuid fallback** via `foundry.cc_uuid` for curated
|
||||
notes whose filename differs.
|
||||
4. `entry = db.byId(ccId)` — from the LevelDB **snapshot**.
|
||||
5. `recommend(...)`: `seed` | `sync-cc` (vault newer) | `repull` (cc newer) | `conflict` |
|
||||
`in-sync` | `import` | `review`.
|
||||
|
||||
## Drift is LOCAL-baseline-based, not live
|
||||
- `refinedChanged` = `foundry.contentHash` != `contentHash(body)` — vault newer than the LAST
|
||||
BASELINE (set by a prior push/sync). Not "newer than live Foundry."
|
||||
- `ccChanged` = `cc_sync_hash` != cc body hash — cc file newer than its baseline.
|
||||
- `entry` is the snapshot, so pull ops (`rePullRow`) reflect the snapshot, not live Foundry.
|
||||
|
||||
## Implications
|
||||
- The index is a fast, local, offline drift signal — great for "what have I edited since I
|
||||
last pushed?" It does NOT know about live Foundry edits someone else made.
|
||||
- Live Foundry edits are only surfaced via the relay push/refresh path. Detecting a live
|
||||
content change requires `/get` per entry (no hash in `/search`) — see [[005-autosync-o-to-f-only]].
|
||||
|
||||
## Related
|
||||
- [[content-hash]], [[foundry-block]] — the baselines this compares.
|
||||
- [[push-flow]] / [[refresh-flow]] — the live paths that complement this offline index.
|
||||
39
wiki/flows/push-flow.md
Normal file
39
wiki/flows/push-flow.md
Normal file
@@ -0,0 +1,39 @@
|
||||
---
|
||||
type: flow
|
||||
status: active
|
||||
purpose: "Push one refined note's content into LIVE Foundry without stopping Foundry."
|
||||
actors: [[push]], [[relay-client]], [[foundry-block]], [[name-uuid-resolver]], foundry/assets.ts
|
||||
steps: [read note, process body images, read foundry.cc_uuid, resolve name↔uuid, relay /get live entry, build minimal diff, backup live entry, relay /update, baseline note]
|
||||
tags: [flow]
|
||||
created: 2026-06-22
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# push-flow
|
||||
|
||||
Obsidian → live Foundry. Driven by [[push]] `pushNote`, invoked from the dashboard's push /
|
||||
push-all buttons and from [[autosync]].
|
||||
|
||||
## Steps
|
||||
1. Read the refined note; pre-process body `![[image]]` embeds → upload to Foundry's uploads
|
||||
dir, rewrite to `` (drop non-local). Frontmatter untouched.
|
||||
2. `readNoteFoundryMeta` → `foundry.cc_uuid` (required — else "run seed first") + portrait.
|
||||
3. Resolve the name↔uuid map (preloaded → `<out>/name-uuid.json` → build via `/search`).
|
||||
4. `relay.getEntry(cc_uuid)` — the LIVE entry (keeps ownership/folder/pages/existing image).
|
||||
5. Upload portrait if present (needs `FOUNDRY_DATA_DIR`/`WORLD`).
|
||||
6. `buildPushPayload` → minimal diff `{ name, "flags.campaign-codex": cc }` (dot-path merge
|
||||
preserves sibling flags; no `_id`/`pages`/`ownership`).
|
||||
7. dryRun → return diff. Apply → back up live entry to `<out>/bak/<name>.<stamp>.json`, then
|
||||
`relay.updateEntry(uuid, diff)`.
|
||||
8. (caller) `baselineNote` → set `foundry.contentHash = contentHash(body)` so a re-run skips.
|
||||
|
||||
## Edge cases / failure modes
|
||||
- No `foundry.cc_uuid` → throws (seed/link first).
|
||||
- No connected Foundry client → relay `404`/`400` ("Invalid client ID" / lists clients). Start
|
||||
the headless session.
|
||||
- Relay timeout `408`/`504` (~10s WS round-trip).
|
||||
- Image upload skipped when `FOUNDRY_DATA_DIR`/`WORLD` unset (existing image kept).
|
||||
|
||||
## Related
|
||||
- [[autosync-watch-loop]] — same flow, triggered by a vault save.
|
||||
- [[refresh-flow]] — builds the name↔uuid map this depends on.
|
||||
37
wiki/flows/refresh-flow.md
Normal file
37
wiki/flows/refresh-flow.md
Normal file
@@ -0,0 +1,37 @@
|
||||
---
|
||||
type: flow
|
||||
status: active
|
||||
purpose: "Rebuild the cached name↔uuid map live via relay /search — zero Foundry downtime."
|
||||
actors: [[relay-client]], [[name-uuid-resolver]], [[server]], [[cli]]
|
||||
steps: [relay /search minified, nameUuidIndexFromEntries, save name-uuid.json]
|
||||
tags: [flow]
|
||||
created: 2026-06-22
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# refresh-flow
|
||||
|
||||
`handleRefresh` (dashboard `POST /api/refresh`) and `cmd refresh` (CLI). Builds `name-uuid.json`
|
||||
so pushes can resolve note mentions to Foundry UUIDs.
|
||||
|
||||
## Steps
|
||||
1. `relay.searchJournalEntries()` → `GET /search?filter=documentType:JournalEntry&minified=true`
|
||||
→ `results: [{uuid,id,name,img}]`.
|
||||
2. `nameUuidIndexFromEntries(results.map(r => ({name, uuid})))` → `{nameToUuid, uuidToName}`.
|
||||
3. `saveNameUuidIndex(idx, <out>/name-uuid.json)`.
|
||||
4. Returns `{pairs, path}`.
|
||||
|
||||
## Why /search (not the LevelDB)
|
||||
`/search` is minified and live — it lists every journal entry with zero Foundry downtime
|
||||
(no Docker stop, no LevelDB lock). The heavy docker-stop full LevelDB read is CLI-only
|
||||
(`refresh --full-index`) and only needed for the dashboard's full `indexAll` (which needs full
|
||||
entry docs, not minified). See [[003-relay-no-journal-db-no-docker-stop]].
|
||||
|
||||
## Edge cases / failure modes
|
||||
- 0 connected clients → `404`; >1 → `400` (pass `clientId`).
|
||||
- Minified results have NO content hash — so refresh detects new/renamed entries but NOT
|
||||
content changes. That asymmetry is why Foundry→Obsidian isn't auto ([[005-autosync-o-to-f-only]]).
|
||||
|
||||
## Related
|
||||
- [[push-flow]] — consumes the map.
|
||||
- [[index-recommend]] — the index uses the LevelDB snapshot, not this map.
|
||||
17
wiki/modules/_index.md
Normal file
17
wiki/modules/_index.md
Normal file
@@ -0,0 +1,17 @@
|
||||
---
|
||||
type: meta
|
||||
title: "Modules Index"
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# Modules
|
||||
|
||||
One note per major module / package. See `_templates/module.md`.
|
||||
|
||||
- [[server]] — HTTP dashboard + JSON API
|
||||
- [[autosync]] — vault watcher → live push
|
||||
- [[push]] — single-note live push
|
||||
- [[relay-client]] — ThreeHats relay client
|
||||
- [[batch]] — index + row builders
|
||||
- [[cli]] — commands
|
||||
- [[config]] — env-driven config
|
||||
50
wiki/modules/autosync.md
Normal file
50
wiki/modules/autosync.md
Normal file
@@ -0,0 +1,50 @@
|
||||
---
|
||||
type: module
|
||||
path: "src/server.ts (AutoSyncController)"
|
||||
status: active
|
||||
language: typescript
|
||||
purpose: "Watch the refined vault and push each saved linked+seeded note into live Foundry instantly (Obsidian→Foundry only)."
|
||||
maintainer: Kaysser Kayyali
|
||||
last_updated: 2026-06-22
|
||||
linked_issues: []
|
||||
depends_on: [[push]], [[relay-client]], [[content-hash]], [[foundry-block]]
|
||||
used_by: [[server]]
|
||||
tags: [module]
|
||||
created: 2026-06-22
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# autosync
|
||||
|
||||
`AutoSyncController` in `src/server.ts`. Toggled via the dashboard's "Auto-sync" button
|
||||
(`POST /api/autosync {enabled}`); status + activity log at `GET /api/autosync` (the UI polls
|
||||
every 2s while on). See [[005-autosync-o-to-f-only]] for why it's one-directional.
|
||||
|
||||
## How it works
|
||||
1. `fs.watch(refinedDir, {recursive:true})` (per-dir fallback for older Node) → `onChange`.
|
||||
2. Filter to `*.md`, skip `.obsidian/` and `.bak`. Debounce 700ms per file.
|
||||
3. `process(relPath)`:
|
||||
- read the note via `resolveRefined` (mirror-aware);
|
||||
- **guard**: skip unless `foundry.cc_uuid` exists (linked), `foundry.contentHash` exists
|
||||
(seeded), and `contentHash(body) !== foundry.contentHash` (actually changed);
|
||||
- `pushNote` (live, `dryRun:false`) → `baselineNote` (rewrite `foundry.contentHash` to the
|
||||
new body hash).
|
||||
4. Bounded concurrency (3), per-file in-flight mutex, 100-event ring buffer (`events`).
|
||||
|
||||
## The guard is the loop/idempotency key
|
||||
`pushNote` always PUTs. The guard skips no-op saves (body hash == baseline). After a push,
|
||||
`baselineNote` sets the baseline to the new body hash, so the watcher's own baseline write
|
||||
(also a `.md` change) reads as unchanged → skipped. No feedback loop, no redundant push.
|
||||
|
||||
## Mode behavior
|
||||
- **dev**: baseline lands in the `--out` mirror (consistent with dev semantics); the mirror
|
||||
then wins `resolveRefined` until the real vault is edited again.
|
||||
- **apply**: baseline lands in the real vault (with `.bak`).
|
||||
- The push itself goes live via the relay regardless of mode.
|
||||
|
||||
## Notes / gotchas
|
||||
- Needs the relay + a live headless session (`activeClients > 0`), or `pushNote` → `relay.getEntry`
|
||||
404s ("Invalid client ID" / no connected clients) and logs an `error` (no crash, no Foundry write).
|
||||
- Verified 2026-06-22: no-op save → `skipped: unchanged`; scratch note with fake uuid + body
|
||||
change → guard passed → push attempted → relay 404 → `error` logged. Real-push end-to-end
|
||||
pending the dev headless session coming back up.
|
||||
56
wiki/modules/batch.md
Normal file
56
wiki/modules/batch.md
Normal file
@@ -0,0 +1,56 @@
|
||||
---
|
||||
type: module
|
||||
path: "src/batch.ts"
|
||||
status: active
|
||||
language: typescript
|
||||
purpose: "Index both corpora against the journal DB; produce match rows + recommendations; build per-row outputs."
|
||||
maintainer: Kaysser Kayyali
|
||||
last_updated: 2026-06-22
|
||||
linked_issues: []
|
||||
depends_on: [[content-hash]], [[foundry-block]], db.ts, toObsidian.ts
|
||||
used_by: [[server]], [[cli]]
|
||||
tags: [module]
|
||||
created: 2026-06-22
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# batch
|
||||
|
||||
`src/batch.ts` — `indexAll`, `recommend`, and the row builders (`seedRow`, `syncRow`,
|
||||
`rePullRow`, `importRow`). See [[index-recommend]].
|
||||
|
||||
## indexAll(db, ccDir, refinedDir, refinedOverlay?, ccOverlay?)
|
||||
Walks refined + cc dirs (with dev-mode overlays), reads each file's meta, matches by
|
||||
**basename** (canonical — the exporter names cc files by entry name), with **uuid fallback**
|
||||
for curated notes whose filename differs but are already linked by `foundry.cc_uuid`. Produces:
|
||||
`matched`, `ccOnly`, `refinedOnly`, `counts`, `byRecommendation`. In dev mode the server
|
||||
passes the `<out>/refined` + `<out>/cc` mirrors as overlays so dev writes are reflected.
|
||||
|
||||
## recommend(params) → Recommendation
|
||||
```
|
||||
refined-only → review
|
||||
cc-only (entry?) → import : review
|
||||
matched-unlinked → review
|
||||
matched+linked, !seeded → seed
|
||||
!hasCc | !ccSynced → sync-cc
|
||||
refinedChanged && ccChanged → conflict
|
||||
refinedChanged → sync-cc (vault newer → push)
|
||||
ccChanged → repull (cc/foundry newer → pull)
|
||||
else → in-sync
|
||||
```
|
||||
`refinedChanged` = note's `foundry.contentHash` != note body hash (vault newer than last
|
||||
baseline). `ccChanged` = cc file's `cc_sync_hash` != cc body hash. **Both compare against
|
||||
LOCAL baselines, not live Foundry.**
|
||||
|
||||
## Row builders
|
||||
- `seedRow` — inject/refresh the `foundry:` block from the journal entry (curation untouched).
|
||||
- `syncRow` — regenerate cc.md from the refined note AND refresh the refined `foundry:` block;
|
||||
baselines both sides.
|
||||
- `rePullRow` — regenerate the refined body from Foundry (`entryToObsidian(row.entry, …)`),
|
||||
preserving curated type/aliases/status tags. Uses the snapshot `row.entry`.
|
||||
- `importRow` — cc-only entry → new un-curated refined note under `refined/imported/<folder>/`.
|
||||
|
||||
## Notes / gotchas
|
||||
- `row.entry` comes from `db.byId(ccId)` — the LevelDB **snapshot**, not live Foundry. So
|
||||
pull ops reflect the snapshot; live Foundry edits need a fresh snapshot (docker-stop full
|
||||
index) or a future live-pull path. See [[003-relay-no-journal-db-no-docker-stop]].
|
||||
32
wiki/modules/cli.md
Normal file
32
wiki/modules/cli.md
Normal file
@@ -0,0 +1,32 @@
|
||||
---
|
||||
type: module
|
||||
path: "src/cli.ts"
|
||||
status: active
|
||||
language: typescript
|
||||
purpose: "Entry point (tsx). Commands: ui (dashboard), refresh, push. Env-driven via [[config]]."
|
||||
maintainer: Kaysser Kayyali
|
||||
last_updated: 2026-06-22
|
||||
linked_issues: []
|
||||
depends_on: [[server]], [[batch]], [[relay-client]], [[push]], [[config]]
|
||||
used_by: []
|
||||
tags: [module]
|
||||
created: 2026-06-22
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# cli
|
||||
|
||||
`src/cli.ts` — run via `npx tsx src/cli.ts <cmd>` or `./sync.sh <cmd>` (which sources `.env`
|
||||
and injects `--journal`).
|
||||
|
||||
## Commands (to document fully)
|
||||
- `ui` → `startServer` ([[server]]). `--vault`, `--cc` (optional — auto-builds stubs),
|
||||
`--out`, `--host` (default `0.0.0.0`), `--apply` (write real dirs; default dev/dry-run).
|
||||
Relay enabled when `RELAY_API_KEY` is set. Binds `0.0.0.0:7788`.
|
||||
- `refresh` → `name-uuid.json` via relay `/search` ([[refresh-flow]]). `--full-index` does
|
||||
the heavy docker-stop LevelDB read (CLI-only by design).
|
||||
- `push` → [[push]] one note live.
|
||||
|
||||
## Notes
|
||||
- `sync.sh` loads `.env` (`set -a; . ./.env`) and injects `--journal` from `$JOURNAL`.
|
||||
- This page is a stub — expand with exact flags + the `cmdUi`/`cmdRefresh`/`cmdPush` bodies.
|
||||
29
wiki/modules/config.md
Normal file
29
wiki/modules/config.md
Normal file
@@ -0,0 +1,29 @@
|
||||
---
|
||||
type: module
|
||||
path: "src/config.ts"
|
||||
status: active
|
||||
language: typescript
|
||||
purpose: "Env-driven config loaders for the relay + Foundry host control."
|
||||
maintainer: Kaysser Kayyali
|
||||
last_updated: 2026-06-22
|
||||
linked_issues: []
|
||||
depends_on: []
|
||||
used_by: [[server]], [[cli]], [[relay-client]]
|
||||
tags: [module]
|
||||
created: 2026-06-22
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# config
|
||||
|
||||
`src/config.ts` — `loadRelayConfig` / `loadFoundryConfig` (and the `RelayConfig` /
|
||||
`FoundryHostConfig` types). Reads `RELAY_URL`, `RELAY_API_KEY`, `RELAY_CLIENT_ID` for the
|
||||
relay; `FOUNDRY_CONTAINER`, `FOUNDRY_DATA_DIR`, world for host control. See `.env.example`
|
||||
for the full var list.
|
||||
|
||||
## Notes
|
||||
- The relay is enabled in the dashboard iff `RELAY_API_KEY` is set.
|
||||
- `FOUNDRY_CONTAINER` / `FOUNDRY_DATA_DIR` are only used by `refresh --full-index` (stop/start
|
||||
a local Foundry container + read its live LevelDB). Live push and plain refresh go through
|
||||
the relay and never touch them. See [[003-relay-no-journal-db-no-docker-stop]].
|
||||
- This page is a stub — expand with the exact env→field mapping.
|
||||
48
wiki/modules/push.md
Normal file
48
wiki/modules/push.md
Normal file
@@ -0,0 +1,48 @@
|
||||
---
|
||||
type: module
|
||||
path: "src/push.ts"
|
||||
status: active
|
||||
language: typescript
|
||||
purpose: "Push one refined note into LIVE Foundry via the relay (Foundry keeps running)."
|
||||
maintainer: Kaysser Kayyali
|
||||
last_updated: 2026-06-22
|
||||
linked_issues: []
|
||||
depends_on: [[relay-client]], [[name-uuid-resolver]], [[foundry-block]], foundry/assets.ts, toFoundry.ts
|
||||
used_by: [[server]], [[autosync]], [[cli]]
|
||||
tags: [module]
|
||||
created: 2026-06-22
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# push
|
||||
|
||||
`src/push.ts` — `pushNote(deps)` and the pure `buildPushPayload(md, noteName, liveEntry, resolver, imageOverride?)`.
|
||||
|
||||
## pushNote flow (see [[push-flow]])
|
||||
1. Read the note; pre-process body `![[image]]` embeds (upload co-located files to Foundry's
|
||||
uploads dir, rewrite to ``, drop non-local). Frontmatter untouched.
|
||||
2. `readNoteFoundryMeta` → `foundry.cc_uuid` (throws if absent — "run seed first") + portrait.
|
||||
3. Resolve the name↔uuid map: preloaded `resolver`, else `<out>/name-uuid.json`, else build via
|
||||
relay `/search`.
|
||||
4. `relay.getEntry(id)` — fetch the LIVE entry (preserves ownership/folder/pages/existing image).
|
||||
5. Portrait upload if the note has one (and `foundryDataDir`/`world` set).
|
||||
6. `buildPushPayload` → `obsidianToFoundryJsonLive` (full entry with `name` + `flags.campaign-codex`
|
||||
overridden, links resolved) → minimal diff `{ name, "flags.campaign-codex": cc }` (dot-path merge
|
||||
preserves sibling flags; never echoes `_id`/`pages`/`ownership`).
|
||||
7. `dryRun` → return diff. Apply → back up the live entry to `<out>/bak/<name>.<stamp>.json`
|
||||
(reversible), then `relay.updateEntry(id, diff)`.
|
||||
|
||||
## Critical: no internal idempotency
|
||||
`pushNote` ALWAYS PUTs (unless `dryRun`). It does NOT compare the note body to anything. The
|
||||
caller is responsible for gating: the dashboard's push-all and [[autosync]] call `baselineNote`
|
||||
afterward to set `foundry.contentHash = contentHash(body)` so a re-run skips unchanged notes.
|
||||
Without that, repeated pushes re-send the same diff.
|
||||
|
||||
## Depends on
|
||||
- [[relay-client]], [[name-uuid-resolver]], [[foundry-block]] (`splitFrontmatter`, `readFoundryBlock`),
|
||||
`src/foundry/assets.ts` (portrait + body-image upload), `src/toFoundry.ts` (`obsidianToFoundryJsonLive`).
|
||||
|
||||
## Notes / gotchas
|
||||
- Image upload needs `FOUNDRY_DATA_DIR` + `FOUNDRY_WORLD`; without them, images are skipped and
|
||||
existing ones kept. See [[foundry-uploads-convention]] (in the user's memory, not this wiki).
|
||||
- `skipImageUpload` is used for fast batch dry-run previews that shouldn't touch the Foundry data dir.
|
||||
43
wiki/modules/relay-client.md
Normal file
43
wiki/modules/relay-client.md
Normal file
@@ -0,0 +1,43 @@
|
||||
---
|
||||
type: module
|
||||
path: "src/relay/client.ts"
|
||||
status: active
|
||||
language: typescript
|
||||
purpose: "HTTP client over the ThreeHats foundryvtt-rest-api-relay (x-api-key auth, clientId query param)."
|
||||
maintainer: Kaysser Kayyali
|
||||
last_updated: 2026-06-22
|
||||
linked_issues: []
|
||||
depends_on: [[threehats-relay]], [[config]]
|
||||
used_by: [[push]], [[server]], [[cli]]
|
||||
tags: [module]
|
||||
created: 2026-06-22
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# relay-client
|
||||
|
||||
`src/relay/client.ts` — `RelayClient(cfg)` wraps the relay's pass-through REST API. See
|
||||
`docs/relay-api.md` for envelope shapes.
|
||||
|
||||
## Methods
|
||||
| Method | Relay call | Returns |
|
||||
|---|---|---|
|
||||
| `getEntry(uuid)` | `GET /get?uuid=` | `data: <JournalEntry>` |
|
||||
| `updateEntry(uuid, diff)` | `PUT /update?uuid=` body `{data:<diff>}` | `entity: [<doc>]` |
|
||||
| `createEntry(entityType, data)` | `POST /create` | `{uuid, data}` |
|
||||
| `searchJournalEntries()` | `GET /search?filter=documentType:JournalEntry&minified=true` | `results: [{uuid,id,name,img}]` |
|
||||
|
||||
## Mechanics
|
||||
- Base URL: no `/api` prefix. Auth: `x-api-key` header on every request.
|
||||
- **World selection**: `clientId` query param. Omit only when exactly one Foundry client is
|
||||
connected to the key (relay auto-resolves); 0 → `404`, >1 → `400` listing clients.
|
||||
- The relay forwards to Foundry's rest-api module over WebSocket and returns the module's
|
||||
response verbatim. Envelopes are per-endpoint (`data` / `entity` / `results`).
|
||||
- WS round-trip timeout ~10s → `408`/`504`.
|
||||
|
||||
## Notes / gotchas
|
||||
- For `update`, send a minimal diff (dot-path `flags.campaign-codex`), never the full doc —
|
||||
echoing `_id`/`pages`/`ownership` would clobber the live entry.
|
||||
- `/search` is **minified** (`{uuid,id,name,img}`) — no content hash. So you cannot detect a
|
||||
Foundry content change from `/search` alone; you'd need `/get` per entry. This is why
|
||||
Foundry→Obsidian auto-sync wasn't built ([[005-autosync-o-to-f-only]]).
|
||||
59
wiki/modules/server.md
Normal file
59
wiki/modules/server.md
Normal file
@@ -0,0 +1,59 @@
|
||||
---
|
||||
type: module
|
||||
path: "src/server.ts"
|
||||
status: active
|
||||
language: typescript
|
||||
purpose: "Local review dashboard + JSON API over the batch engine; binds 0.0.0.0:7788."
|
||||
maintainer: Kaysser Kayyali
|
||||
last_updated: 2026-06-22
|
||||
linked_issues: []
|
||||
depends_on: [[batch]], [[push]], [[relay-client]], [[content-hash]], [[foundry-block]], [[name-uuid-resolver]]
|
||||
used_by: [[cli]]
|
||||
tags: [module]
|
||||
created: 2026-06-22
|
||||
updated: 2026-06-22
|
||||
---
|
||||
|
||||
# server
|
||||
|
||||
`src/server.ts` — `startServer(cfg)` builds the index once, serves `dashboard.html` at `/`
|
||||
and a JSON API under `/api/*`. The mode (`dev` | `apply`) is fixed at startup; the UI cannot
|
||||
escalate to apply unless the server was started with `--apply`.
|
||||
|
||||
## Mode + mirror-aware reads
|
||||
- **dev**: writes land under `--out/<bucket>/<relPath>` (a mirror). `resolveRefined` /
|
||||
`resolveCc` read the mirror only if it exists AND is at least as new (mtime) as the real
|
||||
file — otherwise the real vault (edited in Obsidian) wins. This makes dev a true preview
|
||||
that never masks live vault edits.
|
||||
- **apply**: `targetPath` writes the real refined/cc dir; `writeWithBackup` copies a
|
||||
`.bak-<stamp>` first (reversible).
|
||||
- dry-run never writes; it collects `preview` entries.
|
||||
|
||||
## Routes
|
||||
| Method | Path | Handler |
|
||||
|---|---|---|
|
||||
| GET | `/` | `dashboard.html` |
|
||||
| GET | `/api/index` | rebuild index, return `IndexResult` (rebuilt every request — hash-only, cheap) |
|
||||
| GET | `/api/status` | `{mode, refinedDir, ccDir, outDir}` |
|
||||
| GET | `/api/file?name=` | per-row detail + seed/sync/re-pull previews |
|
||||
| GET | `/api/entries` | Foundry journal entries (name+uuid+type) from the snapshot, for the Link picker |
|
||||
| POST | `/api/action` | `runAction` — seed/sync/repull/import + All variants |
|
||||
| POST | `/api/push` | [[push]] one note (live) |
|
||||
| POST | `/api/push-all` | push every vault-newer matched note (live), baseline each |
|
||||
| POST | `/api/link` | inject the `foundry:` identity block (link a refined-only note by uuid) |
|
||||
| POST | `/api/refresh` | rebuild `name-uuid.json` via relay `/search` |
|
||||
| GET/POST | `/api/autosync` | [[autosync]] status / toggle |
|
||||
|
||||
## State
|
||||
`{ db, cfg, index, autosync }`. `index` is rebuilt on every `/api/index` and after mutating
|
||||
ops (push-all, link) so recommendations reflect current files.
|
||||
|
||||
## Depends on
|
||||
- [[batch]] (`indexAll`, row builders), [[push]] (`pushNote`), [[relay-client]], [[content-hash]],
|
||||
[[foundry-block]] (`readFoundryBlock`, `splitFrontmatter`), [[name-uuid-resolver]].
|
||||
|
||||
## Notes / gotchas
|
||||
- The index's drift detection compares against LOCAL baselines + the LevelDB snapshot — not
|
||||
live Foundry. Live edits are only seen via the relay push/refresh path. See [[index-recommend]].
|
||||
- `pushNote` has no internal idempotency; the dashboard's push-all (and [[autosync]]) baseline
|
||||
the note's `foundry.contentHash` afterward so re-runs skip unchanged notes.
|
||||
Reference in New Issue
Block a user