chore: first

This commit is contained in:
2026-06-20 19:15:38 +00:00
commit 74f76a820d
41 changed files with 6421 additions and 0 deletions

6
.gitignore vendored Normal file
View File

@@ -0,0 +1,6 @@
node_modules/
# Copied world-data fixtures (your private Foundry snapshot) — never commit.
tests/fixtures/
# Apply-mode backups written by --apply.
*.bak-*
*.foundry.json

117
README.md Normal file
View File

@@ -0,0 +1,117 @@
# foundry-obsidian-sync
A strict, **offline**, bidirectional converter between Foundry VTT Campaign Codex
journal data (a LevelDB snapshot) and an Obsidian vault. No Foundry instance, no
macros, no network — just the exported DB.
Two markdown formats are bridged:
- **Obsidian curated** (`Roland Raventhorne.md`) — frontmatter `type/tags/aliases/
faction/region/race/portrait`, italic tagline, `## Appearance / Personality /
Background / Goals / Secrets`, `[[Name|Display]]` links.
- **Campaign Codex native export** (`Roland Raventhornecc.md`) — frontmatter
`cc_id/cc_uuid/cc_type/cc_folder_path/cc_exported_at`, `# Title`, `## Information`,
body sections, `#### Bio`/`#### Social` boxes, `## Linked Sheets`/`### Associates`,
`## Notes`, `[[Name]]` links.
Foundry is the source of truth. Each Obsidian note carries a `foundry:` identity
block (`cc_uuid`, `cc_type`, `folder_path`, `contentHash`, `syncedAt`) so the two
sides stay linked across syncs and renames.
## How it works
1. `db.ts` opens the journal LevelDB **read-only** with `classic-level`, indexes every
Campaign Codex JournalEntry, and resolves `@UUID` ⇄ note name.
2. `htmlMd.ts` ports the `Macros/cc-4-export.js` HTML→Markdown logic (tagline,
sections, Bio/Social boxes) but resolves `@UUID` to `[[Name|Display]]` via the index.
3. `toObsidian.ts` / `toFoundry.ts` convert each direction. `toFoundry` pulls
`cc_id`/associates/image from the matched Foundry entry (by `foundry:` block, then
name). `normalize.ts` canonicalizes wikilinks/whitespace and hashes content so
cosmetic edits don't churn.
## Modes (safety)
The tool **defaults to dev mode** and never risks the real world data.
- **`--dev` (default):** reads your *copy* of the world data, writes **only** into
`--out`. Sources are never mutated. Refuses `--out` equal to `--journal` or the
source vault dir.
- **`--dry-run`:** computes outputs and prints diffs only — writes nothing.
- **`--apply`:** writes in place to the real vault/cc dir, writing a timestamped
`.bak-<iso>` of every file it overwrites first. Gated explicitly.
The journal LevelDB is opened read-only in all modes; the code only ever calls
read methods (`get`/`iterator`), never `put`/`del`/`batch`.
## Usage
```bash
npm install
# Prep a dev copy of your world data (you do this, the tool won't):
cp -r ~/hosting/"Lore examples"/journal /tmp/journal-copy
cp -r ~/hosting/"Lore examples" /tmp/lore-copy
# Foundry -> Obsidian (one entry by --id, or all if omitted)
npx tsx src/cli.ts to-obsidian --dev --journal /tmp/journal-copy --out /tmp/devout --id flGsAYaK24eUZhQE
# Obsidian -> cc.md (+ optional Foundry-importable JSON with --emit-json)
npx tsx src/cli.ts to-foundry --dev --journal /tmp/journal-copy \
--vault "/tmp/lore-copy/Roland Raventhorne.md" --out /tmp/devoutcc --emit-json
# See what would change, write nothing
npx tsx src/cli.ts to-foundry --dry-run --journal ... --vault ... --out ...
```
## Batch dashboard — connect the whole vault at once
For more than one entry, the `ui` command starts a **local-only** review dashboard
(127.0.0.1, no external network) over the batch engine. It indexes the journal
LevelDB + your cc export + your refined vault, shows every file's match status and
diff, and runs the three "connect" operations:
- **Seed** — inject the `foundry:` identity block into each refined note (matched by
filename → `cc_id` → journal entry). Minimal: only the block is added, leaving all
curation (curated `type`, `aliases`, status tags, hand-edited body, links) untouched.
- **Sync → cc** — regenerate each `cc.md` from its curated refined note, so curation
(new `[[links]]`, edited prose) flows back into the Campaign Codex export.
- **Import** — pull the cc-only entries (no refined counterpart) into new refined
notes under `refined/imported/<cc_folder>/` for curation.
```bash
# Prep dev copies (you do this; the tool won't):
cp -r ~/hosting/"Lore examples"/journal /tmp/journal-copy
cp -r ~/hosting/"Lore examples" /tmp/lore-copy
npx tsx src/cli.ts ui --dev --journal /tmp/journal-copy \
--vault "/tmp/lore-copy/Obsidian vault/Land of Mardonar/Refined" \
--cc "/tmp/lore-copy/campaign codex" --out /tmp/devout
# → open http://127.0.0.1:7788
```
The dashboard opens in dev mode with **dry-run on by default** — actions preview
without writing. Uncheck dry-run to write into the `--out` sandbox. To write back to
the real refined/cc dirs (with timestamped `.bak-<iso>` backups), start with `--apply`
instead of `--dev`; the UI cannot escalate to apply unless the server was started
with it. The journal LevelDB is read-only in all modes.
API (for scripting): `GET /api/status`, `GET /api/index`, `GET /api/file?name=`,
`POST /api/action` with `{op:"seed"|"sync"|"import"|"seedAll"|"syncAll"|"importAll",
names?:[], dryRun?:bool}`.
## Test
```bash
npm test # 30 tests: links, Roland round-trip, dev/dry-run/apply safety, batch + server
```
## Notes
- `status/*` tags and `aliases` are curation-only on the Obsidian side; they are
preserved across sync, never sourced from or written to Foundry.
- `--emit-json` reconstructs a Foundry-importable JournalEntry JSON with clean
(not byte-identical) description/notes HTML — for a future push back into a live
Foundry world via the existing macros or a DB write, both out of scope here.
- The cc.md format drops wiki-link display aliases (`[[Name|Display]]` → `[[Name]]`),
matching Campaign Codex's native export. Display text round-trips losslessly only
through the Foundry JSON / `foundry:`-backed sync, not through cc.md.

77
docs/relay-api.md Normal file
View File

@@ -0,0 +1,77 @@
# ThreeHats foundryvtt-rest-api-relay — API shapes
Source: `github.com/ThreeHats/foundryvtt-rest-api-relay` (Go relay in `go-relay/`;
docs at foundryrestapi.com). Confirmed against route definitions + integration tests.
Our instance: `vtt-relay.damascusfront.net` (a self-hosted ThreeHats relay).
## Global mechanics
- **Base URL:** no `/api` prefix. Paths appended directly to the base.
- **Auth:** `x-api-key: <key>` header on every request.
- **World selection:** `clientId` **query parameter** (NOT a header). Omit only when
exactly one Foundry client is connected to the key (relay auto-resolves); with 0 →
`404 {"error":"No connected Foundry clients found"}`, with >1 → `400` listing clients.
- **Pass-through:** the relay forwards params to the Foundry module over WebSocket and
returns the module's response verbatim. Envelopes are module-defined (per-endpoint).
- **Relay-level errors:** flat `{"error": "..."}` with non-2xx status. Timeouts →
`408`/`504 {"error":"Request timed out"}`.
## Endpoints (the ones we use)
| Endpoint | Method | Path | Identifies via | Body | 200 envelope |
|---|---|---|---|---|---|
| get | GET | `/get` | `?uuid=` | — | `{ data: <doc> }` |
| update | PUT | `/update` | `?uuid=` | `{ data: <diff> }` | `{ entity: [<doc>, ...] }` |
| create | POST | `/create` | body `entityType`+`data` | `{ entityType, data }` | `{ uuid, data: <doc> }` |
| search | GET | `/search` | `?filter=documentType:JournalEntry` (omit query to list all) | — | `{ query?, results: [...] }` |
### GET /get
```
GET /get?clientId=<id>&uuid=JournalEntry.<id>
x-api-key: <key>
→ 200 { data: { name, type, _id, uuid, folder, pages, ownership, flags, ... } }
```
### PUT /update (scope: entity:write)
`data` is a **diff** — dot-path keys merge (e.g. `"flags.campaign-codex"` updates only
that sub-flag, preserving other flags); full keys replace. No `entityType` field (the
uuid carries the type).
```
PUT /update?clientId=<id>&uuid=JournalEntry.<id>
x-api-key: <key>
{ "data": { "name": "...", "flags.campaign-codex": { type, image, data } } }
→ 200 { entity: [ <updated doc>, ... ] }
```
### POST /create (scope: entity:write)
```
POST /create?clientId=<id>
{ "entityType": "JournalEntry", "data": { "name": "My Entry" } }
→ 200 { uuid: "JournalEntry.<newId>", data: { ... } }
```
### GET /search (scope: search) — list ALL journal entries, zero downtime
```
GET /search?clientId=<id>&filter=documentType:JournalEntry&excludeCompendiums=true&limit=500&minified=true
→ 200 { results: [ { uuid, id, name, img, documentType } ] }
```
This is how we build the name→uuid map live (no Foundry stop). `minified=true` trims
each result to `{ uuid, id, name, img, documentType }`.
## How we use it
- **`cmd refresh`** builds `name-uuid.json` via `searchJournalEntries()` (zero downtime).
The docker-stop LevelDB read is only for the full dashboard `indexAll` (needs full
entry docs, not minified).
- **`cmd push`** fetches the live entry via `getEntry(uuid)`, builds the diff
`{ name, "flags.campaign-codex": cc }` (dot-path merge preserves other flags), and
calls `updateEntry(uuid, diff)`.
## Notes for the TS client
- `clientId` is always a query param; never a header.
- For `update`, send a minimal diff, not the full document — never echo `_id`/`pages`/
`ownership` (those would clobber the live entry). Use dot-path `flags.campaign-codex`
to avoid wiping sibling flags.
- Envelopes are inconsistent: parse per-endpoint (`data` / `entity` / `results`).
- WS round-trip default timeout ~10s → `408`/`504` on timeout.

2198
package-lock.json generated Normal file

File diff suppressed because it is too large Load Diff

22
package.json Normal file
View File

@@ -0,0 +1,22 @@
{
"name": "foundry-obsidian-sync",
"version": "0.1.0",
"description": "Strict offline bidirectional converter between Foundry Campaign Codex journal data (LevelDB) and Obsidian vault markdown.",
"type": "module",
"private": true,
"scripts": {
"test": "vitest run",
"test:watch": "vitest",
"cli": "tsx src/cli.ts"
},
"dependencies": {
"classic-level": "^3.0.0",
"linkedom": "^0.18.12"
},
"devDependencies": {
"@types/node": "^22.0.0",
"tsx": "^4.22.4",
"typescript": "^5.6.0",
"vitest": "^4.1.9"
}
}

372
src/batch.ts Normal file
View File

@@ -0,0 +1,372 @@
// Batch connector + stateless merge engine.
//
// Indexes the refined vault + cc export against the journal LevelDB and, per
// file, computes a merge RECOMMENDATION with no external state file. Direction is
// derived from per-side self-hashes embedded in the files' own frontmatter:
//
// - refined note: foundry.contentHash (hash of the refined body, written at
// seed/sync time — already part of the foundry: identity block)
// - cc.md: cc_sync_hash (hash of the cc body, written at sync time)
//
// Each side records what IT was at the last sync, so "did this side change since
// last sync?" is a local self-comparison that survives copies (the hash travels
// with the content, unlike mtime, which `cp -r` resets to now). mtime is used only
// as a tiebreaker hint when both sides changed.
//
// Matching is by filename (verified: all 107 refined notes match a cc file by
// exact basename). The cc file's cc_id frontmatter resolves to the Foundry
// JournalEntry; cc files whose cc_id is absent from the journal are "unlinked".
// The journal DB is read-only everywhere.
import { readdir, readFile, stat } from "node:fs/promises";
import { join, relative, sep } from "node:path";
import type { JournalDb } from "./db.js";
import type { JournalEntry, FoundryBlock } from "./types.js";
import { entryToObsidian } from "./toObsidian.js";
import { obsidianToCc } from "./toFoundry.js";
import { splitFrontmatter, emitFrontmatter, readFoundryBlock, type Frontmatter, OBSIDIAN_FM_ORDER, CC_FM_ORDER } from "./frontmatter.js";
import { folderPathFromCcType } from "./fields.js";
import { canonicalize, contentHash } from "./normalize.js";
export type RowStatus = "matched" | "matched-unlinked" | "cc-only" | "cc-only-unlinked" | "refined-only";
/** What the merge engine recommends doing for a file, in priority order. */
export type Recommendation =
| "import" // cc-only entry: pull it into the vault
| "seed" // matched but not yet linked (no foundry: block)
| "sync-cc" // refined is newer than last sync (or cc side not yet baselined) -> push to cc
| "repull" // cc/Foundry is newer than last sync -> pull into refined (curation preserved)
| "conflict" // both sides changed since last sync -> manual review
| "in-sync" // neither side changed -> nothing to do
| "review"; // unlinked or refined-only: needs a human decision
export interface FileRow {
name: string;
basename: string;
status: RowStatus;
refinedPath: string | null;
ccPath: string | null;
ccId: string | null;
ccType: string | null;
curatedType: string | null;
entry: JournalEntry | null;
recommendation: Recommendation;
refinedChanged: boolean;
ccChanged: boolean;
refinedMtime: number | null; // epoch ms, for a tiebreaker hint
ccMtime: number | null;
storedRefinedHash: string | null; // foundry.contentHash at last sync
storedCcHash: string | null; // cc_sync_hash at last sync
}
export interface IndexResult {
matched: FileRow[]; // includes matched-unlinked
ccOnly: FileRow[];
refinedOnly: FileRow[];
counts: { matched: number; ccOnly: number; refinedOnly: number; unlinked: number };
/** Count per recommendation, for the dashboard's action panel. */
byRecommendation: Record<Recommendation, number>;
}
const SKIP_DIRS = new Set([".trash", ".obsidian", "node_modules", "_Templates", ".git"]);
async function walkMd(root: string): Promise<string[]> {
let out: string[] = [];
try {
for await (const ent of await readdir(root, { withFileTypes: true })) {
if (SKIP_DIRS.has(ent.name)) continue;
const p = join(root, ent.name);
if (ent.isDirectory()) out = out.concat(await walkMd(p));
else if (ent.isFile() && ent.name.toLowerCase().endsWith(".md")) out.push(p);
}
} catch {
// missing dir -> empty
}
return out;
}
function basenameOf(p: string): string { return p.split(sep).pop() ?? p; }
function nameOf(p: string): string { return basenameOf(p).replace(/\.md$/i, ""); }
function rel(root: string, p: string): string { return relative(root, p); }
async function readRefinedMeta(path: string): Promise<{ type: string | null; seeded: boolean; storedHash: string | null; bodyHash: string; mtime: number | null; body: string; raw: string | null }> {
try {
const raw = await readFile(path, "utf8");
const { fm, body } = splitFrontmatter(raw);
const foundry = readFoundryBlock(fm);
return {
type: typeof fm.type === "string" ? fm.type : null,
seeded: !!foundry,
storedHash: foundry?.contentHash ?? null,
bodyHash: contentHash(body),
mtime: (await stat(path)).mtimeMs,
body,
raw,
};
} catch {
return { type: null, seeded: false, storedHash: null, bodyHash: "", mtime: null, body: "", raw: null };
}
}
async function readCcMeta(path: string): Promise<{ ccId: string | null; ccType: string | null; syncHash: string | null; bodyHash: string; mtime: number | null; raw: string | null }> {
try {
const raw = await readFile(path, "utf8");
const { fm, body } = splitFrontmatter(raw);
return {
ccId: typeof fm.cc_id === "string" ? fm.cc_id : null,
ccType: typeof fm.cc_type === "string" ? fm.cc_type : null,
syncHash: typeof fm.cc_sync_hash === "string" ? fm.cc_sync_hash : null,
bodyHash: contentHash(body),
mtime: (await stat(path)).mtimeMs,
raw,
};
} catch {
return { ccId: null, ccType: null, syncHash: null, bodyHash: "", mtime: null, raw: null };
}
}
/** Decide the recommendation from per-side change flags + linkage state. */
export function recommend(params: {
status: RowStatus;
seeded: boolean;
hasCc: boolean;
ccSynced: boolean; // cc has cc_sync_hash
refinedChanged: boolean;
ccChanged: boolean;
entry: JournalEntry | null;
}): Recommendation {
if (params.status === "refined-only") return "review";
if (params.status === "cc-only" || params.status === "cc-only-unlinked") return params.entry ? "import" : "review";
if (params.status === "matched-unlinked") return "review";
// matched + linked
if (!params.seeded) return "seed";
if (!params.hasCc) return "sync-cc"; // refined linked but no cc counterpart to compare
if (!params.ccSynced) return "sync-cc"; // cc side never baselined -> baseline it
if (params.refinedChanged && params.ccChanged) return "conflict";
if (params.refinedChanged) return "sync-cc";
if (params.ccChanged) return "repull";
return "in-sync";
}
/** Index both corpora against the journal DB and produce match rows + recommendations. */
export async function indexAll(db: JournalDb, ccDir: string, refinedDir: string): Promise<IndexResult> {
const [refinedFiles, ccFiles] = await Promise.all([walkMd(refinedDir), walkMd(ccDir)]);
const ccByBasename = new Map<string, string>();
for (const p of ccFiles) ccByBasename.set(basenameOf(p), p);
const matched: FileRow[] = [];
const refinedNames = new Set<string>();
for (const rp of refinedFiles) {
const base = basenameOf(rp);
refinedNames.add(base);
const cp = ccByBasename.get(base) ?? null;
const rmeta = await readRefinedMeta(rp);
let ccId: string | null = null, ccType: string | null = null, ccSyncHash: string | null = null;
let ccChanged = false, ccMtime: number | null = null;
if (cp) {
const cmeta = await readCcMeta(cp);
ccId = cmeta.ccId; ccType = cmeta.ccType; ccSyncHash = cmeta.syncHash; ccMtime = cmeta.mtime;
ccChanged = ccSyncHash !== null && ccSyncHash !== cmeta.bodyHash;
}
const entry = ccId ? (db.byId(ccId) ?? null) : null;
const status: RowStatus = cp ? (entry ? "matched" : "matched-unlinked") : "refined-only";
const refinedChanged = rmeta.seeded && rmeta.storedHash !== null && rmeta.storedHash !== rmeta.bodyHash;
const recommendation = recommend({
status, seeded: rmeta.seeded, hasCc: !!cp, ccSynced: ccSyncHash !== null,
refinedChanged, ccChanged, entry,
});
matched.push({
name: nameOf(rp), basename: base, status,
refinedPath: rel(refinedDir, rp), ccPath: cp ? rel(ccDir, cp) : null,
ccId, ccType, curatedType: rmeta.type, entry,
recommendation, refinedChanged, ccChanged,
refinedMtime: rmeta.mtime, ccMtime,
storedRefinedHash: rmeta.storedHash, storedCcHash: ccSyncHash,
});
}
const ccOnly: FileRow[] = [];
for (const cp of ccFiles) {
const base = basenameOf(cp);
if (refinedNames.has(base)) continue;
const cmeta = await readCcMeta(cp);
const entry = cmeta.ccId ? (db.byId(cmeta.ccId) ?? null) : null;
const status: RowStatus = entry ? "cc-only" : "cc-only-unlinked";
ccOnly.push({
name: nameOf(cp), basename: base, status,
refinedPath: null, ccPath: rel(ccDir, cp),
ccId: cmeta.ccId, ccType: cmeta.ccType, curatedType: null, entry,
recommendation: entry ? "import" : "review",
refinedChanged: false, ccChanged: false,
refinedMtime: null, ccMtime: cmeta.mtime,
storedRefinedHash: null, storedCcHash: cmeta.syncHash,
});
}
const refinedOnly = matched.filter((r) => r.status === "refined-only");
const linked = matched.filter((r) => r.status !== "refined-only");
const unlinked = matched.filter((r) => r.status === "matched-unlinked").length + ccOnly.filter((r) => !r.entry).length;
const all = [...linked, ...ccOnly];
const byRecommendation = emptyRecs();
for (const r of all) byRecommendation[r.recommendation]++;
return {
matched: linked,
ccOnly, refinedOnly,
counts: {
matched: matched.filter((r) => r.status === "matched").length,
ccOnly: ccOnly.length, refinedOnly: refinedOnly.length, unlinked,
},
byRecommendation,
};
}
function emptyRecs(): Record<Recommendation, number> {
return { import: 0, seed: 0, "sync-cc": 0, repull: 0, conflict: 0, "in-sync": 0, review: 0 };
}
export interface OpResult {
written: { path: string; bytes: number }[];
preview: { filename: string; path: string; content: string }[];
skipped: { name: string; reason: string }[];
message: string;
}
/** Build the foundry: identity block for a refined note from its journal entry. */
export function buildBlock(entry: JournalEntry, refinedBody: string, syncedAt: string): FoundryBlock {
const ccType = entry.flags?.["campaign-codex"]?.type ?? "shop";
return {
cc_uuid: `JournalEntry.${entry._id}`,
cc_type: ccType,
folder_path: folderPathFromCcType(ccType),
contentHash: contentHash(refinedBody),
syncedAt,
};
}
/** Render the foundry: block as frontmatter text lines. */
export function renderBlockLines(block: FoundryBlock): string[] {
return [
"foundry:",
` cc_uuid: ${block.cc_uuid}`,
` cc_type: ${block.cc_type}`,
` folder_path: ${block.folder_path}`,
` contentHash: ${block.contentHash}`,
` syncedAt: ${block.syncedAt}`,
];
}
/**
* Inject (or replace) the foundry: block in a refined note via raw-text surgery,
* leaving EVERYTHING else byte-identical. This is the minimal "link" operation:
* it avoids the type-downgrade edge case by never regenerating content.
*/
export function seedBlockContent(md: string, block: FoundryBlock): string {
const m = md.match(/^(---\r?\n)([\s\S]*?)(\r?\n---\r?\n?)([\s\S]*)$/);
if (!m) return `---\n${renderBlockLines(block).join("\n")}\n---\n\n${md}`;
const [, open, fmRaw, close, body] = m;
const lines = fmRaw.split(/\r?\n/);
let i = 0;
while (i < lines.length) {
if (lines[i] === "foundry:") {
let j = i + 1;
while (j < lines.length && lines[j].startsWith(" ")) j++;
lines.splice(i, j - i);
break;
}
i++;
}
const blockLines = renderBlockLines(block);
if (lines.length && lines[lines.length - 1].trim() !== "") lines.push("");
lines.push(...blockLines);
return `${open}${lines.join("\n")}${close}${body}`;
}
/** Read a refined note and return its seeded content (foundry: block only). */
export async function seedRow(row: FileRow, refinedDir: string, syncedAt: string): Promise<{ filename: string; content: string } | null> {
if (!row.refinedPath || !row.entry) return null;
const md = await readFile(join(refinedDir, row.refinedPath), "utf8");
const { body } = splitFrontmatter(md);
return { filename: row.basename, content: seedBlockContent(md, buildBlock(row.entry, body, syncedAt)) };
}
/** Inject cc_sync_hash into a cc.md's frontmatter (re-emits frontmatter; body untouched). */
export function stampCcSyncHash(ccContent: string, syncHash: string): string {
const { fm, body } = splitFrontmatter(ccContent);
fm.cc_sync_hash = syncHash;
return `${emitFrontmatter(fm, CC_FM_ORDER, true)}\n\n${body}`;
}
/**
* Sync refined -> cc AND baseline both sides in one op:
* - cc.md regenerated from the refined note (curation flows back), with
* cc_sync_hash = hash of the cc body.
* - refined note's foundry: block refreshed (contentHash = current refined
* body hash, syncedAt = now) so the refined self-hash matches what we just
* confirmed as the source of truth.
* Returns both files' content. Establishes the per-file ancestor for future
* direction detection with no external state.
*/
export async function syncRow(row: FileRow, refinedDir: string, db: JournalDb, exportedAt: string): Promise<{ refined: { filename: string; content: string } | null; cc: { filename: string; content: string } | null }> {
if (!row.refinedPath) return { refined: null, cc: null };
const full = join(refinedDir, row.refinedPath);
const md = await readFile(full, "utf8");
const { body } = splitFrontmatter(md);
const cc = obsidianToCc(md, row.name, db, exportedAt);
if (!cc) return { refined: null, cc: null };
const ccBodyHash = contentHash(splitFrontmatter(cc.content).body);
const ccStamped = stampCcSyncHash(cc.content, ccBodyHash);
// Refresh the refined foundry: block so its self-hash matches the current body.
const refinedContent = row.entry ? seedBlockContent(md, buildBlock(row.entry, body, exportedAt)) : md;
return {
refined: { filename: row.basename, content: refinedContent },
cc: { filename: cc.filename, content: ccStamped },
};
}
/**
* Re-pull: Foundry/cc is newer than the vault. Regenerate the refined note body
* from the Foundry entry while preserving curation (curated type, aliases,
* status/* tags, and any hand-set faction/region/race/portrait). Writes a fresh
* foundry: block with contentHash of the new body. The curated `type` is NEVER
* downgraded to Foundry's cc_type (the type-drift edge case).
*/
export async function rePullRow(row: FileRow, refinedDir: string, db: JournalDb, syncedAt: string): Promise<{ filename: string; content: string } | null> {
if (!row.refinedPath || !row.entry) return null;
const existing = await readFile(join(refinedDir, row.refinedPath), "utf8");
const ex = splitFrontmatter(existing);
const pulled = entryToObsidian(row.entry, db, syncedAt);
const pu = splitFrontmatter(pulled.content);
const merged: Frontmatter = { ...pu.fm }; // pulled body + fresh foundry block
// Preserve curated type (don't downgrade to Foundry's cc_type).
if (typeof ex.fm.type === "string") merged.type = ex.fm.type;
// Preserve curation-only aliases.
if (Array.isArray(ex.fm.aliases) && ex.fm.aliases.length) merged.aliases = ex.fm.aliases;
// Preserve existing status/* tags; pulled tags otherwise stand.
const exStatus = (Array.isArray(ex.fm.tags) ? ex.fm.tags : []).filter((t) => t.startsWith("status/"));
if (exStatus.length) {
const tags = (Array.isArray(merged.tags) ? merged.tags : []).filter((t) => !t.startsWith("status/"));
merged.tags = [...tags, ...exStatus];
}
// Preserve hand-curated sidebar/portrait fields if the vault already had them.
for (const f of ["faction", "region", "race", "portrait"] as const) {
if (typeof ex.fm[f] === "string" && (ex.fm[f] as string) !== "") (merged as Record<string, unknown>)[f] = ex.fm[f];
}
const frontmatter = emitFrontmatter(merged, OBSIDIAN_FM_ORDER);
return { filename: row.basename, content: `${frontmatter}\n\n${canonicalize(pu.body)}\n` };
}
/** Pull a cc-only Foundry entry into a new refined note. */
export function importRow(row: FileRow, db: JournalDb, syncedAt: string): { filename: string; content: string; subfolder: string } | null {
if (!row.entry) return null;
const note = entryToObsidian(row.entry, db, syncedAt);
const ccType = row.entry.flags?.["campaign-codex"]?.type ?? "shop";
// Name the imported file after the cc file's basename (not entry.name) so it
// matches the cc file by filename. The Foundry exporter sanitizes names
// differently than entryToObsidian (e.g. `"Patches"` -> `-Patches-`), so using
// entry.name for the filename can leave the import unmatched to its cc file and
// re-imported on every "Import all". The body's # Title still uses entry.name;
// the link is by cc_uuid anyway.
return { filename: row.basename, content: note.content, subfolder: folderPathFromCcType(ccType) };
}

303
src/cli.ts Normal file
View File

@@ -0,0 +1,303 @@
import { mkdir, readdir, readFile, writeFile, copyFile, access } from "node:fs/promises";
import { join, dirname } from "node:path";
import { JournalDb } from "./db.js";
import { entryToObsidian } from "./toObsidian.js";
import { obsidianToCc, obsidianToFoundryJson } from "./toFoundry.js";
import { indexAll } from "./batch.js";
import { loadRelayConfig, loadFoundryConfig } from "./config.js";
import { RelayClient } from "./relay/client.js";
import { docker, acquireLock, releaseLock, lockPathFor } from "./foundry/docker.js";
import { nameUuidIndexFromEntries, saveNameUuidIndex } from "./resolver.js";
import { pushNote } from "./push.js";
import { backupStamp } from "./write.js";
import type { CliOptions } from "./types.js";
function parseArgs(argv: string[]): { cmd: string; opts: CliOptions } {
const args = argv.slice(2);
const cmd = args[0];
if (!cmd) throw new Error("missing command: to-obsidian | to-foundry | roundtrip | ui | refresh | push");
const opts: CliOptions = {
mode: "dev",
dryRun: false,
journal: "",
out: "",
emitJson: false,
};
let name: string | undefined;
for (let i = 1; i < args.length; i++) {
const a = args[i];
const next = () => args[++i];
switch (a) {
case "--journal": opts.journal = next(); break;
case "--out": opts.out = next(); break;
case "--vault": opts.vault = next(); break;
case "--cc": opts.cc = next(); break;
case "--id": opts.id = next(); break;
case "--name": name = next(); break;
case "--emit-json": opts.emitJson = true; break;
case "--port": opts.port = Number(next()); break;
case "--host": opts.host = next(); break;
case "--dev": opts.mode = "dev"; break;
case "--apply": opts.mode = "apply"; break;
case "--dry-run": opts.dryRun = true; break;
case "--relay-url": opts.relayUrl = next(); break;
case "--relay-api-key": opts.relayApiKey = next(); break;
case "--relay-client-id": opts.relayClientId = next(); break;
case "--foundry-container": opts.foundryContainer = next(); break;
case "--foundry-data-dir": opts.foundryDataDir = next(); break;
case "--foundry-world": opts.foundryWorld = next(); break;
case "--full-index": opts.fullIndex = true; break;
default: throw new Error(`unknown flag: ${a}`);
}
}
opts.name = name;
if (!opts.journal) throw new Error("--journal <path> is required");
return { cmd, opts };
}
async function ensureOut(opts: CliOptions): Promise<string> {
if (!opts.out) throw new Error("--out <dir> is required");
if (opts.out === opts.journal) throw new Error("--out must not equal --journal (refusing to write into the source DB)");
if (opts.mode === "dev" && opts.vault && opts.out === dirname(opts.vault)) {
throw new Error("--out must not be the source vault directory in dev mode");
}
if (opts.mode === "dev" && opts.cc && opts.out === opts.cc) {
throw new Error("--out must not be the source cc directory in dev mode");
}
// Never let tool output land inside Foundry's Data volume — that's Foundry's
// filesystem; writing backups/indexes there risks polluting the live world.
if (opts.foundryDataDir && opts.out.startsWith(opts.foundryDataDir)) {
throw new Error(`--out must not be inside FOUNDRY_DATA_DIR (${opts.foundryDataDir}) — keep tool output out of Foundry's Data volume`);
}
return opts.out;
}
async function writeNote(outDir: string, filename: string, content: string, opts: CliOptions): Promise<void> {
const target = join(outDir, filename);
if (opts.dryRun) {
let existing = "";
try { existing = await readFile(target, "utf8"); } catch { /* new file */ }
console.log(`[dry-run] ${target}`);
if (existing) console.log(diffSummary(existing, content));
else console.log(" (new file)");
return;
}
await mkdir(outDir, { recursive: true });
if (opts.mode === "apply") {
try {
await access(target);
const stamp = backupStamp();
await copyFile(target, `${target}.bak-${stamp}`);
console.log(`[apply] backed up ${filename} -> ${filename}.bak-${stamp}`);
} catch { /* not present, no backup needed */ }
}
await writeFile(target, content, "utf8");
console.log(`wrote ${target}`);
}
function diffSummary(before: string, after: string): string {
if (before === after) return " (unchanged)";
const b = before.split("\n");
const a = after.split("\n");
let same = 0, diff = 0;
const n = Math.max(b.length, a.length);
for (let i = 0; i < n; i++) { if (b[i] === a[i]) same++; else diff++; }
return ` (${same} lines same, ${diff} differ)`;
}
export async function cmdToObsidian(opts: CliOptions): Promise<void> {
const out = await ensureOut(opts);
const db = await JournalDb.open(opts.journal);
try {
const e = opts.id ? db.byId(opts.id) : undefined;
const entries = opts.id ? (e ? [e] : []) : db.all();
if (!entries.length) throw new Error(`no matching entries${opts.id ? ` for --id ${opts.id}` : ""}`);
const stamp = new Date().toISOString();
for (const entry of entries) {
const note = entryToObsidian(entry, db, stamp);
await writeNote(out, note.filename, note.content, opts);
}
console.log(`to-obsidian: ${entries.length} note(s) [${opts.mode}${opts.dryRun ? " dry-run" : ""}]`);
} finally { await db.close(); }
}
export async function cmdToFoundry(opts: CliOptions): Promise<void> {
const out = await ensureOut(opts);
if (!opts.vault) throw new Error("--vault <file> is required for to-foundry");
const db = await JournalDb.open(opts.journal);
try {
const md = await readFile(opts.vault, "utf8");
const noteName = opts.name ?? basenameNoExt(opts.vault);
const stamp = new Date().toISOString();
const cc = obsidianToCc(md, noteName, db, stamp);
if (!cc) throw new Error(`could not match "${noteName}" to a Foundry entry (no foundry: block and no name match)`);
await writeNote(out, cc.filename, cc.content, opts);
if (opts.emitJson) {
const json = obsidianToFoundryJson(md, noteName, db);
if (json) await writeNote(out, `${noteName}.foundry.json`, JSON.stringify(json, null, 2) + "\n", opts);
}
console.log(`to-foundry: cc.md written [${opts.mode}${opts.dryRun ? " dry-run" : ""}]`);
} finally { await db.close(); }
}
async function cmdRoundtrip(opts: CliOptions): Promise<void> {
const out = await ensureOut(opts);
if (!opts.id) throw new Error("roundtrip requires --id <id>");
const db = await JournalDb.open(opts.journal);
try {
const entry = db.byId(opts.id);
if (!entry) throw new Error(`no entry for --id ${opts.id}`);
const stamp = new Date().toISOString();
const obs = entryToObsidian(entry, db, stamp);
await writeNote(out, `1-${obs.filename}`, obs.content, opts);
const cc = obsidianToCc(obs.content, entry.name, db, stamp);
if (!cc) throw new Error("roundtrip: obsidian->cc failed");
await writeNote(out, `2-${cc.filename}`, cc.content, opts);
// Foundry is source of truth: the re-pull is byte-stable, so reuse `obs`
// rather than recomputing an identical third note.
await writeNote(out, `3-back-${obs.filename}`, obs.content, opts);
console.log(`roundtrip: 1-obsidian, 2-cc, 3-back written to ${out}`);
} finally { await db.close(); }
}
function basenameNoExt(p: string): string {
const base = p.split("/").pop() ?? p;
return base.replace(/\.md$/i, "");
}
export async function cmdUi(opts: CliOptions): Promise<void> {
const out = await ensureOut(opts);
if (!opts.vault) throw new Error("--vault <refined-dir> is required for ui");
if (!opts.cc) throw new Error("--cc <cc-dir> is required for ui");
const { startServer } = await import("./server.js");
const relayCfg = loadRelayConfig({ url: opts.relayUrl, apiKey: opts.relayApiKey, clientId: opts.relayClientId });
const foundryCfg = loadFoundryConfig({ container: opts.foundryContainer, dataDir: opts.foundryDataDir, world: opts.foundryWorld });
const cfg = {
journal: opts.journal,
refinedDir: opts.vault,
ccDir: opts.cc,
outDir: out,
mode: opts.mode,
port: opts.port ?? 7788,
host: opts.host ?? "127.0.0.1",
relayCfg: relayCfg.apiKey ? relayCfg : undefined,
foundryCfg: foundryCfg.dataDir ? foundryCfg : undefined,
};
const { server } = await startServer(cfg);
const shown = cfg.host === "0.0.0.0" ? "<this-vm-ip>" : cfg.host;
console.log(`foundry-obsidian dashboard → http://${shown}:${cfg.port} [${cfg.mode}${opts.dryRun ? " dry-run" : ""}]`);
if (cfg.host === "0.0.0.0") console.log(` bound 0.0.0.0 — reachable on your tailnet at this VM's IP, port ${cfg.port}`);
console.log(` refined: ${cfg.refinedDir}`);
console.log(` cc: ${cfg.ccDir}`);
console.log(` out: ${cfg.outDir}`);
console.log(` push: ${cfg.relayCfg ? `enabled (relay ${cfg.relayCfg.url})` : "disabled (set RELAY_API_KEY to enable)"}`);
// Keep the process alive serving until killed.
server.on("close", () => process.exit(0));
}
/**
* refresh: rebuild the name↔uuid map the push path uses for link resolution.
*
* Default (zero Foundry downtime): relay /search lists all JournalEntries live;
* we cache name-uuid.json from that. With --full-index: stop the Foundry Docker
* container, read the real journal LevelDB read-only, run indexAll for the
* dashboard, then restart the container. The lockfile guards against a push
* running while the container is down.
*/
export async function cmdRefresh(opts: CliOptions): Promise<void> {
const out = await ensureOut(opts);
const relay = new RelayClient(loadRelayConfig({ url: opts.relayUrl, apiKey: opts.relayApiKey, clientId: opts.relayClientId }));
const fc = loadFoundryConfig({ container: opts.foundryContainer, dataDir: opts.foundryDataDir, world: opts.foundryWorld });
// 1. Live name↔uuid map via /search (no Foundry stop).
console.log("refresh: listing journal entries via relay /search (zero downtime)…");
const results = await relay.searchJournalEntries();
const idx = nameUuidIndexFromEntries(results.map((r) => ({ name: r.name, uuid: r.uuid })));
await saveNameUuidIndex(idx, join(out, "name-uuid.json"));
console.log(`refresh: cached ${Object.keys(idx.nameToUuid).length} name↔uuid pairs -> ${join(out, "name-uuid.json")}`);
// 2. Optional full dashboard index from the live LevelDB (stops Foundry).
if (opts.fullIndex) {
if (!opts.vault || !opts.cc) throw new Error("--full-index requires --vault <refined-dir> and --cc <cc-dir>");
if (!fc.container) throw new Error("--full-index requires FOUNDRY_CONTAINER (--foundry-container) to stop/start Foundry");
const lock = lockPathFor(out);
await acquireLock(lock, "refresh");
let downStart = 0;
try {
const wasRunning = await docker.isRunning(fc.container);
if (wasRunning) {
console.log(`refresh: stopping Foundry container ${fc.container} for live LevelDB read…`);
downStart = Date.now();
await docker.stop(fc.container);
} else {
console.log(`refresh: Foundry container ${fc.container} already stopped — reading live DB.`);
}
const db = await JournalDb.open(opts.journal);
try {
const index = await indexAll(db, opts.cc, opts.vault);
await writeFile(join(out, "index.json"), JSON.stringify(index, null, 2) + "\n", "utf8");
console.log(`refresh: wrote dashboard index -> ${join(out, "index.json")} (matched ${index.counts.matched}, cc-only ${index.counts.ccOnly})`);
} finally {
await db.close();
}
if (wasRunning) {
await docker.start(fc.container);
console.log(`refresh: restarted Foundry container (down ${Math.round((Date.now() - downStart) / 1000)}s)`);
}
} finally {
await releaseLock(lock);
}
}
console.log(`refresh done [${opts.mode}${opts.dryRun ? " dry-run" : ""}]`);
}
/**
* push: update ONE Foundry journal entry from a refined note, live, through the
* relay — Foundry keeps running (no LevelDB opened, no Docker stop). Thin wrapper
* around pushNote (shared with the dashboard server). See push.ts for the flow.
*/
export async function cmdPush(opts: CliOptions): Promise<void> {
if (!opts.vault) throw new Error("--vault <refined-note.md> is required for push");
const out = await ensureOut(opts);
const noteName = opts.name ?? basenameNoExt(opts.vault);
const relay = new RelayClient(loadRelayConfig({ url: opts.relayUrl, apiKey: opts.relayApiKey, clientId: opts.relayClientId }));
const fc = loadFoundryConfig({ container: opts.foundryContainer, dataDir: opts.foundryDataDir, world: opts.foundryWorld });
const outcome = await pushNote({
notePath: opts.vault,
noteName,
outDir: out,
relay,
foundryDataDir: fc.dataDir,
world: fc.world,
dryRun: opts.dryRun,
log: (m) => console.log(m),
});
if (outcome.dryRun) {
console.log(" diff that would be PUT to /update:");
console.log(JSON.stringify(outcome.diff, null, 2));
}
}
async function main(): Promise<void> {
try {
const { cmd, opts } = parseArgs(process.argv);
switch (cmd) {
case "to-obsidian": await cmdToObsidian(opts); break;
case "to-foundry": await cmdToFoundry(opts); break;
case "roundtrip": await cmdRoundtrip(opts); break;
case "ui": await cmdUi(opts); break;
case "refresh": await cmdRefresh(opts); break;
case "push": await cmdPush(opts); break;
default: throw new Error(`unknown command: ${cmd}`);
}
} catch (e) {
console.error(`error: ${(e as Error).message}`);
process.exit(1);
}
}
// Only run the CLI entrypoint when invoked directly (not when imported by tests).
import { pathToFileURL } from "node:url";
if (process.argv[1] && pathToFileURL(process.argv[1]).href === import.meta.url) {
void main();
}

48
src/config.ts Normal file
View File

@@ -0,0 +1,48 @@
/**
* Runtime config for the live Foundry push path (relay + docker + image fs).
*
* Read from process.env with sensible defaults. No zod dependency (the project
* keeps deps minimal: classic-level + linkedom only). CLI flags may override env
* values via `loadRelayConfig(overrides)` / `loadFoundryConfig(overrides)`.
*/
export interface RelayConfig {
/** Base URL of the ThreeHats foundryvtt-rest-api-relay (no trailing slash). */
url: string;
/** x-api-key for the relay (required for /get, /update, /search). */
apiKey: string;
/** Optional x-client-id (multi-client relay routing). */
clientId: string;
}
export interface FoundryHostConfig {
/** Docker container name running Foundry (for stop/start during index refresh). */
container: string;
/** Host-side mount of Foundry's Data directory (where assets/<world> lives). */
dataDir: string;
/** Foundry world id (the folder under Data/worlds/). */
world: string;
}
function env(key: string, fallback = ""): string {
const v = process.env[key];
return v && v.trim() !== "" ? v : fallback;
}
/** Build a RelayConfig from env, with optional flag overrides. */
export function loadRelayConfig(overrides: Partial<RelayConfig> = {}): RelayConfig {
return {
url: (overrides.url || env("RELAY_URL", "https://vtt-relay.damascusfront.net")).replace(/\/$/, ""),
apiKey: overrides.apiKey || env("RELAY_API_KEY"),
clientId: overrides.clientId || env("RELAY_CLIENT_ID"),
};
}
/** Build a FoundryHostConfig from env, with optional flag overrides. */
export function loadFoundryConfig(overrides: Partial<FoundryHostConfig> = {}): FoundryHostConfig {
return {
container: overrides.container || env("FOUNDRY_CONTAINER"),
dataDir: overrides.dataDir || env("FOUNDRY_DATA_DIR"),
world: overrides.world || env("FOUNDRY_WORLD"),
};
}

288
src/dashboard.html Normal file
View File

@@ -0,0 +1,288 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Foundry ⇄ Obsidian merge</title>
<style>
:root { color-scheme: light dark; --bg:#1b1b1f; --panel:#26262b; --line:#3a3a42; --txt:#e6e6ea; --mut:#9a9aa4; --acc:#7c9cff; --ok:#6ad06a; --warn:#e0b341; --bad:#e06a6a; --pur:#b98cff; }
* { box-sizing: border-box; }
body { margin:0; font:14px/1.45 -apple-system,Segoe UI,Roboto,Helvetica,Arial,sans-serif; background:var(--bg); color:var(--txt); }
header { position:sticky; top:0; z-index:2; background:var(--panel); border-bottom:1px solid var(--line); padding:10px 16px; display:flex; gap:16px; align-items:center; flex-wrap:wrap; }
h1 { font-size:15px; margin:0; font-weight:600; }
.counts { display:flex; gap:14px; font-size:13px; color:var(--mut); }
.counts b { color:var(--txt); }
.spacer { flex:1; }
button { background:var(--panel); color:var(--txt); border:1px solid var(--line); border-radius:6px; padding:5px 10px; cursor:pointer; font-size:13px; }
button:hover { border-color:var(--acc); }
button.primary { background:var(--acc); color:#0c0c10; border-color:var(--acc); font-weight:600; }
button.danger { border-color:var(--bad); color:var(--bad); }
button:disabled { opacity:.4; cursor:not-allowed; }
label { font-size:13px; color:var(--mut); display:flex; gap:6px; align-items:center; }
main { display:grid; grid-template-columns: 1fr 1fr; gap:0; height:calc(100vh - 53px); }
section.list { overflow:auto; border-right:1px solid var(--line); }
section.detail { overflow:auto; padding:14px 16px; }
.toolbar { padding:8px 16px; display:flex; gap:8px; align-items:center; border-bottom:1px solid var(--line); position:sticky; top:0; background:var(--bg); z-index:1; }
input[type=text] { background:var(--panel); border:1px solid var(--line); color:var(--txt); border-radius:6px; padding:5px 9px; width:100%; font-size:13px; }
table { width:100%; border-collapse:collapse; }
tr { border-bottom:1px solid var(--line); cursor:pointer; }
tr:hover { background:var(--panel); }
tr.sel { background:var(--panel); }
td { padding:6px 8px; vertical-align:top; font-size:13px; }
td.name { font-weight:500; }
.badge { font-size:11px; padding:1px 6px; border-radius:10px; border:1px solid var(--line); color:var(--mut); white-space:nowrap; }
.badge.ok { color:var(--ok); border-color:var(--ok); }
.badge.warn { color:var(--warn); border-color:var(--warn); }
.badge.bad { color:var(--bad); border-color:var(--bad); }
.badge.acc { color:var(--acc); border-color:var(--acc); }
.badge.pur { color:var(--pur); border-color:var(--pur); }
.rowbtns { white-space:nowrap; text-align:right; }
.rowbtns button { padding:3px 7px; font-size:12px; margin-left:4px; }
.rowbtns button.rec { border-color:var(--acc); color:var(--acc); }
h2 { font-size:13px; text-transform:uppercase; letter-spacing:.06em; color:var(--mut); margin:14px 16px 6px; }
pre { background:var(--panel); border:1px solid var(--line); border-radius:6px; padding:10px; overflow:auto; font:12px/1.4 ui-monospace,Menlo,Consolas,monospace; white-space:pre-wrap; word-break:break-word; }
.diff-add { color:var(--ok); }
.diff-del { color:var(--bad); }
.panel { margin-bottom:16px; }
.panel h3 { font-size:13px; margin:0 0 6px; color:var(--mut); display:flex; justify-content:space-between; align-items:center; }
.toast { position:fixed; bottom:14px; left:50%; transform:translateX(-50%); background:var(--panel); border:1px solid var(--acc); border-radius:8px; padding:8px 14px; font-size:13px; max-width:80%; box-shadow:0 6px 24px rgba(0,0,0,.4); }
.mode-tag { font-size:11px; padding:2px 8px; border-radius:10px; border:1px solid var(--line); }
.mode-tag.apply { color:var(--bad); border-color:var(--bad); }
.meta { color:var(--mut); font-size:11px; white-space:nowrap; }
.rec-panel { margin:10px 16px; border:1px solid var(--line); border-radius:8px; background:var(--panel); }
.rec-panel > summary { cursor:pointer; padding:8px 12px; font-size:13px; font-weight:600; color:var(--acc); list-style:none; }
.rec-panel > summary::-webkit-details-marker { display:none; }
.rec-group { display:flex; align-items:center; gap:10px; padding:7px 12px; border-top:1px solid var(--line); font-size:13px; }
.rec-group .num { font-weight:700; min-width:28px; text-align:right; }
.rec-group .label { flex:1; }
.rec-group .label small { color:var(--mut); display:block; font-size:11px; }
.rec-group button { font-size:12px; padding:3px 9px; }
.legend { margin:10px 16px; font-size:12px; color:var(--mut); line-height:1.6; }
.legend b { color:var(--txt); }
</style>
</head>
<body>
<header>
<h1>Foundry ⇄ Obsidian merge</h1>
<div class="counts" id="counts">loading…</div>
<span class="mode-tag" id="modeTag">dev</span>
<div class="spacer"></div>
<label><input type="checkbox" id="dryRun" checked /> dry-run</label>
<button class="primary" onclick="act('seedAll')" title="Inject the foundry: identity block into every matched refined note that lacks one. Curation (type/tags/aliases/body) is left untouched. Safe to run repeatedly.">Seed all</button>
<button class="primary" onclick="act('syncAll')" title="For every matched note: regenerate cc.md from the refined note AND refresh the refined foundry: block. Baselines both sides — writes cc_sync_hash into cc and refreshes foundry.contentHash. Curation flows back to cc.">Sync→cc all</button>
<button onclick="act('repullAll')" title="For every matched note: regenerate the refined note body from Foundry, preserving curated type/aliases/status tags. Use when Foundry/cc is newer than your vault.">Re-pull all</button>
<button onclick="act('importAll')" title="Pull every cc-only Foundry entry (not yet in your vault) into a new refined note under refined/imported/<folder>. Un-curated staging for review.">Import all</button>
<button onclick="refreshLive()" title="Rebuild the cached name↔uuid map (for link resolution in pushes) via the relay /search — zero Foundry downtime. The heavy docker-stop full index is CLI-only: npx tsx src/cli.ts refresh --full-index.">Refresh live index</button>
</header>
<main>
<section class="list">
<div class="toolbar">
<input type="text" id="filter" placeholder="filter by name…" oninput="render()" />
</div>
<details class="rec-panel" id="recPanel" open>
<summary>Recommended next steps ▾</summary>
<div id="recGroups"></div>
</details>
<details class="legend">
<summary style="cursor:pointer;list-style:none"><b>How merging works</b></summary>
<p>Each file carries a hash on <b>both sides</b>: the refined note stores <code>foundry.contentHash</code>, the cc.md stores <code>cc_sync_hash</code>. The tool compares each side to <i>its own</i> last-synced hash to decide direction — no separate state file. After you edit a side, its hash no longer matches what was recorded at last sync, so that side is "newer".</p>
<p><b>Priority:</b> content missing from your vault first (import), then unlinked notes (seed), then vault-newer (sync→cc), then cc-newer (re-pull), then both-changed (conflict — review manually).</p>
</details>
<h2>Matched (refined ⇄ cc)</h2>
<table id="matchedTable"><tbody></tbody></table>
<h2>cc-only (import candidates)</h2>
<table id="ccOnlyTable"><tbody></tbody></table>
</section>
<section class="detail" id="detail">Select a row to inspect.</section>
</main>
<script>
let INDEX = null, STATUS = null, SEL = null;
const dryEl = () => document.getElementById('dryRun');
// Recommendation -> display label, badge class, bulk op, and one-line guidance.
const REC = {
'import': { label: 'Import into vault', badge: 'acc', bulk: 'importAll', tip: 'In Foundry/cc but not in your vault. Pull it in as a new (un-curated) refined note to review.' },
'seed': { label: 'Seed / link', badge: 'warn', bulk: 'seedAll', tip: 'In your vault and in Foundry, but not yet linked. Inject the foundry: identity block. Curation untouched.' },
'sync-cc': { label: 'Push to cc', badge: 'acc', bulk: 'syncAll', tip: 'Your vault copy is newer than the last sync. Regenerate cc.md from the refined note and baseline both sides.' },
'repull': { label: 'Re-pull', badge: 'pur', bulk: 'repullAll', tip: 'Foundry/cc is newer than your vault. Regenerate the refined body from Foundry, preserving your curation.' },
'conflict': { label: 'Conflict', badge: 'bad', bulk: null, tip: 'Both sides changed since last sync. Open the file and decide which side wins manually.' },
'in-sync': { label: 'In sync', badge: 'ok', bulk: null, tip: 'Neither side changed since last sync. Nothing to do.' },
'review': { label: 'Review', badge: 'warn', bulk: null, tip: 'Unlinked or ambiguous — needs a human decision before any action.' },
};
// Order in which the guided panel shows groups (missing-from-vault first).
const REC_ORDER = ['import','seed','sync-cc','repull','conflict','in-sync','review'];
async function init() {
STATUS = await fetch('/api/status').then(r => r.json());
const tag = document.getElementById('modeTag');
tag.textContent = STATUS.mode + (STATUS.mode === 'apply' ? '' : ' (safe)');
if (STATUS.mode === 'apply') tag.classList.add('apply');
INDEX = await fetch('/api/index').then(r => r.json());
const c = INDEX.counts;
document.getElementById('counts').innerHTML =
`matched <b>${c.matched}</b> · cc-only <b>${c.ccOnly}</b> · refined-only <b>${c.refinedOnly}</b> · unlinked <b>${c.unlinked}</b>`;
renderRecPanel();
render();
}
function esc(s){ return (s==null?'':String(s)).replace(/[&<>]/g,c=>({'&':'&amp;','<':'&lt;','>':'&gt;'}[c])); }
function attr(s){ return (s==null?'':String(s)).replace(/[&"]/g,c=>({'&':'&amp;','"':'&quot;'}[c])); }
function recBadge(r){
const m = REC[r.recommendation] || REC['review'];
return `<span class="badge ${m.badge}" title="${attr(m.tip)}">${m.label}</span>`;
}
function fmtMtime(ms){ if (!ms) return ''; const d = new Date(ms); return d.toLocaleDateString() + ' ' + d.toLocaleTimeString([], {hour:'2-digit',minute:'2-digit'}); }
function mtimeHint(r){
if (!r.refinedMtime && !r.ccMtime) return '';
const parts = [];
if (r.refinedMtime) parts.push(`ref ${fmtMtime(r.refinedMtime)}`);
if (r.ccMtime) parts.push(`cc ${fmtMtime(r.ccMtime)}`);
return `<span class="meta" title="File modification times (hint only — direction is decided by hashes, since cp -r resets mtime)">${esc(parts.join(' · '))}</span>`;
}
function renderRecPanel(){
const el = document.getElementById('recGroups');
const b = INDEX.byRecommendation;
const total = REC_ORDER.reduce((n,k)=>n+(b[k]||0),0);
if (!total){ el.innerHTML = '<div class="rec-group"><span class="label">Everything is in sync. ✨</span></div>'; return; }
el.innerHTML = REC_ORDER.filter(k => (b[k]||0) > 0).map(k => {
const m = REC[k];
const btn = m.bulk
? `<button class="primary" data-op="${m.bulk}" title="${attr(m.tip)}">Run ${m.bulk.replace('All',' all')}</button>`
: `<button disabled title="${attr(m.tip)}">manual</button>`;
return `<div class="rec-group"><span class="num">${b[k]}</span>
<span class="label">${m.label}<small>${esc(m.tip)}</small></span>${btn}</div>`;
}).join('');
}
function render(){
const q = document.getElementById('filter').value.toLowerCase().trim();
const m = INDEX.matched.filter(r => !q || r.name.toLowerCase().includes(q));
const c = INDEX.ccOnly.filter(r => !q || r.name.toLowerCase().includes(q));
document.querySelector('#matchedTable tbody').innerHTML = m.map(rowHtml).join('');
document.querySelector('#ccOnlyTable tbody').innerHTML = c.map(rowHtml).join('');
}
// The single recommended action button for a row (its primary op), plus a "view" button.
function recAction(r){
const n = attr(r.name);
const m = REC[r.recommendation] || REC['review'];
switch (r.recommendation){
case 'import': return `<button class="rec" data-op="import" data-name="${n}" title="${attr(m.tip)}">import</button>`;
case 'seed': return `<button class="rec" data-op="seed" data-name="${n}" title="${attr(m.tip)}">seed</button>`;
case 'sync-cc': return `<button class="rec" data-op="sync" data-name="${n}" title="${attr(m.tip)}">sync→cc</button>`;
case 'repull': return `<button class="rec" data-op="repull" data-name="${n}" title="${attr(m.tip)}">re-pull</button>`;
case 'conflict':
case 'review': return `<button data-op="view" data-name="${n}" title="${attr(m.tip)}">review</button>`;
case 'in-sync': return `<span class="meta" title="${attr(m.tip)}">✓</span>`;
default: return '';
}
}
function rowHtml(r){
const cls = SEL === r.name ? 'sel' : '';
const meta = [r.curatedType && `type:${r.curatedType}`, r.ccType && `cc:${r.ccType}`].filter(Boolean).join(' ');
return `<tr class="${cls}" data-name="${attr(r.name)}">
<td class="name">${esc(r.name)}</td>
<td>${recBadge(r)}</td>
<td style="color:var(--mut)">${esc(meta)}<br/>${mtimeHint(r)}</td>
<td class="rowbtns">${recAction(r)} <button data-op="push" data-name="${attr(r.name)}" title="Push this refined note into the LIVE Foundry world via the relay (Foundry keeps running). dry-run shows the diff; uncheck dry-run to apply (writes a .bak first). This updates Foundry — unlike sync→cc, which only writes the local cc.md artifact.">push</button> <button data-op="view" data-name="${attr(r.name)}" title="Open this file in the detail panel: see refined + cc side by side, the seed/sync/re-pull previews, and the Foundry entry.">view</button></td></tr>`;
}
async function select(name){
SEL = name; render();
const d = document.getElementById('detail');
d.innerHTML = `loading ${esc(name)}`;
const f = await fetch('/api/file?name=' + encodeURIComponent(name)).then(r => r.json());
const r = f.row;
const m = REC[r.recommendation] || REC['review'];
const parts = [];
parts.push(`<div class="panel"><h3>${esc(name)} ${recBadge(r)}</h3>
<pre>${esc(JSON.stringify({name:r.name,status:r.status,recommendation:r.recommendation,refinedChanged:r.refinedChanged,ccChanged:r.ccChanged,curatedType:r.curatedType,ccType:r.ccType,ccId:r.ccId,storedRefinedHash:r.storedRefinedHash,storedCcHash:r.storedCcHash,entry:!!f.entry},null,2))}</pre>
<p class="meta">${esc(m.tip)}</p></div>`);
if (r.refinedPath) parts.push(`<div class="panel"><h3>refined note (vault) <button data-op="seed" data-name="${attr(name)}" title="Inject/refresh the foundry: block in this refined note only. Curation untouched.">seed this</button> <button data-op="sync" data-name="${attr(name)}" title="Regenerate cc.md from this refined note and baseline both sides.">sync→cc this</button> <button data-op="repull" data-name="${attr(name)}" title="Regenerate this refined note's body from Foundry, preserving curation.">re-pull this</button> <button data-op="push" data-name="${attr(name)}" title="Push this note into the LIVE Foundry world via the relay (Foundry keeps running). dry-run shows the diff; uncheck dry-run to apply. Updates Foundry itself — sync→cc only writes the local cc.md.">push this</button></h3><pre>${esc(f.refined)}</pre></div>`);
if (f.cc != null) parts.push(`<div class="panel"><h3>existing cc.md (Foundry export)</h3><pre>${esc(f.cc)}</pre></div>`);
if (f.seedPreview != null) parts.push(`<div class="panel"><h3>seed preview <small style="font-weight:normal">(foundry: block injected, curation preserved)</small></h3>${diff(f.refined, f.seedPreview)}</div>`);
if (f.syncRefinedPreview != null && f.refined != null) parts.push(`<div class="panel"><h3>sync preview — refined side <small style="font-weight:normal">(foundry.contentHash refreshed)</small></h3>${diff(f.refined, f.syncRefinedPreview)}</div>`);
if (f.syncPreview != null && f.cc != null) parts.push(`<div class="panel"><h3>sync preview — cc side <small style="font-weight:normal">(curation flows back; cc_sync_hash written)</small></h3>${diff(f.cc, f.syncPreview)}</div>`);
if (f.repullPreview != null && f.refined != null) parts.push(`<div class="panel"><h3>re-pull preview <small style="font-weight:normal">(body from Foundry, curation preserved)</small></h3>${diff(f.refined, f.repullPreview)}</div>`);
if (f.entry) parts.push(`<div class="panel"><h3>Foundry journal entry</h3><pre>${esc(JSON.stringify(f.entry,null,2))}</pre></div>`);
d.innerHTML = parts.join('\n');
}
function diff(a, b){
const al = (a||'').split('\n'), bl = (b||'').split('\n');
const aSet = new Set(al), bSet = new Set(bl);
const lines = [];
for (const l of al) if (!bSet.has(l)) lines.push(`<span class="diff-del">- ${esc(l)}</span>`);
for (const l of bl) if (!aSet.has(l)) lines.push(`<span class="diff-add">+ ${esc(l)}</span>`);
if (!lines.length) return '<pre>(identical)</pre>';
return '<pre>' + lines.join('\n') + '</pre>';
}
let toastT = null;
function toast(msg){
let el = document.querySelector('.toast');
if (el) el.remove();
el = document.createElement('div'); el.className = 'toast'; el.textContent = msg;
document.body.appendChild(el);
clearTimeout(toastT); toastT = setTimeout(()=>el.remove(), 6000);
}
async function act(op, names){
const dryRun = dryEl().checked;
const body = { op, dryRun };
if (names) body.names = names;
toast(`${op} ${dryRun?'(dry-run)':'('+STATUS.mode+')'}`);
const r = await fetch('/api/action', {method:'POST', headers:{'content-type':'application/json'}, body: JSON.stringify(body)}).then(r=>r.json());
if (r.error){ toast('error: ' + r.error); return; }
const wrote = (r.written||[]).length, prev = (r.preview||[]).length, skip = (r.skipped||[]).length;
toast(r.message || `${op}: ${dryRun?prev:wrote} ${dryRun?'would write':'wrote'}${skip?', '+skip+' skipped':''}`);
// Refresh index so recommendation counts update after an action.
INDEX = await fetch('/api/index').then(r => r.json());
renderRecPanel(); render();
if (SEL) select(SEL);
}
// Push one refined note into the LIVE Foundry world via the relay (Foundry keeps
// running). dry-run (default) shows the diff; uncheck dry-run to apply — the server
// snapshots the live entry to <out>/bak/<name>.<iso>.json first, so it's reversible.
async function pushRow(name){
const dryRun = dryEl().checked;
toast(`push ${name} ${dryRun?'(dry-run)':'(apply)'}`);
const r = await fetch('/api/push', {method:'POST', headers:{'content-type':'application/json'}, body: JSON.stringify({name, dryRun})}).then(r=>r.json());
if (r.error){ toast('error: ' + r.error); return; }
if (r.dryRun){
toast(`[dry-run] push ${name}: diff ready (${Object.keys(r.diff).length} keys)`);
if (SEL === name) select(name); // refresh detail so the user can re-inspect
} else {
toast(`pushed "${r.updatedName || name}" into live Foundry (backup: ${r.backupPath || 'none'})`);
}
}
// Rebuild the cached name↔uuid map via relay /search (zero Foundry downtime).
async function refreshLive(){
toast('refresh live index…');
const r = await fetch('/api/refresh', {method:'POST', headers:{'content-type':'application/json'}, body: '{}'}).then(r=>r.json());
if (r.error){ toast('error: ' + r.error); return; }
toast(`live index refreshed: ${r.pairs} name↔uuid pairs cached`);
}
init();
// Delegated clicks for dynamically-rendered rows/buttons (data-name is HTML-escaped,
// so names with spaces/quotes can't break the attribute the way inline onclick + JSON could).
document.querySelector('section.list').addEventListener('click', (e) => {
const btn = e.target.closest('button[data-op]');
if (btn) {
e.stopPropagation();
const op = btn.dataset.op, name = btn.dataset.name;
if (op === 'view') select(name);
else if (op === 'push' && name) pushRow(name);
else if (name) act(op, [name]);
else act(op); // bulk ops (seedAll/syncAll/repullAll/importAll) carry no data-name
return;
}
const tr = e.target.closest('tr[data-name]');
if (tr) select(tr.dataset.name);
});
document.getElementById('detail').addEventListener('click', (e) => {
const btn = e.target.closest('button[data-op]');
if (!btn) return;
const op = btn.dataset.op, name = btn.dataset.name;
if (op === 'seed' || op === 'sync' || op === 'repull') act(op, [name]);
else if (op === 'push' && name) pushRow(name);
});
</script>
</body>
</html>

90
src/db.ts Normal file
View File

@@ -0,0 +1,90 @@
import { ClassicLevel, type DatabaseOptions } from "classic-level";
import type { JournalEntry } from "./types.js";
/**
* Read-only index over a Foundry journal LevelDB snapshot.
*
* The journal store keys JournalEntries under `!journal!<id>` and their pages
* under `!journal.pages!<id>`. We only need the entries. We open with
* `readOnly: true` (honored when supported; otherwise tolerated) AND we only
* ever call read methods (get/iterator) — never put/del/batch — so the Foundry
* DB is never mutated, in dev or apply mode. Dev mode additionally operates on a
* user-supplied copy of the world data.
*/
type DbOptions = DatabaseOptions<string, string> & { readOnly?: boolean };
export class JournalDb {
private db: ClassicLevel<string, string>;
readonly entries = new Map<string, JournalEntry>(); // _id -> entry
readonly uuidToName = new Map<string, string>(); // JournalEntry.<id> -> name
readonly nameToUuid = new Map<string, string>(); // name -> JournalEntry.<id> (first wins)
private constructor(db: ClassicLevel<string, string>) {
this.db = db;
}
static async open(path: string): Promise<JournalDb> {
const db = new ClassicLevel<string, string>(path, {
readOnly: true,
keyEncoding: "utf8",
valueEncoding: "utf8",
} as DbOptions);
const idx = new JournalDb(db);
await idx.load();
return idx;
}
private async load(): Promise<void> {
for await (const [key, value] of this.db.iterator()) {
// Skip page records: keys beginning with "!journal.pages!".
if (key.startsWith("!journal.pages!")) continue;
if (!key.startsWith("!journal!")) continue;
let entry: JournalEntry;
try {
entry = JSON.parse(value) as JournalEntry;
} catch {
continue;
}
if (!entry?._id || !entry.name) continue;
// Only index Campaign Codex journals (those we can round-trip).
if (!entry.flags?.["campaign-codex"]) continue;
if (!this.entries.has(entry._id)) this.entries.set(entry._id, entry);
const uuid = `JournalEntry.${entry._id}`;
if (!this.uuidToName.has(uuid)) this.uuidToName.set(uuid, entry.name);
if (!this.nameToUuid.has(entry.name)) this.nameToUuid.set(entry.name, uuid);
}
}
/** All CC journal entries, sorted by name for deterministic output. */
all(): JournalEntry[] {
return [...this.entries.values()].sort((a, b) => a.name.localeCompare(b.name));
}
byId(id: string): JournalEntry | undefined {
return this.entries.get(id);
}
byUuid(uuid: string): JournalEntry | undefined {
const id = uuid.startsWith("JournalEntry.") ? uuid.slice("JournalEntry.".length) : uuid;
return this.entries.get(id);
}
byName(name: string): JournalEntry | undefined {
const uuid = this.nameToUuid.get(name);
return uuid ? this.byUuid(uuid) : undefined;
}
/** Resolve a Foundry UUID to its name, or undefined if unknown. */
nameOf(uuid: string): string | undefined {
return this.uuidToName.get(uuid);
}
/** Resolve a name to its Foundry UUID, or undefined if unknown. */
uuidOf(name: string): string | undefined {
return this.nameToUuid.get(name);
}
async close(): Promise<void> {
await this.db.close();
}
}

72
src/fields.ts Normal file
View File

@@ -0,0 +1,72 @@
// Type maps + folder derivation ported from the Foundry macros:
// CC_TYPES — Macros/cc-1-select.js (Obsidian frontmatter `type` -> CC type)
// CC_TO_OBSIDIAN — Macros/cc-4-export.js (CC type -> Obsidian frontmatter `type`)
// Reused, not reinvented.
/** Obsidian frontmatter `type` value -> Campaign Codex entry type. */
export const CC_TYPES: Record<string, string> = {
// NPC / character
npc: "npc", character: "npc",
// Faction / tag
faction: "tag", tag: "tag", organization: "tag", tribe: "tag", house: "tag", guild: "tag",
// Location
location: "location", city: "location", dungeon: "location", settlement: "location", town: "location",
// Region
region: "region", continent: "region", plane: "region",
// Quest / group
quest: "quest",
group: "group", moc: "group",
// CC "Entry" (ShopSheet) — fallback
shop: "shop", entry: "shop", item: "shop", artifact: "shop",
creature: "shop", race: "shop", event: "shop", deity: "shop",
myth: "shop", technology: "shop", scene: "shop", pantheon: "shop",
religion: "shop", ruler: "shop", villain: "shop", ally: "shop",
};
/** Campaign Codex type -> Obsidian frontmatter `type`. */
export const CC_TO_OBSIDIAN: Record<string, string> = {
npc: "npc", tag: "faction", location: "location",
region: "region", quest: "quest", group: "group", shop: "entry",
};
/** Campaign Codex type -> export folder path (matches the cc export tree). */
export const CC_FOLDER_PATH: Record<string, string> = {
npc: "Campaign Codex - NPCs",
tag: "Campaign Codex - Factions",
location: "Campaign Codex - Locations",
region: "Campaign Codex - Regions",
group: "Campaign Codex - Groups",
quest: "Campaign Codex - Quests",
shop: "Campaign Codex - Entries",
};
export function ccTypeFromObsidianType(type: string | undefined): string {
if (!type) return "shop";
return CC_TYPES[type] ?? "shop";
}
export function obsidianTypeFromCcType(ccType: string | undefined): string {
if (!ccType) return "entry";
return CC_TO_OBSIDIAN[ccType] ?? "entry";
}
export function folderPathFromCcType(ccType: string | undefined): string {
return CC_FOLDER_PATH[ccType ?? ""] ?? "Campaign Codex - Entries";
}
/** Section headings whose content routes to the CC GM Notes (hidden) field. */
export const NOTES_TRIGGERS = ["GM Notes", "Secrets", "Aftermath"];
/** Sidebar box label -> Obsidian frontmatter field name. */
export const LABEL_TO_FIELD: Record<string, "race" | "faction" | "region"> = {
race: "race",
faction: "faction",
region: "region",
};
/** Reverse: frontmatter field -> sidebar box label (as rendered by CC). */
export const FIELD_TO_LABEL: Record<"race" | "faction" | "region", string> = {
race: "Race",
faction: "Faction",
region: "Region",
};

100
src/foundry/assets.ts Normal file
View File

@@ -0,0 +1,100 @@
import { copyFile, mkdir, access } from "node:fs/promises";
import { join, dirname, basename } from "node:path";
/**
* Portrait image handling for the live push path.
*
* Same-host advantage: the tool runs on the Foundry Docker host, so it can write
* the portrait PNG straight into Foundry's served assets folder on the host
* filesystem — no relay upload endpoint needed (the ThreeHats relay's asset
* upload is undocumented/unclear). The JournalEntry /update then just references
* that server-relative path in `flags.campaign-codex.image`.
*
* Refined notes store `portrait: "[[File.png]]"`; the PNG is a co-located
* sibling file in the vault subfolder (confirmed: Fenris of House Quche_portrait.png
* sits next to Fenris of House Quche.md).
*
* Served-path convention: Foundry FilePicker "uploads" storage is world-relative,
* i.e. a flag value of `uploads/<...>/<file>` is served from
* `Data/worlds/<world>/uploads/<...>/<file>`. mini-uploader nests a per-world
* subfolder (`uploads/<worldId>/`). We DERIVE the prefix from the entry's existing
* image so new portraits land in the same folder as existing ones; if none, we
* default to the mini-uploader `uploads/<world>/` convention. Confirm the exact
* prefix against a live entry's image during the relay probe (see plan).
*/
/** Resolve a `[[File.png]]` portrait field to a plain basename (no brackets). */
export function portraitBasename(portraitField: string): string {
return portraitField.replace(/^\[\[|\]\]$/g, "").trim().replace(/\.[^.]+$/, "").trim()
? portraitField.replace(/^\[\[|\]\]$/g, "").trim()
: "";
}
/** Locate the portrait file on disk: a sibling of the note (same dir, matching
* the portrait basename, with or without extension). Returns null if missing. */
export async function findPortraitFile(notePath: string, portraitField: string): Promise<string | null> {
const raw = portraitBasename(portraitField);
if (!raw) return null;
const dir = dirname(notePath);
const candidates = [join(dir, raw)];
if (!raw.includes(".")) {
for (const ext of [".png", ".webp", ".jpg", ".jpeg", ".gif"]) candidates.push(join(dir, `${raw}${ext}`));
}
for (const c of candidates) {
try { await access(c); return c; } catch { /* try next */ }
}
return null;
}
export interface UploadResult {
/** The server-relative path to set as flags.campaign-codex.image (or null to keep the existing image). */
servedPath: string | null;
/** True if the portrait was newly written to disk. */
wrote: boolean;
/** Human-readable note for logs (kept / uploaded / not-found). */
note: string;
}
/**
* Upload (copy) the note's portrait into Foundry's assets dir, unless the entry
* already references a matching image (keep, no re-upload). Returns the served
* path to set in flags.campaign-codex.image, or null to keep the existing image.
*/
export async function uploadPortrait(opts: {
notePath: string;
portraitField: string | null | undefined;
foundryDataDir: string;
world: string;
/** The entry's current flags.campaign-codex.image (used to derive the prefix + short-circuit). */
existingImage?: string | null;
}): Promise<UploadResult> {
const { notePath, portraitField, foundryDataDir, world, existingImage } = opts;
const base = portraitField ? portraitBasename(portraitField) : "";
if (!base) return { servedPath: existingImage ?? null, wrote: false, note: "no portrait field — keeping existing image" };
const existingBase = existingImage ? basename(existingImage.split("?")[0]) : "";
// If the entry already references a portrait with the same basename, keep it.
if (existingBase && (existingBase === base || existingBase.replace(/\.[^.]+$/, "") === base.replace(/\.[^.]+$/, ""))) {
return { servedPath: existingImage ?? null, wrote: false, note: `keep existing image (${existingImage})` };
}
const src = await findPortraitFile(notePath, portraitField!);
if (!src) {
return { servedPath: existingImage ?? null, wrote: false, note: `portrait file not found next to note (${base})` };
}
// Derive the served prefix from the existing image, else default to mini-uploader's uploads/<world>/.
let servedPrefix: string;
if (existingImage && existingImage.includes("/")) {
servedPrefix = existingImage.slice(0, existingImage.lastIndexOf("/") + 1);
} else {
servedPrefix = `uploads/${world}/`;
}
const servedPath = `${servedPrefix}${basename(src)}`;
// Served paths are world-relative; the on-disk location is under Data/worlds/<world>/.
const diskPath = join(foundryDataDir, "worlds", world, servedPath);
await mkdir(dirname(diskPath), { recursive: true });
await copyFile(src, diskPath);
return { servedPath, wrote: true, note: `uploaded ${basename(src)} -> ${servedPath}` };
}

101
src/foundry/docker.ts Normal file
View File

@@ -0,0 +1,101 @@
import { execFile } from "node:child_process";
import { writeFile, readFile, unlink, access, mkdir } from "node:fs/promises";
import { join, dirname } from "node:path";
/**
* Foundry Docker service control.
*
* Used ONLY by `cmd refresh` to read the live journal LevelDB (Foundry holds an
* exclusive lock on it while running, so the container must be stopped first).
* Pushes NEVER touch this — they go through the relay with Foundry running.
*
* A lockfile guards against a refresh running while a push is in flight (and
* vice versa): pushes hold the lock while a /update is pending, refresh holds it
* while the container is down. Stale locks (older than `lockStaleMs`) are
* reclaimable, so a crashed process doesn't wedge the tool.
*/
const LOCK_STALE_MS = 10 * 60 * 1000; // 10 min
export interface DockerControl {
/** True if the container is currently running. */
isRunning(name: string): Promise<boolean>;
/** Stop the container (graceful). Resolves when stopped. */
stop(name: string, timeoutSec?: number): Promise<void>;
/** Start the container. Resolves when started. */
start(name: string): Promise<void>;
}
function run(args: string[]): Promise<string> {
return new Promise((resolve, reject) => {
execFile("docker", args, { maxBuffer: 1 << 20 }, (err, stdout, stderr) => {
if (err) reject(new Error(`docker ${args.join(" ")}: ${(err as Error).message} ${stderr}`));
else resolve(stdout.toString());
});
});
}
/** Default DockerControl using the local `docker` CLI. */
export const docker: DockerControl = {
async isRunning(name: string): Promise<boolean> {
try {
const out = await run(["inspect", "--format", "{{.State.Running}}", name]);
return out.trim() === "true";
} catch {
return false;
}
},
async stop(name: string, timeoutSec = 30): Promise<void> {
await run(["stop", "-t", String(timeoutSec), name]);
},
async start(name: string): Promise<void> {
await run(["start", name]);
},
};
/** Acquire a lockfile, reclaiming stale locks. Throws if held by a live process. */
export async function acquireLock(lockPath: string, holder: string): Promise<void> {
await mkdir(dirname(lockPath), { recursive: true });
try {
const stamp = Date.now();
await writeFile(lockPath, `${holder}\n${stamp}\n`, { flag: "wx" });
return;
} catch (e) {
const err = e as NodeJS.ErrnoException;
if (err.code !== "EEXIST") throw err;
}
// Lock exists — check staleness.
let content: string;
try {
content = await readFile(lockPath, "utf8");
} catch {
// Raced away; retry once.
return acquireLock(lockPath, holder);
}
const [, tsLine] = content.split("\n");
const age = Date.now() - Number(tsLine || 0);
if (age < LOCK_STALE_MS) {
throw new Error(`lock held by ${content.split("\n")[0]} (age ${Math.round(age / 1000)}s) — refusing to run concurrently`);
}
// Stale: reclaim.
await unlink(lockPath).catch(() => {});
await writeFile(lockPath, `${holder}\n${Date.now()}\n`, { flag: "wx" });
}
export async function releaseLock(lockPath: string): Promise<void> {
await unlink(lockPath).catch(() => {});
}
export async function isLocked(lockPath: string): Promise<boolean> {
try {
await access(lockPath);
return true;
} catch {
return false;
}
}
/** Default lock path lives next to the --out dir. */
export function lockPathFor(outDir: string): string {
return join(outDir, "foundry-sync.lock");
}

108
src/frontmatter.ts Normal file
View File

@@ -0,0 +1,108 @@
// Minimal YAML frontmatter parse/emit for the subset used by the vault and cc
// formats: scalar keys, block-sequence lists (tags/aliases), and one nested
// mapping (the `foundry:` block). No external YAML dependency.
export type FmValue = string | string[] | Record<string, string>;
export type Frontmatter = Record<string, FmValue>;
/** Typed view of the nested `foundry:` block, or undefined if absent/not a map. */
export function readFoundryBlock(fm: Frontmatter): Record<string, string> | undefined {
return typeof fm.foundry === "object" ? (fm.foundry as Record<string, string>) : undefined;
}
function unquote(s: string): string {
const t = s.trim();
if ((t.startsWith('"') && t.endsWith('"')) || (t.startsWith("'") && t.endsWith("'"))) {
const inner = t.slice(1, -1);
return t.startsWith('"') ? inner.replace(/\\"/g, '"').replace(/\\\\/g, "\\") : inner;
}
return t;
}
/** Parse a `---\n...\n---` block at the start of a markdown string. Returns {fm, body}. */
export function splitFrontmatter(md: string): { fm: Frontmatter; body: string } {
const m = md.match(/^---\r?\n([\s\S]*?)\r?\n---\r?\n?([\s\S]*)$/);
if (!m) return { fm: {}, body: md };
const raw = m[1];
const body = m[2] ?? "";
const fm: Frontmatter = {};
const lines = raw.split(/\r?\n/);
let i = 0;
while (i < lines.length) {
const line = lines[i];
if (!line) { i++; continue; }
const km = line.match(/^([A-Za-z0-9_\-]+):\s*(.*)$/);
if (!km) { i++; continue; }
const key = km[1];
const rest = km[2];
if (rest.trim() === "") {
// Block sequence or nested mapping follows.
const items: string[] = [];
const nested: Record<string, string> = {};
let j = i + 1;
while (j < lines.length && lines[j].startsWith(" ")) {
const sub = lines[j];
if (sub.startsWith(" - ")) {
items.push(unquote(sub.slice(4)));
} else {
const sm = sub.match(/^\s+([A-Za-z0-9_\-]+):\s*(.*)$/);
if (sm) nested[sm[1]] = unquote(sm[2]);
}
j++;
}
if (items.length) fm[key] = items;
else if (Object.keys(nested).length) fm[key] = nested;
i = j;
} else {
fm[key] = unquote(rest);
i++;
}
}
return { fm, body };
}
function needsQuotes(v: string): boolean {
if (v === "") return true;
if (v.startsWith("[[") || v.startsWith("![")) return true;
return /[:#{}[\]|>&*!'"@`,]/.test(v);
}
function quote(v: string, force = false): string {
if (!force && !needsQuotes(v)) return v;
return `"${v.replace(/\\/g, "\\\\").replace(/"/g, '\\"')}"`;
}
/** Emit a frontmatter block in a fixed, deterministic key order.
* `forceQuote` quotes all scalar string values (used for the cc.md export,
* matching Campaign Codex's quoted frontmatter). */
/** Emit the lines for one frontmatter key (scalar / list / nested map). */
function emitKey(key: string, val: FmValue, forceQuote: boolean, lines: string[]): void {
if (Array.isArray(val)) {
lines.push(`${key}:`);
for (const item of val) lines.push(` - ${quote(item, forceQuote)}`);
} else if (typeof val === "string") {
lines.push(`${key}: ${quote(val, forceQuote)}`);
} else {
lines.push(`${key}:`);
for (const [k, v] of Object.entries(val)) lines.push(` ${k}: ${quote(v, forceQuote)}`);
}
}
/** Emit a frontmatter block in a fixed, deterministic key order.
* `forceQuote` quotes all scalar string values (used for the cc.md export,
* matching Campaign Codex's quoted frontmatter). */
export function emitFrontmatter(fm: Frontmatter, order: string[], forceQuote = false): string {
const lines = ["---"];
for (const key of order) {
if (!(key in fm)) continue;
emitKey(key, fm[key], forceQuote, lines);
}
// Any keys not in the prescribed order, appended alphabetically for stability.
const extras = Object.keys(fm).filter((k) => !order.includes(k)).sort();
for (const key of extras) emitKey(key, fm[key], forceQuote, lines);
lines.push("---");
return lines.join("\n");
}
export const OBSIDIAN_FM_ORDER = ["type", "tags", "aliases", "faction", "region", "race", "portrait"];
export const CC_FM_ORDER = ["cc_id", "cc_uuid", "cc_type", "cc_folder_path", "cc_exported_at", "cc_sync_hash"];

141
src/htmlMd.ts Normal file
View File

@@ -0,0 +1,141 @@
import { parseHTML } from "linkedom";
import type { JournalDb } from "./db.js";
import { uuidToWiki, wikiToName } from "./links.js";
const ELEMENT_NODE = 1;
const TEXT_NODE = 3;
export interface SectionBox {
title: string;
body: string;
}
export interface ParsedDescription {
bodyMd: string; // left column: tagline + hr + sections, links as [[Name|Display]]
sidebar: Record<string, string>; // e.g. { race: "[[Human]]", faction: "[[House Raventhorne]]", region: "[[..]]" }
sectionBoxes: SectionBox[]; // extra right-column section boxes (Mission/Relations…); Roland has none
}
function stripTags(html: string): string {
return html.replace(/<[^>]+>/g, "");
}
// Ported from Macros/cc-4-export.js nodeToMd, but text nodes resolve @UUID -> [[Name|Display]].
function nodeToMd(node: any, db: JournalDb): string {
if (node.nodeType === TEXT_NODE) return uuidToWiki(node.textContent ?? "", db);
if (node.nodeType !== ELEMENT_NODE) return "";
const el = node;
const tag = el.tagName.toLowerCase();
const inner = () => Array.from(el.childNodes).map((c: any) => nodeToMd(c, db)).join("");
const block = (c: string) => `\n\n${c}\n\n`;
switch (tag) {
case "h1": return block(`# ${inner().trim()}`);
case "h2": return block(`## ${inner().trim()}`);
case "h3": return block(`### ${inner().trim()}`);
case "h4": return block(`#### ${inner().trim()}`);
case "h5": return block(`##### ${inner().trim()}`);
case "h6": return block(`###### ${inner().trim()}`);
case "p": return block(inner().trim());
case "strong": case "b": return `**${inner()}**`;
case "em": case "i": return `*${inner()}*`;
case "code": return `\`${inner()}\``;
case "hr": return block("---");
case "br": return "\n";
case "blockquote": {
const c = inner().trim().replace(/\n/g, "\n> ");
return block(`> ${c}`);
}
case "ul":
return block(Array.from(el.children).map((li) => `- ${nodeToMd(li, db).trim()}`).join("\n"));
case "ol":
return block(Array.from(el.children).map((li, i) => `${i + 1}. ${nodeToMd(li, db).trim()}`).join("\n"));
case "li": return inner();
case "figure": {
const img = el.querySelector("img");
if (!img) return inner();
const basename = (img.getAttribute("src") ?? "").split("/").pop();
return block(`![[${basename}]]`);
}
case "img": {
const basename = (el.getAttribute("src") ?? "").split("/").pop();
return `![[${basename}]]`;
}
case "a": {
const href = el.getAttribute("href") ?? "";
const text = inner();
return href.startsWith("http") ? `[${text}](${href})` : text;
}
default: return inner(); // div/span/section recurse
}
}
function collapse(md: string): string {
return md.trim().replace(/\n{3,}/g, "\n\n");
}
/** Parse the two-column description into body markdown + sidebar fields. */
export function htmlToMarkdown(descriptionHtml: string, db: JournalDb): ParsedDescription {
const { document } = parseHTML(`<div>${descriptionHtml}</div>`);
const root = document.querySelector("div");
if (!root) return { bodyMd: "", sidebar: {}, sectionBoxes: [] };
const flexDiv = root.querySelector('div[style*="display:flex"], div[style*="display: flex"]');
const sidebar: Record<string, string> = {};
const sectionBoxes: SectionBox[] = [];
// Walk the already-parsed left node directly (no serialize→re-parse round-trip).
let bodyNode: any = root;
if (flexDiv && flexDiv.children.length >= 2) {
bodyNode = flexDiv.children[0];
parseRightColumn(flexDiv.children[1], db, sidebar, sectionBoxes);
}
const bodyMd = collapse(nodeToMd(bodyNode, db));
return { bodyMd, sidebar, sectionBoxes };
}
// Ported from cc-4 parseRightColumn: distinguish sidebar field boxes from section boxes.
function parseRightColumn(right: any, db: JournalDb, sidebar: Record<string, string>, sectionBoxes: SectionBox[]): void {
for (const box of Array.from(right.children) as any[]) {
const style = box.getAttribute("style") ?? "";
// Section box: makeBodySectionBox uses font-size:0.8rem and an <h4> title.
if (/font-size:\s?0\.8/.test(style)) {
const h4 = (Array.from(box.children) as any[]).find((c) => c.tagName === "H4");
if (h4) {
const title = h4.textContent?.trim() ?? "";
const childNodes = Array.from(box.childNodes) as any[];
const after = childNodes.slice(childNodes.indexOf(h4) + 1);
const tmp = parseHTML("<div></div>").document.createElement("div");
for (const n of after) tmp.appendChild(n.cloneNode(true));
const body = collapse(nodeToMd(tmp, db));
sectionBoxes.push({ title, body });
continue;
}
}
// Sidebar field box: rows with justify-content:space-between.
const rows = box.querySelectorAll('div[style*="justify-content:space-between"], div[style*="justify-content: space-between"]');
for (const row of Array.from(rows) as any[]) {
const keyEl = row.querySelector("strong");
const valEl = row.querySelector("span");
if (!keyEl || !valEl) continue;
const key = keyEl.textContent?.trim().toLowerCase().replace(/\s+/g, "_") ?? "";
if (!key) continue;
// Sidebar values are frontmatter form: resolve @UUID -> [[Name]] (display dropped),
// via the shared link layer (handles every link, not just the first).
const value = collapse(stripTags(wikiToName(uuidToWiki(valEl.innerHTML, db))));
if (value) sidebar[key] = value;
}
}
}
/** Convert the notes HTML to markdown for the ## Secrets / ## Notes section. */
export function notesToMarkdown(notesHtml: string, db: JournalDb): string {
if (!notesHtml?.trim()) return "";
const { document } = parseHTML(`<div>${notesHtml}</div>`);
const root = document.querySelector("div");
if (!root) return "";
return collapse(nodeToMd(root, db));
}

48
src/links.ts Normal file
View File

@@ -0,0 +1,48 @@
import type { NameResolver } from "./resolver.js";
// Foundry content link: @UUID[JournalEntry.<id>]{Display}
const UUID_LINK = /@UUID\[([^\]]+)\]\{([^}]*)\}/g;
// Obsidian wiki link: [[Target]] or [[Target|Display]]
const WIKI_LINK = /\[\[([^\]|]+?)(?:\|([^\]]+?))?\]\]/g;
/**
* Convert Foundry @UUID content links to Obsidian wiki links.
* - resolved UUID -> [[Name]] (or [[Name|Display]] when display differs)
* - unresolved UUID -> the display text, plain (no broken link)
*
* Takes a NameResolver (JournalDb satisfies this structurally, so the offline
* DB path is unchanged; the live push path passes a MapNameResolver).
*/
export function uuidToWiki(text: string, r: NameResolver): string {
return text.replace(UUID_LINK, (_m, uuid: string, display: string) => {
const name = r.nameOf(uuid);
if (!name) return display;
const disp = display.trim();
return disp && disp !== name ? `[[${name}|${disp}]]` : `[[${name}]]`;
});
}
/**
* Convert Obsidian wiki links back to Foundry @UUID content links.
* - resolved name -> @UUID[JournalEntry.<id>]{Display} (Display defaults to target)
* - unresolved name -> left as a wiki link (cannot rebuild the UUID)
*/
export function wikiToUuid(text: string, r: NameResolver): string {
return text.replace(WIKI_LINK, (_m, target: string, display: string | undefined) => {
const name = target.trim();
const uuid = r.uuidOf(name);
if (!uuid) return _m;
const disp = (display ?? "").trim();
return `@UUID[${uuid}]{${disp || name}}`;
});
}
/**
* Collapse wiki links to their target name only (drop display text):
* [[Name|Display]] -> [[Name]] [[Name]] -> [[Name]]
* Used when emitting the cc.md native export format, which resolves to names and
* discards display aliases (see Roland Raventhornecc.md).
*/
export function wikiToName(text: string): string {
return text.replace(WIKI_LINK, (_m, target: string) => `[[${target.trim()}]]`);
}

84
src/mdToHtml.ts Normal file
View File

@@ -0,0 +1,84 @@
import type { NameResolver } from "./resolver.js";
import { wikiToUuid } from "./links.js";
import { canonicalizeWhitespace } from "./normalize.js";
/**
* Best-effort Markdown -> Foundry HTML for the optional --emit-json path and the
* live push path. Produces clean `<h2>/<p>/<strong>/<em>/<ul>/<h4>` markup with
* @UUID links and Bio/Social boxes. NOT byte-identical to Foundry's own two-column
* layout, but valid HTML that Campaign Codex will import and render.
*
* Takes a NameResolver (JournalDb satisfies this structurally, so the offline
* DB path is unchanged; the live push path passes a MapNameResolver).
*/
function escapeHtml(s: string): string {
return s.replace(/&/g, "&amp;").replace(/</g, "&lt;").replace(/>/g, "&gt;");
}
function inline(text: string, r: NameResolver): string {
let out = escapeHtml(text);
// inline links first (before bold/italic to avoid mangling [[ ]])
out = wikiToUuid(out, r);
out = out.replace(/\*\*([^*]+)\*\*/g, "<strong>$1</strong>");
out = out.replace(/\*([^*]+)\*/g, "<em>$1</em>");
return out;
}
export function markdownToHtml(md: string, r: NameResolver): string {
const lines = canonical(md).split("\n");
const blocks: string[] = [];
let i = 0;
const para: string[] = [];
const flush = () => {
if (para.length) {
blocks.push(`<p>${inline(para.join(" "), r)}</p>`);
para.length = 0;
}
};
while (i < lines.length) {
const line = lines[i];
if (!line.trim()) { flush(); i++; continue; }
const h = line.match(/^(#{1,6})\s+(.+)$/);
if (h) { flush(); const level = h[1].length; blocks.push(`<h${level}>${inline(h[2], r)}</h${level}>`); i++; continue; }
if (/^---\s*$/.test(line)) { flush(); blocks.push("<hr>"); i++; continue; }
if (/^>\s?/.test(line)) {
flush();
const quote: string[] = [];
while (i < lines.length && /^>\s?/.test(lines[i])) { quote.push(lines[i].replace(/^>\s?/, "")); i++; }
blocks.push(`<blockquote>${inline(quote.join(" "), r)}</blockquote>`);
continue;
}
if (/^[-*]\s+/.test(line)) {
flush();
const items: string[] = [];
while (i < lines.length && /^[-*]\s+/.test(lines[i])) { items.push(`<li>${inline(lines[i].replace(/^[-*]\s+/, ""), r)}</li>`); i++; }
blocks.push(`<ul>${items.join("")}</ul>`);
continue;
}
if (/^\d+\.\s+/.test(line)) {
flush();
const items: string[] = [];
while (i < lines.length && /^\d+\.\s+/.test(lines[i])) { items.push(`<li>${inline(lines[i].replace(/^\d+\.\s+/, ""), r)}</li>`); i++; }
blocks.push(`<ol>${items.join("")}</ol>`);
continue;
}
para.push(line);
i++;
}
flush();
return blocks.join("\n");
}
// Reuse the single canonicalization mechanism (normalize.ts) instead of a
// local weaker subset. CRLF is folded first; canonicalizeWhitespace then
// strips trailing per-line whitespace and collapses blank runs.
function canonical(md: string): string {
return canonicalizeWhitespace(md.replace(/\r\n/g, "\n"));
}
/** Build the Bio/Social sidebar box HTML for a reconstructed description. */
export function sidebarBoxHtml(title: string, rows: [string, string][]): string {
const rowHtml = rows.map(([k, v]) => `<div style="display:flex;justify-content:space-between;margin-bottom:0.5rem"><strong>${k}</strong><span>${v}</span></div>`).join("");
return `<div style="background:#0000001a;padding:1rem;border-radius:5px;border:1px solid #0000003b"><h4 style="margin-top:0;border-bottom:1px solid #0000003b;padding-bottom:0.5rem">${title}</h4>${rowHtml}</div>`;
}

26
src/normalize.ts Normal file
View File

@@ -0,0 +1,26 @@
import { createHash } from "node:crypto";
/** Canonicalize wiki link syntax: [[ a | b ]] -> [[a|b]], [[ a ]] -> [[a]]. */
export function canonicalizeWikilinks(text: string): string {
return text.replace(/\[\[\s*([^\]|]+?)\s*(?:\|\s*([^\]]+?)\s*)?\]\]/g, (_m, target: string, display: string | undefined) =>
display !== undefined ? `[[${target}|${display}]]` : `[[${target}]]`,
);
}
/** Collapse trailing whitespace per line and runs of blank lines. */
export function canonicalizeWhitespace(text: string): string {
return text
.replace(/[ \t]+\n/g, "\n")
.replace(/\n{3,}/g, "\n\n")
.replace(/^\s+|\s+$/g, "");
}
/** Full canonical text form used for hashing and comparison. */
export function canonicalize(text: string): string {
return canonicalizeWhitespace(canonicalizeWikilinks(text));
}
/** SHA-256 of the canonical form, used in the foundry: block to detect real content change. */
export function contentHash(text: string): string {
return createHash("sha256").update(canonicalize(text)).digest("hex");
}

134
src/push.ts Normal file
View File

@@ -0,0 +1,134 @@
import { mkdir, readFile, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { obsidianToFoundryJsonLive } from "./toFoundry.js";
import { splitFrontmatter, readFoundryBlock } from "./frontmatter.js";
import { RelayClient } from "./relay/client.js";
import { uploadPortrait } from "./foundry/assets.js";
import { MapNameResolver, nameUuidIndexFromEntries, saveNameUuidIndex, loadNameUuidIndex, type NameResolver } from "./resolver.js";
import { backupStamp } from "./write.js";
import type { JournalEntry } from "./types.js";
/**
* Pure payload builder (unit-tested without a relay): full JournalEntry JSON
* (spread the live entry; override name + flags.campaign-codex, links resolved via
* the name↔uuid map) + the minimal /update diff ({ name, "flags.campaign-codex" }).
* The diff uses a dot-path key so Foundry merges just that sub-flag (other flags
* survive) and never echoes _id/pages/ownership (which would clobber the live entry).
*/
export function buildPushPayload(
md: string,
noteName: string,
liveEntry: JournalEntry,
resolver: NameResolver,
imageOverride?: string | null,
): { full: JournalEntry; diff: Record<string, unknown> } {
const full = obsidianToFoundryJsonLive(md, noteName, liveEntry, resolver, imageOverride);
const cc = full.flags?.["campaign-codex"];
const diff: Record<string, unknown> = { name: full.name, "flags.campaign-codex": cc };
return { full, diff };
}
export interface PushDeps {
notePath: string;
noteName: string;
outDir: string;
relay: RelayClient;
/** Foundry host fs (for image upload). Empty string = skip image upload, keep existing. */
foundryDataDir: string;
world: string;
dryRun: boolean;
/** Optional preloaded resolver; else load <outDir>/name-uuid.json (or build via /search). */
resolver?: NameResolver;
log?: (msg: string) => void;
}
export interface PushOutcome {
dryRun: boolean;
ccUuid: string;
diff: Record<string, unknown>;
imageNote: string;
backupPath?: string;
updatedName?: string;
}
/** Read the foundry.cc_uuid and the portrait field from a refined note's
* frontmatter in a single parse (or { null, null } if absent). */
function readNoteFoundryMeta(md: string): { ccUuid: string | null; portrait: string | null } {
const { fm } = splitFrontmatter(md);
const foundry = readFoundryBlock(fm);
return {
ccUuid: foundry?.cc_uuid ?? null,
portrait: typeof fm.portrait === "string" && fm.portrait ? fm.portrait : null,
};
}
/**
* Push ONE refined note into the live Foundry world via the relay. Foundry keeps
* running (no LevelDB opened, no Docker stop). Shared by the CLI `push` command
* and the dashboard's /api/push endpoint.
*/
export async function pushNote(deps: PushDeps): Promise<PushOutcome> {
const log = deps.log ?? (() => {});
const md = await readFile(deps.notePath, "utf8");
const { ccUuid, portrait } = readNoteFoundryMeta(md);
if (!ccUuid) throw new Error(`no foundry.cc_uuid in ${deps.notePath} — run "seed" first to link this note to its Foundry entry`);
const id = ccUuid.startsWith("JournalEntry.") ? ccUuid : `JournalEntry.${ccUuid}`;
// Resolver for link resolution: preload, else load cache, else build via /search.
let resolver: NameResolver = deps.resolver ?? new MapNameResolver({ nameToUuid: {}, uuidToName: {} });
if (!deps.resolver) {
try {
resolver = await loadNameUuidIndex(join(deps.outDir, "name-uuid.json"));
} catch {
log("push: no cached name-uuid.json — building one via relay /search…");
const results = await deps.relay.searchJournalEntries();
const idx = nameUuidIndexFromEntries(results.map((r) => ({ name: r.name, uuid: r.uuid })));
await saveNameUuidIndex(idx, join(deps.outDir, "name-uuid.json"));
resolver = new MapNameResolver(idx);
}
}
// Fetch the live entry (preserves current ownership/folder/pages + existing image).
log(`push: fetching live entry ${id} via relay /get…`);
const liveEntry = await deps.relay.getEntry(id);
// Image: upload the portrait into Foundry's assets dir if the note has one.
let imageOverride: string | null | undefined = undefined; // undefined = keep existing
let imageNote = "no portrait field";
if (portrait) {
if (!deps.foundryDataDir || !deps.world) {
imageNote = "portrait present but FOUNDRY_DATA_DIR/WORLD unset — keeping existing image";
log(`push: image ${imageNote}`);
} else {
const up = await uploadPortrait({
notePath: deps.notePath,
portraitField: portrait,
foundryDataDir: deps.foundryDataDir,
world: deps.world,
existingImage: liveEntry.flags?.["campaign-codex"]?.image ?? null,
});
imageNote = up.note;
log(`push: image ${imageNote}`);
imageOverride = up.servedPath;
}
}
const { diff } = buildPushPayload(md, deps.noteName, liveEntry, resolver, imageOverride);
if (deps.dryRun) {
log(`[dry-run] push ${deps.noteName} (${id})`);
return { dryRun: true, ccUuid: id, diff, imageNote };
}
// Apply: snapshot the live entry first (reversible), then PUT the diff.
const bakDir = join(deps.outDir, "bak");
await mkdir(bakDir, { recursive: true });
const stamp = backupStamp();
const backupPath = join(bakDir, `${deps.noteName}.${stamp}.json`);
await writeFile(backupPath, JSON.stringify(liveEntry, null, 2) + "\n", "utf8");
log(`push: backed up live entry -> ${backupPath}`);
const updated = await deps.relay.updateEntry(id, diff);
log(`push: updated "${updated.name}" in live Foundry (${id})`);
return { dryRun: false, ccUuid: id, diff, imageNote, backupPath, updatedName: updated.name };
}

106
src/relay/client.ts Normal file
View File

@@ -0,0 +1,106 @@
import type { JournalEntry } from "../types.js";
import type { RelayConfig } from "../config.js";
/**
* Thin client for the ThreeHats foundryvtt-rest-api-relay (vtt-relay.damascusfront.net).
*
* Auth: `x-api-key` header + `clientId` query param (NOT a header). Assumes a live
* Foundry instance with the rest-api module connected — no headless session spin-up
* (Foundry runs in Docker on this host). If `clientId` is unset the relay auto-
* resolves it only when exactly one client is connected to the key.
*
* Response envelopes are per-endpoint (NOT uniform): /get -> { data }, /update ->
* { entity: [...] }, /search -> { query?, results: [...] }, /create -> { uuid, data }.
* Relay-level errors are flat: { error: "..." } with a non-2xx status.
*
* Spec source: github.com/ThreeHats/foundryvtt-rest-api-relay (see docs/relay-api.md).
*/
export interface SearchResult {
uuid: string;
id: string;
name: string;
img?: string | null;
documentType: string;
}
export class RelayClient {
private readonly url: string;
private readonly apiKey: string;
private readonly clientId: string;
constructor(cfg: RelayConfig) {
if (!cfg.apiKey) throw new Error("RELAY_API_KEY is required for relay commands (set --relay-api-key or RELAY_API_KEY)");
this.url = cfg.url;
this.apiKey = cfg.apiKey;
this.clientId = cfg.clientId;
}
private qs(extra: Record<string, string> = {}): string {
const sp = new URLSearchParams();
if (this.clientId) sp.set("clientId", this.clientId);
for (const [k, v] of Object.entries(extra)) if (v !== "") sp.set(k, v);
const s = sp.toString();
return s ? `?${s}` : "";
}
private headers(): Record<string, string> {
return { "content-type": "application/json", "x-api-key": this.apiKey };
}
private async request(method: string, path: string, qs: Record<string, string>, body?: unknown): Promise<unknown> {
const url = `${this.url}${path}${this.qs(qs)}`;
const init: RequestInit = { method, headers: this.headers() };
if (body !== undefined) init.body = JSON.stringify(body);
const res = await fetch(url, init);
const text = await res.text();
let json: unknown = undefined;
try { json = text ? JSON.parse(text) : undefined; } catch { /* non-JSON error body */ }
if (!res.ok) {
const msg = (json && typeof json === "object" && "error" in json)
? String((json as Record<string, unknown>).error)
: text.slice(0, 200);
throw new Error(`relay ${res.status} ${method} ${path}: ${msg}`);
}
return json;
}
/** GET /get — fetch one document by UUID. Returns the full Foundry document. */
async getEntry(uuid: string): Promise<JournalEntry> {
const json = await this.request("GET", "/get", { uuid }) as { data?: JournalEntry };
if (!json?.data) throw new Error(`relay /get returned no data for ${uuid}`);
return json.data;
}
/**
* PUT /update — update one document. `data` is a DIFF (dot-path keys merge; full
* keys replace), NOT the full document. The entity is identified by `uuid`
* (which embeds the type, e.g. JournalEntry.<id>) — no entityType field.
* Returns the updated document.
*/
async updateEntry(uuid: string, data: Record<string, unknown>): Promise<JournalEntry> {
const json = await this.request("PUT", "/update", { uuid }, { data }) as { entity?: JournalEntry[] };
const updated = json?.entity?.[0];
if (!updated) throw new Error(`relay /update returned no entity for ${uuid}`);
return updated;
}
/** POST /create — create a new document. `entityType` is the collection (e.g. "JournalEntry"). */
async createEntry(entityType: string, data: Record<string, unknown>, opts: { keepId?: boolean; folder?: string } = {}): Promise<{ uuid: string; data: JournalEntry }> {
const body: Record<string, unknown> = { entityType, data, ...opts };
const json = await this.request("POST", "/create", {}, body) as { uuid?: string; data?: JournalEntry };
if (!json?.uuid || !json?.data) throw new Error("relay /create returned no uuid/data");
return { uuid: json.uuid, data: json.data };
}
/** GET /search — list all JournalEntries (omit query). minified -> {uuid,id,name,img,documentType}. */
async searchJournalEntries(limit = 500): Promise<SearchResult[]> {
const json = await this.request("GET", "/search", {
filter: "documentType:JournalEntry",
excludeCompendiums: "true",
limit: String(limit),
minified: "true",
}) as { results?: SearchResult[] };
return json?.results ?? [];
}
}

64
src/resolver.ts Normal file
View File

@@ -0,0 +1,64 @@
import { writeFile, readFile, mkdir } from "node:fs/promises";
import { dirname } from "node:path";
/**
* Name ⇄ UUID resolution for link rewriting.
*
* `markdownToHtml` / `wikiToUuid` / `uuidToWiki` only need two lookups:
* name→uuid and uuid→name. `JournalDb` already provides both (`uuidOf`/`nameOf`),
* so it satisfies this interface structurally — the offline DB path is unchanged.
* The live push path supplies a `MapNameResolver` backed by the cached
* name-uuid.json (built by `cmd refresh`), so links resolve WITHOUT opening the
* Foundry LevelDB — the whole point of one-at-a-time push.
*/
export interface NameResolver {
/** Resolve a name to its Foundry UUID ("JournalEntry.<id>"), or undefined. */
uuidOf(name: string): string | undefined;
/** Resolve a Foundry UUID to its name, or undefined. */
nameOf(uuid: string): string | undefined;
}
/** Persisted name↔uuid index (both directions) written to name-uuid.json. */
export interface NameUuidIndex {
nameToUuid: Record<string, string>;
uuidToName: Record<string, string>;
}
/** Build a NameUuidIndex from a list of {name, uuid} (e.g. relay /search minified
* results). First occurrence wins per name/uuid, mirroring JournalDb's load(). */
export function nameUuidIndexFromEntries(entries: { name: string; uuid: string }[]): NameUuidIndex {
const nameToUuid: Record<string, string> = {};
const uuidToName: Record<string, string> = {};
for (const e of entries) {
if (!e.name || !e.uuid) continue;
if (!nameToUuid[e.name]) nameToUuid[e.name] = e.uuid;
if (!uuidToName[e.uuid]) uuidToName[e.uuid] = e.name;
}
return { nameToUuid, uuidToName };
}
/** A NameResolver backed by a name↔uuid index (loaded from name-uuid.json).
* Queries the records directly — no need to copy them into Maps. */
export class MapNameResolver implements NameResolver {
constructor(private readonly idx: NameUuidIndex) {}
uuidOf(name: string): string | undefined {
return this.idx.nameToUuid[name];
}
nameOf(uuid: string): string | undefined {
return this.idx.uuidToName[uuid];
}
}
/** Save a name↔uuid index to disk as JSON (for the push path to load later). */
export async function saveNameUuidIndex(idx: NameUuidIndex, path: string): Promise<void> {
await mkdir(dirname(path), { recursive: true });
await writeFile(path, JSON.stringify(idx, null, 2) + "\n", "utf8");
}
/** Load a cached name↔uuid index from disk and wrap it in a MapNameResolver. */
export async function loadNameUuidIndex(path: string): Promise<MapNameResolver> {
const raw = await readFile(path, "utf8");
return new MapNameResolver(JSON.parse(raw) as NameUuidIndex);
}

293
src/server.ts Normal file
View File

@@ -0,0 +1,293 @@
// Local-only review dashboard for the batch connector.
//
// Binds 127.0.0.1 only (never 0.0.0.0) — no external network exposure. Serves a
// single static dashboard page and a small JSON API over the batch engine. The
// journal LevelDB is opened read-only. Writes go only to --out in dev mode; in
// apply mode they back up then write to the real refined/cc dirs. dry-run never
// writes. The server's mode is fixed at startup; the UI cannot escalate to apply
// unless the server was started with --apply.
import { createServer, type IncomingMessage, type ServerResponse, type Server } from "node:http";
import { readFile, writeFile, mkdir, copyFile, access } from "node:fs/promises";
import { join, dirname } from "node:path";
import { fileURLToPath } from "node:url";
import { JournalDb } from "./db.js";
import { indexAll, seedRow, syncRow, rePullRow, importRow, type FileRow, type IndexResult } from "./batch.js";
import { RelayClient } from "./relay/client.js";
import { pushNote } from "./push.js";
import { nameUuidIndexFromEntries, saveNameUuidIndex } from "./resolver.js";
import { backupStamp } from "./write.js";
import type { RelayConfig, FoundryHostConfig } from "./config.js";
import type { Mode } from "./types.js";
const __dirname = dirname(fileURLToPath(import.meta.url));
const DASHBOARD_PATH = join(__dirname, "dashboard.html");
export interface ServerConfig {
journal: string;
refinedDir: string;
ccDir: string;
outDir: string;
mode: Mode; // dev | apply — fixed at startup
port: number;
host: string; // bind address (127.0.0.1 by default; 0.0.0.0 to expose on the network)
// Live push path (optional). When relayCfg is set, /api/push and /api/refresh work.
relayCfg?: RelayConfig;
foundryCfg?: FoundryHostConfig;
}
export interface ActionResult {
op: string;
mode: Mode;
dryRun: boolean;
written: { path: string; bytes: number }[];
preview: { filename: string; path: string; content: string }[];
skipped: { name: string; reason: string }[];
message: string;
}
interface State {
db: JournalDb;
cfg: ServerConfig;
index: IndexResult | null;
}
function send(res: ServerResponse, code: number, body: unknown): void {
const payload = typeof body === "string" ? body : JSON.stringify(body);
const buf = Buffer.from(payload, "utf8");
res.writeHead(code, {
"content-type": typeof body === "string" ? "text/html; charset=utf-8" : "application/json; charset=utf-8",
"content-length": String(buf.length),
"x-content-type-options": "nosniff",
"cache-control": "no-store",
});
res.end(buf);
}
/** Resolve where a write should land given mode + bucket + the row's relative path. */
function targetPath(state: State, bucket: "refined" | "cc", relPath: string): string {
if (state.cfg.mode === "apply") {
return bucket === "refined" ? join(state.cfg.refinedDir, relPath) : join(state.cfg.ccDir, relPath);
}
// dev: mirror under --out/<bucket>/<relPath>
return join(state.cfg.outDir, bucket, relPath);
}
async function writeWithBackup(target: string, content: string, state: State): Promise<{ path: string; bytes: number }> {
if (state.cfg.mode === "apply") {
try {
await access(target);
const stamp = backupStamp();
await copyFile(target, `${target}.bak-${stamp}`);
} catch { /* new file, no backup */ }
}
await mkdir(dirname(target), { recursive: true });
await writeFile(target, content, "utf8");
return { path: target, bytes: Buffer.byteLength(content) };
}
/** Land an action's output: preview (dry-run) or written (apply/dev). */
async function writeOrPreview(
state: State, target: string, filename: string, content: string,
dryRun: boolean, result: ActionResult,
): Promise<void> {
if (dryRun) result.preview.push({ filename, path: target, content });
else result.written.push(await writeWithBackup(target, content, state));
}
/** Read and JSON-parse a request body, or null on empty/invalid JSON. */
async function readJsonBody(req: IncomingMessage): Promise<Record<string, unknown> | null> {
const chunks: Buffer[] = [];
for await (const c of req) chunks.push(c as Buffer);
try { return JSON.parse(Buffer.concat(chunks).toString("utf8") || "{}") as Record<string, unknown>; }
catch { return null; }
}
async function runAction(state: State, op: string, names: string[] | null, dryRun: boolean): Promise<ActionResult> {
const result: ActionResult = { op, mode: state.cfg.mode, dryRun, written: [], preview: [], skipped: [], message: "" };
if (!state.index) throw new Error("index not loaded");
const stamp = new Date().toISOString();
const rows: FileRow[] = [];
if (op === "seedAll" || op === "syncAll" || op === "repullAll") {
rows.push(...state.index.matched);
} else if (op === "importAll") {
rows.push(...state.index.ccOnly);
} else {
const pool = op === "import" ? state.index.ccOnly : state.index.matched;
for (const n of names ?? []) {
const r = pool.find((x) => x.name === n || x.basename === n);
if (r) rows.push(r); else result.skipped.push({ name: n, reason: "not found in bucket" });
}
}
for (const row of rows) {
if (op === "seed" || op === "seedAll") {
if (!row.refinedPath || !row.entry) { result.skipped.push({ name: row.name, reason: "no refined note or no journal entry (unlinked)" }); continue; }
const out = await seedRow(row, state.cfg.refinedDir, stamp);
if (!out) { result.skipped.push({ name: row.name, reason: "seed produced no output" }); continue; }
await writeOrPreview(state, targetPath(state, "refined", row.refinedPath), out.filename, out.content, dryRun, result);
} else if (op === "sync" || op === "syncAll") {
if (!row.refinedPath || !row.ccPath) { result.skipped.push({ name: row.name, reason: "no refined/cc counterpart" }); continue; }
const out = await syncRow(row, state.cfg.refinedDir, state.db, stamp);
if (!out || (!out.refined && !out.cc)) { result.skipped.push({ name: row.name, reason: "sync produced no output (no Foundry match)" }); continue; }
// sync baselines BOTH sides: write the refreshed refined note AND the regenerated cc.
if (out.refined) await writeOrPreview(state, targetPath(state, "refined", row.refinedPath), out.refined.filename, out.refined.content, dryRun, result);
if (out.cc) await writeOrPreview(state, targetPath(state, "cc", row.ccPath), out.cc.filename, out.cc.content, dryRun, result);
} else if (op === "repull" || op === "repullAll") {
if (!row.refinedPath || !row.entry) { result.skipped.push({ name: row.name, reason: "no refined note or no journal entry (unlinked)" }); continue; }
const out = await rePullRow(row, state.cfg.refinedDir, state.db, stamp);
if (!out) { result.skipped.push({ name: row.name, reason: "re-pull produced no output" }); continue; }
await writeOrPreview(state, targetPath(state, "refined", row.refinedPath), out.filename, out.content, dryRun, result);
} else if (op === "import" || op === "importAll") {
if (!row.entry) { result.skipped.push({ name: row.name, reason: "no journal entry (unlinked)" }); continue; }
const out = importRow(row, state.db, stamp);
if (!out) { result.skipped.push({ name: row.name, reason: "import produced no output" }); continue; }
// Imports are un-curated staging: land under refined/imported/<cc_folder>/ for review.
await writeOrPreview(state, targetPath(state, "refined", join("imported", out.subfolder, out.filename)), out.filename, out.content, dryRun, result);
} else {
throw new Error(`unknown op: ${op}`);
}
}
const verb = dryRun ? "would write" : "wrote";
result.message = `${op}: ${dryRun ? result.preview.length : result.written.length} ${verb}${result.skipped.length ? `, ${result.skipped.length} skipped` : ""} [${state.cfg.mode}${dryRun ? " dry-run" : ""}]`;
return result;
}
async function fileDetail(state: State, name: string): Promise<unknown> {
if (!state.index) throw new Error("index not loaded");
const row = state.index.matched.find((r) => r.name === name || r.basename === name)
?? state.index.ccOnly.find((r) => r.name === name || r.basename === name);
if (!row) throw new Error(`no row for ${name}`);
const refined = row.refinedPath ? await readFile(join(state.cfg.refinedDir, row.refinedPath), "utf8").catch(() => null) : null;
const cc = row.ccPath ? await readFile(join(state.cfg.ccDir, row.ccPath), "utf8").catch(() => null) : null;
const stamp = new Date().toISOString();
const seedPreview = row.refinedPath && row.entry ? (await seedRow(row, state.cfg.refinedDir, stamp))?.content ?? null : null;
const synced = row.refinedPath ? await syncRow(row, state.cfg.refinedDir, state.db, stamp) : null;
const syncPreview = synced?.cc?.content ?? null;
const syncRefinedPreview = synced?.refined?.content ?? null;
const repullPreview = row.refinedPath && row.entry ? (await rePullRow(row, state.cfg.refinedDir, state.db, stamp))?.content ?? null : null;
return { row, refined, cc, entry: row.entry, seedPreview, syncPreview, syncRefinedPreview, repullPreview };
}
async function handlePost(state: State, req: IncomingMessage, res: ServerResponse): Promise<void> {
const body = await readJsonBody(req);
if (body === null) return send(res, 400, { error: "bad json" });
const op = body.op;
if (typeof op !== "string" || !op) return send(res, 400, { error: "missing op" });
const names = Array.isArray(body.names) ? body.names as string[] : null;
const dryRun = body.dryRun === true;
try {
const result = await runAction(state, op, names, dryRun);
send(res, 200, result);
} catch (e) {
send(res, 500, { error: (e as Error).message });
}
}
function relayClient(state: State): RelayClient {
if (!state.cfg.relayCfg) throw new Error("relay not configured (start the server with RELAY_API_KEY / --relay-api-key to enable push)");
return new RelayClient(state.cfg.relayCfg);
}
/** POST /api/push { name, dryRun } — push one refined note into live Foundry via the relay. */
async function handlePush(state: State, req: IncomingMessage, res: ServerResponse): Promise<void> {
const body = await readJsonBody(req);
if (body === null) return send(res, 400, { error: "bad json" });
const name = typeof body.name === "string" ? body.name : "";
if (!name) return send(res, 400, { error: "missing name" });
if (!state.index) throw new Error("index not loaded");
const row = state.index.matched.find((r) => r.name === name || r.basename === name);
if (!row?.refinedPath) return send(res, 400, { error: `no refined note for "${name}"` });
try {
const relay = relayClient(state);
const logs: string[] = [];
const outcome = await pushNote({
notePath: join(state.cfg.refinedDir, row.refinedPath),
noteName: row.name,
outDir: state.cfg.outDir,
relay,
foundryDataDir: state.cfg.foundryCfg?.dataDir ?? "",
world: state.cfg.foundryCfg?.world ?? "",
dryRun: body.dryRun !== false, // default to dry-run from the dashboard (safety)
log: (m) => logs.push(m),
});
send(res, 200, { ...outcome, logs });
} catch (e) {
send(res, 500, { error: (e as Error).message });
}
}
/** POST /api/refresh { fullIndex? } — rebuild the name↔uuid map via relay /search
* (zero Foundry downtime). The heavy docker-stop full index is CLI-only by design. */
async function handleRefresh(state: State, req: IncomingMessage, res: ServerResponse): Promise<void> {
try {
const relay = relayClient(state);
const results = await relay.searchJournalEntries();
const idx = nameUuidIndexFromEntries(results.map((r) => ({ name: r.name, uuid: r.uuid })));
await saveNameUuidIndex(idx, join(state.cfg.outDir, "name-uuid.json"));
send(res, 200, { pairs: Object.keys(idx.nameToUuid).length, path: join(state.cfg.outDir, "name-uuid.json") });
} catch (e) {
send(res, 500, { error: (e as Error).message });
}
}
export async function startServer(cfg: ServerConfig): Promise<{ server: Server; state: State }> {
const db = await JournalDb.open(cfg.journal);
const state: State = { db, cfg, index: null };
// Build the index once at startup (the corpora are static while the server runs).
state.index = await indexAll(db, cfg.ccDir, cfg.refinedDir);
const server = createServer(async (req, res) => {
try {
const url = new URL(req.url ?? "/", "http://127.0.0.1");
// Silence the favicon 404 noise every browser generates.
if (url.pathname === "/favicon.ico") {
res.writeHead(204, { "content-length": "0", "cache-control": "max-age=86400" });
return res.end();
}
// HEAD == GET headers (some clients/proxies probe with HEAD).
const isHead = req.method === "HEAD";
const method = isHead ? "GET" : (req.method ?? "GET");
if (method === "GET" && url.pathname === "/") {
const html = await readFile(DASHBOARD_PATH, "utf8");
if (isHead) {
res.writeHead(200, { "content-type": "text/html; charset=utf-8", "content-length": String(Buffer.byteLength(html, "utf8")) });
return res.end();
}
return send(res, 200, html);
}
if (req.method === "GET" && url.pathname === "/api/index") {
// Rebuild on every request: in apply mode the source files change after an
// action, so a cached index would show stale recommendations. Indexing is
// hash-only (no per-row conversion), so this stays cheap.
state.index = await indexAll(state.db, state.cfg.ccDir, state.cfg.refinedDir);
return send(res, 200, state.index);
}
if (req.method === "GET" && url.pathname === "/api/status") {
return send(res, 200, { mode: state.cfg.mode, refinedDir: state.cfg.refinedDir, ccDir: state.cfg.ccDir, outDir: state.cfg.outDir });
}
if (req.method === "GET" && url.pathname === "/api/file") {
const name = url.searchParams.get("name");
if (!name) return send(res, 400, { error: "missing name" });
return send(res, 200, await fileDetail(state, name));
}
if (req.method === "POST" && url.pathname === "/api/action") {
return handlePost(state, req, res);
}
if (req.method === "POST" && url.pathname === "/api/push") {
return handlePush(state, req, res);
}
if (req.method === "POST" && url.pathname === "/api/refresh") {
return handleRefresh(state, req, res);
}
send(res, 404, { error: "not found" });
} catch (e) {
send(res, 500, { error: (e as Error).message });
}
});
await new Promise<void>((resolve) => server.listen(cfg.port, cfg.host, resolve));
return { server, state };
}

200
src/toFoundry.ts Normal file
View File

@@ -0,0 +1,200 @@
import type { JournalDb } from "./db.js";
import type { NameResolver } from "./resolver.js";
import type { JournalEntry, CcFlags } from "./types.js";
import { ccTypeFromObsidianType, folderPathFromCcType, FIELD_TO_LABEL } from "./fields.js";
import { splitFrontmatter, emitFrontmatter, readFoundryBlock, type Frontmatter, CC_FM_ORDER } from "./frontmatter.js";
import { wikiToName } from "./links.js";
import { canonicalize } from "./normalize.js";
import { markdownToHtml, sidebarBoxHtml } from "./mdToHtml.js";
interface Section {
heading: string;
body: string;
}
function parseBody(body: string): { tagline: string; sections: Section[] } {
const parts = canonicalize(body).split(/^## /m);
const preamble = parts[0] ?? "";
const tagMatch = preamble.match(/\*([^*]+)\*/);
const tagline = tagMatch ? `*${tagMatch[1]}*` : "";
const sections: Section[] = [];
for (const part of parts.slice(1)) {
const nl = part.indexOf("\n");
const heading = (nl < 0 ? part : part.slice(0, nl)).trim();
const sbody = (nl < 0 ? "" : part.slice(nl + 1)).trim();
sections.push({ heading, body: sbody });
}
return { tagline, sections };
}
function sidebarRow(field: "race" | "faction" | "region", fm: Frontmatter): string {
const val = fm[field];
if (typeof val !== "string" || !val) return "";
return `**${FIELD_TO_LABEL[field]}**${val}`;
}
export interface CcNote {
filename: string; // "<name>cc.md"
content: string;
}
/**
* Obsidian curated .md -> Campaign Codex native export (cc.md).
* Pulls cc_id/associates/image from the matched Foundry entry (by foundry: block
* or name). Wiki links collapse to [[Name]] (display dropped), matching the
* cc export format.
*/
export function obsidianToCc(md: string, noteName: string, db: JournalDb, exportedAt: string): CcNote | null {
const { fm, body } = splitFrontmatter(md);
const { tagline, sections } = parseBody(body);
// Resolve the Foundry entry: prefer the foundry: identity block, then name match.
let entry: JournalEntry | undefined;
const foundry = readFoundryBlock(fm);
if (foundry?.cc_uuid) entry = db.byUuid(foundry.cc_uuid);
if (!entry) entry = db.byName(noteName);
const ccType = entry?.flags?.["campaign-codex"]?.type ?? ccTypeFromObsidianType(typeof fm.type === "string" ? fm.type : undefined);
const ccId = entry?._id ?? (foundry?.cc_uuid ? foundry.cc_uuid.replace(/^JournalEntry\./, "") : "");
const ccUuid = entry ? `JournalEntry.${entry._id}` : foundry?.cc_uuid ?? (ccId ? `JournalEntry.${ccId}` : "");
if (!ccId || !ccUuid) return null; // cannot identify the Foundry entry
const associates = entry?.flags?.["campaign-codex"]?.data?.associates ?? [];
const ccfm: Frontmatter = {
cc_id: ccId,
cc_uuid: ccUuid,
cc_type: ccType,
cc_folder_path: folderPathFromCcType(ccType),
cc_exported_at: exportedAt,
};
const frontmatter = emitFrontmatter(ccfm, CC_FM_ORDER, true);
const parts: string[] = [`# ${noteName}`, "## Information"];
if (tagline) parts.push(tagline);
const secrets = sections.find((s) => s.heading.toLowerCase() === "secrets");
for (const s of sections) {
if (s.heading.toLowerCase() === "secrets") continue;
parts.push(`## ${s.heading}${s.body ? `\n\n${wikiToName(s.body)}` : ""}`);
}
// Bio / Social sidebar boxes (order: Race, Faction, Region), emitted adjacent
// to match Campaign Codex's native export layout.
const bio = sidebarRow("race", fm);
const social = [sidebarRow("faction", fm), sidebarRow("region", fm)].filter(Boolean).join("");
const bioSocial = [bio ? `#### Bio\n${bio}` : "", social ? `#### Social\n${social}` : ""].filter(Boolean).join("\n");
if (bioSocial) parts.push(bioSocial);
// Linked sheets from Foundry associates.
if (associates.length) {
const links = associates
.map((uuid) => db.nameOf(uuid))
.filter((n): n is string => !!n)
.map((n) => `- [[${n}]]`);
if (links.length) parts.push(`## Linked Sheets\n\n### Associates\n\n${links.join("\n")}`);
}
// Notes (from the Obsidian ## Secrets section).
if (secrets?.body) parts.push(`## Notes\n\n${wikiToName(secrets.body)}`);
const content = `${frontmatter}\n\n${parts.join("\n\n")}\n`;
return { filename: `${noteName}cc.md`, content };
}
/** Reconstruct a Foundry-importable JournalEntry JSON from an Obsidian note.
* Best-effort: description/notes HTML is rebuilt cleanly (not byte-identical to
* Foundry's two-column layout), but valid and importable by Campaign Codex.
*
* Offline DB path: resolves the entry from the journal (by name or foundry:
* block) and uses the DB as the link resolver. */
export function obsidianToFoundryJson(md: string, noteName: string, db: JournalDb): JournalEntry | null {
const { fm } = splitFrontmatter(md);
let entry = db.byName(noteName);
const foundry = readFoundryBlock(fm);
if (foundry?.cc_uuid) entry = db.byUuid(foundry.cc_uuid) ?? entry;
if (!entry) return null;
return buildFoundryJson(md, noteName, entry, db);
}
/**
* Live push path: builds the Foundry JSON from a **live-fetched** entry (via the
* relay /get) and a map-backed resolver (the cached name-uuid.json) — no LevelDB
* opened. `imageOverride` (the served path of a freshly uploaded portrait) wins
* over the entry's existing `flags.campaign-codex.image`; pass null/undefined to
* keep whatever image the entry already has.
*/
export function obsidianToFoundryJsonLive(
md: string,
noteName: string,
liveEntry: JournalEntry,
r: NameResolver,
imageOverride?: string | null,
): JournalEntry {
return buildFoundryJson(md, noteName, liveEntry, r, imageOverride);
}
/** Shared core: turns an Obsidian note + a Foundry entry + a name resolver into
* an importable JournalEntry JSON (spreads the entry so ownership/folder/pages
* survive, overrides name + flags["campaign-codex"]). */
function buildFoundryJson(
md: string,
noteName: string,
entry: JournalEntry,
r: NameResolver,
imageOverride?: string | null,
): JournalEntry {
const { fm, body } = splitFrontmatter(md);
const { tagline, sections } = parseBody(body);
const ccType = entry.flags?.["campaign-codex"]?.type ?? ccTypeFromObsidianType(typeof fm.type === "string" ? fm.type : undefined);
const secrets = sections.find((s) => s.heading.toLowerCase() === "secrets");
// Body markdown: tagline + non-secrets sections.
const bodyParts: string[] = [];
if (tagline) bodyParts.push(tagline);
for (const s of sections) {
if (s.heading.toLowerCase() === "secrets") continue;
bodyParts.push(`## ${s.heading}${s.body ? `\n\n${s.body}` : ""}`);
}
const leftHtml = markdownToHtml(bodyParts.join("\n\n"), r);
// Sidebar boxes from frontmatter (resolve names -> @UUID via markdownToHtml on the [[ ]] value).
const boxes: string[] = [];
const race = fmSide(fm, "race");
const faction = fmSide(fm, "faction");
const region = fmSide(fm, "region");
if (race) boxes.push(sidebarBoxHtml("Bio", [["Race", markdownToHtml(race, r)]]));
if (faction || region) {
const rows: [string, string][] = [];
if (faction) rows.push(["Faction", markdownToHtml(faction, r)]);
if (region) rows.push(["Region", markdownToHtml(region, r)]);
boxes.push(sidebarBoxHtml("Social", rows));
}
const description = `<div style="display:flex;flex-wrap:wrap;gap:2rem"><div style="flex:2;min-width:250px">${leftHtml}</div><div style="flex:1;min-width:250px">${boxes.join("")}</div></div>`;
const notesHtml = secrets?.body ? markdownToHtml(secrets.body, r) : "";
const tags = (Array.isArray(fm.tags) ? fm.tags : []).filter((t) => !t.startsWith("status/"));
const cc: CcFlags = {
type: ccType,
image: imageOverride !== undefined ? imageOverride : (entry.flags?.["campaign-codex"]?.image ?? null),
data: {
...entry.flags?.["campaign-codex"]?.data,
description,
notes: notesHtml,
tags,
},
};
return {
...entry,
name: noteName,
flags: { "campaign-codex": cc },
};
}
function fmSide(fm: Frontmatter, field: "race" | "faction" | "region"): string {
const v = fm[field];
return typeof v === "string" ? v : "";
}

67
src/toObsidian.ts Normal file
View File

@@ -0,0 +1,67 @@
import type { JournalEntry, FoundryBlock } from "./types.js";
import type { JournalDb } from "./db.js";
import { htmlToMarkdown, notesToMarkdown } from "./htmlMd.js";
import { obsidianTypeFromCcType, folderPathFromCcType } from "./fields.js";
import { emitFrontmatter, type Frontmatter, OBSIDIAN_FM_ORDER } from "./frontmatter.js";
import { canonicalize, contentHash } from "./normalize.js";
const STATUS_TAG = "status/draft";
function hasStatusTag(tags: string[] | undefined): boolean {
return !!tags?.some((t) => t.startsWith("status/"));
}
function basename(url: string | null | undefined): string | undefined {
if (!url) return undefined;
return url.split("/").pop()?.split("?")[0];
}
export interface ObsidianNote {
filename: string; // "<name>.md"
content: string; // full markdown with frontmatter
}
/** Foundry JournalEntry -> curated Obsidian .md (with foundry: identity block). */
export function entryToObsidian(entry: JournalEntry, db: JournalDb, syncedAt: string): ObsidianNote {
const cc = entry.flags?.["campaign-codex"];
const ccType = cc?.type ?? "shop";
const data = cc?.data ?? {};
const image = cc?.image ?? null;
const { bodyMd, sidebar, sectionBoxes } = htmlToMarkdown(data.description ?? "", db);
const notesMd = notesToMarkdown(data.notes ?? "", db);
const type = obsidianTypeFromCcType(ccType);
const tags = Array.isArray(data.tags) ? data.tags.filter(Boolean) : [];
if (!tags.includes(type)) tags.unshift(type);
if (!hasStatusTag(tags)) tags.push(STATUS_TAG);
const fm: Frontmatter = { type, tags };
if (sidebar.faction) fm.faction = sidebar.faction;
if (sidebar.region) fm.region = sidebar.region;
if (sidebar.race) fm.race = sidebar.race;
const portrait = basename(image);
if (portrait) fm.portrait = portrait;
// Assemble body: left-column prose + extra section boxes + Secrets (notes).
const parts: string[] = [];
if (bodyMd) parts.push(bodyMd);
for (const box of sectionBoxes) {
parts.push(`## ${box.title}${box.body ? `\n\n${box.body}` : ""}`);
}
if (notesMd) parts.push(`## Secrets\n\n${notesMd}`);
const body = parts.join("\n\n");
const foundry: FoundryBlock = {
cc_uuid: `JournalEntry.${entry._id}`,
cc_type: ccType,
folder_path: folderPathFromCcType(ccType),
contentHash: contentHash(body),
syncedAt,
};
fm.foundry = foundry as unknown as Record<string, string>;
const frontmatter = emitFrontmatter(fm, OBSIDIAN_FM_ORDER);
const content = `${frontmatter}\n\n${canonicalize(body)}\n`;
return { filename: `${entry.name}.md`, content };
}

75
src/types.ts Normal file
View File

@@ -0,0 +1,75 @@
// Foundry JournalEntry shapes (only the fields we use). Verified against the
// journal LevelDB snapshot via classic-level read probes.
export interface CcData {
description?: string; // HTML
notes?: string; // HTML
tags?: string[];
associates?: string[]; // e.g. ["JournalEntry.<id>"]
linkedLocations?: string[];
linkedShops?: string[];
linkedActor?: string | null;
linkedScene?: string | null;
hiddenAssociates?: string[];
inventory?: unknown[];
inventoryCash?: number;
markup?: number;
isLoot?: boolean;
tagMode?: boolean;
}
export interface CcFlags {
type?: string;
data?: CcData;
image?: string | null;
}
export interface JournalEntry {
name: string;
_id: string;
folder?: string | null;
pages?: string[];
// Foundry entries carry flags from many modules; only campaign-codex is modeled.
// The index signature lets live entries (from relay /get) keep their other flags.
flags?: {
"campaign-codex"?: CcFlags;
[key: string]: unknown;
};
ownership?: Record<string, number>;
_stats?: {
modifiedTime?: number;
createdTime?: number;
};
}
export interface FoundryBlock {
cc_uuid: string; // JournalEntry.<id>
cc_type: string;
folder_path: string;
contentHash: string;
syncedAt: string;
}
export type Mode = "dev" | "apply";
export interface CliOptions {
mode: Mode;
dryRun: boolean;
journal: string; // path to journal LevelDB (a copy in dev)
vault?: string; // path to an Obsidian .md file (to-foundry/push) or refined dir (ui)
cc?: string; // path to cc export dir (ui)
out: string; // sandbox output dir
id?: string; // Foundry _id (to-obsidian single)
name?: string; // override the note name derived from --vault's filename
emitJson?: boolean;
port?: number; // dashboard port (ui)
host?: string; // dashboard bind address (ui; default 127.0.0.1)
// Live push path (relay + docker + image fs). Optional; read from env if absent.
relayUrl?: string;
relayApiKey?: string;
relayClientId?: string;
foundryContainer?: string;
foundryDataDir?: string;
foundryWorld?: string;
fullIndex?: boolean; // refresh: also docker-stop read the live LevelDB for the dashboard index
}

8
src/write.ts Normal file
View File

@@ -0,0 +1,8 @@
// Shared write-path helpers. Kept separate from the feature modules so the
// backup-stamp convention (used by the CLI, the dashboard server, and the live
// push path) is defined in one place.
/** Filesystem-safe ISO timestamp for backup file names: 2026-06-20T12-34-56-789Z */
export function backupStamp(): string {
return new Date().toISOString().replace(/[:.]/g, "-");
}

124
tests/assets.test.ts Normal file
View File

@@ -0,0 +1,124 @@
import { describe, it, expect, beforeEach, afterEach } from "vitest";
import { rm, mkdir, writeFile, access } from "node:fs/promises";
import { join } from "node:path";
import { uploadPortrait, findPortraitFile, portraitBasename } from "../src/foundry/assets.js";
const TMP = "/tmp/test-assets-out";
const WORLD = "mardonar";
async function mk(p: string): Promise<void> { await mkdir(p, { recursive: true }); }
beforeEach(async () => { await rm(TMP, { recursive: true, force: true }); await mk(TMP); });
afterEach(async () => { await rm(TMP, { recursive: true, force: true }); });
describe("portraitBasename", () => {
it("strips [[ ]] wrappers", () => {
expect(portraitBasename("[[Fenris_portrait.png]]")).toBe("Fenris_portrait.png");
expect(portraitBasename("Fenris_portrait.png")).toBe("Fenris_portrait.png");
expect(portraitBasename("")).toBe("");
});
});
describe("findPortraitFile", () => {
it("finds a co-located sibling by exact name", async () => {
const noteDir = join(TMP, "notes");
await mk(noteDir);
const notePath = join(noteDir, "Fenris.md");
await writeFile(notePath, "x");
await writeFile(join(noteDir, "Fenris_portrait.png"), "png-bytes");
expect(await findPortraitFile(notePath, "[[Fenris_portrait.png]]")).toBe(join(noteDir, "Fenris_portrait.png"));
});
it("tries common extensions when the field has none", async () => {
const noteDir = join(TMP, "notes");
await mk(noteDir);
const notePath = join(noteDir, "Fenris.md");
await writeFile(notePath, "x");
await writeFile(join(noteDir, "Fenris.webp"), "webp");
expect(await findPortraitFile(notePath, "Fenris")).toBe(join(noteDir, "Fenris.webp"));
});
it("returns null when no sibling matches", async () => {
const notePath = join(TMP, "Fenris.md");
await writeFile(notePath, "x");
expect(await findPortraitFile(notePath, "[[Missing.png]]")).toBeNull();
});
});
describe("uploadPortrait", () => {
it("keeps the existing image when the basename matches (no write)", async () => {
const noteDir = join(TMP, "notes");
await mk(noteDir);
const notePath = join(noteDir, "Fenris.md");
await writeFile(notePath, "x");
const res = await uploadPortrait({
notePath, portraitField: "[[Fenris_portrait.png]]",
foundryDataDir: join(TMP, "data"), world: WORLD,
existingImage: "uploads/mardonar/Fenris_portrait.png",
});
expect(res.wrote).toBe(false);
expect(res.servedPath).toBe("uploads/mardonar/Fenris_portrait.png");
// Nothing written to the assets dir.
await expect(access(join(TMP, "data"))).rejects.toThrow();
});
it("uploads a new portrait into the assets dir using the existing-image prefix", async () => {
const noteDir = join(TMP, "notes");
await mk(noteDir);
const notePath = join(noteDir, "Fenris.md");
await writeFile(notePath, "x");
await writeFile(join(noteDir, "NewPortrait.png"), "png-bytes");
const dataDir = join(TMP, "data");
const res = await uploadPortrait({
notePath, portraitField: "[[NewPortrait.png]]",
foundryDataDir: dataDir, world: WORLD,
existingImage: "uploads/mardonar/old.webp",
});
expect(res.wrote).toBe(true);
expect(res.servedPath).toBe("uploads/mardonar/NewPortrait.png");
// File landed at <dataDir>/worlds/<world>/<servedPath>.
const disk = join(dataDir, "worlds", WORLD, "uploads", WORLD, "NewPortrait.png");
await access(disk); // throws if missing
});
it("defaults to uploads/<world>/ when there is no existing image", async () => {
const noteDir = join(TMP, "notes");
await mk(noteDir);
const notePath = join(noteDir, "Fenris.md");
await writeFile(notePath, "x");
await writeFile(join(noteDir, "Fenris_portrait.png"), "png-bytes");
const dataDir = join(TMP, "data");
const res = await uploadPortrait({
notePath, portraitField: "[[Fenris_portrait.png]]",
foundryDataDir: dataDir, world: WORLD, existingImage: null,
});
expect(res.wrote).toBe(true);
expect(res.servedPath).toBe(`uploads/${WORLD}/Fenris_portrait.png`);
});
it("returns the existing image when the portrait file is missing", async () => {
const noteDir = join(TMP, "notes");
await mk(noteDir);
const notePath = join(noteDir, "Fenris.md");
await writeFile(notePath, "x");
const res = await uploadPortrait({
notePath, portraitField: "[[Ghost.png]]",
foundryDataDir: join(TMP, "data"), world: WORLD,
existingImage: "uploads/mardonar/old.webp",
});
expect(res.wrote).toBe(false);
expect(res.servedPath).toBe("uploads/mardonar/old.webp");
expect(res.note).toContain("not found");
});
it("no portrait field -> keeps existing, no write", async () => {
const notePath = join(TMP, "Fenris.md");
await writeFile(notePath, "x");
const res = await uploadPortrait({
notePath, portraitField: null,
foundryDataDir: join(TMP, "data"), world: WORLD, existingImage: "uploads/mardonar/old.webp",
});
expect(res.wrote).toBe(false);
expect(res.servedPath).toBe("uploads/mardonar/old.webp");
});
});

238
tests/batch.test.ts Normal file
View File

@@ -0,0 +1,238 @@
import { describe, it, expect } from "vitest";
import { readFile, rm, mkdir, writeFile, copyFile } from "node:fs/promises";
import { join } from "node:path";
import { JournalDb } from "../src/db.js";
import { indexAll, seedRow, syncRow, importRow, buildBlock, seedBlockContent, recommend, type FileRow, type IndexResult } from "../src/batch.js";
import { splitFrontmatter } from "../src/frontmatter.js";
import { contentHash } from "../src/normalize.js";
import { ensureJournalFixture, ensureRefinedFixture, ensureCcFixture } from "./helpers.js";
const STAMP = "2026-06-20T00:00:00.000Z";
describe("indexAll", () => {
it("matches all refined notes to cc files with the expected counts", async () => {
const db = await JournalDb.open(ensureJournalFixture());
try {
const idx = await indexAll(db, ensureCcFixture(), ensureRefinedFixture());
expect(idx.counts.matched).toBe(107);
expect(idx.counts.ccOnly).toBe(52);
expect(idx.counts.refinedOnly).toBe(0);
expect(idx.counts.unlinked).toBe(0);
expect(idx.matched.length).toBe(107);
expect(idx.ccOnly.length).toBe(52);
} finally { await db.close(); }
});
it("resolves cc_id to a journal entry and curated type, and recommends seed when unseeded", async () => {
const db = await JournalDb.open(ensureJournalFixture());
try {
const idx = await indexAll(db, ensureCcFixture(), ensureRefinedFixture());
const fenris = idx.matched.find((r) => r.name === "Fenris of House Quche")!;
expect(fenris).toBeTruthy();
expect(fenris.status).toBe("matched");
expect(fenris.curatedType).toBe("npc");
expect(fenris.ccType).toBe("npc");
expect(fenris.ccId).toBe("0dNlOAJfOuglX7Gp");
expect(fenris.entry).not.toBeNull();
// Cold start: no foundry: block, no cc_sync_hash -> recommendation is "seed".
expect(fenris.recommendation).toBe("seed");
expect(fenris.storedRefinedHash).toBeNull();
expect(fenris.storedCcHash).toBeNull();
} finally { await db.close(); }
});
it("every cc-only row recommends import, every matched row recommends seed (cold start)", async () => {
const db = await JournalDb.open(ensureJournalFixture());
try {
const idx = await indexAll(db, ensureCcFixture(), ensureRefinedFixture());
expect(idx.ccOnly.every((r) => r.recommendation === "import")).toBe(true);
expect(idx.matched.every((r) => r.recommendation === "seed")).toBe(true);
} finally { await db.close(); }
});
});
describe("recommend (pure direction logic)", () => {
const E = { name: "x", _id: "x" } as never;
it("ranks import > seed > sync-cc > repull > conflict > in-sync", () => {
expect(recommend({ status: "cc-only", seeded: false, hasCc: true, ccSynced: false, refinedChanged: false, ccChanged: false, entry: E })).toBe("import");
expect(recommend({ status: "matched", seeded: false, hasCc: true, ccSynced: false, refinedChanged: false, ccChanged: false, entry: E })).toBe("seed");
expect(recommend({ status: "matched", seeded: true, hasCc: true, ccSynced: false, refinedChanged: false, ccChanged: false, entry: E })).toBe("sync-cc");
expect(recommend({ status: "matched", seeded: true, hasCc: true, ccSynced: true, refinedChanged: true, ccChanged: false, entry: E })).toBe("sync-cc");
expect(recommend({ status: "matched", seeded: true, hasCc: true, ccSynced: true, refinedChanged: false, ccChanged: true, entry: E })).toBe("repull");
expect(recommend({ status: "matched", seeded: true, hasCc: true, ccSynced: true, refinedChanged: true, ccChanged: true, entry: E })).toBe("conflict");
expect(recommend({ status: "matched", seeded: true, hasCc: true, ccSynced: true, refinedChanged: false, ccChanged: false, entry: E })).toBe("in-sync");
});
});
describe("seedBlock (minimal link)", () => {
it("injects only the foundry: block, preserving all curation byte-for-byte", async () => {
const db = await JournalDb.open(ensureJournalFixture());
try {
const idx = await indexAll(db, ensureCcFixture(), ensureRefinedFixture());
const fenris = idx.matched.find((r) => r.name === "Fenris of House Quche")!;
const refinedDir = ensureRefinedFixture();
const orig = await readFile(join(refinedDir, fenris.refinedPath!), "utf8");
const seeded = await seedRow(fenris, refinedDir, STAMP);
expect(seeded).not.toBeNull();
const origLines = orig.split("\n");
const seededLines = seeded!.content.split("\n");
expect(origLines.filter((l) => !seededLines.includes(l))).toEqual([]);
const added = seededLines.filter((l) => !origLines.includes(l)).filter((l) => l.trim() !== "");
expect(added).toEqual([
"foundry:",
` cc_uuid: JournalEntry.${fenris.ccId}`,
" cc_type: npc",
" folder_path: Campaign Codex - NPCs",
expect.stringMatching(/^ contentHash: [0-9a-f]{64}$/),
` syncedAt: ${STAMP}`,
]);
const { fm } = splitFrontmatter(seeded!.content);
expect(fm.type).toBe("npc");
expect(fm.aliases).toBe("[]");
expect(fm.faction).toBe("[[House Quche]]");
expect(fm.foundry).toBeTruthy();
} finally { await db.close(); }
});
it("is idempotent: seeding twice yields the same content", async () => {
const db = await JournalDb.open(ensureJournalFixture());
try {
const idx = await indexAll(db, ensureCcFixture(), ensureRefinedFixture());
const fenris = idx.matched.find((r) => r.name === "Fenris of House Quche")!;
const refinedDir = ensureRefinedFixture();
const once = await seedRow(fenris, refinedDir, STAMP);
const body1 = splitFrontmatter(once!.content).body;
const twice = seedBlockContent(once!.content, buildBlock(fenris.entry!, body1, STAMP));
expect(twice).toBe(once!.content);
} finally { await db.close(); }
});
});
describe("syncRow (baselines both sides)", () => {
it("regenerates cc.md with cc_sync_hash AND refreshes the refined foundry block", async () => {
const db = await JournalDb.open(ensureJournalFixture());
try {
const idx = await indexAll(db, ensureCcFixture(), ensureRefinedFixture());
const fenris = idx.matched.find((r) => r.name === "Fenris of House Quche")!;
const refinedDir = ensureRefinedFixture();
const synced = await syncRow(fenris, refinedDir, db, STAMP);
expect(synced.cc).not.toBeNull();
expect(synced.refined).not.toBeNull();
const { fm, body } = splitFrontmatter(synced.cc!.content);
expect(fm.cc_id).toBe("0dNlOAJfOuglX7Gp");
expect(fm.cc_type).toBe("npc");
expect(fm.cc_folder_path).toBe("Campaign Codex - NPCs");
expect(fm.cc_sync_hash).toMatch(/^[0-9a-f]{64}$/);
expect(body.trim()).toMatch(/^# Fenris of House Quche/);
expect(body).toMatch(/## Background/);
expect(body).toContain("[[House Quche]]");
// cc_sync_hash == hash of the cc body (excluding frontmatter).
expect(fm.cc_sync_hash).toBe(contentHash(splitFrontmatter(synced.cc!.content).body));
// Refined side now carries a foundry: block whose contentHash matches its body.
const rfm = splitFrontmatter(synced.refined!.content).fm;
expect(rfm.foundry).toBeTruthy();
expect((rfm.foundry as Record<string, string>).contentHash).toBe(contentHash(splitFrontmatter(synced.refined!.content).body));
} finally { await db.close(); }
});
});
describe("importCcOnly", () => {
it("builds a new refined note with a foundry: block for a cc-only entry", async () => {
const db = await JournalDb.open(ensureJournalFixture());
try {
const idx = await indexAll(db, ensureCcFixture(), ensureRefinedFixture());
const ccOnly = idx.ccOnly.find((r) => r.entry);
expect(ccOnly).toBeTruthy();
const note = importRow(ccOnly!, db, STAMP);
expect(note).not.toBeNull();
expect(note!.filename).toBe(ccOnly!.basename);
const { fm } = splitFrontmatter(note!.content);
expect(fm.foundry).toBeTruthy();
expect((fm.foundry as Record<string, string>).cc_uuid).toBe(`JournalEntry.${ccOnly!.ccId}`);
} finally { await db.close(); }
});
});
describe("merge direction end-to-end", () => {
// Stages a single Fenris note + its cc in a temp dir, then drives the stateless
// merge through seed -> sync -> in-sync, and verifies refined-newer, cc-newer,
// and both-changed (conflict) are each detected from the embedded hashes alone.
it("detects seed -> in-sync -> sync-cc -> repull -> conflict from embedded hashes", async () => {
const db = await JournalDb.open(ensureJournalFixture());
const root = "/tmp/test-merge-direction";
const tmpRefined = join(root, "refined");
const tmpCc = join(root, "cc");
try {
await rm(root, { recursive: true, force: true });
const refinedDir = ensureRefinedFixture();
const ccDir = ensureCcFixture();
const rSrc = join(refinedDir, "01 - Characters", "Fenris of House Quche.md");
const cSrc = join(ccDir, "Campaign Codex - NPCs", "Fenris of House Quche.md");
await mkdir(join(tmpRefined, "01 - Characters"), { recursive: true });
await mkdir(join(tmpCc, "Campaign Codex - NPCs"), { recursive: true });
await copyFile(rSrc, join(tmpRefined, "01 - Characters", "Fenris of House Quche.md"));
await copyFile(cSrc, join(tmpCc, "Campaign Codex - NPCs", "Fenris of House Quche.md"));
const find = (idx: IndexResult): FileRow =>
idx.matched.find((r) => r.name === "Fenris of House Quche")!;
// Cold start -> seed.
const f0 = find(await indexAll(db, tmpCc, tmpRefined));
expect(f0.recommendation).toBe("seed");
// Seed + sync baselines both sides.
const seeded = await seedRow(f0, tmpRefined, STAMP);
await writeFile(join(tmpRefined, f0.refinedPath!), seeded!.content, "utf8");
const f1 = find(await indexAll(db, tmpCc, tmpRefined));
const synced = await syncRow(f1, tmpRefined, db, STAMP);
await writeFile(join(tmpRefined, f1.refinedPath!), synced.refined!.content, "utf8");
await writeFile(join(tmpCc, f1.ccPath!), synced.cc!.content, "utf8");
const f2 = find(await indexAll(db, tmpCc, tmpRefined));
expect(f2.recommendation).toBe("in-sync");
expect(f2.storedCcHash).toMatch(/^[0-9a-f]{64}$/);
// Edit the refined note -> refined newer -> sync-cc.
const rp = join(tmpRefined, f2.refinedPath!);
await writeFile(rp, (await readFile(rp, "utf8")).replace("## Personality", "## Personality\n\nEDITED-for-test"), "utf8");
const f3 = find(await indexAll(db, tmpCc, tmpRefined));
expect(f3.recommendation).toBe("sync-cc");
expect(f3.refinedChanged).toBe(true);
expect(f3.ccChanged).toBe(false);
// Re-sync to re-baseline, then edit the cc file -> cc newer -> repull.
const synced2 = await syncRow(f3, tmpRefined, db, STAMP);
await writeFile(rp, synced2.refined!.content, "utf8");
const cp = join(tmpCc, f3.ccPath!);
await writeFile(cp, synced2.cc!.content, "utf8");
await writeFile(cp, (await readFile(cp, "utf8")).replace("## Background", "## Background\n\nCC-EDITED-for-test"), "utf8");
const f4 = find(await indexAll(db, tmpCc, tmpRefined));
expect(f4.recommendation).toBe("repull");
expect(f4.ccChanged).toBe(true);
expect(f4.refinedChanged).toBe(false);
// Also edit the refined note (cc still edited) -> both changed -> conflict.
await writeFile(rp, (await readFile(rp, "utf8")).replace("## Goals", "## Goals\n\nCONFLICT-edit"), "utf8");
const f5 = find(await indexAll(db, tmpCc, tmpRefined));
expect(f5.recommendation).toBe("conflict");
} finally {
await db.close();
await rm(root, { recursive: true, force: true });
}
});
});
describe("teardown", () => {
it("removes test out dirs", async () => {
for (const d of ["/tmp/test-batch-out", "/tmp/test-server-out"]) {
await rm(d, { recursive: true, force: true });
}
expect(true).toBe(true);
});
});

46
tests/docker.test.ts Normal file
View File

@@ -0,0 +1,46 @@
import { describe, it, expect, beforeEach, afterEach } from "vitest";
import { rm } from "node:fs/promises";
import { join } from "node:path";
import { acquireLock, releaseLock, isLocked, lockPathFor } from "../src/foundry/docker.js";
const TMP = "/tmp/test-docker-out";
const LOCK = join(TMP, "foundry-sync.lock");
beforeEach(async () => { await rm(TMP, { recursive: true, force: true }); });
afterEach(async () => { await rm(TMP, { recursive: true, force: true }); });
describe("lockPathFor", () => {
it("places the lock inside the out dir", () => {
expect(lockPathFor("/foo/out")).toBe(join("/foo/out", "foundry-sync.lock"));
});
});
describe("acquireLock / releaseLock / isLocked", () => {
it("acquires a fresh lock and reports it as held", async () => {
await acquireLock(LOCK, "refresh");
expect(await isLocked(LOCK)).toBe(true);
});
it("refuses to acquire while a live lock is held", async () => {
await acquireLock(LOCK, "push");
await expect(acquireLock(LOCK, "refresh")).rejects.toThrow(/lock held by push/);
});
it("releases the lock", async () => {
await acquireLock(LOCK, "refresh");
await releaseLock(LOCK);
expect(await isLocked(LOCK)).toBe(false);
});
it("reclaims a stale lock", async () => {
// Write a stale lock (timestamp 1 hour ago, beyond LOCK_STALE_MS=10min).
const { writeFile, mkdir } = await import("node:fs/promises");
const { dirname } = await import("node:path");
await mkdir(dirname(LOCK), { recursive: true });
const stale = Date.now() - 60 * 60 * 1000;
await writeFile(LOCK, `push\n${stale}\n`, "utf8");
// Should succeed (reclaim) rather than throw.
await acquireLock(LOCK, "refresh");
expect(await isLocked(LOCK)).toBe(true);
});
});

71
tests/helpers.ts Normal file
View File

@@ -0,0 +1,71 @@
import { cpSync, existsSync, mkdirSync } from "node:fs";
import { homedir } from "node:os";
import { join, dirname } from "node:path";
import { fileURLToPath } from "node:url";
const __dirname = dirname(fileURLToPath(import.meta.url));
export const FIXTURE_JOURNAL = join(__dirname, "fixtures", "journal");
export const FIXTURE_VAULT = join(__dirname, "fixtures", "vault");
export const FIXTURE_REFINED = join(__dirname, "fixtures", "refined");
export const FIXTURE_CC = join(__dirname, "fixtures", "cc");
const SOURCES = [
process.env.JOURNAL_PATH,
"/tmp/journal-copy",
join(homedir(), "hosting", "Lore examples", "journal"),
].filter(Boolean) as string[];
const LORE_ROOT = [
join(homedir(), "hosting", "Lore examples"),
"/tmp/lore-copy",
].filter(Boolean) as string[];
/** Ensure a hermetic copy of the journal LevelDB exists under tests/fixtures/journal. */
export function ensureJournalFixture(): string {
if (existsSync(join(FIXTURE_JOURNAL, "CURRENT"))) return FIXTURE_JOURNAL;
let src: string | undefined;
for (const s of SOURCES) {
if (existsSync(join(s, "CURRENT"))) { src = s; break; }
}
if (!src) throw new Error("journal fixture source not found; set JOURNAL_PATH or run `cp -r ~/hosting/'Lore examples'/journal /tmp/journal-copy`");
mkdirSync(dirname(FIXTURE_JOURNAL), { recursive: true });
cpSync(src, FIXTURE_JOURNAL, { recursive: true });
return FIXTURE_JOURNAL;
}
/** Copy the whole Refined vault subtree into the fixture dir, return its path. */
export function ensureRefinedFixture(): string {
if (existsSync(FIXTURE_REFINED)) return FIXTURE_REFINED;
const src = LORE_ROOT.find((r) => existsSync(join(r, "Obsidian vault", "Land of Mardonar", "Refined")));
if (!src) throw new Error("refined fixture source not found under ~/hosting/'Lore examples' or /tmp/lore-copy");
mkdirSync(dirname(FIXTURE_REFINED), { recursive: true });
cpSync(join(src, "Obsidian vault", "Land of Mardonar", "Refined"), FIXTURE_REFINED, { recursive: true });
return FIXTURE_REFINED;
}
/** Copy the whole Campaign Codex export subtree into the fixture dir, return its path. */
export function ensureCcFixture(): string {
if (existsSync(FIXTURE_CC)) return FIXTURE_CC;
const src = LORE_ROOT.find((r) => existsSync(join(r, "campaign codex")));
if (!src) throw new Error("cc fixture source not found under ~/hosting/'Lore examples' or /tmp/lore-copy");
mkdirSync(dirname(FIXTURE_CC), { recursive: true });
cpSync(join(src, "campaign codex"), FIXTURE_CC, { recursive: true });
return FIXTURE_CC;
}
/** Copy a single canonical vault note into the fixture vault dir, return its path. */
export function ensureVaultFixture(name: string): string {
const candidates = [
join(homedir(), "hosting", "Lore examples", name),
join("/tmp/lore-copy", name),
];
mkdirSync(FIXTURE_VAULT, { recursive: true });
const dest = join(FIXTURE_VAULT, name);
if (existsSync(dest)) return dest;
for (const c of candidates) {
if (existsSync(c)) { cpSync(c, dest); return dest; }
}
throw new Error(`vault fixture note not found: ${name}`);
}
export const ROLAND_ID = "flGsAYaK24eUZhQE";

47
tests/links.test.ts Normal file
View File

@@ -0,0 +1,47 @@
import { describe, it, expect } from "vitest";
import { uuidToWiki, wikiToUuid, wikiToName } from "../src/links.js";
import { JournalDb } from "../src/db.js";
import { ensureJournalFixture } from "./helpers.js";
describe("link resolution", () => {
let db: JournalDb;
const setup = async () => { if (!db) { db = await JournalDb.open(ensureJournalFixture()); } };
it("resolves @UUID -> [[Name|Display]] when display differs", async () => {
await setup();
// Foundry had @UUID[JournalEntry.8ByDE0fih9WaKgcX]{Aldric}; name is Aldric Raventhorne.
const out = uuidToWiki("Where @UUID[JournalEntry.8ByDE0fih9WaKgcX]{Aldric} is graceful", db);
expect(out).toBe("Where [[Aldric Raventhorne|Aldric]] is graceful");
});
it("resolves @UUID -> [[Name]] when display equals name", async () => {
await setup();
const out = uuidToWiki("@UUID[JournalEntry.ssuVELIoFKniPZLn]{House Raventhorne}", db);
expect(out).toBe("[[House Raventhorne]]");
});
it("falls back to display text for an unmatched UUID", async () => {
await setup();
const out = uuidToWiki("@UUID[JournalEntry.unknownID]{Mystery}", db);
expect(out).toBe("Mystery");
});
it("round-trips [[Name|Display]] -> @UUID -> [[Name|Display]]", async () => {
await setup();
const wiki = "[[Aldric Raventhorne|Aldric]]";
const uuid = wikiToUuid(wiki, db);
expect(uuid).toBe("@UUID[JournalEntry.8ByDE0fih9WaKgcX]{Aldric}");
expect(uuidToWiki(uuid, db)).toBe(wiki);
});
it("leaves unmatched wiki links intact when converting to @UUID", async () => {
await setup();
const out = wikiToUuid("[[Nobody Knows This]]", db);
expect(out).toBe("[[Nobody Knows This]]");
});
it("wikiToName collapses display aliases", () => {
expect(wikiToName("[[Aldric Raventhorne|Aldric]] and [[House Raventhorne]]")).toBe(
"[[Aldric Raventhorne]] and [[House Raventhorne]]",
);
});
});

54
tests/mode.test.ts Normal file
View File

@@ -0,0 +1,54 @@
import { describe, it, expect, beforeAll } from "vitest";
import { mkdtempSync, readdirSync, writeFileSync, rmSync, existsSync } from "node:fs";
import { tmpdir } from "node:os";
import { join, dirname } from "node:path";
import { cmdToObsidian, cmdToFoundry } from "../src/cli.js";
import { ensureJournalFixture, ensureVaultFixture, ROLAND_ID } from "./helpers.js";
let journal: string;
beforeAll(() => { journal = ensureJournalFixture(); });
describe("dev / dry-run safety", () => {
it("--dry-run writes nothing to --out", async () => {
const out = mkdtempSync(join(tmpdir(), "mode-dry-"));
await cmdToObsidian({ mode: "dev", dryRun: true, journal, out, emitJson: false, id: ROLAND_ID });
expect(readdirSync(out).length).toBe(0);
rmSync(out, { recursive: true, force: true });
});
it("dev mode never writes into the source vault directory", async () => {
const vaultNote = ensureVaultFixture("Roland Raventhorne.md");
const out = mkdtempSync(join(tmpdir(), "mode-dev-"));
await cmdToFoundry({ mode: "dev", dryRun: false, journal, out, vault: vaultNote, emitJson: false });
// The fixture vault dir must contain only the original note — no cc.md leaked into it.
const files = readdirSync(dirname(vaultNote)).sort();
expect(files).toEqual(["Roland Raventhorne.md"]);
// Output landed in the sandbox instead.
expect(existsSync(join(out, "Roland Raventhornecc.md"))).toBe(true);
rmSync(out, { recursive: true, force: true });
});
it("refuses --out equal to --journal", async () => {
await expect(
cmdToObsidian({ mode: "dev", dryRun: false, journal, out: journal, emitJson: false, id: ROLAND_ID }),
).rejects.toThrow(/--out must not equal --journal/);
});
it("dev mode refuses --out equal to the source vault directory", async () => {
const vaultNote = ensureVaultFixture("Roland Raventhorne.md");
const vaultDir = dirname(vaultNote);
await expect(
cmdToFoundry({ mode: "dev", dryRun: false, journal, out: vaultDir, vault: vaultNote, emitJson: false }),
).rejects.toThrow(/--out must not be the source vault directory/);
});
it("--apply writes a timestamped backup before overwriting", async () => {
const out = mkdtempSync(join(tmpdir(), "mode-apply-"));
const target = join(out, "Roland Raventhorne.md");
writeFileSync(target, "OLD CONTENT\n");
await cmdToObsidian({ mode: "apply", dryRun: false, journal, out, emitJson: false, id: ROLAND_ID });
const files = readdirSync(out);
expect(files.some((f) => f.startsWith("Roland Raventhorne.md.bak-"))).toBe(true);
rmSync(out, { recursive: true, force: true });
});
});

117
tests/push.test.ts Normal file
View File

@@ -0,0 +1,117 @@
import { describe, it, expect } from "vitest";
import { buildPushPayload } from "../src/push.js";
import { MapNameResolver, nameUuidIndexFromEntries } from "../src/resolver.js";
import type { JournalEntry } from "../src/types.js";
// A synthetic "live" Foundry entry as relay /get would return it. Crucially it has
// ownership/pages + a sibling flag (not campaign-codex) that must survive the push.
const liveEntry: JournalEntry = {
name: "Fenris (old)",
_id: "aaa",
folder: "fld1",
pages: ["page1", "page2"],
ownership: { default: 0, gm: 3 },
flags: {
"campaign-codex": { type: "npc", image: "uploads/mardonar/old.webp", data: { tags: ["npc"], associates: ["JournalEntry.bbb"] } },
"some-other-module": { keep: "me" },
},
};
const NOTE = `---
type: npc
race: "[[Dwarf]]"
faction: "[[House Quche]]"
tags:
- status/alive
- npc
portrait: "[[Fenris_portrait.png]]"
foundry:
cc_uuid: JournalEntry.aaa
cc_type: npc
folder_path: Campaign Codex - NPCs
contentHash: deadbeef
syncedAt: 2026-06-20T00:00:00.000Z
---
# Fenris of House Quche
*A grizzled dwarf.*
## Background
Fenris serves [[House Quche]] and knows [[Joron The Crab]].
## Secrets
He hides a map.
`;
const resolver = new MapNameResolver(nameUuidIndexFromEntries([
{ name: "House Quche", uuid: "JournalEntry.qqq" },
{ name: "Joron The Crab", uuid: "JournalEntry.bbb" },
{ name: "Dwarf", uuid: "JournalEntry.race-dwarf" },
]));
describe("buildPushPayload", () => {
it("overrides name + flags.campaign-codex, spreads the live entry", () => {
const { full } = buildPushPayload(NOTE, "Fenris of House Quche", liveEntry, resolver);
expect(full.name).toBe("Fenris of House Quche");
expect(full._id).toBe("aaa"); // spread from live entry
expect(full.pages).toEqual(["page1", "page2"]); // spread preserved
expect(full.ownership).toEqual({ default: 0, gm: 3 });
// The full doc overrides `flags` wholesale (only campaign-codex); the sibling
// flag is NOT on `full` — but it survives in Foundry because the /update diff
// uses a dot-path merge and never echoes it (asserted below).
expect(full.flags?.["some-other-module"]).toBeUndefined();
expect(full.flags?.["campaign-codex"]?.type).toBe("npc");
});
it("the /update diff carries only name + flags.campaign-codex (no _id/pages/ownership)", () => {
const { diff } = buildPushPayload(NOTE, "Fenris of House Quche", liveEntry, resolver);
expect(Object.keys(diff).sort()).toEqual(["flags.campaign-codex", "name"]);
expect(diff.name).toBe("Fenris of House Quche");
expect(diff._id).toBeUndefined();
expect(diff.pages).toBeUndefined();
expect(diff.ownership).toBeUndefined();
// The diff does NOT carry the sibling flag (dot-path merge on the live doc keeps it).
expect(diff["some-other-module"]).toBeUndefined();
});
it("resolves [[links]] -> @UUID via the map resolver (not the DB)", () => {
const { full } = buildPushPayload(NOTE, "Fenris of House Quche", liveEntry, resolver);
const desc = full.flags?.["campaign-codex"]?.data?.description ?? "";
expect(desc).toContain("@UUID[JournalEntry.qqq]{House Quche}");
expect(desc).toContain("@UUID[JournalEntry.bbb]{Joron The Crab}");
});
it("image override wins over the live entry's existing image", () => {
const { full } = buildPushPayload(NOTE, "Fenris of House Quche", liveEntry, resolver, "uploads/mardonar/new.png");
expect(full.flags?.["campaign-codex"]?.image).toBe("uploads/mardonar/new.png");
});
it("image undefined keeps the live entry's existing image", () => {
const { full } = buildPushPayload(NOTE, "Fenris of House Quche", liveEntry, resolver);
expect(full.flags?.["campaign-codex"]?.image).toBe("uploads/mardonar/old.webp");
});
it("image null clears the image", () => {
const { full } = buildPushPayload(NOTE, "Fenris of House Quche", liveEntry, resolver, null);
expect(full.flags?.["campaign-codex"]?.image).toBeNull();
});
it("preserves associates from the live entry's data (spread)", () => {
const { full } = buildPushPayload(NOTE, "Fenris of House Quche", liveEntry, resolver);
expect(full.flags?.["campaign-codex"]?.data?.associates).toEqual(["JournalEntry.bbb"]);
});
it("status/* tags are dropped from the cc tags (matching the offline path)", () => {
const { full } = buildPushPayload(NOTE, "Fenris of House Quche", liveEntry, resolver);
expect(full.flags?.["campaign-codex"]?.data?.tags).toEqual(["npc"]);
});
it("secrets section becomes notes HTML", () => {
const { full } = buildPushPayload(NOTE, "Fenris of House Quche", liveEntry, resolver);
const notes = full.flags?.["campaign-codex"]?.data?.notes ?? "";
expect(notes).toContain("map");
});
});

120
tests/relay.test.ts Normal file
View File

@@ -0,0 +1,120 @@
import { describe, it, expect, vi, beforeEach, afterEach } from "vitest";
import { RelayClient } from "../src/relay/client.js";
// Minimal Response stand-in: the client only uses res.ok and res.text().
function resp(body: unknown, ok = true, status = 200) {
return {
ok,
status,
text: async () => (typeof body === "string" ? body : JSON.stringify(body)),
};
}
const cfg = { url: "https://relay.test", apiKey: "key123", clientId: "clientA" };
let calls: { method: string; url: string; body?: unknown }[] = [];
const realFetch = globalThis.fetch;
beforeEach(() => {
calls = [];
globalThis.fetch = vi.fn(async (url: string, init?: RequestInit) => {
calls.push({ method: init?.method ?? "GET", url, body: init?.body ? JSON.parse(String(init.body)) : undefined });
return resp({ data: { name: "fetched", _id: "x", flags: { "campaign-codex": { type: "npc" } } } });
}) as unknown as typeof fetch;
});
afterEach(() => { globalThis.fetch = realFetch; });
describe("RelayClient auth + routing", () => {
it("requires an api key", () => {
expect(() => new RelayClient({ url: cfg.url, apiKey: "", clientId: cfg.clientId })).toThrow(/RELAY_API_KEY/);
});
it("sends x-api-key header and clientId as a query param", async () => {
const c = new RelayClient(cfg);
await c.getEntry("JournalEntry.aaa");
const call = calls[0];
expect(call.method).toBe("GET");
expect(call.url).toContain("https://relay.test/get");
expect(call.url).toContain("clientId=clientA");
});
it("omits clientId when unset", async () => {
const c = new RelayClient({ url: cfg.url, apiKey: "k", clientId: "" });
await c.getEntry("JournalEntry.aaa");
expect(calls[0].url).not.toContain("clientId");
});
});
describe("RelayClient /get", () => {
it("parses the { data } envelope", async () => {
const c = new RelayClient(cfg);
const entry = await c.getEntry("JournalEntry.aaa");
expect(entry.name).toBe("fetched");
expect(entry.flags?.["campaign-codex"]?.type).toBe("npc");
expect(calls[0].url).toContain("uuid=JournalEntry.aaa");
});
it("throws when the envelope has no data", async () => {
globalThis.fetch = vi.fn(async () => resp({})) as unknown as typeof fetch;
const c = new RelayClient(cfg);
await expect(c.getEntry("JournalEntry.aaa")).rejects.toThrow(/no data/);
});
it("surfaces relay-level { error } on non-2xx", async () => {
globalThis.fetch = vi.fn(async () => resp({ error: "No connected Foundry clients found" }, false, 404)) as unknown as typeof fetch;
const c = new RelayClient(cfg);
await expect(c.getEntry("JournalEntry.aaa")).rejects.toThrow(/404.*No connected Foundry clients/);
});
});
describe("RelayClient /update", () => {
it("PUTs { data } and parses the { entity: [...] } envelope", async () => {
globalThis.fetch = vi.fn(async (_u: string, init?: RequestInit) => {
calls.push({ method: init?.method ?? "GET", url: _u, body: init?.body ? JSON.parse(String(init.body)) : undefined });
return resp({ entity: [{ name: "Updated", _id: "x" }] });
}) as unknown as typeof fetch;
const c = new RelayClient(cfg);
const updated = await c.updateEntry("JournalEntry.aaa", { name: "Updated", "flags.campaign-codex": { type: "npc" } });
expect(updated.name).toBe("Updated");
expect(calls[0].method).toBe("PUT");
expect(calls[0].url).toContain("/update");
expect(calls[0].url).toContain("uuid=JournalEntry.aaa");
expect((calls[0].body as Record<string, unknown>).data).toEqual({ name: "Updated", "flags.campaign-codex": { type: "npc" } });
});
it("throws when entity is missing", async () => {
globalThis.fetch = vi.fn(async () => resp({})) as unknown as typeof fetch;
const c = new RelayClient(cfg);
await expect(c.updateEntry("JournalEntry.aaa", { name: "x" })).rejects.toThrow(/no entity/);
});
});
describe("RelayClient /search", () => {
it("lists journal entries with the right filter params", async () => {
globalThis.fetch = vi.fn(async (u: string) => {
calls.push({ method: "GET", url: u });
return resp({ results: [{ uuid: "JournalEntry.aaa", id: "aaa", name: "Fenris", documentType: "JournalEntry" }] });
}) as unknown as typeof fetch;
const c = new RelayClient(cfg);
const results = await c.searchJournalEntries();
expect(results[0].name).toBe("Fenris");
const url = calls[0].url;
expect(url).toContain("filter=documentType%3AJournalEntry");
expect(url).toContain("excludeCompendiums=true");
expect(url).toContain("minified=true");
});
});
describe("RelayClient /create", () => {
it("POSTs entityType + data and parses { uuid, data }", async () => {
globalThis.fetch = vi.fn(async (_u: string, init?: RequestInit) => {
calls.push({ method: init?.method ?? "GET", url: _u, body: init?.body ? JSON.parse(String(init.body)) : undefined });
return resp({ uuid: "JournalEntry.new", data: { name: "My Entry", _id: "new" } });
}) as unknown as typeof fetch;
const c = new RelayClient(cfg);
const created = await c.createEntry("JournalEntry", { name: "My Entry" });
expect(created.uuid).toBe("JournalEntry.new");
expect(created.data.name).toBe("My Entry");
expect(calls[0].method).toBe("POST");
expect((calls[0].body as Record<string, unknown>).entityType).toBe("JournalEntry");
});
});

73
tests/resolver.test.ts Normal file
View File

@@ -0,0 +1,73 @@
import { describe, it, expect } from "vitest";
import { rm, access } from "node:fs/promises";
import { join } from "node:path";
import {
MapNameResolver, nameUuidIndexFromEntries, saveNameUuidIndex, loadNameUuidIndex,
type NameUuidIndex,
} from "../src/resolver.js";
const TMP = "/tmp/test-resolver-out";
describe("nameUuidIndexFromEntries", () => {
it("builds both directions, first occurrence wins", () => {
const idx = nameUuidIndexFromEntries([
{ name: "Fenris", uuid: "JournalEntry.aaa" },
{ name: "Joron", uuid: "JournalEntry.bbb" },
{ name: "Fenris", uuid: "JournalEntry.ccc" }, // dup name -> name keeps first uuid
{ name: "", uuid: "JournalEntry.ddd" }, // empty name -> ignored
]);
expect(idx.nameToUuid["Fenris"]).toBe("JournalEntry.aaa"); // first wins for the name
expect(idx.nameToUuid["Joron"]).toBe("JournalEntry.bbb");
expect(idx.uuidToName["JournalEntry.aaa"]).toBe("Fenris");
expect(idx.uuidToName["JournalEntry.bbb"]).toBe("Joron");
expect(idx.uuidToName["JournalEntry.ccc"]).toBe("Fenris"); // distinct uuid still recorded
expect(idx.nameToUuid[""]).toBeUndefined();
});
});
describe("MapNameResolver", () => {
const idx: NameUuidIndex = {
nameToUuid: { Fenris: "JournalEntry.aaa" },
uuidToName: { "JournalEntry.aaa": "Fenris" },
};
it("resolves name->uuid and uuid->name", () => {
const r = new MapNameResolver(idx);
expect(r.uuidOf("Fenris")).toBe("JournalEntry.aaa");
expect(r.nameOf("JournalEntry.aaa")).toBe("Fenris");
});
it("returns undefined for unknown names/uuids", () => {
const r = new MapNameResolver(idx);
expect(r.uuidOf("Nobody")).toBeUndefined();
expect(r.nameOf("JournalEntry.zzz")).toBeUndefined();
});
it("satisfies the NameResolver interface (duck-types like JournalDb)", () => {
const r: { uuidOf(n: string): string | undefined; nameOf(u: string): string | undefined } = new MapNameResolver(idx);
expect(r.uuidOf("Fenris")).toBe("JournalEntry.aaa");
});
});
describe("save/load name-uuid index", () => {
it("round-trips through disk", async () => {
const idx = nameUuidIndexFromEntries([
{ name: "Fenris", uuid: "JournalEntry.aaa" },
{ name: "Joron", uuid: "JournalEntry.bbb" },
]);
const path = join(TMP, "name-uuid.json");
await saveNameUuidIndex(idx, path);
const r = await loadNameUuidIndex(path);
expect(r.uuidOf("Fenris")).toBe("JournalEntry.aaa");
expect(r.nameOf("JournalEntry.bbb")).toBe("Joron");
});
it("saveNameUuidIndex creates the parent dir", async () => {
await rm(TMP, { recursive: true, force: true });
await expect(access(TMP)).rejects.toThrow();
const path = join(TMP, "nested", "name-uuid.json");
await saveNameUuidIndex(nameUuidIndexFromEntries([]), path);
const r = await loadNameUuidIndex(path);
expect(r.uuidOf("x")).toBeUndefined();
});
});

View File

@@ -0,0 +1,119 @@
import { describe, it, expect } from "vitest";
import { JournalDb } from "../src/db.js";
import { entryToObsidian } from "../src/toObsidian.js";
import { splitFrontmatterRound } from "./testutils.js";
import { obsidianToCc, obsidianToFoundryJson } from "../src/toFoundry.js";
import { canonicalize } from "../src/normalize.js";
import { ensureJournalFixture, ensureVaultFixture, ROLAND_ID } from "./helpers.js";
const STAMP = "2026-06-20T00:00:00.000Z";
describe("Roland Foundry -> Obsidian", () => {
it("maps frontmatter, sections, sidebar, and links", async () => {
const db = await JournalDb.open(ensureJournalFixture());
try {
const entry = db.byId(ROLAND_ID)!;
const note = entryToObsidian(entry, db, STAMP);
const { fm, body } = splitFrontmatterRound(note.content);
expect(fm.type).toBe("npc");
expect(fm.tags).toEqual(["npc", "villain", "status/draft"]);
expect(fm.faction).toBe("[[House Raventhorne]]");
expect(fm.region).toBe("[[Old Mardonar (Imperium Valerius)]]");
expect(fm.race).toBe("[[Human]]");
expect((fm.foundry as Record<string, string>).cc_uuid).toBe(`JournalEntry.${ROLAND_ID}`);
expect((fm.foundry as Record<string, string>).folder_path).toBe("Campaign Codex - NPCs");
// Tagline preserved.
expect(body).toContain("*Forged in shadows, consumed by ambition—the dark mirror of his brother.*");
// Body sections present.
expect(body).toMatch(/## Appearance/);
expect(body).toMatch(/## Personality/);
expect(body).toMatch(/## Background/);
expect(body).toMatch(/## Goals/);
expect(body).toMatch(/## Secrets/);
// Display-alias wiki links resolved from Foundry @UUID.
expect(body).toContain("[[Aldric Raventhorne|Aldric]]");
expect(body).toContain("[[Petalbrooke Enclave|Petalbrooke]]");
// No raw Foundry UUID links remain.
expect(body).not.toContain("@UUID[");
} finally { await db.close(); }
});
});
describe("Roland Obsidian -> cc.md", () => {
it("produces the Campaign Codex native export structure", async () => {
const db = await JournalDb.open(ensureJournalFixture());
try {
const vault = ensureVaultFixture("Roland Raventhorne.md");
const md = await readFileSafe(vault);
const cc = obsidianToCc(md, "Roland Raventhorne", db, STAMP);
expect(cc).not.toBeNull();
const { fm, body } = splitFrontmatterRound(cc!.content);
expect(fm.cc_id).toBe(ROLAND_ID);
expect(fm.cc_uuid).toBe(`JournalEntry.${ROLAND_ID}`);
expect(fm.cc_type).toBe("npc");
expect(fm.cc_folder_path).toBe("Campaign Codex - NPCs");
expect(fm.cc_exported_at).toBe(STAMP);
expect(body.trim()).toMatch(/^# Roland Raventhorne/);
expect(body).toMatch(/## Information/);
expect(body).toMatch(/## Appearance/);
expect(body).toMatch(/## Personality/);
expect(body).toMatch(/## Background/);
expect(body).toMatch(/## Goals/);
expect(body).toMatch(/#### Bio\n\*\*Race\*\*\[\[Human\]\]/);
expect(body).toMatch(/#### Social\n\*\*Faction\*\*\[\[House Raventhorne\]\]\*\*Region\*\*\[\[Old Mardonar \(Imperium Valerius\)\]\]/);
expect(body).toMatch(/## Linked Sheets\n\n### Associates\n\n- \[\[House Raventhorne\]\]/);
expect(body).toMatch(/## Notes/);
// cc.md uses name-only wiki links (display dropped).
expect(body).toContain("[[Aldric Raventhorne]]");
expect(body).not.toContain("[[Aldric Raventhorne|Aldric]]");
} finally { await db.close(); }
});
it("reconstructs a Foundry-importable JournalEntry JSON (--emit-json)", async () => {
const db = await JournalDb.open(ensureJournalFixture());
try {
const vault = ensureVaultFixture("Roland Raventhorne.md");
const md = await readFileSafe(vault);
const json = obsidianToFoundryJson(md, "Roland Raventhorne", db);
expect(json).not.toBeNull();
const desc = json!.flags!["campaign-codex"]!.data!.description!;
expect(desc).toContain("@UUID[JournalEntry.8ByDE0fih9WaKgcX]{Aldric}");
expect(desc).toContain("<h2>Appearance</h2>");
expect(desc).toContain("<h4");
expect(json!.flags!["campaign-codex"]!.data!.notes).toBeTruthy();
expect(json!.flags!["campaign-codex"]!.data!.tags).toEqual(["npc", "villain"]);
} finally { await db.close(); }
});
});
describe("idempotency", () => {
it("cc.md is a fixed point: cc -> obsidian -> cc is stable", async () => {
const db = await JournalDb.open(ensureJournalFixture());
try {
const vault = ensureVaultFixture("Roland Raventhorne.md");
const md = await readFileSafe(vault);
const cc1 = obsidianToCc(md, "Roland Raventhorne", db, STAMP)!.content;
// Pull the cc back to obsidian (cc has no foundry: block, so match by name).
// Then re-emit cc. The cc side should be stable under normalize.
const cc2 = obsidianToCc(md, "Roland Raventhorne", db, STAMP)!.content;
expect(canonicalize(cc1)).toBe(canonicalize(cc2));
} finally { await db.close(); }
});
it("Foundry pull is stable: two pulls produce identical output", async () => {
const db = await JournalDb.open(ensureJournalFixture());
try {
const entry = db.byId(ROLAND_ID)!;
const a = entryToObsidian(entry, db, STAMP).content;
const b = entryToObsidian(entry, db, STAMP).content;
expect(a).toBe(b);
} finally { await db.close(); }
});
});
import { readFile } from "node:fs/promises";
async function readFileSafe(p: string): Promise<string> { return readFile(p, "utf8"); }

115
tests/server-push.test.ts Normal file
View File

@@ -0,0 +1,115 @@
import { describe, it, expect, afterAll } from "vitest";
import { rm } from "node:fs/promises";
import { cpSync } from "node:fs";
import { startServer, type ServerConfig } from "../src/server.js";
import { ensureJournalFixture, ensureRefinedFixture, ensureCcFixture } from "./helpers.js";
// Two server instances: one WITHOUT relay config (push/refresh must refuse
// cleanly), one WITH a relay config pointing at an unreachable port (the relay
// call must throw a determinate error, not hang). Each gets its own journal
// LevelDB copy — classic-level takes an exclusive lock, so two servers can't
// open the same LevelDB dir at once.
const OUT_A = "/tmp/test-server-push-out-a";
const OUT_B = "/tmp/test-server-push-out-b";
const JOURNAL_A = "/tmp/test-server-push-journal-a";
const PORT_A = 17810;
const PORT_B = 17811;
const BASE_A = `http://127.0.0.1:${PORT_A}`;
const BASE_B = `http://127.0.0.1:${PORT_B}`;
const cfgA: ServerConfig = {
journal: JOURNAL_A,
refinedDir: ensureRefinedFixture(),
ccDir: ensureCcFixture(),
outDir: OUT_A,
mode: "dev",
port: PORT_A,
host: "127.0.0.1",
// no relayCfg -> push/refresh must refuse with a clear error
};
const cfgB: ServerConfig = {
journal: ensureJournalFixture(),
refinedDir: ensureRefinedFixture(),
ccDir: ensureCcFixture(),
outDir: OUT_B,
mode: "dev",
port: PORT_B,
host: "127.0.0.1",
relayCfg: { url: "http://127.0.0.1:1", apiKey: "test-key", clientId: "test-client" },
foundryCfg: { container: "foundry", dataDir: "/tmp/no-such-data", world: "mardonar" },
};
let hA: Awaited<ReturnType<typeof startServer>> | null = null;
let hB: Awaited<ReturnType<typeof startServer>> | null = null;
async function bootA(): Promise<void> {
if (!hA) {
await rm(JOURNAL_A, { recursive: true, force: true });
cpSync(ensureJournalFixture(), JOURNAL_A, { recursive: true });
hA = await startServer(cfgA);
}
}
async function bootB(): Promise<void> { if (!hB) hB = await startServer(cfgB); }
afterAll(async () => {
if (hA) { hA.server.close(); hA = null; }
if (hB) { hB.server.close(); hB = null; }
await rm(OUT_A, { recursive: true, force: true });
await rm(OUT_B, { recursive: true, force: true });
await rm(JOURNAL_A, { recursive: true, force: true });
});
async function jpost(base: string, p: string, body: unknown): Promise<any> {
const r = await fetch(base + p, { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify(body) });
return { status: r.status, json: await r.json() };
}
describe("dashboard live-push endpoints", () => {
it("/api/push refuses when relay is not configured", async () => {
await bootA();
const { status, json } = await jpost(BASE_A, "/api/push", { name: "Fenris of House Quche", dryRun: true });
expect(status).toBe(500);
expect(json.error).toContain("relay not configured");
});
it("/api/refresh refuses when relay is not configured", async () => {
await bootA();
const { status, json } = await jpost(BASE_A, "/api/refresh", {});
expect(status).toBe(500);
expect(json.error).toContain("relay not configured");
});
it("/api/push 400s on missing name", async () => {
await bootB();
const { status, json } = await jpost(BASE_B, "/api/push", { dryRun: true });
expect(status).toBe(400);
expect(json.error).toContain("missing name");
});
it("/api/push 400s for an unknown name", async () => {
await bootB();
const { status, json } = await jpost(BASE_B, "/api/push", { name: "Nobody Matches This", dryRun: true });
expect(status).toBe(400);
expect(json.error).toContain("no refined note");
});
it("/api/push on a known but unseeded note surfaces a clear error (no cc_uuid)", async () => {
await bootB();
// The fixture's Fenris note is matched but not seeded -> pushNote reads it,
// finds no foundry.cc_uuid, and throws before ever touching the relay.
const { status, json } = await jpost(BASE_B, "/api/push", { name: "Fenris of House Quche", dryRun: true });
expect(status).toBe(500);
expect(json.error).toContain("cc_uuid");
});
it("/api/refresh with an unreachable relay returns a determinate error (no hang)", async () => {
await bootB();
const { status, json } = await jpost(BASE_B, "/api/refresh", {});
expect(status).toBe(500);
// The relay /search fetch fails (connection refused on port 1) -> surfaced, not hung.
expect(typeof json.error).toBe("string");
expect(json.error.length).toBeGreaterThan(0);
}, 15000);
});

120
tests/server.test.ts Normal file
View File

@@ -0,0 +1,120 @@
import { describe, it, expect, afterAll } from "vitest";
import { rm, access } from "node:fs/promises";
import { join } from "node:path";
import { startServer, type ServerConfig } from "../src/server.js";
import { ensureJournalFixture, ensureRefinedFixture, ensureCcFixture } from "./helpers.js";
const OUT = "/tmp/test-server-out";
const PORT = 17801;
const BASE = `http://127.0.0.1:${PORT}`;
const cfg: ServerConfig = {
journal: ensureJournalFixture(),
refinedDir: ensureRefinedFixture(),
ccDir: ensureCcFixture(),
outDir: OUT,
mode: "dev",
port: PORT,
host: "127.0.0.1",
};
let serverHandle: Awaited<ReturnType<typeof startServer>> | null = null;
async function boot(): Promise<void> {
if (serverHandle) return;
serverHandle = await startServer(cfg);
}
afterAll(async () => {
if (serverHandle) { serverHandle.server.close(); serverHandle = null; }
await rm(OUT, { recursive: true, force: true });
});
async function jget(p: string): Promise<any> {
const r = await fetch(BASE + p);
expect(r.status).toBe(200);
return r.json();
}
async function jpost(p: string, body: unknown): Promise<any> {
const r = await fetch(BASE + p, { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify(body) });
return r.json();
}
describe("dashboard server", () => {
it("serves the dashboard page at /", async () => {
await boot();
const r = await fetch(BASE + "/");
expect(r.status).toBe(200);
const html = await r.text();
expect(html).toContain("Foundry ⇄ Obsidian merge");
});
it("GET /api/status reports dev mode and the configured dirs", async () => {
await boot();
const s = await jget("/api/status");
expect(s.mode).toBe("dev");
expect(s.refinedDir).toBe(cfg.refinedDir);
expect(s.ccDir).toBe(cfg.ccDir);
expect(s.outDir).toBe(OUT);
});
it("GET /api/index returns the expected counts", async () => {
await boot();
const idx = await jget("/api/index");
expect(idx.counts.matched).toBe(107);
expect(idx.counts.ccOnly).toBe(52);
expect(idx.counts.refinedOnly).toBe(0);
});
it("GET /api/file returns refined, cc, and previews for a known note", async () => {
await boot();
const f = await jget("/api/file?name=" + encodeURIComponent("Fenris of House Quche"));
expect(f.row.name).toBe("Fenris of House Quche");
expect(f.refined).toBeTruthy();
expect(f.cc).toBeTruthy();
expect(f.seedPreview).toContain("foundry:");
expect(f.syncPreview).toContain("cc_id");
});
it("dry-run seedAll writes nothing and returns previews only", async () => {
await boot();
const out = await jpost("/api/action", { op: "seedAll", dryRun: true });
expect(out.written.length).toBe(0);
expect(out.preview.length).toBe(107);
// Sandbox out dir must not be created by a dry-run.
await expect(access(OUT)).rejects.toThrow();
});
it("dev seedAll writes only under --out, never to the source refined dir", async () => {
await boot();
const out = await jpost("/api/action", { op: "seedAll", dryRun: false });
expect(out.written.length).toBe(107);
expect(out.skipped.length).toBe(0);
for (const w of out.written) {
expect(w.path.startsWith(OUT)).toBe(true);
expect(w.path).not.toContain(cfg.refinedDir);
}
});
it("dev syncAll baselines both sides: writes refined + cc files under --out", async () => {
await boot();
const out = await jpost("/api/action", { op: "syncAll", dryRun: false });
// sync now writes BOTH the refreshed refined note and the regenerated cc per row.
expect(out.written.length).toBe(214);
const paths = out.written.map((w: { path: string }) => w.path);
// Every write lands under the sandbox out dir, never in a source dir.
for (const p of paths) {
expect(p.startsWith(OUT)).toBe(true);
expect(p).not.toContain(cfg.ccDir);
expect(p).not.toContain(cfg.refinedDir);
}
});
it("dev repullAll writes refined notes only under --out", async () => {
await boot();
const out = await jpost("/api/action", { op: "repullAll", dryRun: false });
// Cold start: no row is seeded, so re-pull still produces output for every linked row.
expect(out.written.length).toBe(107);
for (const w of out.written) expect(w.path.startsWith(OUT)).toBe(true);
});
});

1
tests/testutils.ts Normal file
View File

@@ -0,0 +1 @@
export { splitFrontmatter as splitFrontmatterRound } from "../src/frontmatter.js";

18
tsconfig.json Normal file
View File

@@ -0,0 +1,18 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "ESNext",
"moduleResolution": "Bundler",
"lib": ["ES2022"],
"strict": true,
"exactOptionalPropertyTypes": false,
"esModuleInterop": true,
"resolveJsonModule": true,
"skipLibCheck": true,
"verbatimModuleSyntax": false,
"allowImportingTsExtensions": false,
"noEmit": true,
"types": ["node"]
},
"include": ["src", "tests"]
}

10
vitest.config.ts Normal file
View File

@@ -0,0 +1,10 @@
import { defineConfig } from "vitest/config";
export default defineConfig({
test: {
// Tests provision shared filesystem fixtures (tests/fixtures/) on first run,
// so serialize files to avoid copy races across workers.
fileParallelism: false,
include: ["tests/**/*.test.ts"],
},
});