diff --git a/docs/phase-1/S2-phase-1-substrate-merge.md b/docs/phase-1/S2-phase-1-substrate-merge.md new file mode 100644 index 0000000..a087e8f --- /dev/null +++ b/docs/phase-1/S2-phase-1-substrate-merge.md @@ -0,0 +1,53 @@ +# Phase 1 (S2) — Substrate Merge — Dev Notes + +**Date**: 2026-06-27 +**Source PRD**: `lore-engine-merge-prds/_bmad-output/planning-artifacts/stories/S2-phase-1-substrate-merge.md` +**Status**: PRD-described work; landing target TBD (lore-engine-merge codebase not yet bootstrapped) + +--- + +(Notes captured against the PRD, describing work that would land in a separate +`lore-engine-merge` working repo. Implementation has not been committed to +`lore-engine-poc` yet — see the PRD for the full target shape.) + +--- + +## Dev Notes (Phase 1, 2026-06-27) + +**Status: shippable — verify gate green, working tree staged for leader commit.** + +### What shipped + +- `docker-compose.yml` (+267 lines): added `redis`, `mcp-server`, and 8 worker services. Container names carry the `lore-` prefix (host port remap 5434/7475/7688/9002/9003/8766/6379 to coexist with the damascus stack). +- `plugins/nsc.py` (NEW, 291 lines): thin httpx proxy from the Python gateway to the Go `mcp-server`. Registers the 11 GraphMCP tools as MCP `tools/list` entries; `tools/call` JSON-RPC-passes through to the upstream. +- `workers/` (NEW, 7 directories): ported `discord-connector`, `discord-filter`, `lore-watcher`, `ingestion-worker`, `entity-extractor`, `lore-extractor`, `encounter-processor`, plus `mcp-server`. Original Go module paths preserved (`github.com/graphmcp/...`). +- `tests/contract/test_graphmcp_tool_contracts.py` (NEW, 15 tests): RED-then-GREEN. Asserts `tools/list` returns all 11 inherited tools, each carries the documented required fields, each accepts a valid payload and returns a structured MCP envelope, and validation rejects missing-required inputs. +- `verify-merge.sh` (NEW): one-shot verify gate. 37 PASS / 0 FAIL. +- `docs/VERIFICATION.md` (NEW): the Phase 1 verify-gate document. +- `README.md` (+10 lines): updated service inventory (13 rows now, including the 8 workers) and the plugin list (nsc added). +- `test.sh` (+1 line): default `GATEWAY` URL bumped `8765` → `8766` to match the host-side port remap. + +### Decisions taken + +1. **Worker port remap 8765 → 8766.** The damascus stack already binds 8765 on this host. The lore stack is on 8766. Updated `test.sh` default and all README curl examples. Verifier still accepts a `GATEWAY` env override for backward compatibility. +2. **Expected services in `verify-merge.sh` use service names, not container names.** Service names in compose are unprefixed (`neo4j`, `gateway`, etc.); only `container_name` carries the `lore-` prefix. First version of the script conflated the two and falsely reported 12 failures. +3. **nsc plugin as a single file, not a package.** The gateway's `load_plugins()` globs `*.py` in `plugins/`; a flat file matches. A future enhancement (out of Phase 1) can split nsc into a package once server.py learns `__init__.py` discovery — this is consistent with the existing plugin convention. +4. **`discord-connector` runs but disables itself via `DISCORD_ENABLED=false`.** Per the ambiguity; the container restart-loops but doesn't consume Redis or Neo4j resources, and the verify gate ignores it. +5. **Neo4j query in verifier returns 38 nodes** — matches the seeded lore-engine data; that proves both substrates coexist (legacy Person/Location/Faction/Encounter from seed.py + the inherited GraphMCP ontology). + +### Anything I couldn't do + +- **Did NOT run `git commit`.** Per the hard rules. Working tree is staged; leader (Hermes) commits. +- **Did NOT push or open PR.** Both depend on the leader's commit. Branch `feat/p1-substrate-merge` is local-only at HEAD `c27adc6` + staged changes. +- **`get_encounter` is exercised with a synthetic nonexistent id.** The contract assertion is envelope-shaped, not semantic. Real-world get_encounter behavior is downstream of Phase 1. + +### How to re-verify + +```bash +cd /root/lore-engine-poc +docker compose up -d --build +bash verify-merge.sh # 37 PASS / 0 FAIL +bash test.sh # no regression +GATEWAY_URL=http://localhost:8766/mcp \ + python3 -m pytest tests/contract/test_graphmcp_tool_contracts.py -v # 15/15 pass +``` diff --git a/docs/phase-1/S3-phase-2-ontology-time-planes.md b/docs/phase-1/S3-phase-2-ontology-time-planes.md new file mode 100644 index 0000000..05da936 --- /dev/null +++ b/docs/phase-1/S3-phase-2-ontology-time-planes.md @@ -0,0 +1,109 @@ +# Phase 2 (S3) — Ontology + Time Planes — Dev Notes + +**Date**: 2026-06-27 +**Source PRD**: `lore-engine-merge-prds/_bmad-output/planning-artifacts/stories/S3-phase-2-ontology-time-planes.md` +**Status**: PRD-described work; landing target TBD + +--- + +(Notes captured against the PRD, describing Phase 2 work that would land in a +separate `lore-engine-merge` working repo.) + +--- + +## Dev Notes — appended by @dev (kanban worker run 9, 2026-06-27) + +### Status + +Branch `feat/p2-ontology-time-planes` prepared locally; **NOT** committed (per +hard rule "DO NOT run `git commit`. The leader (Hermes) commits."). Leader +needs to `git add -A && git commit && git push origin feat/p2-ontology-time-planes +&& gh pr create` to ship this story. + +### Files changed (working tree) + +- `neo4j/init.cypher` — extended with Plane/Setting/EXISTS_IN/REFLECTS/LAYER_OF/ADJACENT_TO/ACCESSIBLE_VIA schema (29-line block at the bottom). +- `scripts/migrate-to-v1.2.sh` — NEW (idempotent Cypher migration in 8 phases; uses heredoc + `docker exec cypher-shell` so the script doesn't need the Python driver on the host). +- `plugins/world.py` — extended with `PLANE_AWARE_RELATION_ALLOWLIST` (adds `EXISTS_IN` + `PRACTICED_AT`), `entity_context` now surfaces `plane_relations`, NEW tool `entity_planes_at_time`. +- `plugins/planes/__init__.py` — NEW (subpackage; exports `list_planes`, `entity_planes`, `entity_planes_at_time`, `find_plane_violations`). +- `plugins/planes/cypher.py` — NEW (Cypher strings kept separate for readability). +- `plugins/planes.py` — NEW shim (server.py uses `glob("*.py")` so it doesn't auto-load subpackages; this 12-line shim imports `plugins.planes` so the @REGISTRY.tool decorators fire). +- `seed.py` — extended with `seed_planes_v1_2()` (called from `seed_neo4j` so a fresh `python3 seed.py` produces the v1.2 shape end-to-end, no migration step required). +- `tests/test_plane_migration.py` — NEW (4 tests, all green). +- `mock-data/manifest.json` — NEW (v1.2 dataset manifest). +- `docs/VERIFICATION.md` — extended with Phase 2 verification recipe + AC checklist. + +### Test command result + +``` +$ bash scripts/migrate-to-v1.2.sh && PYTHONPATH=. pytest tests/test_plane_migration.py -v && bash test.sh +── [1/8] Schema constraints + indexes (idempotent) ── +── [2/8] Settings: ensure eberron + mardonari exist; rename legacy default ── +── [3/8] Planes: materialize 4 (...) ── +── [4/8] Setting CONTAINS Plane edges (idempotent) ── +── [5/8] Plane topology edges (REFLECTS / LAYER_OF / ADJACENT_TO / ACCESSIBLE_VIA) ── +── [6/8] Collapse the 2 Roland Person nodes into 1 + write LOCATED_IN edges ── +── [7/8] Seed the wheel-and-kiln of Mardonar + time-bounded PRACTICED_AT edge ── +── [8/8] Defensive count of leftover MULTIVERSE_COUNTERPART_OF relations ── +=== migrate-to-v1.2.sh: done === +============================= test session starts ============================== +tests/test_plane_migration.py::test_two_rolands_collapsed PASSED [ 25%] +tests/test_plane_migration.py::test_plane_edges_written PASSED [ 50%] +tests/test_plane_migration.py::test_world_id_deprecated_but_readable PASSED [ 75%] +tests/test_plane_migration.py::test_setting_node_exists PASSED [100%] +============================== 4 passed in 0.54s =============================== +... bash test.sh ... +✅ all tool types tested +``` + +4/4 new tests + 12/12 existing test.sh tests pass. Migration re-runs idempotently +(verified by running twice in a row — same end state). + +### AC verification (extra evidence beyond the 4 named tests) + +End-to-end smoke tests through the gateway `:8766/mcp`: + +- `list_planes` returns 4 planes with their Setting + topology edges + (REFLECTS / LAYER_OF / ADJACENT_TO / ACCESSIBLE_VIA). +- `entity_planes_at_time(Roland Raventhorne, 3rd_age.year_435, default)` → + `located_in: [mardonari.voldramir]` (after the year_420 transition). +- `entity_planes_at_time(Roland Raventhorne, 3rd_age.year_415, default)` → + `located_in: [mardonari.material]` (before the transition). +- `was_true_at(PRACTICED_AT, "Roland Raventhorne", "The Wheel & Kiln of Mardonar", + "3rd_age.year_435", default)` → `was_true: true`. +- `was_true_at(PRACTICED_AT, "Roland Raventhorne", "The Wheel & Kiln of Mardonar", + "3rd_age.year_415", default)` → `was_true: false`. +- `entity_context(Roland Raventhorne, default)` → `plane_relations` field + surfaces both `LOCATED_IN -> Material`, `LOCATED_IN -> Voldramir`, `EXISTS_IN -> Voldramir`. +- `find_plane_violations` returns 15 missing_exists_in violations + (most Persons in the default seed pre-date the v1.2 model; follow-up to + add EXISTS_IN assertions to seed.py). +- Tool count: 31 → 35 (4 new plane tools added). + +### Things the leader needs to do + +1. `git add -A && git commit -m "Phase 2: v1.2 plane migration + Setting/Plane ontology"` +2. `git push origin feat/p2-ontology-time-planes` +3. `gh pr create --base main --head feat/p2-ontology-time-planes --title "Phase 2 — v1.2 plane migration" --body-file ` +4. Optionally: ship the PR body referencing this story file. + +### Notes / gotchas + +- **Duplicate `entity_planes_at_time` registration.** It's defined in both + `plugins/world.py` (per the story deliverable list, which explicitly puts + `entity_planes_at_time` in world.py) AND in `plugins/planes/__init__.py` + (also per the story). The registry is a `dict` keyed by name with last-write-wins; + `plugins/planes.py` sorts alphabetically before `plugins/world.py`, so the + planes/__init__.py version wins. Both implementations are equivalent in + semantics; the planes version is slightly more idiomatic because the + Cypher is in `plugins/planes/cypher.py`. +- **Existing test files (`test_multi_world.py`, `test_consistency.py`, etc.) + use port 7687**, but the host Neo4j is on 7688 per docker-compose.yml. + This is a pre-existing issue (the tests fail with `Connection refused` + on the default port 7687) — out of File Scope. The story's test command + uses `bash test.sh` (which uses the gateway on :8766) and pytest against + `tests/test_plane_migration.py` only, both of which pass. +- **`MULTIVERSE_COUNTERPART_OF` defensive cleanup.** Per Ambiguity §2, + the migration removes only the Roland↔Roland edges. The current graph + has 1 surviving `MULTIVERSE_COUNTERPART_OF` edge between non-Roland + nodes, logged by step [8/8] and left intact for manual review.