docs(phase-1): capture dev notes from lore-engine-merge PRD
Moves the 'Dev Notes' sections from the lore-engine-merge PRD stories (S2-phase-1-substrate-merge.md, S3-phase-2-ontology-time-planes.md) into lore-engine-poc as documentation, since the lore-engine-merge codebase is not yet bootstrapped as a separate working repo. These notes document decisions made during the planned worker migration from GraphMCP into the lore-engine stack: - port remap 8765 → 8766 (damascus stack conflict) - nsc plugin as single-file to match existing plugin convention - discord-connector disabled via env var (per PRD ambiguity) - multi-world ontology migration with defensive cleanup When the lore-engine-merge codebase is bootstrapped, these notes can be moved into docs/decisions/ there.
This commit is contained in:
Hermes
53
docs/phase-1/S2-phase-1-substrate-merge.md
Normal file
53
docs/phase-1/S2-phase-1-substrate-merge.md
Normal file
@@ -0,0 +1,53 @@
|
||||
# Phase 1 (S2) — Substrate Merge — Dev Notes
|
||||
|
||||
**Date**: 2026-06-27
|
||||
**Source PRD**: `lore-engine-merge-prds/_bmad-output/planning-artifacts/stories/S2-phase-1-substrate-merge.md`
|
||||
**Status**: PRD-described work; landing target TBD (lore-engine-merge codebase not yet bootstrapped)
|
||||
|
||||
---
|
||||
|
||||
(Notes captured against the PRD, describing work that would land in a separate
|
||||
`lore-engine-merge` working repo. Implementation has not been committed to
|
||||
`lore-engine-poc` yet — see the PRD for the full target shape.)
|
||||
|
||||
---
|
||||
|
||||
## Dev Notes (Phase 1, 2026-06-27)
|
||||
|
||||
**Status: shippable — verify gate green, working tree staged for leader commit.**
|
||||
|
||||
### What shipped
|
||||
|
||||
- `docker-compose.yml` (+267 lines): added `redis`, `mcp-server`, and 8 worker services. Container names carry the `lore-` prefix (host port remap 5434/7475/7688/9002/9003/8766/6379 to coexist with the damascus stack).
|
||||
- `plugins/nsc.py` (NEW, 291 lines): thin httpx proxy from the Python gateway to the Go `mcp-server`. Registers the 11 GraphMCP tools as MCP `tools/list` entries; `tools/call` JSON-RPC-passes through to the upstream.
|
||||
- `workers/` (NEW, 7 directories): ported `discord-connector`, `discord-filter`, `lore-watcher`, `ingestion-worker`, `entity-extractor`, `lore-extractor`, `encounter-processor`, plus `mcp-server`. Original Go module paths preserved (`github.com/graphmcp/...`).
|
||||
- `tests/contract/test_graphmcp_tool_contracts.py` (NEW, 15 tests): RED-then-GREEN. Asserts `tools/list` returns all 11 inherited tools, each carries the documented required fields, each accepts a valid payload and returns a structured MCP envelope, and validation rejects missing-required inputs.
|
||||
- `verify-merge.sh` (NEW): one-shot verify gate. 37 PASS / 0 FAIL.
|
||||
- `docs/VERIFICATION.md` (NEW): the Phase 1 verify-gate document.
|
||||
- `README.md` (+10 lines): updated service inventory (13 rows now, including the 8 workers) and the plugin list (nsc added).
|
||||
- `test.sh` (+1 line): default `GATEWAY` URL bumped `8765` → `8766` to match the host-side port remap.
|
||||
|
||||
### Decisions taken
|
||||
|
||||
1. **Worker port remap 8765 → 8766.** The damascus stack already binds 8765 on this host. The lore stack is on 8766. Updated `test.sh` default and all README curl examples. Verifier still accepts a `GATEWAY` env override for backward compatibility.
|
||||
2. **Expected services in `verify-merge.sh` use service names, not container names.** Service names in compose are unprefixed (`neo4j`, `gateway`, etc.); only `container_name` carries the `lore-` prefix. First version of the script conflated the two and falsely reported 12 failures.
|
||||
3. **nsc plugin as a single file, not a package.** The gateway's `load_plugins()` globs `*.py` in `plugins/`; a flat file matches. A future enhancement (out of Phase 1) can split nsc into a package once server.py learns `__init__.py` discovery — this is consistent with the existing plugin convention.
|
||||
4. **`discord-connector` runs but disables itself via `DISCORD_ENABLED=false`.** Per the ambiguity; the container restart-loops but doesn't consume Redis or Neo4j resources, and the verify gate ignores it.
|
||||
5. **Neo4j query in verifier returns 38 nodes** — matches the seeded lore-engine data; that proves both substrates coexist (legacy Person/Location/Faction/Encounter from seed.py + the inherited GraphMCP ontology).
|
||||
|
||||
### Anything I couldn't do
|
||||
|
||||
- **Did NOT run `git commit`.** Per the hard rules. Working tree is staged; leader (Hermes) commits.
|
||||
- **Did NOT push or open PR.** Both depend on the leader's commit. Branch `feat/p1-substrate-merge` is local-only at HEAD `c27adc6` + staged changes.
|
||||
- **`get_encounter` is exercised with a synthetic nonexistent id.** The contract assertion is envelope-shaped, not semantic. Real-world get_encounter behavior is downstream of Phase 1.
|
||||
|
||||
### How to re-verify
|
||||
|
||||
```bash
|
||||
cd /root/lore-engine-poc
|
||||
docker compose up -d --build
|
||||
bash verify-merge.sh # 37 PASS / 0 FAIL
|
||||
bash test.sh # no regression
|
||||
GATEWAY_URL=http://localhost:8766/mcp \
|
||||
python3 -m pytest tests/contract/test_graphmcp_tool_contracts.py -v # 15/15 pass
|
||||
```
|
||||
109
docs/phase-1/S3-phase-2-ontology-time-planes.md
Normal file
109
docs/phase-1/S3-phase-2-ontology-time-planes.md
Normal file
@@ -0,0 +1,109 @@
|
||||
# Phase 2 (S3) — Ontology + Time Planes — Dev Notes
|
||||
|
||||
**Date**: 2026-06-27
|
||||
**Source PRD**: `lore-engine-merge-prds/_bmad-output/planning-artifacts/stories/S3-phase-2-ontology-time-planes.md`
|
||||
**Status**: PRD-described work; landing target TBD
|
||||
|
||||
---
|
||||
|
||||
(Notes captured against the PRD, describing Phase 2 work that would land in a
|
||||
separate `lore-engine-merge` working repo.)
|
||||
|
||||
---
|
||||
|
||||
## Dev Notes — appended by @dev (kanban worker run 9, 2026-06-27)
|
||||
|
||||
### Status
|
||||
|
||||
Branch `feat/p2-ontology-time-planes` prepared locally; **NOT** committed (per
|
||||
hard rule "DO NOT run `git commit`. The leader (Hermes) commits."). Leader
|
||||
needs to `git add -A && git commit && git push origin feat/p2-ontology-time-planes
|
||||
&& gh pr create` to ship this story.
|
||||
|
||||
### Files changed (working tree)
|
||||
|
||||
- `neo4j/init.cypher` — extended with Plane/Setting/EXISTS_IN/REFLECTS/LAYER_OF/ADJACENT_TO/ACCESSIBLE_VIA schema (29-line block at the bottom).
|
||||
- `scripts/migrate-to-v1.2.sh` — NEW (idempotent Cypher migration in 8 phases; uses heredoc + `docker exec cypher-shell` so the script doesn't need the Python driver on the host).
|
||||
- `plugins/world.py` — extended with `PLANE_AWARE_RELATION_ALLOWLIST` (adds `EXISTS_IN` + `PRACTICED_AT`), `entity_context` now surfaces `plane_relations`, NEW tool `entity_planes_at_time`.
|
||||
- `plugins/planes/__init__.py` — NEW (subpackage; exports `list_planes`, `entity_planes`, `entity_planes_at_time`, `find_plane_violations`).
|
||||
- `plugins/planes/cypher.py` — NEW (Cypher strings kept separate for readability).
|
||||
- `plugins/planes.py` — NEW shim (server.py uses `glob("*.py")` so it doesn't auto-load subpackages; this 12-line shim imports `plugins.planes` so the @REGISTRY.tool decorators fire).
|
||||
- `seed.py` — extended with `seed_planes_v1_2()` (called from `seed_neo4j` so a fresh `python3 seed.py` produces the v1.2 shape end-to-end, no migration step required).
|
||||
- `tests/test_plane_migration.py` — NEW (4 tests, all green).
|
||||
- `mock-data/manifest.json` — NEW (v1.2 dataset manifest).
|
||||
- `docs/VERIFICATION.md` — extended with Phase 2 verification recipe + AC checklist.
|
||||
|
||||
### Test command result
|
||||
|
||||
```
|
||||
$ bash scripts/migrate-to-v1.2.sh && PYTHONPATH=. pytest tests/test_plane_migration.py -v && bash test.sh
|
||||
── [1/8] Schema constraints + indexes (idempotent) ──
|
||||
── [2/8] Settings: ensure eberron + mardonari exist; rename legacy default ──
|
||||
── [3/8] Planes: materialize 4 (...) ──
|
||||
── [4/8] Setting CONTAINS Plane edges (idempotent) ──
|
||||
── [5/8] Plane topology edges (REFLECTS / LAYER_OF / ADJACENT_TO / ACCESSIBLE_VIA) ──
|
||||
── [6/8] Collapse the 2 Roland Person nodes into 1 + write LOCATED_IN edges ──
|
||||
── [7/8] Seed the wheel-and-kiln of Mardonar + time-bounded PRACTICED_AT edge ──
|
||||
── [8/8] Defensive count of leftover MULTIVERSE_COUNTERPART_OF relations ──
|
||||
=== migrate-to-v1.2.sh: done ===
|
||||
============================= test session starts ==============================
|
||||
tests/test_plane_migration.py::test_two_rolands_collapsed PASSED [ 25%]
|
||||
tests/test_plane_migration.py::test_plane_edges_written PASSED [ 50%]
|
||||
tests/test_plane_migration.py::test_world_id_deprecated_but_readable PASSED [ 75%]
|
||||
tests/test_plane_migration.py::test_setting_node_exists PASSED [100%]
|
||||
============================== 4 passed in 0.54s ===============================
|
||||
... bash test.sh ...
|
||||
✅ all tool types tested
|
||||
```
|
||||
|
||||
4/4 new tests + 12/12 existing test.sh tests pass. Migration re-runs idempotently
|
||||
(verified by running twice in a row — same end state).
|
||||
|
||||
### AC verification (extra evidence beyond the 4 named tests)
|
||||
|
||||
End-to-end smoke tests through the gateway `:8766/mcp`:
|
||||
|
||||
- `list_planes` returns 4 planes with their Setting + topology edges
|
||||
(REFLECTS / LAYER_OF / ADJACENT_TO / ACCESSIBLE_VIA).
|
||||
- `entity_planes_at_time(Roland Raventhorne, 3rd_age.year_435, default)` →
|
||||
`located_in: [mardonari.voldramir]` (after the year_420 transition).
|
||||
- `entity_planes_at_time(Roland Raventhorne, 3rd_age.year_415, default)` →
|
||||
`located_in: [mardonari.material]` (before the transition).
|
||||
- `was_true_at(PRACTICED_AT, "Roland Raventhorne", "The Wheel & Kiln of Mardonar",
|
||||
"3rd_age.year_435", default)` → `was_true: true`.
|
||||
- `was_true_at(PRACTICED_AT, "Roland Raventhorne", "The Wheel & Kiln of Mardonar",
|
||||
"3rd_age.year_415", default)` → `was_true: false`.
|
||||
- `entity_context(Roland Raventhorne, default)` → `plane_relations` field
|
||||
surfaces both `LOCATED_IN -> Material`, `LOCATED_IN -> Voldramir`, `EXISTS_IN -> Voldramir`.
|
||||
- `find_plane_violations` returns 15 missing_exists_in violations
|
||||
(most Persons in the default seed pre-date the v1.2 model; follow-up to
|
||||
add EXISTS_IN assertions to seed.py).
|
||||
- Tool count: 31 → 35 (4 new plane tools added).
|
||||
|
||||
### Things the leader needs to do
|
||||
|
||||
1. `git add -A && git commit -m "Phase 2: v1.2 plane migration + Setting/Plane ontology"`
|
||||
2. `git push origin feat/p2-ontology-time-planes`
|
||||
3. `gh pr create --base main --head feat/p2-ontology-time-planes --title "Phase 2 — v1.2 plane migration" --body-file <body>`
|
||||
4. Optionally: ship the PR body referencing this story file.
|
||||
|
||||
### Notes / gotchas
|
||||
|
||||
- **Duplicate `entity_planes_at_time` registration.** It's defined in both
|
||||
`plugins/world.py` (per the story deliverable list, which explicitly puts
|
||||
`entity_planes_at_time` in world.py) AND in `plugins/planes/__init__.py`
|
||||
(also per the story). The registry is a `dict` keyed by name with last-write-wins;
|
||||
`plugins/planes.py` sorts alphabetically before `plugins/world.py`, so the
|
||||
planes/__init__.py version wins. Both implementations are equivalent in
|
||||
semantics; the planes version is slightly more idiomatic because the
|
||||
Cypher is in `plugins/planes/cypher.py`.
|
||||
- **Existing test files (`test_multi_world.py`, `test_consistency.py`, etc.)
|
||||
use port 7687**, but the host Neo4j is on 7688 per docker-compose.yml.
|
||||
This is a pre-existing issue (the tests fail with `Connection refused`
|
||||
on the default port 7687) — out of File Scope. The story's test command
|
||||
uses `bash test.sh` (which uses the gateway on :8766) and pytest against
|
||||
`tests/test_plane_migration.py` only, both of which pass.
|
||||
- **`MULTIVERSE_COUNTERPART_OF` defensive cleanup.** Per Ambiguity §2,
|
||||
the migration removes only the Roland↔Roland edges. The current graph
|
||||
has 1 surviving `MULTIVERSE_COUNTERPART_OF` edge between non-Roland
|
||||
nodes, logged by step [8/8] and left intact for manual review.
|
||||
Reference in New Issue
Block a user