lore-engine-poc/meta/prd.md

PRD — Lore Engine × GraphMCP Substrate Merge

Template: BMAD PRD at _bmad-output/meta/prd.md. Companion to planning-artifacts/architecture.md and meta/epics.md.

Project: kaykayyali/lore-engine-poc (runtime) — Gitea project: lore-engine-merge Author: hermes-agent (BMAD Phase 4 epic) Date: 2026-06-26 Status: Draft v1 — pending architecture review gate Companion ADR: 2026-06-26 Lore Engine GraphMCP Merge (Decision); 2026-06-26 Lore Engine GraphMCP Merge Research (research)

Phase index (linked stories):

• Phase 0 — Inventory of GraphMCP-Example substrate: docs/merge/00-inventory.md. Gate story — nothing else ships until the inventory lands.
• Phase 1 — Substrate merge (deferred to story S2)
• Phase 2 — Ontology + time planes (S3)
• Phase 5 — Bot integration (S6)
• Phase 6 — Connector template (S7)

---

1. Goal

Produce a single merged MCP runtime that:

• Hosts the 14-node lore ontology + time-bounded relations + v1.2 Setting/Plane model on top of GraphMCP-Example's existing Person/Location/Faction/Event/Encounter graph
• Preserves all 7 GraphMCP ingestion workers (Go, Redis Streams) — they're proven, production, and the user has stated "I'll add more"
• Adds 2 new Go workers (structured-ingestor + dialogue-processor) for the YAML/Dialogue paths that lore-engine designed but never shipped
• Exposes a unified MCP tool surface (~24 tools): 8 inherited GraphMCP + 12 lore-engine POC plugins + 4 v1.2 plane tools + consistency generalizations
• Lets kaykayyali/mardonar-npcs (the new Discord bot) call query_as_npc + log_encounter and publish NPC dialogue to raw.dialogue

The smallest end-state we can ship: Phase 0 + Phase 1 + Phase 5 — the bot can run a Mardonar encounter and the NPCs remember across sessions via the merged MCP server, even without the v1.2 plane model or the consistency engine. Phases 2-4 and 6 are additive.

Repo destinations post-merge:

Repo	Post-merge role
kaykayyali/mardonar-specs	YAML encounter corpus (content only, unchanged)
kaykayyali/mardonar-npcs (NEW)	Discord bot runtime; consumes mardonar-specs at build time; calls the merged MCP server
kaykayyali/lore-engine	Design docs (17 docs + ongoing). Unchanged in this epic.
kaykayyali/lore-engine-poc	Merged runtime home. Gains Redis + 7 workers + 2 new workers + the merged MCP surface.
kaykayyali/GraphMCP-Example	Deprecated. Once lore-engine-poc reaches feature parity, archive this repo (do NOT delete — historical value).

---

2. Personas

Persona	What they want
Kay (operator / DM)	Author a new encounter in mardonar-specs, commit, rebuild bot image. The bot runs it. NPCs remember across sessions. The lore graph is consistent and historically accurate.
World-builder	Write timeline.yaml / family_tree.yaml / gazetteer.yaml / magic_system.yaml and ingest via POST /ingest/structured. The graph updates deterministically (no LLM in the loop for structured facts).
LLM (LLM as DM)	Open-ended world queries through MCP: "what did House Vyr rule in 340 TA?", "where is Roland currently?", "did Aldric witness the Battle of Black Spire?". Get precise, source-attributed, contradiction-checked answers.
NPC (in-character via bot)	Be queried for what they personally know (query_as_npc). Their answers are scoped to WITNESSED edges. They remember across sessions.
Future ingestion source author (Slack connector, RSS feed, PDF watcher, etc.)	Copy connector-template/, set the env vars, point at a stream. New workers appear in docker compose ps healthy.

---

3. User Stories (v1)

P0 — must have for v1

• U1: As Kay, I author a YAML encounter spec, commit it to mardonar-specs, and rebuild the bot image with SPECS_GIT_URL. The bot loads it and runs the encounter in Discord.
• U2: As the LLM DM, I query the merged MCP server with query_as_npc(name="Bram", question="what's my opening line?") and get an answer scoped to Bram's WITNESSED-edge knowledge.
• U3: As the bot, I call log_encounter(title=..., participants=..., summary=..., location=...) after each scene. The next query_as_npc returns this encounter in the NPC's witness graph.
• U4: As Kay, I ask the MCP server "where was Roland Raventhorne in 430 TA?" and get a precise, time-bounded answer using was_true_at.
• U5: As world-builder, I POST timeline.yaml to /ingest/structured. The graph gains Date/Event/Edge nodes within 1s; no LLM in the loop.
• U6: As Kay, I open Neo4j Browser at :7474 and inspect the merged graph — I see Settings, Planes, Persons, Factions, Eras, Encounters, LoreFragments with proper edges.

P1 — nice-to-have for v1

• U7: As Kay, I run verify-merge.sh and it exercises every plugin and every inherited tool end-to-end (green).
• U8: As Kay, I add a new ingestion source by copying connector-template/. New workers appear in docker compose ps healthy.
• U9: As Kay, I ask "find contradictions about Aldric's whereabouts" and the consistency engine surfaces planted contradictions from the seed.
• U10: As LLM DM, I ask "what plane is Roland on right now?" and get entity_planes_at_time(Roland, "now") → mardonari.material.

Out of scope for v1

• In-fiction physics (dice rolls, combat resolution) — Foundry owns this; the bot calls into Foundry, not the lore engine.
• Plane-traversal mechanics ("can I Plane Shift to Voldramir?") — the engine knows planes exist; it doesn't compute the spell.
• Real-time collaboration — the bot is single-session, single-party. Multi-party concurrent sessions are v1.5.
• Voice/audio NPC dialogue — text-only for v1. Audio is a v2 expansion.
• Auto-scaling workers — single-instance per worker for v1. The dual-LLM arbitration pattern (*-2 replicas) handles quality, not load.

---

4. Functional Requirements

4.1 MCP server (the merged surface)

The merged MCP server MUST expose at least these 24 tools:

Inherited from GraphMCP-Example (8)	Inherited from lore-engine-poc (12)	New from v1.2 plane model (4)
semantic_search	entity_context	list_planes
graph_traverse	was_true_at	entity_planes
get_context	state_at	entity_planes_at_time
get_person_profile	ancestors_of	find_plane_violations
query_as_npc	descendants_of
log_encounter	lineage_of
get_unresolved	log_trade
get_contradictions	trades_by_buyer
	market_price
	register_image
	recall_images
	search_images_by_caption
	embed_images
	search_images_semantic
	find_contradictions (was get_contradictions)
	find_anachronisms
	find_ontology_violations
	find_orphans

Generalization requirement: get_contradictions from GraphMCP is REPLACED by find_contradictions from lore-engine-poc (which is a generalization — same subject/limit params, plus optional since/severity filters). All tool discovery surfaces show the new name. get_contradictions becomes an alias if any consumer still references it.

4.2 Ingestion layer

The runtime MUST run all 9 worker processes (7 inherited + 2 new) connected to the 4 (now 5) Redis Streams:

Stream	Producers	Consumers
raw.discord	discord-connector	discord-filter
raw.messages	discord-filter, ingestion-worker (HTTP)	entity-extractor, entity-extractor-2
raw.lore	lore-watcher, ingestion-worker (HTTP)	lore-extractor, lore-extractor-2
raw.encounters	discord-connector, ingestion-worker (HTTP)	encounter-processor, encounter-processor-2
raw.structured (NEW)	ingestion-worker (HTTP /ingest/structured)	structured-ingestor
raw.dialogue (NEW)	ingestion-worker (HTTP /ingest/dialogue)	dialogue-processor

Dual-LLM arbitration contract: every -extractor has a -2 replica in the same consumer group. They race on the same stream entry; both write to Neo4j with source_lv: 1 / source_lv: 2. find_contradictions surfaces entries where source_lv: 1 and source_lv: 2 disagree on the same claim.

4.3 Graph ontology

The merged graph MUST support both legacy GraphMCP nodes (Person/Location/Faction/Event/Encounter/Chunk) and lore-engine ontology extensions (Era/Lineage/Calendar/Culture/Deity/MagicSystem/Spell/Language/Title/Artifact/Region/Plane/Setting) plus the temporal relations and the v1.2 plane model.

Migration contract: existing lore-engine-poc data (2 Settings, 4 Planes, Roland Raventhorne, the seed) MUST be migrated to the v1.2 model without loss. Specifically: the 2 Roland Person nodes collapse to 1 with two LOCATED_IN edges, the MULTIVERSE_COUNTERPART_OF relation goes away, and the world_id string property is deprecated (still readable for backwards compat).

4.4 Bot integration

mardonar-npcs MUST:

• Validate encounter specs against EncounterSpecSchema (Pydantic v2) at /encounter start
• Call query_as_npc(name, question) before every NPC reply
• Call log_encounter(...) synchronously on encounter resolve
• Publish every in-character NPC line to POST /ingest/dialogue

Build contract: SPECS_GIT_URL and SPECS_GIT_REF Docker build args clone mardonar-specs into ./specs/ at build time. The image is fully self-contained; no runtime fetch.

---

5. Non-Functional Requirements

• Cost: total LLM cost ≤ $20 for the full Phase 0-6 epic at minimax-m3 rates. Comfortably within Ollama $100/mo + Gemini $20/day caps.
• Latency: query_as_npc returns in <500ms p95 for encounter-scoped queries (NPC witness graph is small). log_encounter is synchronous and MUST complete in <200ms p95.
• Reliability: log_encounter is the source of truth for NPC memory. Failure returns an error to the bot; the bot retries; the encounter graph is consistent. No silent data loss.
• Backward compatibility: GraphMCP-Example's existing 8 MCP tools keep the same input/output contracts (no breaking schema changes). Lore-engine-poc's existing 12 plugins extend, not break, their surface.
• Observability: every worker logs structured JSON with worker, stream, group, msg_id, latency_ms per consumed message. Logs land in Docker's JSON log driver; docker compose logs shows them.

---

6. Risks + Mitigations

Risk	Likelihood	Impact	Mitigation
Lore-engine-poc plugins break when GraphMCP worker writes hit Neo4j	Medium	High	Phase 1 includes verify-merge.sh that exercises every plugin against the merged stack before phase 2 starts
WITNESSED-edge semantics drift when plane model lands	Medium	High	Spell out: WITNESSED is Person↔Encounter, NOT Person↔Plane; orthogonal
Two-LLM arbitration writes conflicting nodes	Medium	Medium	Add source_lv property check in find_contradictions
world_id → plane migration corrupts existing Mardonar data	Low	High	One-shot Cypher migration with rollback, run against v1.2 seed; 2 Roland nodes collapse to 1 with two LOCATED_IN
Bot log_encounter writes fail during active DM	Low	High	Sync write is the contract; failure → bot retries; encounter graph is source of truth

---

7. Phased execution (summary — full detail in meta/epics.md)

Phase	What	Owner profile	Wall clock	Cost
P0	Inventory of GraphMCP workers/tools/streams	dev	30 min	~$0.40
P1	Substrate merge (Redis + 7 workers + nsc plugin)	dev + tester	2 h	~$1.60
P2	Ontology + time + planes	dev + tester	2 h	~$1.60
P3	Consistency engine	dev + tester	1.5 h	~$1.20
P4	Structured + dialogue ingestion	dev + tester	2 h	~$1.60
P5	Bot integration (mardonar-npcs ↔ merged MCP)	dev + tester	2 h	~$1.60
P6	Connector template + first new source	dev + tester	1.5 h	~$1.20
Total			~11.5 h	~$9.20

Linear chain P0 → P1 → ... → P6. P6 can fan out into multiple connector-* workers once the template ships.

---

8. Definition of Done (rolled up)

• All 6 phases done on the damascus orchestrator
• Each phase's PR is merged to main on kaykayyali/lore-engine-poc
• bash verify-merge.sh exits 0 (exercises every plugin + every inherited tool)
• docker compose ps shows all 11 services healthy (neo4j, postgres, minio, redis, gateway, 7 workers)
• Neo4j Browser at :7474 shows the merged graph (Settings, Planes, all ontology nodes, Encounter + WITNESSED)
• Live URL http://hp-grey-public.tailcb2b60.ts.net:8765/mcp returns the full 24-tool surface on tools/list
• kaykayyali/mardonar-npcs image built + tested end-to-end against the merged runtime
• Wiki page Projects/Lore Engine.md updated with the merge state + URL
• ADR at wiki/Decisions/2026-06-26 Lore Engine GraphMCP Merge.md updated with "Merged" status + merge SHA