Lore Engine POC — Time-Aware Query Slice

A working slice of the Lore Engine on top of Cognee. The slice proves the load-bearing primitives end-to-end: typed ontology ingest, time-bounded edges, the was_true_at query, and source attribution.

The seed data is a real D&D campaign codex (Mardonar / Voldramir) with 159 entities — NPCs, locations, regions, factions, lore entries. It lives in lore_engine_poc/seed/.

What's in the slice

1. Codex parser (lore_engine_poc/parsers.py) — walks the Obsidian-style markdown under seed/, reads YAML frontmatter (entity type, faction, region), pulls [[wiki links]] from the body, and emits typed (subject, relation, object) triples.
2. Time model (lore_engine_poc/time_model.py) — Python port of the time_in_window(at, valid_from, valid_until) UDF from docs/02-time-model.md. 13/13 self-tests pass. Era-tree membership (3rd_age matches 3rd_age.year_345), the current token, and open-ended bounds are all handled.
3. One read tool (lore_engine_poc/tools.py) — was_true_at(relation, subject, object, at_time). Returns was_true, valid_from, valid_until, sources, confidence.
4. Cognee integration (in scripts/01_ingest.py) — best-effort call to cognee.add() + cognee.cognify() over every markdown file. Skipped automatically when no LLM API key is configured; the slice is fully functional without it because the structured path is exact.

What's NOT in the slice

The 44 other MCP tools, the consistency engine, the TypeTemplate polymorphic extension, the plane model, the MCP server wiring. All deferred to follow-up slices per the design.

Storage backends (slice 5)

The 36 MCP tools and the consistency engine read from a GraphBackend Protocol (lore_engine_poc/graph_backend.py) with two implementations:

• InMemoryGraph (default, pickle-backed). The in-memory dict-of-dicts from slices 1-11. Used for fast tests, the CI test suite, and the Docker image's baked-in .graph.pkl (slice 11).
• Neo4jGraph (slice 5.3+). A neo4j:5 container with the reified :Relation shape (ADR 0009). The production graph substrate per ADR 0008.

The MCP entry scripts (scripts/05_mcp_server.py, scripts/06_mcp_http_server.py) select the backend at startup via LORE_GRAPH_BACKEND:

# Pickle (default) — load .graph.pkl
python3 scripts/05_mcp_server.py
# or
python3 scripts/06_mcp_http_server.py

# Neo4j — connect to bolt://$LORE_NEO4J_URI
LORE_GRAPH_BACKEND=neo4j LORE_NEO4J_URI=bolt://localhost:7687 \
  python3 scripts/06_mcp_http_server.py

Compose brings up the Neo4j stack with a profile:

# Pickle-backed MCP (slice 11 default)
docker compose --profile pickle up -d

# Neo4j-backed MCP (production substrate, slice 5)
docker compose --profile neo4j up -d

The Neo4j compose stack includes:

• neo4j:5 — the database (NEO4J_AUTH=none for loopback; switch to a username/password before any non-loopback exposure).
• lore-engine-ingest — one-shot job that runs 01_ingest.py --write-neo4j --skip-cognee after Neo4j is healthy.
• lore-engine-mcp-neo4j — the MCP HTTP server reading from Neo4j (depends on both neo4j healthy and lore-engine-ingest exited 0).

The pickle profile keeps working exactly as in slice 11.4 (pickup unchanged, MCP server on port 8765, baked .graph.pkl).

Run

# 1. Install Cognee (one-time)
pip3 install --user cognee

# 2. Build the in-memory graph from the codex
python3 scripts/01_ingest.py            # try Cognee (fails fast w/o LLM key)
python3 scripts/01_ingest.py --skip-cognee   # structured path only

# 3. Run the demo
python3 scripts/02_demo.py
#   -> 7 sample queries, e.g.
#      was_true_at(MEMBER_OF, "Roland Raventhorne", "House Raventhorne", "3rd_age.year_345")
#      was_true_at(SIBLING_OF, "Roland Raventhorne", "Aldric Raventhorne", "3rd_age.year_345")
#      was_true_at(PART_OF, "Voldramir", "Underdark", "3rd_age.year_345")

# 4. Run a one-off query
python3 scripts/02_demo.py \
  --query "MEMBER_OF,Elysia Petalbrooke,Petalbrooke Enclave,3rd_age.year_345"

# 5. Reset (wipe the graph cache and the Cognee dataset)
python3 scripts/03_reset.py

Demo output (excerpt)

Query: SIBLING_OF,Roland Raventhorne,Aldric Raventhorne,3rd_age.year_345
{
  "was_true": true,
  "relation": "SIBLING_OF",
  "subject": "Roland Raventhorne",
  "object": "Aldric Raventhorne",
  "at_time": "3rd_age.year_345",
  "valid_from": null,
  "valid_until": null,
  "sources": [".../Roland Raventhorne.md"],
  "confidence": 1.0,
  "edges_examined": 2
}

The codex

The seed is a 168-file D&D campaign codex. The richest content is in the NPC backstories; the faction and location files are mostly stubs. The parser handles both — stubs produce no edges, and the demo's "negative" queries exercise that case.

The structured path extracted 81 typed triples from the codex:

Relation	Count
LOCATED_IN	34
MEMBER_OF	27
SIBLING_OF	12
ENEMY_OF	4
ALLIED_WITH	3
PART_OF	1

SIBLING_OF and PART_OF are inferred from body-text wikilinks (spouse/parent/sibling heuristic for sibling edges; a low-confidence PART_OF is generated when a region body mentions another region without a frontmatter field).

Why this proves the design

• The structured YAML path (extended to markdown) is exact: every edge traces to a specific source file with confidence 1.0.
• The time model is a working port of the spec, with self-tests.
• One Lore Engine tool is implementable in ~80 lines of Python on top of an in-memory graph. The Cognee integration is a parallel path that materialises the same triples into Cognee's graph DB; once an LLM is configured, the prose path lights up alongside it.
• The time filter actually works — the time_in_window test suite passes 13/13 cases (era-tree, current, open bounds, sub-era).

Limitations

• All extracted edges have valid_from = valid_until = null because the codex doesn't have temporal metadata on relationships. A richer codex (or a family_tree.yaml style structured input) would carry time bounds per edge.
• The sibling/parent/spouse heuristic is naive; it confuses "mentioned in the same paragraph" with "actually related". The full design uses a family_tree.yaml for lineage — always structured, never inferred.
• Cognee's cognify() requires an LLM API key (OpenAI or OpenAI-compatible). The slice runs without one.

Next slices (per docs/09-roadmap.md)

• Slice 2 — extend the parser to handle family_tree.yaml and timeline.yaml (or a + syntax in the codex for time bounds).
• Slice 3 — add the consistency engine (Contradiction, Anachronism, Orphan) on top of the typed graph.
• Slice 4 — wire the remaining 44 tools on the same Graph primitive used here.

MCP transports

The server speaks the standard MCP protocol over two transports. The dispatcher is identical — both are thin transport adapters over MCPServer.handle_message.

stdio (slice 2.6, default)

python3 scripts/05_mcp_server.py

Used by stdlib stdio clients (Claude Desktop, Continue, Cline). No network.

Streamable HTTP (slice 11)

python3 scripts/06_mcp_http_server.py --host 127.0.0.1 --port 8765

Single POST /mcp endpoint. The response body is application/json by default (one-shot) or text/event-stream if the client sends Accept: text/event-stream. Stateless in v1 (no Mcp-Session-Id).

Round-trip with curl

# initialize
curl -s http://127.0.0.1:8765/mcp \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json' \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}'

# was_true_at against the real graph
curl -s http://127.0.0.1:8765/mcp \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json' \
  -d '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"was_true_at","arguments":{"relation":"MEMBER_OF","subject":"Roland Raventhorne","object":"House Raventhorne","at_time":"3rd_age.year_345"}}}'

# SSE-upgraded response (single 'event: message\ndata: <json>\n\n' frame)
curl -sN http://127.0.0.1:8765/mcp \
  -H 'Content-Type: application/json' \
  -H 'Accept: text/event-stream' \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}'

Docker (slice 11.4)

# Pre-step: build the cached graph on the host
python3 scripts/01_ingest.py --skip-cognee   # produces lore_engine_poc/.graph.pkl
docker build -t lore-engine-mcp .
docker run --rm -p 8765:8765 lore-engine-mcp

The image is python:3.12-slim with the 36-tool registry + cached graph baked in (.graph.pkl is bundled via the source-tree COPY, so 01_ingest.py must run on the host before docker build for the graph to be present). A HEALTHCHECK polls POST /mcp initialize every 30s. To override the graph at runtime (e.g. after re-running scripts/01_ingest.py):

docker run --rm -p 8765:8765 \
  -v $PWD/lore_engine_poc/data:/data:ro \
  -e LORE_GRAPH_PATH=/data/.graph.pkl \
  lore-engine-mcp

Docker Compose (slice 11.5)

docker compose up

Brings the service up on port 8765. The compose file exposes LORE_HTTP_PORT (host-side port mapping) and TAG (image tag) as environment overrides:

LORE_HTTP_PORT=9000 TAG=dev docker compose up

Limitations

• Write tools (add_entity, add_relation, add_lore_source, slice 10's update_entity / delete_entity / set_alias / …) mutate the in-memory graph only. Restart the server to reset. This is the same caveat as the stdio path.
• Single-process uvicorn. Multi-worker is intentionally not exposed — the graph is in-memory per-process and multi-worker would diverge silently.
• In-container Cognee ingest is not in scope. The container serves a pre-built graph. Run scripts/01_ingest.py on the host and bind-mount the result.
• No CORS. The HTTP transport is consumable from server-side proxies (Open WebUI, LiteLLM) and curl/httpx clients. A browser-side consumer would need CORSMiddleware added later.