Lore Engine Dev 6510e767c6 slice 4: 16 read tools + 3 write tools + Graph reverse indexes (92/92 tests; 341/341)
- Graph reverse indexes (slice 4.0): edges_by_object (O(1) reverse lookups)
  and entities_by_type (type-filtered queries) added to lore_engine_poc/tools.py
- responses.py (slice 4.1): shared edge_to_fact / entity_summary / envelope
  helpers — single source of truth for tool response shape
- read_tools.py (slice 4.2-4.6): 16 read tools across 5 groups
  - Group 1: lookup, entity_context
  - Group 2: true_during, entities_present, timeline (state_at deferred to 4.6+)
  - Group 3: list_lineage, list_offspring, ancestors_of, descendants_of,
    location_hierarchy
  - Group 4: event_chain, events_during
  - Group 5: lore_about (cite deferred to 4.7+)
- write_tools.py (slice 4.7): 3 minimal world-builder tools
  (add_entity, add_relation, add_lore_source) with allowlist + envelope
- scripts/02_demo.py: now exercises every read tool + write/read round-trip
- 92 new tests (7 graph_indexes + 7 responses + 10 group1 + 15 group2 +
  19 group3 + 11 group4 + 7 group5 + 16 write_tools). 341/341 green.

Excluded: state_at (composes 4 tools + consistency engine), summarize_chain
+ narrate_arc (LLM-required), cite (vector store), and Group 8 write tools
beyond the 3 minimal — all deferred per slice 4 plan.
2026-06-18 09:41:02 -04:00

Lore Engine POC — Time-Aware Query Slice

A working slice of the Lore Engine on top of Cognee. The slice proves the load-bearing primitives end-to-end: typed ontology ingest, time-bounded edges, the was_true_at query, and source attribution.

The seed data is a real D&D campaign codex (Mardonar / Voldramir) with 159 entities — NPCs, locations, regions, factions, lore entries. It lives in lore_engine_poc/seed/.

What's in the slice

  1. Codex parser (lore_engine_poc/parsers.py) — walks the Obsidian-style markdown under seed/, reads YAML frontmatter (entity type, faction, region), pulls [[wiki links]] from the body, and emits typed (subject, relation, object) triples.
  2. Time model (lore_engine_poc/time_model.py) — Python port of the time_in_window(at, valid_from, valid_until) UDF from docs/02-time-model.md. 13/13 self-tests pass. Era-tree membership (3rd_age matches 3rd_age.year_345), the current token, and open-ended bounds are all handled.
  3. One read tool (lore_engine_poc/tools.py) — was_true_at(relation, subject, object, at_time). Returns was_true, valid_from, valid_until, sources, confidence.
  4. Cognee integration (in scripts/01_ingest.py) — best-effort call to cognee.add() + cognee.cognify() over every markdown file. Skipped automatically when no LLM API key is configured; the slice is fully functional without it because the structured path is exact.

What's NOT in the slice

The 44 other MCP tools, the consistency engine, the TypeTemplate polymorphic extension, the plane model, the MCP server wiring. All deferred to follow-up slices per the design.

Run

# 1. Install Cognee (one-time)
pip3 install --user cognee

# 2. Build the in-memory graph from the codex
python3 scripts/01_ingest.py            # try Cognee (fails fast w/o LLM key)
python3 scripts/01_ingest.py --skip-cognee   # structured path only

# 3. Run the demo
python3 scripts/02_demo.py
#   -> 7 sample queries, e.g.
#      was_true_at(MEMBER_OF, "Roland Raventhorne", "House Raventhorne", "3rd_age.year_345")
#      was_true_at(SIBLING_OF, "Roland Raventhorne", "Aldric Raventhorne", "3rd_age.year_345")
#      was_true_at(PART_OF, "Voldramir", "Underdark", "3rd_age.year_345")

# 4. Run a one-off query
python3 scripts/02_demo.py \
  --query "MEMBER_OF,Elysia Petalbrooke,Petalbrooke Enclave,3rd_age.year_345"

# 5. Reset (wipe the graph cache and the Cognee dataset)
python3 scripts/03_reset.py

Demo output (excerpt)

Query: SIBLING_OF,Roland Raventhorne,Aldric Raventhorne,3rd_age.year_345
{
  "was_true": true,
  "relation": "SIBLING_OF",
  "subject": "Roland Raventhorne",
  "object": "Aldric Raventhorne",
  "at_time": "3rd_age.year_345",
  "valid_from": null,
  "valid_until": null,
  "sources": [".../Roland Raventhorne.md"],
  "confidence": 1.0,
  "edges_examined": 2
}

The codex

The seed is a 168-file D&D campaign codex. The richest content is in the NPC backstories; the faction and location files are mostly stubs. The parser handles both — stubs produce no edges, and the demo's "negative" queries exercise that case.

The structured path extracted 81 typed triples from the codex:

Relation Count
LOCATED_IN 34
MEMBER_OF 27
SIBLING_OF 12
ENEMY_OF 4
ALLIED_WITH 3
PART_OF 1

SIBLING_OF and PART_OF are inferred from body-text wikilinks (spouse/parent/sibling heuristic for sibling edges; a low-confidence PART_OF is generated when a region body mentions another region without a frontmatter field).

Why this proves the design

  • The structured YAML path (extended to markdown) is exact: every edge traces to a specific source file with confidence 1.0.
  • The time model is a working port of the spec, with self-tests.
  • One Lore Engine tool is implementable in ~80 lines of Python on top of an in-memory graph. The Cognee integration is a parallel path that materialises the same triples into Cognee's graph DB; once an LLM is configured, the prose path lights up alongside it.
  • The time filter actually works — the time_in_window test suite passes 13/13 cases (era-tree, current, open bounds, sub-era).

Limitations

  • All extracted edges have valid_from = valid_until = null because the codex doesn't have temporal metadata on relationships. A richer codex (or a family_tree.yaml style structured input) would carry time bounds per edge.
  • The sibling/parent/spouse heuristic is naive; it confuses "mentioned in the same paragraph" with "actually related". The full design uses a family_tree.yaml for lineage — always structured, never inferred.
  • Cognee's cognify() requires an LLM API key (OpenAI or OpenAI-compatible). The slice runs without one.

Next slices (per docs/09-roadmap.md)

  • Slice 2 — extend the parser to handle family_tree.yaml and timeline.yaml (or a + syntax in the codex for time bounds).
  • Slice 3 — add the consistency engine (Contradiction, Anachronism, Orphan) on top of the typed graph.
  • Slice 4 — wire the remaining 44 tools on the same Graph primitive used here.
Description
No description provided
Readme 5.6 MiB
Languages
Python 99.8%
Dockerfile 0.2%