docs: reframe consistency engine as from-scratch on Cognee; add CONTEXT.md glossary
Research into Cognee's actual API (docs.cognee.ai) confirmed the docs made a load-bearing false claim: that the Lore Engine 'inherits and generalizes' a Contradiction node, get_contradictions tool, 8 inherited MCP tools, and neo4j-init.cypher from the substrate. Cognee ships NONE of that. Cognee provides DataPoint + custom graph models + remember/recall + a Cypher/APOC graph-rule pattern. So: - Slice 2 (consistency) is a from-scratch BUILD, not a generalization - Categories A/B/D (Contradiction/Anachronism/Orphan) are ours - Category C (declarative OntologyRule) rides Cognee's Cypher pattern - '8 inherited tools' -> '8 base tools' (one wraps cognee.recall) - '7 inherited labels' -> '7 base types' (Lore Engine originals on DataPoint) Fixed across 04-consistency, 01-ontology, 05-mcp-tools, 00-overview, 09-roadmap, 15-related-work, 16-comparison. Historical GraphMCP comparisons left intact. Added CONTEXT.md (glossary) — the grill-with-docs skill mandates it and 6 ADRs' worth of resolved terms (Lineage/Faction/Region/Plane/ LoreSource/extraction+source confidence/disputed edge/retcon/Setting/ ConsistencyRun/Cognee) had no single home. New readers no longer mine ADR prose for the vocabulary. Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
70
CONTEXT.md
Normal file
70
CONTEXT.md
Normal file
@@ -0,0 +1,70 @@
|
||||
# Lore Engine
|
||||
|
||||
A reasoning substrate for a high-fantasy world. Its single product: when an LLM asks a question about the world, it gets a precise, time-bound, contradiction-checked answer traceable to the document it came from. Built on Cognee (the substrate); the Lore Engine is the domain layer on top.
|
||||
|
||||
## Language
|
||||
|
||||
### Entities & structure
|
||||
|
||||
**Lineage**:
|
||||
A blood relationship — descent from a founding ancestor. A person has at most one Lineage.
|
||||
_Avoid_: family, house (a house is a Faction, not a Lineage)
|
||||
|
||||
**Faction**:
|
||||
An association — a House, Order, or Guild. A person can belong to many Factions over time.
|
||||
_Avoid_: house (when meaning blood — that's Lineage)
|
||||
|
||||
**Region**:
|
||||
A geographic concept — a continent, kingdom, forest, dungeon. Exists *inside* a Plane. `PART_OF` between Regions means "geographically contains."
|
||||
_Avoid_: plane (a plane is not geographic)
|
||||
|
||||
**Plane**:
|
||||
A mode of existence — Material, Shadow, Voldramir, the Feywild. Plane-relation edges (`REFLECTS`, `LAYER_OF`, `ADJACENT_TO`, `ACCESSIBLE_VIA`) are between Planes.
|
||||
_Avoid_: region, world
|
||||
|
||||
**Setting**:
|
||||
A top-level namespace — a campaign or world. Hosts one or more Planes. The v1.2 replacement for the flat `world_id` string.
|
||||
_Avoid_: world, world_id (deprecated)
|
||||
|
||||
**LoreSource**:
|
||||
A document ingested into the engine (prose, timeline YAML, dialogue log). Carries a `reliability` and is the unit of source attribution.
|
||||
|
||||
### Confidence & conflict
|
||||
|
||||
**extraction_confidence**:
|
||||
Did we extract this triple correctly? A per-triple score (0.0–1.0) from the LLM extractor.
|
||||
|
||||
**source_confidence**:
|
||||
How reliable is the document a triple came from? Derived from `LoreSource.reliability` (canonical 1.0 → fanon 0.3).
|
||||
|
||||
**aggregate_confidence**:
|
||||
The floor — `min(extraction_confidence × source_confidence)` across every source on an edge. Not a mean or max. (ADR 0001.)
|
||||
_Avoid_: confidence (unqualified — say which dimension)
|
||||
|
||||
**Disputed edge**:
|
||||
When two sources produce the same (subject, relation, object) with *conflicting* time bounds. Kept as separate Edge records, both marked `is_disputed: true`, linked via `disputed_with` — not merged. (ADR 0002.)
|
||||
|
||||
**Contradiction**:
|
||||
A first-class *node* flagging incompatible claims between LoreSources. Built from scratch in slice 2 — Cognee ships no contradiction machinery.
|
||||
|
||||
### Time
|
||||
|
||||
**at_time / as_of**:
|
||||
`at_time` scopes a query to an in-world moment. `as_of` scopes it to a *retcon* boundary — what the world-builder knew on a given date, before a retcon.
|
||||
|
||||
**Retcon**:
|
||||
A declared revision that supersedes — never deletes — an old fact. Old edges are marked `superseded_by`; both pre- and post-retcon states stay answerable.
|
||||
|
||||
### Operations
|
||||
|
||||
**ConsistencyRun**:
|
||||
A record of one execution of the consistency engine (started_at, rules_run, violations_found). The LLM checks `latest_run()` before answering historical questions.
|
||||
|
||||
**Cognee**:
|
||||
The substrate — the MIT-licensed framework that owns storage (Neo4j/Kuzu), extraction, embedding, and the `remember`/`recall`/`forget` API. Pinned at v1.1.2 (ADR 0006).
|
||||
|
||||
## Avoid list (global)
|
||||
|
||||
- **world_id** — use `setting_id`.
|
||||
- **"inherited from GraphMCP-Example"** — the prior substrate; no longer used. Cognee is the substrate. GraphMCP-Example appears only in historical comparisons ("the original design," "cheaper than the prior stack").
|
||||
- **confidence** (unqualified) — say `extraction_confidence`, `source_confidence`, or `aggregate_confidence`.
|
||||
@@ -28,7 +28,7 @@ The Lore Engine is built on top of [Cognee](https://github.com/topoteretes/cogne
|
||||
|
||||
## What we add
|
||||
|
||||
- **Deeper ontology** — Era, Calendar, Lineage, Culture, Deity, MagicSystem, Spell, Language, Title, Item (covers Artifact), Region, plus 2 v1.2 nodes (Plane, Setting) and 5 consistency nodes. Roughly 36 node labels total (7 inherited + 19 v1 core + 2 v1.2 planes + 6 v1.1 polymorphic + 5 consistency). The v1.1 docs (`11-extensibility.md`) add the polymorphic `DomainEntity`, `Relation`, `TypeTemplate`, `NPC`, `PC`, and `Human` labels.
|
||||
- **Deeper ontology** — Era, Calendar, Lineage, Culture, Deity, MagicSystem, Spell, Language, Title, Item (covers Artifact), Region, plus 2 v1.2 nodes (Plane, Setting) and 5 consistency nodes. Roughly 36 node labels total (7 base + 19 v1 core + 2 v1.2 planes + 6 v1.1 polymorphic + 5 consistency). The 7 base types are Lore Engine originals built on Cognee's `DataPoint` — not inherited as-is (Cognee ships no `Contradiction`/`Message`/etc. nodes of its own). The v1.1 docs (`11-extensibility.md`) add the polymorphic `DomainEntity`, `Relation`, `TypeTemplate`, `NPC`, `PC`, and `Human` labels.
|
||||
- **Time as a first-class concept** — temporal validity windows on relations, era filters, "was X true at time T?" via dedicated tools. The LLM no longer has to guess which version of Aldric it is talking to.
|
||||
- **Structured ingestion** — not just `.md` files. We ingest `timeline.yaml`, `family-tree.yaml`, `gazetteer.yaml`, `bestiary.yaml` as first-class sources. Free prose stays, but is no longer the only path.
|
||||
- **A consistency engine** — Cypher-based rules that flag anachronisms (Aldric can't be at the Battle of Black Spire if it happened 200 years before his birth), missing lineage (a noble with no recorded parents), and ontological violations (a region claimed to be inside two non-overlapping kingdoms).
|
||||
|
||||
@@ -13,7 +13,9 @@ The world is modeled as a typed, temporal, source-attributed property graph. Thi
|
||||
|
||||
## Node labels
|
||||
|
||||
### Inherited from GraphMCP-Example
|
||||
### Base types (Lore Engine originals on Cognee's `DataPoint`)
|
||||
|
||||
> Cognee provides the `DataPoint` base class and its own chunking/embedding — that is what's genuinely inherited. The labels below are **Lore Engine types** built on top of `DataPoint`, not Cognee primitives. In particular, `Contradiction` (and the other consistency nodes) are built from scratch in slice 2 — Cognee ships no contradiction machinery. `Message`/`Topic` are legacy ingestion concepts retained for dialogue logs.
|
||||
|
||||
| Label | Purpose | Key properties |
|
||||
|---|---|---|
|
||||
@@ -133,7 +135,7 @@ All edges are directed, typed, and (where meaningful) carry a temporal validity
|
||||
| `OCCURRED_AT` | Event → Location | yes (start) | Where the event happened. |
|
||||
| `OCCURRED_DURING` | Event → Era | yes | Which era the event belongs to. |
|
||||
| `PARTICIPATED_IN` | Person/Faction → Event | yes | Direct involvement. |
|
||||
| `WITNESSED` | Person → Encounter | yes | Same as existing GraphMCP pattern, kept for in-fiction events. |
|
||||
| `WITNESSED` | Person → Encounter | yes | Lore Engine pattern for in-fiction events. |
|
||||
| `CAUSED` | Event → Event | yes | Causal chain. "The Sundering caused the Long Winter." |
|
||||
| `PRECEDED` | Event → Event | yes | Pure chronological. |
|
||||
| `CONCURRENT_WITH` | Event → Event | yes | Overlap, not causation. |
|
||||
@@ -219,7 +221,7 @@ The MCP tool layer turns `valid_from`/`valid_until` into answers to "was X true
|
||||
}
|
||||
```
|
||||
|
||||
`lore_verified: false` is a critical flag. The LLM is told to treat those nodes as provisional, and there's a dedicated `get_unresolved` tool (already in GraphMCP-Example) that lists them.
|
||||
`lore_verified: false` is a critical flag. The LLM is told to treat those nodes as provisional, and there's a dedicated `get_unresolved` tool (built in slice 2) that lists them.
|
||||
|
||||
## Constraint strategy
|
||||
|
||||
@@ -230,7 +232,7 @@ We **do not** use uniqueness constraints on `name` for everything — same names
|
||||
- Same for every typed label
|
||||
- A separate `(e:Entity {canonical_id: "uuid"})` hub node can be created when a name needs to span types — the engine uses this to register *ambiguous* references and resolve them.
|
||||
|
||||
For the full Cypher that creates the schema, see `08-architecture.md#schema-bootstrap` and the existing `neo4j-init.cypher` in GraphMCP-Example for the parts that carry over.
|
||||
For the full Cypher that creates the schema, see `08-architecture.md#schema-bootstrap`. Cognee manages its own base schema; our typed constraints and indexes live alongside it.
|
||||
|
||||
## What the ontology deliberately leaves out
|
||||
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
# 04 — Consistency Engine
|
||||
|
||||
The Consistency Engine is the part of the Lore Engine that *catches the LLM before it lies*. It runs a set of Cypher-defined rules over the graph and surfaces violations as first-class nodes (`Contradiction` already exists in GraphMCP-Example; we add `Anachronism`, `Orphan`, `OntologyViolation`).
|
||||
The Consistency Engine is the part of the Lore Engine that *catches the LLM before it lies*. It runs a set of Cypher-defined rules over the graph and surfaces violations as first-class nodes. Cognee ships **no** contradiction machinery of its own — `Contradiction`, `Anachronism`, `Orphan`, and `OntologyViolation` are all Lore Engine types, built from scratch in slice 2 on top of Cognee's `DataPoint`.
|
||||
|
||||
This document catalogs the rule categories, the node types they produce, and the MCP tools the LLM uses to inspect them.
|
||||
|
||||
## The three failure modes we catch
|
||||
|
||||
1. **Contradictions** — two sources make incompatible claims about the same fact. (Already partially handled in GraphMCP-Example for `LOCATED_AT` and `RULES`; we generalize.)
|
||||
1. **Contradictions** — two sources make incompatible claims about the same fact. (Built from scratch on Cognee — there is no inherited contradiction handler to generalize.)
|
||||
2. **Anachronisms** — a claim requires a person/faction/thing to exist at a time it could not have. Aldric is at the Battle of Black Spire, but Black Spire happened 200 years before his birth.
|
||||
3. **Ontology violations** — the graph is internally inconsistent in ways that violate domain rules. A region is inside two non-overlapping kingdoms. A spell is in a magic system that doesn't exist in this era.
|
||||
|
||||
@@ -51,7 +51,7 @@ All four are *first-class nodes*, not flags. The LLM can query them, the user ca
|
||||
|
||||
### Category A: Source-claim contradictions
|
||||
|
||||
A `LoreSource` makes a claim about an entity that another `LoreSource` contradicts. The existing GraphMCP-Example code handles this for `LOCATED_AT` and `RULES`. We extend to:
|
||||
A `LoreSource` makes a claim about an entity that another `LoreSource` contradicts. Cognee provides only coreference resolution in its extraction prompt; the contradiction detector is ours to build. We cover:
|
||||
|
||||
- `MEMBER_OF` (Person can't be in two factions at once, unless `valid_from`/`valid_until` differ)
|
||||
- `RULES` (a Location can't have two simultaneous rulers — `valid_from`/`valid_until` must not overlap)
|
||||
@@ -105,7 +105,7 @@ The user can define arbitrary rules as Cypher strings. They get stored as `:Onto
|
||||
})
|
||||
```
|
||||
|
||||
A nightly batch job iterates over all `:OntologyRule` nodes and runs each one, materializing `:OntologyViolation` nodes for any non-empty result.
|
||||
A nightly batch job iterates over all `:OntologyRule` nodes and runs each one, materializing `:OntologyViolation` nodes for any non-empty result. This leans on Cognee's documented pattern for graph-level consistency (Cypher rules with APOC's `apoc.util.validate`) — Category C is the one part of the consistency engine that rides a Cognee-native mechanism rather than being built entirely from scratch.
|
||||
|
||||
**Out of the box**, we ship a starter set of ~10 rules. See `05-mcp-tools.md#starter-rules` for the list.
|
||||
|
||||
@@ -148,7 +148,7 @@ These are the tools the LLM uses to *ask* about consistency, not just the tools
|
||||
|
||||
| Tool | Purpose |
|
||||
|---|---|
|
||||
| `get_contradictions(subject?, severity?, limit)` | List flagged contradictions, optionally filtered. **Already in GraphMCP-Example; we generalize.** |
|
||||
| `get_contradictions(subject?, severity?, limit)` | List flagged contradictions, optionally filtered. **Built in slice 2 (no Cognee equivalent).** |
|
||||
| `get_anachronisms(entity?, limit)` | List flagged anachronisms. |
|
||||
| `get_ontology_violations(rule_id?, severity?, limit)` | List ontology rule violations. |
|
||||
| `get_orphans(reason?, limit)` | List entities with missing structural data. |
|
||||
|
||||
@@ -2,18 +2,18 @@
|
||||
|
||||
The complete tool surface the LLM uses to reason about the world. Each tool has one job. Higher-level patterns are compositions, not bigger tools.
|
||||
|
||||
Existing tools from GraphMCP-Example (kept as-is, marked with `[inherited]`):
|
||||
Base tools (none are "inherited as-is" — Cognee ships no MCP tool catalog; all 45 are Lore Engine handlers. `semantic_search` delegates to `cognee.recall`; the rest are Lore Engine originals):
|
||||
|
||||
- `semantic_search` — vector search over chunks
|
||||
- `semantic_search` — vector search over chunks (wraps `cognee.recall`)
|
||||
- `graph_traverse` — n-hop traversal
|
||||
- `get_context` — full context for a single message
|
||||
- `get_person_profile` — topic/message summary
|
||||
- `query_as_npc` — scoped query (kept, generalized)
|
||||
- `log_encounter` — write an encounter
|
||||
- `get_unresolved` — list provisional entities
|
||||
- `get_contradictions` — list contradictions (kept, generalized)
|
||||
- `get_context` — full context for a single entity
|
||||
- `get_person_profile` — entity summary
|
||||
- `query_as_npc` — NPC-scoped query
|
||||
- `log_encounter` — write an encounter (world-builder write tool)
|
||||
- `get_unresolved` — list provisional entities (built in slice 2)
|
||||
- `get_contradictions` — list contradictions (built in slice 2)
|
||||
|
||||
New tools are grouped by function. Signatures use TypeScript-ish notation for clarity; the actual JSON-RPC schemas live in the Go MCP server source.
|
||||
New tools are grouped by function. Signatures use TypeScript-ish notation for clarity; the actual JSON-RPC schemas live in the MCP server source.
|
||||
|
||||
---
|
||||
|
||||
@@ -382,9 +382,9 @@ The user can disable any rule by ID, and add new ones via `add_ontology_rule`.
|
||||
|
||||
---
|
||||
|
||||
## Tool count: 45 total (8 inherited + 37 new)
|
||||
## Tool count: 45 total (8 base + 37 domain)
|
||||
|
||||
The full catalog: 8 inherited from Cognee (or the prior substrate) + 37 new across Groups 1–8 = **45 MCP tools**. That's well past the empirical LLM tool-use ceiling (~25 in a single system prompt), so the Phase 6 reasoning-harness validation measures usage and collapses the long tail. The LLM uses 5–8 of them 90% of the time; the long tail exists for edge cases the LLM will sometimes need and shouldn't have to fall back to free-text generation for.
|
||||
The full catalog: 8 base tools (one wraps `cognee.recall`; the rest are Lore Engine originals) + 37 domain tools across Groups 1–8 = **45 MCP tools** — all Lore Engine handlers. Cognee ships no MCP tool catalog of its own; a handful of tools delegate to Cognee primitives (`cognee.recall`, `cognee.add`) as a low-precision fallback. That's well past the empirical LLM tool-use ceiling (~25 in a single system prompt), so the Phase 6 reasoning-harness validation measures usage and collapses the long tail. The LLM uses 5–8 of them 90% of the time; the long tail exists for edge cases the LLM will sometimes need and shouldn't have to fall back to free-text generation for.
|
||||
|
||||
This is on the high end of what an LLM can effectively use in a single context. We mitigate by:
|
||||
- The reasoning harness documents which 8 to use first.
|
||||
|
||||
@@ -22,7 +22,7 @@ The substrate. No LLM-facing changes; just the data layer.
|
||||
- [ ] Add 5 starter `:OntologyRule` nodes to the schema (the 5 most common from `05-mcp-tools.md#starter-rules`).
|
||||
- [ ] Document the canonical time format as a comment block in the schema file.
|
||||
|
||||
**Deliverable:** running Neo4j with the extended schema, both UDFs working, a test script that validates UDF behavior against 30+ cases. The existing GraphMCP-Example stack still works.
|
||||
**Deliverable:** running Neo4j with the extended schema, both UDFs working, a test script that validates UDF behavior against 30+ cases. Cognee's default `recall` still works for unstructured queries alongside the typed model.
|
||||
|
||||
**Verify:**
|
||||
```bash
|
||||
|
||||
@@ -84,7 +84,7 @@ The paper evaluates on three multi-hop QA benchmarks (HotPotQA, TwoWikiMultiHop,
|
||||
|
||||
- **Cognitive ontology.** This is the genuinely interesting part. Most KG-RAG systems treat the graph as a bag of triples. Cognee imposes a typed structure derived from cognitive science, which means the LLM can reason over "kinds of things" not just "things."
|
||||
- **Real production deployment.** 17k+ stars. They have paying customers (Cognee Cloud). The system is robust, not a research demo.
|
||||
- **LLM-provider-agnostic.** Works with OpenAI, Anthropic, Ollama, anything. The Lore Engine is currently hard-wired to LiteLLM via the GraphMCP-Example stack.
|
||||
- **LLM-provider-agnostic.** Works with OpenAI, Anthropic, Ollama, anything. The Lore Engine is currently hard-wired to LiteLLM via the Cognee stack.
|
||||
- **Pluggable storage.** Their storage layer supports multiple backends. The Lore Engine's v1.1 multi-store strategy is similar in spirit.
|
||||
- **Agent-native API.** `remember` / `recall` / `forget` is exactly what an LLM agent wants. The Lore Engine's MCP tools are more numerous but less ergonomic for direct agent use.
|
||||
|
||||
@@ -198,7 +198,7 @@ The famous result: starting from the single seed that "Isabella is throwing a Va
|
||||
|
||||
- **Memory stream, not knowledge graph.** Memories are unstructured natural language. There's no way to ask "what was Aldric's lineage?" because lineage isn't a typed thing in the memory stream. The Lore Engine's typed ontology makes this a single Cypher query.
|
||||
- **No temporal reasoning beyond recency.** The memory stream is chronological. The LLM has to *infer* that "X happened before Y" from the text. The Lore Engine's `time_in_window` UDF makes this a single function call.
|
||||
- **Reflections drift.** The paper acknowledges (and the Lore Engine's critique about the existing GraphMCP-Example stack hints at this) that reflections can be wrong, biased, or stale. There's no consistency engine.
|
||||
- **Reflections drift.** The paper acknowledges (and the Lore Engine's consistency engine exists precisely to catch this — the Cognee substrate ships no consistency layer of its own) that reflections can be wrong, biased, or stale. There's no consistency engine.
|
||||
- **Scales badly.** 25 agents with a few days of memory is the published limit. The system slows down as memories accumulate. The Lore Engine's Postgres + Neo4j split scales to many years of history.
|
||||
- **No source attribution.** Memories are generated by the LLM; there's no record of *why* the agent believes something.
|
||||
|
||||
|
||||
@@ -22,9 +22,9 @@ Both systems use a knowledge graph to ground LLM reasoning. Both have ingestion
|
||||
|
||||
4. **Consistency engine.** GraphRAG has no contradiction detection. The Lore Engine's 4-category consistency engine (Contradiction, Anachronism, Orphan, OntologyViolation) surfaces conflicts before the LLM hallucinates around them.
|
||||
|
||||
5. **Extensibility via TypeTemplate.** GraphRAG's graph is whatever the LLM extracted. The Lore Engine's `TypeTemplate` system lets the world-builder define new domain types as YAML. The 30+ tools GraphRAG doesn't have (`state_at`, `event_chain`, `list_lineage`, etc.) come from the typed ontology.
|
||||
5. **Extensibility via TypeTemplate.** GraphRAG's graph is whatever the LLM extracted. The Lore Engine's `TypeTemplate` system lets the world-builder define new domain types as YAML. The 45 tools GraphRAG doesn't have (`state_at`, `event_chain`, `list_lineage`, etc.) come from the typed ontology.
|
||||
|
||||
6. **NPC knowledge scoping.** The Lore Engine's `query_as_npc` tool (inherited from GraphMCP-Example) scopes the LLM's knowledge to what a specific character has witnessed. GraphRAG has nothing equivalent.
|
||||
6. **NPC knowledge scoping.** The Lore Engine's `query_as_npc` tool (a Lore Engine tool, no Cognee equivalent) scopes the LLM's knowledge to what a specific character has witnessed. GraphRAG has nothing equivalent.
|
||||
|
||||
### What GraphRAG does better
|
||||
|
||||
@@ -82,7 +82,7 @@ This is the more dangerous comparison. Cognee is the closest functional comparab
|
||||
|
||||
1. **Production maturity.** 17,843 stars, paying customers (Cognee Cloud), real Discord, real integrations (Claude Code plugin, OpenClaw plugin). The Lore Engine has zero users.
|
||||
|
||||
2. **LLM-provider-agnostic.** Cognee works with OpenAI, Anthropic, Ollama, anything. The Lore Engine inherits GraphMCP-Example's LiteLLM integration, which is good but not as polished as Cognee's.
|
||||
2. **LLM-provider-agnostic.** Cognee works with OpenAI, Anthropic, Ollama, anything. The Lore Engine inherits Cognee's LiteLLM integration, which is good but not as polished as Cognee's native multi-provider routing.
|
||||
|
||||
3. **Agent-native API.** `await cognee.remember(...)` and `await cognee.recall(...)` are beautiful Python async APIs. The Lore Engine's MCP-tool API is HTTP/JSON-RPC. Both are fine; Cognee's is more ergonomic for direct agent use.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user