lore-engine/docs/16-comparison.md

16 — Comparison & Critical Thinking

15-related-work.md surveyed the landscape and built the comparison matrix. This document goes deeper: head-to-head comparisons with the three most relevant systems (GraphRAG, Cognee, Generative Agents), an honest assessment of where the Lore Engine is worse, and the design decisions that are most debatable.

The goal is not to make the Lore Engine look good. The goal is to be specific enough that a future me (or Kay) can decide whether to build this, modify it, or abandon it.

---

Head-to-head #1: Lore Engine vs. Microsoft GraphRAG

The setup

Both systems use a knowledge graph to ground LLM reasoning. Both have ingestion pipelines. Both expose query APIs. They diverge sharply on what the query API looks like and what the data model is.

What the Lore Engine does better

1. Temporal model. This is the single biggest differentiator. GraphRAG has no concept of "was X true at time T." The Lore Engine's time_in_window UDF, era-tree membership, and current token resolution are not in GraphRAG at any level. For a fictional world, this is a non-negotiable feature.

2. Closed-world enforcement. GraphRAG extracts whatever the LLM finds from the corpus. If the corpus has three people named "Aldric," the graph has three Aldric nodes and no way to know they're the same. The Lore Engine's Person.name uniqueness constraint, the alias system, the lore_verified flag, and the loadKnownEntities injection prevent this.

3. Source attribution. GraphRAG generates community summaries from chunks; the summaries lose the source provenance. The Lore Engine's cite tool always traces back to a specific LoreSource and chunk. The Lore Engine can answer "according to which document?" GraphRAG cannot.

4. Consistency engine. GraphRAG has no contradiction detection. The Lore Engine's 4-category consistency engine (Contradiction, Anachronism, Orphan, OntologyViolation) surfaces conflicts before the LLM hallucinates around them.

5. Extensibility via TypeTemplate. GraphRAG's graph is whatever the LLM extracted. The Lore Engine's TypeTemplate system lets the world-builder define new domain types as YAML. The 45 tools GraphRAG doesn't have (state_at, event_chain, list_lineage, etc.) come from the typed ontology.

6. NPC knowledge scoping. The Lore Engine's query_as_npc tool (a Lore Engine tool, no Cognee equivalent) scopes the LLM's knowledge to what a specific character has witnessed. GraphRAG has nothing equivalent.

What GraphRAG does better

1. Global summarization. GraphRAG's pre-generated community summaries are a genuine, validated contribution. "What are the main themes in this world?" is a question the Lore Engine cannot answer well today. The closest is state_at(world_root, current), which only works if the world is fully modeled.

2. Production maturity. GraphRAG has 33,779 stars, a Microsoft Research blog post, a real community, and a v3.1.0 release. The Lore Engine has zero users.

3. Polyglot storage. GraphRAG supports multiple storage backends out of the box. The Lore Engine v1.1 plans 5 stores but only Neo4j is set up at design time.

4. Multimodal. GraphRAG's RAG-Anything integration handles images, tables, equations natively. The Lore Engine's S3-attachment model is rudimentary.

5. Tooling. GraphRAG has a WebUI, a CLI, prompt-tuning guides, and documentation at microsoft.github.io/graphrag. The Lore Engine has a design repo.

6. Faster iteration on the user-facing API. GraphRAG has shipped 3 major versions in 2 years. The Lore Engine is a single design.

7. Existing user base. Microsoft has customers. The Lore Engine has one (potential) world-builder.

What neither does well

Both systems inherit the LLM's tendency to hallucinate. Both can be wrong about facts. Both are limited by the quality of the LLM. Both require the user to be skeptical of the output.

Net assessment

The Lore Engine solves a different problem from GraphRAG. GraphRAG is for global sensemaking over private documents. The Lore Engine is for historical reasoning about a fictional world. The two could coexist; in fact, GraphRAG's community-detection idea could be borrowed for the Lore Engine's "what is this world about?" question (a v2 feature).

If the goal is to answer questions about a fictional world with historical accuracy, the Lore Engine is the right shape and GraphRAG is the wrong tool. If the goal is to summarize a private document collection, GraphRAG is the right tool and the Lore Engine is overkill.

The Lore Engine is not a better GraphRAG. It's a different system that addresses problems GraphRAG doesn't.

---

Head-to-head #2: Lore Engine vs. Cognee

The setup

This is the more dangerous comparison. Cognee is the closest functional comparable: a self-hosted, MIT-licensed, knowledge-graph backend for LLM reasoning, with an agent-native API, real production deployment, and a published paper.

What the Lore Engine does better

1. Domain specificity. Cognee is generic; the Lore Engine is purpose-built for fictional worlds. For the specific question "was House Vyr allied with the Crimson Pact in 340 TA?", the Lore Engine has a one-Cypher-call answer. Cognee has an LLM-extraction pipeline that may or may not produce the right entity, with no temporal model to filter by 340 TA.

2. Temporal model. As above. Cognee has no time. The Lore Engine's time_in_window UDF is a primitive the Lore Engine leans on heavily.

3. Closed-world enforcement. Cognee's "cognitive-science-grounded ontology" is generic (ACT-R, SOAR-inspired). It doesn't have a Person node with a lifespan property. It doesn't have a Faction node with a founded property. The Lore Engine's ontology is specific to worlds, and the consistency rules are specific to world-history problems.

4. NPC knowledge scoping. The Lore Engine has query_as_npc and the v1.1 NPC-tier-gated access control. Cognee has nothing equivalent. Cognee doesn't model who knows what at all.

5. Source attribution. The Lore Engine's sources[] array and lore_verified flag on every node are more rigorous than Cognee's paragraph-level citations.

6. Multi-domain extensibility. The Lore Engine's TypeTemplate system lets a world-builder add a thieves-guild mission type without code. Cognee's "cognitive ontology" is fixed at the library level.

7. Cost. The Lore Engine's structured YAML ingestion is free (no LLM). Cognee uses an LLM at every pipeline stage.

What Cognee does better

1. Production maturity. 17,843 stars, paying customers (Cognee Cloud), real Discord, real integrations (Claude Code plugin, OpenClaw plugin). The Lore Engine has zero users.

2. LLM-provider-agnostic. Cognee works with OpenAI, Anthropic, Ollama, anything. The Lore Engine inherits Cognee's LiteLLM integration, which is good but not as polished as Cognee's native multi-provider routing.

3. Agent-native API. await cognee.remember(...) and await cognee.recall(...) are beautiful Python async APIs. The Lore Engine's MCP-tool API is HTTP/JSON-RPC. Both are fine; Cognee's is more ergonomic for direct agent use.

4. Multimodal. Cognee ingests audio, images, video, code, anything. The Lore Engine's v1.1 plans multimodal via S3 attachments, but the implementation is not done.

5. The Cognee paper benchmarks against standard QA. The paper evaluates on HotPotQA, TwoWikiMultiHop, MuSiQue. The Lore Engine has no published benchmarks. When the Lore Engine is built, the first thing it should do is run on these benchmarks to get comparable numbers.

6. Community and ecosystem. Cognee has integrations, plugins, a hosted offering, a Discord. The Lore Engine has a Gitea repo.

7. Operational tooling. Cognee has a CLI, a UI, observability hooks, multi-tenant isolation. The Lore Engine's v1 has none of this.

What neither does well

Both systems struggle with knowledge that changes over time. Cognee has no time. The Lore Engine's time model works for the data we have, but it can't predict what will happen. Neither system can answer "what events are likely in the next 50 years?" — that's a forecasting problem, not a retrieval problem.

Both systems are heavily dependent on the LLM's quality. If the LLM is bad at reading tables or bad at distinguishing speakers, both systems inherit that.

Net assessment

The Lore Engine is built on Cognee. This is the v1.2 substrate decision. Cognee provides the storage abstraction, the extraction pipeline, the embedding store, and the agent-native remember/recall/forget API. The Lore Engine adds the typed high-fantasy ontology, the time model, the consistency engine, and the TypeTemplate polymorphic extension as a Cognee data-model and tool extension.

Cognee is the substrate. The Lore Engine is the domain layer for fictional-world reasoning. The right move in v1.2 is to build the Lore Engine on top of Cognee, which is what we're doing.

The honest take

Cognee was published in May 2025. The Lore Engine design started June 2026. The Lore Engine is catching up to where Cognee already is, on the dimensions of polish, community, and production-readiness. Where the Lore Engine is ahead is on domain specificity (fictional worlds, temporal model, NPC scoping) — but those are the things Cognee could add. The Lore Engine's moat is not the architecture; it's the domain knowledge baked into the ontology and the consistency rules. Cognee could replicate the temporal model in 6 months of focused work. What Cognee can't replicate overnight is the world-builder's specific lore.

The Lore Engine's strategic bet: the world is the asset, the engine is the substrate. A world-builder with 1000 pages of lore and the Lore Engine's MCP tools has something Cognee can't give them out of the box: a system that knows fictional-world ontology. Cognee gives them a system that knows cognitive ontology. For this specific problem, the fictional-world one is the right shape.

---

Head-to-head #3: Lore Engine vs. Stanford Generative Agents

The setup

Generative Agents is a research paper, not a system you can deploy. But the idea — memory stream + reflection + LLM-driven behavior — is what makes NPCs believable. The Lore Engine is the substrate; generative agents are the behavior layer.

What the Lore Engine does better

1. Knowledge representation. Typed graph vs. memory stream. The Lore Engine can answer "who is Aldric's father?" with a single Cypher query. Generative agents would need the LLM to read the memory stream, find the relevant memory, and reason about it. The Lore Engine is faster, more reliable, and more auditable.

2. Temporal precision. time_in_window UDF vs. recency + LLM inference. The Lore Engine can answer "what was Aldric's relationship to Theron in 300 TA?" Generative agents can't separate "what Aldric remembers" from "what is true."

3. Scales to large worlds. Generative agents hit a wall at ~25 agents and a few days of memory. The Lore Engine scales to thousands of entities and centuries of history.

4. Multi-agent support without combinatorial memory streams. With 25 agents, each having a memory stream, the cross-agent memory is a real problem. The Lore Engine's shared Neo4j graph is the cross-agent state.

5. Consistency checking. Generative agents have no consistency engine. Reflections can be wrong. The Lore Engine flags anachronisms.

What generative agents do better

1. Believability. This is the killer feature. The 25-agents demo is visceral. The Lore Engine's query_as_npc is the substrate but not the implementation. To make NPCs behave believably, you need the generative agents pattern. The Lore Engine currently lacks the reflection mechanism.

2. Simplicity. Generative agents is a few hundred lines of Python. The Lore Engine is a 14-doc design with a multi-service architecture. You can ship a generative-agents demo in a weekend; you can ship the Lore Engine in 19+ days.

3. Planning and behavior synthesis. Generative agents produce plans — sequences of actions the agent will take. The Lore Engine produces facts — what is true about the world. The behavior layer is missing from the Lore Engine.

4. The "magic" of emergent behavior. Generative agents produce emergent social dynamics from simple rules. The Lore Engine produces no emergent behavior; it's a database.

5. Proven in the literature. 4,000+ citations on the original paper. Hundreds of follow-ups. The patterns are well-understood.

What neither does well

Neither handles the world-generation problem. Generative agents assume a world is given (the Smallville sandbox). The Lore Engine assumes a world is given (the YAML ingestion). IVIE generates worlds.

Neither has a strong story on prophecy, deception, unreliable narration. Generative agents' reflections are about what the agent believes, but the system doesn't model what is true in the world separately from what the agent believes. The Lore Engine's lore_verified flag is a crude version of this.

Net assessment

The Lore Engine is the substrate; generative agents are the behavior layer. They are not in competition. The Lore Engine should explicitly borrow the reflection mechanism from generative agents. Add a v2 tool reflect_on(entity, period) that summarizes an entity's recent activity into higher-level observations, the way generative agents synthesize reflections from memory streams. This is a high-value, low-cost addition.

The Lore Engine's claim is not that it does what generative agents do. The claim is that it provides the ground truth that generative agents' memory streams lack. An NPC's memory stream is a projection of the world. The world-graph is the source. Generative agents + Lore Engine = an NPC that has a memory stream backed by a verified world-graph. The two together are stronger than either alone.

This is the most actionable cross-pollination in the whole related-work survey. Build the reflection layer next.

---

Where the Lore Engine is worse — a direct list

This is the critical-thinking section. Where the design choices hurt the Lore Engine, and what we should do about it.

1. Maturity gap (worst weakness)

The Lore Engine is a design. GraphRAG, Cognee, and LightRAG are production systems with paying users, real communities, and years of iteration. The Lore Engine is months of catch-up away from being useful, and any user trying to deploy it today will find bugs, missing features, and rough edges. A user trying to deploy Cognee today will find a polished experience with documentation, integrations, and a Discord.

Mitigation: the Lore Engine's v2 plan is to integrate with Cognee rather than compete. If the Lore Engine is built as a Cognee extension, it inherits Cognee's production maturity. This is the single most important strategic move available.

2. Smaller corpus of validation

GraphRAG has Microsoft's research blog posts validating the approach on real datasets. Cognee has benchmarks (HotPotQA, TwoWikiMultiHop, MuSiQue). WikiChat has 97.3% factual accuracy. The Lore Engine has zero validation. The 5 question types from 07-reasoning-harness.md are unproven.

Mitigation: Phase 10 of the v1 roadmap (see 09-roadmap.md) is the reasoning-harness validation. The first build of the Lore Engine should run on a small hand-crafted world with 50 known facts, 10 known contradictions, 5 known anachronisms, and measure: how often does the LLM answer correctly? how often does it surface the contradiction? how often does it hallucinate? If those numbers aren't good, the design has a bug.

3. No global summarization

This is the worst technical gap. "What is the shape of this world? What are the main themes?" — the Lore Engine can't answer. GraphRAG's pre-generated community summaries are the right primitive and we don't have it.

Mitigation: borrow the community-detection + hierarchical-summarization pattern from GraphRAG. The Lore Engine can run Leiden on its Neo4j graph and pre-generate summaries; this is a v2 feature.

4. No forecasting

"What events are likely to happen in the next 50 years of the Third Age?" is a question the Lore Engine can't answer. The TKG methods (TLogic, Chain of History, TGL-LLM) can.

Mitigation: the Lore Engine is a retrieval system, not a prediction system. Adding forecasting changes the scope. v2 might add a forecast_event tool that uses a small TKG model; v1 doesn't need it.

5. No multimodal

The Lore Engine handles text and (via S3) attachments. GraphRAG/RAG-Anything handles images, tables, equations, audio. Cognee handles audio, video, code. The Lore Engine is text-only at the engine layer.

Mitigation: multimodal at the engine layer requires image/video understanding, which is a different research area. The Lore Engine can store multimodal content; it can't reason over it. A v2 might integrate a vision-language model for image-grounded questions.

6. Single-language extraction prompt

The lore-extractor's prompt is English-only. A Japanese or Spanish world-builder gets English entity names. The Lore Engine's name uniqueness constraint is locale-sensitive.

Mitigation: the extractor can be parameterized by language. The Cypher MERGE keys are still strings; the aliases[] field can hold multiple-script variants. v2.

7. No multi-tenant support

The Lore Engine is single-tenant. If a homelab wanted to host two worlds (one for each player's campaign), they'd need two separate stacks. Cognee supports multi-tenancy.

Mitigation: the v1.2 Setting + Plane model is the right primitive (replaces the v1.1 world_id namespace). Multi-tenant would just be multiple Setting values in the same Neo4j, with row-level access control. v2.

8. No user-facing UI

Cognee has a CLI + WebUI. GraphRAG has a CLI + WebUI. LightRAG has a CLI + WebUI. The Lore Engine has... a design. A world-builder can ingest YAML, but they can't see the graph, browse contradictions, or review anachronisms without using Cypher.

Mitigation: a UI is in the v2 roadmap. v1 is CLI + curl. The world-builder who can write YAML can also run cypher-shell.

9. Weaker fallback for ambiguous queries

When the LLM's question is ambiguous, the Lore Engine's lookup tool returns a disambiguation list. GraphRAG just returns the top-K chunks. Cognee returns whatever the cognitive-ontology traversal finds. The Lore Engine's behavior is better in principle but slower in practice — every ambiguous query pays the cost of a round trip.

Mitigation: cache lookup results in Redis. The active context (per-session working set) is the right place to store resolved entities.

10. The 45-tool surface is high

We hit this in 10-critique.md. The 45+ tools (8 inherited + 37 new) are well past the LLM's tool-use ceiling. GraphRAG exposes fewer tools and lets the LLM reason with them. Cognee exposes 4 operations (remember, recall, forget, improve) and lets the cognitive-ontology routing decide.

Mitigation: Phase 6 will measure tool-selection accuracy with all 45 vs. collapsed to ~15. If collapsed is better, we collapse.

---

Where the Lore Engine is better — a direct list

For balance.

1. Closed-world enforcement for fictional domains

This is the genuine, defensible advantage. No other system treats the world as a closed, typed, time-bounded, contradiction-checked graph. The Lore Engine's ontology is specific to the problem. The closest analog (Cognee's cognitive ontology) is generic.

2. First-class temporal model

The time_in_window UDF, era-tree membership, current token, and time-bounded edge properties are a real primitive that no other system has. For historical reasoning, this is non-negotiable.

3. Source attribution with confidence scoring

The sources[] array, lore_verified flag, source_confidence float, and cite tool together are more rigorous than anything in GraphRAG/Cognee/LightRAG. The Lore Engine can say "this claim has 3 sources, the most recent is from 2024, the confidence is 0.92." No other system does this.

4. Consistency engine

The 4-category consistency engine (Contradiction, Anachronism, Orphan, OntologyViolation) is unique. It surfaces problems before the LLM hallucinates around them. No other system does this at the engine level.

5. NPC knowledge scoping + tier-gated access control

The query_as_npc tool plus the v1.1 npc_knowledge template field is a real primitive for narrative games. The black-market example in 14-examples.md shows it works for access control at the data layer. No other system has this.

6. TypeTemplate polymorphism

Adding a new domain type (thieves-guild missions, war campaigns) is a YAML exercise. No other system has this. GraphRAG/Cognee/LightRAG all require code changes for new entity types.

7. Multi-world/planar support

The Setting + Plane graph model (v1.2) is the right primitive for multi-world / multi-planar campaigns. Most narrative games are single-world, but the high-fantasy genre is often multi-planar. The Lore Engine supports it; the others don't.

8. Organic bootstrap with structural-data surfacing

The :Orphan node is a primitive that says "this entity is missing structural data — the chronicles are silent on it." No other system surfaces this. The world-builder grows the world over time and the engine tells them what's incomplete.

9. Retcon preservation

The retcon Postgres table preserves history. No other system does this. Edits overwrite; retcons don't.

10. Time as a service, not a feature

The Lore Engine's design treats temporal reasoning as a primitive the rest of the system is built on. GraphRAG/Cognee/LightRAG treat time as a property of fields, not a system-wide invariant. The difference shows up in every consistency check.

---

Debatable design decisions

These are choices I made in the design that I think are right, but could be wrong. Calling them out so Kay can push back.

D1. The polymorphic DomainEntity wrapper is the right escape hatch, but it might be over-engineered

The DomainEntity + Relation + TypeTemplate system is 200 lines of Cypher plus a template-watcher service. It solves the "thieves-guild missions" problem. But:

• The world-builder has to learn YAML.
• The template validation is non-trivial.
• The dynamic tool generator adds complexity to the gateway.

Alternative: a simpler model where the world-builder writes a Go plugin that registers entity types. More code, less YAML. The trade-off is world-builder complexity vs engine simplicity.

My read: YAML is the right choice. The world-builder who can write a family_tree.yaml can write a mission.yaml. The YAML approach scales because the engine doesn't grow.

D2. Five stores is a lot

The v1.1 storage strategy is Neo4j + Postgres + pgvector + Redis + S3. That's five services. On a homelab with 58GB RAM, it's fine. On a Raspberry Pi, it's not.

Alternative: collapse Postgres and pgvector (use the same Postgres instance), drop MinIO (use local filesystem for v1), and the storage layer is 3 services.

My read: the five-store split is the right target architecture and the v1 can ship with 3. The v1.1 plan in 09-roadmap.md already says "MinIO can be deferred" and "pgvector can replace Neo4j's vector index."

D3. The 45-tool surface is high

We've discussed this. 45 tools is well past the empirical LLM ceiling.

Alternative: collapse state_at into entity_context(comprehensive=true). Collapse summarize_chain into narrate_arc(style=...). The number drops to ~20.

My read: start big, measure, collapse. The reasoning harness groups tools by function; if the LLM uses 8 of them 95% of the time, the long tail should be deprecated, not built.

D4. The consistency engine might over-flag

We've discussed this. A world with rich, overlapping history will have many temporal overlaps that aren't real contradictions.

Alternative: default severity to warn, and only surface error for things the world-builder has explicitly marked as "must be consistent" (e.g., lineage).

My read: the design already has the right knobs (severity, disable_rules[], confidence_threshold). The question is whether the world-builder uses them. Phase 7's test corpus needs to include "noisy" worlds to validate the defaults.

D5. The structured YAML path is a heavy lift for the world-builder

Writing a family_tree.yaml is real work. The world-builder has to learn the schema, follow the strict format, and update it as the world evolves.

Alternative: use a generative world-builder tool that takes a markdown chapter and proposes a YAML diff, which the world-builder approves. The LLM in the loop makes structured authoring easier.

My read: this is a v2 tool. v1 is "world-builder writes YAML." The LLM-assisted world-builder is high-leverage but needs a working v1 to be useful.

D6. The temporal model's current token is a global mutable

We've discussed this. The :Now config node is a single point of failure and a synchronization problem in multi-user scenarios.

Alternative: every session carries a world_time override. The :Now is the default but sessions can pin a different time. The active context (per-session) is the right place for this.

My read: the design has been updated in v1.1 to include world_time in the active context. The :Now is the fallback, not the only source of truth. This fix is in scope of Phase 2.

---

The strategic question

Is the Lore Engine worth building on Cognee?

Strong case for: the closest functional comparables (Cognee, LightRAG, GraphRAG) are all generic and lack a temporal model. The closest in-spirit comparable (Stanford Generative Agents) lacks a knowledge graph. The closest by use case (IVIE) generates worlds, doesn't reason over them. No system in the literature does what the Lore Engine claims to do. That's an opening. And by building on Cognee, the Lore Engine inherits a production-grade substrate for free, which lets us focus the engineering effort on the domain layer (typed ontology, time model, consistency, TypeTemplate).

Strong case against: GraphRAG has 33,779 stars and Microsoft Research. Cognee has 17,843 stars and a paying customer base. LightRAG has 36,622 stars and an HKU team. The Lore Engine is competing with funded, shipping, polished systems for the same homelab resources. The "niche" of closed-world fictional reasoning is real, but the niche may not be big enough to justify the 33-day build cost when Cognee (with a fictional-world ontology extension) could close 80% of the gap in 2 weeks.

My recommendation:

1. The substrate decision is made: Cognee. The build order is in 09-roadmap.md (Cognee-spike-first). The 1-week Cognee validation spike is the gating decision before any Lore Engine code is written. If the spike fails, the substrate decision is wrong and the v1 needs a different foundation. The 1-week validation is the cheapest insurance available.

2. Ship the time model + typed ontology in the MVP (Phases 1–3). The Lore Engine's value is the domain layer — the time model, the consistency rules, the TypeTemplate polymorphism. These are the things Cognee doesn't give you. Build them first; the substrate is solved.

3. Validate before building more. Build the MVP, ingest one small hand-crafted world, measure: can the LLM answer historical questions correctly? Does the consistency engine surface real problems? Does the TypeTemplate system work? If the answer to any of these is "no," the design has a bug and the v2 should address it. Don't build the v1.1 features on top of an unvalidated v1.

4. The most leveraged next move is a 2-day spike that stands up Cognee locally, ingests a 10-document sample world, and confirms cognee.recall("Who is Aldric?") returns something sensible. If that spike succeeds, the 33-day plan makes sense. If it doesn't, the design needs a rethink before more code is written.

The critical-thinking section of the related work is the part I most want Kay to push back on. The comparison is honest; the strategic call was debatable, but the substrate decision is now made. The right next move depends on what the spike shows.

---

The summary

The Lore Engine is a niche specialization of a solved problem (KG-RAG) with two genuinely novel primitives (temporal model, TypeTemplate polymorphism) and one strong domain story (fictional-world reasoning). It is miles behind the leading systems in polish, community, and production maturity. It is miles ahead in domain specificity, source attribution, and consistency enforcement. It is catching up to a moving target (Cognee is iterating fast) and the right move might be to integrate rather than compete.

The honest recommendation: build the spike, measure, decide.