docs(slice 5T): ship the polymorphic extension + ADR 0012
Slice 5 (TypeTemplate) shipped in 5 sub-slices on lore-engine-poc: 5T.1 data model + ingest, 5T.2 template parser, 5T.3 Cypher allowlist, 5T.4 registry + dynamic tool generator, 5T.5 four example templates + end-to-end killer demo. 632 -> 712 tests. ADR 0012 records the design choices: - :DomainEntity, :Relation, :TypeTemplate as Layer 2 backbone on top of the existing GraphBackend Protocol (ADR 0011) - the Cypher-with-allowlist as the safety boundary - why startup-load + reload() over a file-watcher (no new dependency, deterministic tests, operator-controlled) - acknowledged risks: tool surface explosion (deferred collapse-to-one-tool follow-up), in-memory matcher scope. Plan doc updated to shipped status with cross-references. Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
125
docs/adr/0012-typetemplate-polymorphism.md
Normal file
125
docs/adr/0012-typetemplate-polymorphism.md
Normal file
@@ -0,0 +1,125 @@
|
||||
# TypeTemplate polymorphism: DomainEntity + TypeTemplate + allowlist
|
||||
|
||||
**Status:** accepted.
|
||||
|
||||
The Lore Engine's 36 core labels (`docs/01-ontology.md`)
|
||||
describe the *macro* structure of any world: persons,
|
||||
factions, regions, planes. That ceiling stops the engine
|
||||
from modeling *content* types a world-builder actually cares
|
||||
about — thieves-guild missions, war-campaign battles,
|
||||
black-market lots, NPC secrets. Slice 5T (TypeTemplate) is
|
||||
the resolution: the world-builder adds a new domain type by
|
||||
writing **a YAML file**. The engine reads it, validates the
|
||||
Cypher queries through an allowlist, generates MCP tools,
|
||||
and exposes them in `tools/list` — no Python change, no
|
||||
engine restart, no Neo4j migration.
|
||||
|
||||
## Layer 2 (the polymorphic backbone)
|
||||
|
||||
Three new graph labels ship in v1 alongside the 36 core
|
||||
labels:
|
||||
|
||||
- **`:DomainEntity {id, type, name, setting_id, properties,
|
||||
summary, sources, ...}`** — one node per typed instance.
|
||||
``type`` is the discriminator (``ThievesGuildMission``,
|
||||
``WarCampaign``, ...); ``properties`` is the bag of
|
||||
template-defined fields. A world can have any number of
|
||||
``type`` values; each one comes from a template YAML.
|
||||
- **`:Relation {id, from_id, to_id, type, valid_from,
|
||||
valid_until, ...}`** — the reified relation node (per ADR
|
||||
0009), used as the link from a :DomainEntity to anything
|
||||
else (a :Person, a :Faction, another :DomainEntity, ...).
|
||||
- **`:TypeTemplate {id, version, owner, spec_yaml, spec,
|
||||
status, registered_at}`** — the template itself, stored
|
||||
alongside the data so an LLM can recover the schema
|
||||
(``list_template_tools``) without re-reading the file.
|
||||
|
||||
The choice to layer these on top of the existing GraphBackend
|
||||
Protocol (ADR 0011) — rather than replace it — means both
|
||||
the in-memory backend and the Neo4j backend get
|
||||
polymorphic-instance support for free. The InMemoryGraph
|
||||
ships a tiny in-memory Cypher matcher (the one the example
|
||||
templates need); the Neo4j backend passes the
|
||||
allowlist-validated body straight to the driver.
|
||||
|
||||
## The Cypher-with-allowlist (the safety boundary)
|
||||
|
||||
Templates can ship arbitrary read-only Cypher queries.
|
||||
The body is validated through
|
||||
`templates/cypher_allowlist.py` *at registration time*:
|
||||
|
||||
- Accepted: ``MATCH``, ``OPTIONAL MATCH``, ``WHERE``,
|
||||
``RETURN``, ``ORDER BY``, ``SKIP``, ``LIMIT``, plus
|
||||
allowlisted aggregation functions (``count``,
|
||||
``coalesce``, ``min``, ``max``, ``sum``, ``avg``,
|
||||
``toLower``, ``toUpper``, ``length``, ``id``).
|
||||
- Rejected (with the offending token and line number):
|
||||
``CREATE``, ``MERGE``, ``SET``, ``DELETE``, ``DETACH``,
|
||||
``REMOVE``, ``CALL``, ``UNION``, ``WITH``, ``FOREACH``,
|
||||
``LOAD``, ``USING``, and any variable-length path
|
||||
pattern (``*``, ``*1..3``).
|
||||
- Parameter consistency is enforced: a ``$name`` referenced
|
||||
in the body must be declared in the template's
|
||||
``parameters:`` section, and vice versa. The body is
|
||||
*never* concatenated with parameter values — every
|
||||
``$name`` is bound via the driver's parameter API, so a
|
||||
value like ``"O'Brien); MERGE (n:Bad) RETURN n"`` is a
|
||||
string, not Cypher.
|
||||
|
||||
The allowlist is **the** safety boundary. Everything
|
||||
downstream (the in-memory matcher, the Neo4j driver)
|
||||
trusts the body once ``validate()`` returns.
|
||||
|
||||
## Why a startup-load + ``reload()`` rather than a file-watcher
|
||||
|
||||
The plan doc considered three options: hot-watcher,
|
||||
admin-endpoint, and startup-load + reload. We chose the
|
||||
third for the POC because:
|
||||
|
||||
- **No new dependency.** ``watchdog`` / ``inotify`` are
|
||||
not in the POC's stack. Adding a file-watcher means
|
||||
cross-platform fs events + a debounce policy + a thread
|
||||
per process.
|
||||
- **Determinism.** Tests build a registry against a temp
|
||||
dir, call ``reload()``, and assert exact contents. A
|
||||
file-watcher races the test.
|
||||
- **Operator-controlled.** Production operators don't want
|
||||
the engine silently re-reading templates on every
|
||||
``.swp`` save. ``reload()`` is an explicit call from
|
||||
the entry scripts (``--reload-templates``) or an admin
|
||||
RPC.
|
||||
- **Service-extractable later.** Per the plan doc's
|
||||
out-of-scope note, the production system has a
|
||||
``template-registry/`` Go service with a watch channel.
|
||||
The POC's in-process ``TemplateRegistry`` is the seed
|
||||
for that service's contract: ``templates: list[{id,
|
||||
spec, queries}]``, ``reload() -> count``,
|
||||
``query(template_id, qid, args) -> rows``.
|
||||
|
||||
## Risks (acknowledged)
|
||||
|
||||
- **Tool surface explosion.** Each template adds N tools.
|
||||
The plan doc defers "collapse to a single
|
||||
``query_template(type, filters)`` tool when count > 50"
|
||||
to a later slice. For now we ship the unbounded version
|
||||
and document the threshold.
|
||||
- **In-memory matcher is not a full Cypher engine.** It
|
||||
supports only the patterns the 4 example templates need
|
||||
(single-node MATCH, single-hop rel pattern, WHERE on
|
||||
top-level fields, ORDER BY on those fields, LIMIT, SKIP).
|
||||
A template that wants a broader pattern either expands
|
||||
the matcher (POC follow-up) or runs against Neo4j.
|
||||
The allowlist does not reject these patterns — the
|
||||
matcher surfaces a clear ``NotImplementedError`` at
|
||||
execution time, which the registry surfaces as a tool
|
||||
error.
|
||||
|
||||
## Cross-references
|
||||
|
||||
- `docs/11-extensibility.md` — full design
|
||||
- `docs/14-examples.md` — the 4 worked examples
|
||||
- `docs/plan/05-slice-typetemplate.md` — acceptance criteria
|
||||
- `docs/plan/graceful-growing-clarke.md` — POC plan
|
||||
- ADR 0009 — reified `:Relation` shape
|
||||
- ADR 0011 — GraphBackend Protocol (substrate)
|
||||
- `docs/adr/0008-neo4j-backend.md` — the Neo4j substrate
|
||||
@@ -1,7 +1,11 @@
|
||||
# Slice 5 — TypeTemplate Polymorphic Extension
|
||||
|
||||
**Status:** 📋 planned. The big one. This is what makes new domain
|
||||
types a YAML exercise, not a code change.
|
||||
**Status:** ✅ shipped 2026-06-19. The big one. New domain types are
|
||||
now a YAML exercise, not a code change. POC-scoped shipment in 5
|
||||
sub-slices (5T.1–5T.5); 632 → 712 tests. See
|
||||
`docs/plan/graceful-growing-clarke.md` for the sub-slice
|
||||
decomposition and `docs/adr/0012-typetemplate-polymorphism.md`
|
||||
for the design rationale.
|
||||
|
||||
## Goal
|
||||
|
||||
|
||||
Reference in New Issue
Block a user