Files
lore-engine-poc-v3/docker-compose.yml
Lore Engine Dev 4176b95f21 slice 5.8: docker-compose neo4j service + integration test
Three docker-gated tests for the full Neo4j compose stack:

  * test_compose_neo4j_profile_healthy: docker compose
    --profile neo4j up -d brings neo4j + lore-engine-ingest
    + lore-engine-mcp-neo4j to a healthy state within 60s.
  * test_compose_neo4j_was_true_at_round_trip: was_true_at
    through the Neo4j-backed MCP server returns the same
    answer as the pickle-backed server for a known fact
    (Roland Raventhorne / House Raventhorne / 3rd_age.year_345
    → was_true: true).
  * test_compose_neo4j_down_cleans_volumes: docker compose
    --profile neo4j down -v removes the neo4j_data volume.

docker-compose.yml changes:

  * New neo4j:5 service with NEO4J_AUTH=none, loopback
    HTTP + Bolt ports (17474/17687 by default to avoid
    conflict with a developer's manual neo4j on the standard
    7474/7687 ports), 1GiB mem_limit, pids_limit, healthcheck
    via wget on the HTTP root.
  * New lore-engine-ingest service (profile neo4j) that
    runs scripts/01_ingest.py --skip-cognee --write-neo4j
    after Neo4j is healthy. One-shot; no restart policy.
  * The pickle-backed lore-engine-mcp service moved onto
    the pickle profile (so it doesn't conflict on the
    same host port when the neo4j profile is active).
  * New lore-engine-mcp-neo4j service (profile neo4j)
    that depends on both neo4j (service_healthy) and
    lore-engine-ingest (service_completed_successfully).
    Same hardening as the pickle service: cap_drop ALL,
    no-new-privileges, mem_limit 512m, read_only rootfs,
    tmpfs /tmp.
  * Named volume neo4j_data for the Neo4j store.

Profile split (pickle | neo4j) keeps the two stacks from
colliding on the same host port when both are activated.
Run with docker compose --profile pickle up -d for the
default or --profile neo4j up -d for the production
graph substrate.

Slice 11.4 test update:

  * tests/test_mcp/test_dockerfile.py test_docker_compose_up_and_round_trip
    now uses --profile pickle so the pickle service
    activates only.

Pre-prod hardening noted in compose yml: NEO4J_AUTH=none
is loopback-only; switch to a username/password and update
LORE_NEO4J_URI before exposing beyond loopback. Tracked in
docs/plan/05-slice-neo4j-backend.md.

Suite: 629 -> 632 passed (+3 compose-neo4j tests, all 559
baseline + 50 Neo4j + consistency + ingest + backend-switch
+ compose-neo4j tests preserved). The plan's 632 final-test
target is reached.
2026-06-18 23:42:08 -04:00

159 lines
5.9 KiB
YAML

services:
# Slice 5.8: Neo4j 5 GraphBackend substrate. The lore-engine-mcp-neo4j
# service connects to this via LORE_GRAPH_BACKEND=neo4j +
# LORE_NEO4J_URI=bolt://neo4j:7687.
#
# NEO4J_AUTH=none is loopback-only; **before production**, switch
# to a username/password and update LORE_NEO4J_URI accordingly.
# The slice 5 plan (docs/plan/05-slice-neo4j-backend.md) tracks
# this as a pre-prod hardening item.
neo4j:
image: neo4j:5
profiles: ["neo4j"]
restart: unless-stopped
environment:
NEO4J_AUTH: "none"
# Disable the bundled APOC plugin (community image has it but
# we don't depend on it; reduces memory + startup time).
NEO4J_PLUGINS: "[]"
ports:
# HTTP + Bolt, loopback only. Same rationale as the MCP port.
# Defaults are non-standard ports (17474/17687) so they don't
# conflict with a developer's existing manual neo4j container
# on the standard 7474/7687 ports. Override via
# NEO4J_HTTP_PORT / NEO4J_BOLT_PORT env vars if needed.
- "127.0.0.1:${NEO4J_HTTP_PORT:-17474}:7474"
- "127.0.0.1:${NEO4J_BOLT_PORT:-17687}:7687"
volumes:
- neo4j_data:/data
mem_limit: 1g
pids_limit: 256
healthcheck:
# Neo4j exposes / on the HTTP port when ready. wget is more
# portable across neo4j:5 minor versions. The first 30s after
# container start can be quiet while Neo4j initializes the
# store.
test: ["CMD-SHELL", "wget -q -O - http://127.0.0.1:7474/ >/dev/null 2>&1 || exit 1"]
interval: 10s
timeout: 5s
retries: 6
start_period: 30s
# Slice 5.8: one-shot ingest job that mirrors the codex into
# Neo4j after the database is healthy. Runs to completion and
# exits 0; the MCP server (in the neo4j profile) waits on this
# via depends_on (service_completed_successfully).
lore-engine-ingest:
build: .
image: lore-engine-mcp:${TAG:-slice11}
profiles: ["neo4j"]
depends_on:
neo4j:
condition: service_healthy
environment:
# Ingest writes to the Neo4j container at the compose network
# name. The MCP server reads from the same URI later.
LORE_NEO4J_URI: "bolt://neo4j:7687"
command:
- python
- scripts/01_ingest.py
- --skip-cognee
- --write-neo4j
# Ingest is short-lived — no restart policy, no healthcheck.
restart: "no"
# Pickle profile: pickle-backed MCP server (slice 11 default).
# Read tools + write tools run against the .graph.pkl baked
# into the image at build time. Run with:
# docker compose --profile pickle up -d
lore-engine-mcp:
build: .
image: lore-engine-mcp:${TAG:-slice11}
profiles: ["pickle"]
restart: unless-stopped
# Bind the host port to loopback only. The MCP server has no auth, so
# exposing it on 0.0.0.0 would let anyone on the LAN mutate the
# graph (12 write tools are exposed). If you need LAN reachability,
# add a TLS-terminating reverse proxy (Caddy/Traefik) + bearer token
# in the handler first.
ports:
- "127.0.0.1:${LORE_HTTP_PORT:-8765}:8765"
environment:
LORE_HTTP_HOST: "0.0.0.0"
LORE_HTTP_PORT: "8765"
# LORE_GRAPH_PATH: "/data/.graph.pkl" # uncomment + add volume below to override
# volumes:
# # Optional: mount a fresh graph without rebuilding the image.
# # Build via `python3 scripts/01_ingest.py --out data/.graph.pkl` first.
# - ./lore_engine_poc/data:/data:ro
# Defense in depth: drop all capabilities (the server doesn't need any
# beyond what it inherits for binding >=1024), forbid new privileges,
# cap memory + PIDs, and mark the rootfs read-only.
cap_drop:
- ALL
security_opt:
- no-new-privileges:true
mem_limit: 512m
pids_limit: 256
read_only: true
tmpfs:
- /tmp:size=64m,mode=1777
healthcheck:
# Same path an MCP client would use. Mirrors the Dockerfile HEALTHCHECK.
test:
- "CMD"
- "python"
- "-c"
- "import json, urllib.request; r = urllib.request.urlopen(urllib.request.Request('http://127.0.0.1:8765/mcp', method='POST', data=json.dumps({'jsonrpc':'2.0','id':1,'method':'initialize','params':{}}).encode(), headers={'Content-Type':'application/json','Accept':'application/json'}), timeout=3); assert json.loads(r.read())['result']['protocolVersion'] == '2024-11-05'"
interval: 30s
timeout: 5s
retries: 3
start_period: 5s
# Slice 5.8: Neo4j-backed MCP server. Same image as
# ``lore-engine-mcp``, but selected via the ``neo4j`` profile.
# The depends_on waits for both Neo4j to be healthy AND the
# one-shot ingest job to complete successfully, so the MCP
# server never reads from an empty Neo4j.
#
# Run with: ``docker compose --profile neo4j up -d``
lore-engine-mcp-neo4j:
build: .
image: lore-engine-mcp:${TAG:-slice11}
profiles: ["neo4j"]
depends_on:
neo4j:
condition: service_healthy
lore-engine-ingest:
condition: service_completed_successfully
restart: unless-stopped
ports:
- "127.0.0.1:${LORE_HTTP_PORT:-8765}:8765"
environment:
LORE_HTTP_HOST: "0.0.0.0"
LORE_HTTP_PORT: "8765"
LORE_GRAPH_BACKEND: "neo4j"
LORE_NEO4J_URI: "bolt://neo4j:7687"
cap_drop:
- ALL
security_opt:
- no-new-privileges:true
mem_limit: 512m
pids_limit: 256
read_only: true
tmpfs:
- /tmp:size=64m,mode=1777
healthcheck:
test:
- "CMD"
- "python"
- "-c"
- "import json, urllib.request; r = urllib.request.urlopen(urllib.request.Request('http://127.0.0.1:8765/mcp', method='POST', data=json.dumps({'jsonrpc':'2.0','id':1,'method':'initialize','params':{}}).encode(), headers={'Content-Type':'application/json','Accept':'application/json'}), timeout=3); assert json.loads(r.read())['result']['protocolVersion'] == '2024-11-05'"
interval: 30s
timeout: 5s
retries: 3
start_period: 5s
volumes:
neo4j_data: