From 4fc0e7513c5ba8df1c4b3ab3bbde0afd3a14bfec Mon Sep 17 00:00:00 2001 From: hermes Date: Wed, 24 Jun 2026 11:27:37 +0000 Subject: [PATCH] docs(wiki): entry points contract (P1) + entity page cross-link --- concepts/entry-points-contract.md | 282 ++++++++++++++++++++++++++++++ entities/damascus-orchestrator.md | 4 + 2 files changed, 286 insertions(+) create mode 100644 concepts/entry-points-contract.md diff --git a/concepts/entry-points-contract.md b/concepts/entry-points-contract.md new file mode 100644 index 0000000..f7d2828 --- /dev/null +++ b/concepts/entry-points-contract.md @@ -0,0 +1,282 @@ +--- +title: Entry Points Contract (HTTP API + MCP + UI v1) +created: 2026-06-24 +updated: 2026-06-24 +type: concept +project: damascus-orchestrator +tags: [api, mcp, ui, contract, schema, architecture] +sources: [/tmp/gemini_review_prompt.txt] +related: [damascus-orchestrator, state-resume-protocol, builder-contract, reviewer-contract, spec-refiner-contract] +--- + +# Entry Points Contract (HTTP API + MCP + UI v1) + +The durable contract that P2–P6 build against. The authoritative machine-readable +form is [`src/damascus/api_schemas.py`](../../src/damascus/api_schemas.py); if this +page and the schema disagree, fix this page — the schema is the source of truth +because FastAPI uses it for OpenAPI generation. + +## 1. System diagram + +Three compose services, all reading from the existing `db` (Postgres 16): + +``` + ┌──────────────────────────┐ + Browser ──HTTP──▶ │ damascus-ui (static) │ React/Vite build + │ /opt/damascus/ui │ mounted read-only + └────────────┬─────────────┘ + │ fetch JSON + ▼ + ┌──────────────────────────┐ port 9110 + Claude/Hermes ─MCP─▶ │ damascus-api (FastAPI) │ ◀── HTTP (loopback + (stdio) │ │ /v1/items /v1/issues │ only; token-gated + ▼ │ /v1/events /v1/stats │ on writes) + ┌─────────────────┤ /healthz /v1/cost │ + │ damascus-mcp └────────────┬─────────────┘ + │ (stdio, thin │ + │ HTTP wrapper) │ psycopg3 sync + │ ▼ + │ ┌──────────────────────────┐ + └─────────────────▶│ db (Postgres 16) │ + │ work_items │ + │ human_issues │ + │ cost_ledger │ + │ events_outbox │ + └──────────────────────────┘ +``` + +| Service | Port | Image basis | Notes | +|----------------|------|-----------------------------------|------------------------------------------------------| +| `damascus-api` | 9110 | existing damascus-orchestrator image | FastAPI; also mounts `damascus-ui` build as `/` | +| `damascus-mcp` | — | same image | stdio transport; wraps the HTTP API | +| `damascus-ui` | — | pre-built artifact | Static SPA, baked into `damascus-api` (no separate container) | + +The UI is **not** a separate `python:3.12-slim` container — it ships as a +pre-built Vite bundle mounted into the API container at `/opt/damascus/ui` and +served by FastAPI's `StaticFiles`. One container per concern; the API owns +process lifecycle, CORS, rate limit, and auth in one place. + +## 2. Endpoint catalog + +All paths are prefixed `/v1`. Read endpoints take no auth (LAN trust model, +matches existing `sidecar-status` on :9100). Write endpoints require +`Authorization: Bearer ` (see §4). + +| Method | Path | Auth | Purpose | +|--------|-----------------------------|-------|--------------------------------------------------| +| GET | `/healthz` | none | Liveness (returns `{"status":"ok"}`) | +| GET | `/v1/items` | none | List `work_items` with filters, sort, pagination | +| GET | `/v1/items/{id}` | none | Item + recent events + open issues | +| POST | `/v1/items` | token | Ingest one story (wraps `state.upsert_story`) | +| POST | `/v1/items/bulk` | token | Ingest many stories in one transaction | +| GET | `/v1/issues` | none | List `human_issues` | +| POST | `/v1/issues/{id}/answer` | token | Answer an open question | +| GET | `/v1/events` | none | Stream `events_outbox` (last N, no SSE in v1) | +| GET | `/v1/cost` | none | `cost_ledger` summary | +| GET | `/v1/stats` | none | Phase counts + recent activity | + +Write endpoints require `Authorization: Bearer ` (see §4). + +### Error model + +All errors return `application/problem+json`-shaped `{"error": "", "detail": ""}` +(matches `ErrorResponse` in the schema): + +| HTTP | `error` code | When | +|------|------------------------|------------------------------------------------------------| +| 400 | `bad_request` | Pydantic validation failure (missing field, bad enum) | +| 401 | `unauthorized` | Missing/invalid `Authorization: Bearer` on a write | +| 404 | `not_found` | Item id or issue id does not exist | +| 409 | `conflict` | Story already exists in a terminal phase (bulk ingest) | +| 422 | `unprocessable_entity` | Pydantic accepted input but server-side check failed | +| 429 | `rate_limited` | Write rate bucket exhausted for source IP | +| 500 | `internal_error` | Unhandled exception (logged with trace id) | + +### POST `/v1/items` (single ingest) + +- **Required**: `project` (1–64), `story_id` (1–128), `title` (1–255) +- **Optional**: `file_scope` (list[str], default `[]`), `priority` (0–1000, default 100), + `budget_cycles` (1–10, default 3) +- **Behavior**: wraps `state.upsert_story`. Idempotent: re-submitting the same + `(project, story_id)` returns the existing id (200) rather than creating a + duplicate (no 409 on single-ingest). +- **Errors**: 400 (validation), 401 (no token), 429 (rate limit) + +### POST `/v1/items/bulk` (bulk ingest) + +- **Required**: `items` (list of `IngestStoryRequest`, length 1–500) +- **Behavior**: one Postgres transaction. Either every story inserts or none do. +- **Errors**: 400 (validation), 401, 409 (any story in a terminal phase), 429 + +### POST `/v1/issues/{id}/answer` + +- **Required**: `answer` (1–10_000 chars) +- **Behavior**: sets `answer`, `status = 'answered'`, `answered_at = NOW()`. +- **Errors**: 401, 404 (issue not found / not open), 429 + +### GET `/v1/events` + +- Query params: `work_item_id` (optional UUID), `limit` (default 100, max 1000), + `since_id` (optional BIGINT — return events with `id > since_id` for polling) +- Returns `events` in `id ASC` order. v1 is poll-based; SSE is deferred. + +## 3. Query parameter spec + +### `GET /v1/items` + +| Param | Type | Default | Constraints | +|----------------------|--------|--------------------|----------------------------------------------| +| `project` | str | (none — all) | exact match, max 64 chars | +| `phase` | enum | (none — all) | `WorkItemPhase` | +| `priority_min` | int | 0 | `>= 0` | +| `priority_max` | int | 1000 | `>= 0`, `>= priority_min` (server-enforced) | +| `sort` | enum | `priority_asc` | `priority_asc` \| `priority_desc` \| `updated_desc` \| `attempts_desc` | +| `limit` | int | 50 | `1 <= n <= 500` | +| `offset` | int | 0 | `>= 0` | +| `open_questions_only`| bool | false | filters to items that have at least one open `human_issues` row | + +### `GET /v1/issues` + +| Param | Type | Default | Constraints | +|-----------|--------|----------|----------------------------| +| `status` | enum | (none) | `IssueStatus` | +| `project` | str | (none) | exact match | +| `limit` | int | 50 | `1 <= n <= 500` | +| `offset` | int | 0 | `>= 0` | + +### `GET /v1/events` + +| Param | Type | Default | Constraints | +|-----------------|--------|----------|------------------------------| +| `work_item_id` | str | (none) | UUID | +| `limit` | int | 100 | `1 <= n <= 1000` | +| `since_id` | int | (none) | BIGINT | + +### `GET /v1/cost` + +| Param | Type | Default | Constraints | +|-----------|------|---------|---------------------------------------------------| +| `project` | str | (none) | exact match | +| `since` | str | (none) | ISO timestamp; `recorded_at >= since` | + +## 4. Auth model + +- **Reads (GET)**: no auth. LAN trust model matches existing `sidecar-status` on + :9100 and the FastAPI services behind Traefik (`*.hermes.damascusfront.net`). + The DB itself is bound to loopback, so LAN-only attackers are the realistic + threat. +- **Writes (POST)**: `Authorization: Bearer `. Token lives + in `/root/.hermes/.env` as `DAMASCUS_API_TOKEN`. Read by the API at startup + via `os.environ`. **If the token is empty or unset the API refuses to boot** + (fail-closed; loud log line + exit 1). This is enforced before the FastAPI + app starts listening. +- **Write rate limit**: token bucket per source IP, default **30 req/min**, + configurable via `DAMASCUS_WRITE_RATE_PER_MIN`. Implemented in middleware so + every POST goes through it. Returns 429 with `Retry-After` header on + exhaustion. +- **MCP token pass-through**: the MCP server reads `DAMASCUS_API_TOKEN` from + the same env file and forwards it as `Authorization: Bearer` on every HTTP + call to the API. MCP never connects to Postgres directly. + +Threat-model rationale: an attacker on the LAN who reaches the API can read +data (matches `sidecar-status`) but cannot ingest 10k stories in 30 seconds — +they're capped at 30 writes/minute and would need the static token for any +write at all. A leaked token is rotated by editing `.env` and restarting +`damascus-api`. + +## 5. MCP tool catalog + +The MCP server is a **thin wrapper** — it does not connect to Postgres, does +not hold state, and does not implement business logic. Every tool maps to one +HTTP call. Implementation: `src/damascus/mcp_server.py` using the official +`mcp` Python SDK, stdio transport. + +| MCP tool | Args | Maps to | +|-----------------------|-----------------------------------------------------------------------|-------------------------------------------| +| `list_items` | `project?`, `phase?`, `sort?`, `limit?` | `GET /v1/items` | +| `get_item` | `id` | `GET /v1/items/{id}` | +| `list_open_questions` | `project?` | `GET /v1/issues?status=open` | +| `answer_question` | `issue_id`, `answer` | `POST /v1/issues/{id}/answer` | +| `ingest_story` | `project`, `story_id`, `title`, `file_scope?`, `priority?` | `POST /v1/items` | +| `ingest_project` | `project` | `POST /v1/items/bulk` (scans BMAD artifacts) | +| `system_status` | (none) | `GET /v1/stats` | + +Tool input schemas are derived directly from the request bodies in +`src/damascus/api_schemas.py` (Pydantic v2's `model_json_schema()` is reused +via FastAPI's OpenAPI export — no second source of truth). Tool results are +JSON-encoded `*Response` shapes with one extra `{"ok": true}` envelope the MCP +SDK requires. + +The `ingest_project` tool is the only one with server-side logic beyond +HTTP forwarding — it scans the project's BMAD planning-artifacts directory +(`/data/specs//stories/*.md`) and submits each as an `IngestStoryRequest`. +This logic lives in the API (P3) so the MCP stays a strict wrapper. + +## 6. Connection pool sizing + +`psycopg_pool.ConnectionPool(min_size=2, max_size=5)` shared across the +FastAPI threadpool (sync endpoints). Postgres default `max_connections=100`. + +Budget (Postgres `max_connections=100`): + +| Consumer | Connections | Notes | +|------------------------|-------------|-----------------------------------------| +| Orchestrator worker | 1 | one transaction per cycle | +| Scheduler | 1 | one transaction per tick | +| `damascus-api` | 5 (max) | threadpool shared; min 2 warm | +| Reserve / future | ~93 | headroom for additional workers, MCP, debug psql sessions | + +`min_size=2` keeps two idle connections warm so cold-start latency doesn't +bite on the first request. `max_size=5` is enough for ~5 concurrent reads; +the cycle ticker rate is far below that, so contention isn't expected. A +single process owns the pool — no cross-process sharing needed. + +## 7. "Self-improving" UI — v1 interpretation + +The user's phrase covers a spectrum from "telemetry-tracking UI" to +"feedback loop into the spec refiner." v1 ships **system health trends** — +the operator sees pipeline state at a glance without opening a terminal: + +- **Phase counts** as a stacked bar (live, polled every 5s). Shows the + distribution of `work_items.phase` so the operator sees bottlenecks + (e.g. 40 items in `review` = reviewer stuck or Gitea slow). +- **Open `human_issues`** count + last 5 inline (clickable → drawer). The + answer form lives in the drawer; answering transitions the issue to + `answered` and unblocks the work item. +- **Items in `blocked` phase** with their `last_verdict` and `last_feedback` + rendered as cards. The operator sees *why* items are stuck — `tests_failed` + with stack traces, `spec_ambiguous` with the prompt that confused the + builder, `rebase_conflict` with the conflicting files. +- **Cost per day for the last 7 days** as a sparkline. Cheap LLM cost + awareness without a full dashboard. + +The deeper "feedback updates `wiki_pins` to fix the spec refiner" loop is +**out of scope for v1** and tracked as a separate future task on the board. +The v1 UI does not write to `wiki_pins` and does not push spec refiner +prompts. + +## 8. Phase deliverables (P2–P6) + +- **P2 — `damascus-api` service.** FastAPI on :9110 inside the existing + damascus-orchestrator image. New `damascus serve` CLI. Connection pool, + token check, rate limit middleware, all 10 endpoints with handlers wired + to `state.*` helpers. Compose service added; existing `db` untouched. P2 + owns the contract test (`tests/contract/test_api_schemas_match_db.py`) + that round-trips the schema enums against `schema.sql`. +- **P3 — `damascus-mcp` server.** `src/damascus/mcp_server.py` using `mcp` + Python SDK, stdio transport. Seven tools, each one HTTP call. Token + pass-through. No direct Postgres access. +- **P4 — `damascus-ui` v1.** React 19 + Vite 6. Routes: `/` (dashboard), + `/items` (table), `/items/:id` (drawer). Filter/sort wired to + `/v1/items`. No ingest UI yet. Built bundle mounted into the API at + `/opt/damascus/ui`. +- **P5 — `damascus-ui` v2.** Ingest form (`/ingest`), answer form (inside + the drawer), project-grouped dashboard. All four "self-improving" widgets + from §7 wired live. +- **P6 — E2E verify.** Live cycle + UI screenshot + MCP round-trip. One + integration test that ingests via MCP, watches the item flow through + `spec → build → review → merged`, and asserts the UI reflects each + phase transition. This is the merge gate for v1. + +P2–P6 are blocked on this P1 contract merging. Do not start any of them +until the PR for this page merges into `main`. \ No newline at end of file diff --git a/entities/damascus-orchestrator.md b/entities/damascus-orchestrator.md index e6a9d92..16a4180 100644 --- a/entities/damascus-orchestrator.md +++ b/entities/damascus-orchestrator.md @@ -69,3 +69,7 @@ The system being built. A Postgres/MySQL-backed state machine that schedules BMA - [[multi-project-orchestration-plan-1]] — the design doc - [[damascus-e2e-test-session-2026-06-23]] — current test run - [[spec-refiner-gap-2026-06-23]] — known gap in the spec-refiner prompt + +## Architecture (P1 contract — read first) + +- [[entry-points-contract]] — durable contract for the v1 HTTP API (`damascus-api` on :9110), the MCP server (`damascus-mcp`, stdio), and the React UI (`damascus-ui`). Includes all endpoint shapes, MCP tool catalog, auth model (no auth on reads, `DAMASCUS_API_TOKEN` on writes, 30 req/min/IP rate limit), connection-pool sizing, and the v1 interpretation of "self-improving UI" (system-health trends only — the deeper spec-refiner feedback loop is a separate future task). P2–P6 build against this contract.