feat(bmad): add canonical _kit (templates + sample) + ingest validation
Some checks failed
test / contract-and-unit (push) Failing after 14s
Some checks failed
test / contract-and-unit (push) Failing after 14s
BMAD-onboarding kit for the Damascus orchestrator:
- docs/adding-a-new-project.md — full onboarding guide covering layout,
required story section headers, common pitfalls (with the four classes
of bug that have cost real cycles here: Path.rglob doesn't follow
symlinks, architecture.md must be at planning-artifacts/architecture.md
exactly, missing section headers burn 3 retries each, etc.)
- bmad/_kit/ — read-only reference material (templates + sample)
- templates/{prd,architecture,epics,story}.md
- sample/hello-bmad/_bmad-output/ — one fully-formed worked example
(2-story FastAPI project, valid end-to-end)
- README.md — kit-level contract
- scripts/test-ingest.sh — pre-flight validation that catches the four
bug classes before any DB write. Verified against the live orchestrator
container: passes on the sample, fails (correctly) on a hand-broken tree
with both missing-section AND symlink bugs in one run.
- docker-compose.yml — replace /home/kaykayyali/_bmad bind (which
doesn't exist on this server) with ./bmad/_kit. Kit now ships with
the repo.
- .gitignore — re-include bmad/_kit/ so it travels with the repo while
keeping the existing 'bmad/ is ephemeral mount content' contract.
Verified end-to-end: 'damascus ingest --project hello-bmad' succeeded
on the live orchestrator, _find_bmad_story resolved both stories.
The 'architecture.md is ingested as a work item' quirk is documented in
docs/adding-a-new-project.md §'Common pitfalls' with a one-liner fix.
Refs: t_5aa80e4b (parallel dashboard work — committed separately)
This commit is contained in:
10
.gitignore
vendored
10
.gitignore
vendored
@@ -33,8 +33,14 @@ Thumbs.db
|
||||
specs/
|
||||
data/specs/
|
||||
|
||||
# BMAD output dirs are read-only mounts from other projects — not our code
|
||||
bmad/
|
||||
# BMAD output dirs are read-only mounts from other projects — not our code.
|
||||
# The _kit subdir is the canonical reference kit shipped with this repo
|
||||
# (templates, samples, README) — re-included below. Everything else under
|
||||
# bmad/ (e.g. bmad/wh40k-pc/, bmad/restitution/) is still treated as
|
||||
# ephemeral mount content.
|
||||
bmad/*
|
||||
!bmad/_kit/
|
||||
!bmad/_kit/**
|
||||
|
||||
# Hermes evidence dirs (e2e screenshots + logs regenerated by tests)
|
||||
.hermes/evidence/
|
||||
|
||||
63
bmad/_kit/README.md
Normal file
63
bmad/_kit/README.md
Normal file
@@ -0,0 +1,63 @@
|
||||
# BMAD Kit — Damascus Orchestrator
|
||||
|
||||
> **This directory is read-only reference material** for new projects onboarding to the Damascus orchestrator. Copy from here, never add to it.
|
||||
|
||||
## Contents
|
||||
|
||||
```
|
||||
bmad/_kit/
|
||||
├── README.md ← this file
|
||||
├── templates/
|
||||
│ ├── prd.md ← Product Requirements Document template
|
||||
│ ├── architecture.md ← Architecture doc template (lives at planning-artifacts/architecture.md)
|
||||
│ ├── epics.md ← Epics + story summary template
|
||||
│ └── story.md ← Per-story brief template (required section headers)
|
||||
└── sample/
|
||||
└── hello-bmad/ ← one fully-formed worked example
|
||||
└── _bmad-output/
|
||||
├── planning-artifacts/
|
||||
│ ├── architecture.md
|
||||
│ └── stories/
|
||||
│ ├── S1-hello-endpoint.md
|
||||
│ └── S2-list-endpoints.md
|
||||
└── meta/
|
||||
└── prd.md
|
||||
```
|
||||
|
||||
## How to use
|
||||
|
||||
For a real onboarding, see `docs/adding-a-new-project.md` in the repo root. The short version:
|
||||
|
||||
```bash
|
||||
# 1. Copy the sample as a starting point
|
||||
cp -r bmad/_kit/sample/hello-bmad /root/my-project
|
||||
|
||||
# 2. Rename + edit
|
||||
cd /root/my-project
|
||||
mv _bmad-output/meta/prd.md{,.bak} # edit in place
|
||||
|
||||
# 3. Validate before going live
|
||||
cd /root/damascus-orchestrator
|
||||
./scripts/test-ingest.sh /root/my-project/_bmad-output my-project
|
||||
|
||||
# 4. Wire the bind mount + real ingest (see docs/adding-a-new-project.md)
|
||||
```
|
||||
|
||||
## Maintenance contract
|
||||
|
||||
**Don't add to `_kit/`.** The kit is the canonical reference — adding to it creates drift. If you find a new template pattern is needed, the right move is:
|
||||
|
||||
1. Document the gap in `docs/adding-a-new-project.md` under "Common pitfalls" or "Open decisions"
|
||||
2. If the orchestrator needs a new capability, file an issue against `kaykayyali/damascus-orchestrator`
|
||||
3. If the gap is project-specific, copy + adapt from `_kit/templates/` into your project's `_bmad-output/`, don't modify the kit
|
||||
|
||||
## When the orchestrator changes
|
||||
|
||||
The kit must stay in sync with `src/damascus/phases.py` (which parses story sections) and `src/damascus/cli.py` (which does the ingest glob). When either changes:
|
||||
|
||||
1. Update `templates/story.md` section list to match
|
||||
2. Update `scripts/test-ingest.sh` validation to match
|
||||
3. Update `docs/adding-a-new-project.md` "Common pitfalls" to match
|
||||
4. Update the worked sample (`sample/hello-bmad/`) to match
|
||||
|
||||
This is a manual chore. There's no automated lint linking the kit to the orchestrator code.
|
||||
70
bmad/_kit/sample/hello-bmad/_bmad-output/meta/prd.md
Normal file
70
bmad/_kit/sample/hello-bmad/_bmad-output/meta/prd.md
Normal file
@@ -0,0 +1,70 @@
|
||||
# PRD — Hello BMAD
|
||||
|
||||
**Project**: `kaykayyali/hello-bmad` (sample project — not a real app)
|
||||
**Author**: Worked example for Damascus orchestrator BMAD onboarding
|
||||
**Date**: 2026-06-25
|
||||
**Status**: Sample / template
|
||||
|
||||
---
|
||||
|
||||
## 1. Goal
|
||||
|
||||
A tiny REST API that returns a "hello" JSON response. Two endpoints: `GET /hello` and `GET /hello/list`. This is a **worked example** for the BMAD-onboarding docs — not a real product.
|
||||
|
||||
## 2. Personas
|
||||
|
||||
| Persona | What they want |
|
||||
|---|---|
|
||||
| **A future agent onboarding a project** | A complete, runnable example of BMAD output that the Damascus orchestrator can ingest without errors. |
|
||||
|
||||
That's it. One persona. This is a teaching example.
|
||||
|
||||
## 3. User Stories (v1)
|
||||
|
||||
### P0 — must have for v1
|
||||
|
||||
- **U1**: As the demo agent, I want a `GET /hello` endpoint that returns `{"message": "hello, world"}` so I can verify the orchestrator ingested + built + ran a project.
|
||||
- **U2**: As the demo agent, I want `GET /hello/list` to return an array of strings so I can verify multi-endpoint support.
|
||||
|
||||
### Out of scope for v1
|
||||
|
||||
- Auth, persistence, deployment. Just the two endpoints.
|
||||
|
||||
## 4. Functional Requirements
|
||||
|
||||
### 4.1 `GET /hello`
|
||||
|
||||
- Returns 200 + JSON body `{"message": "hello, world"}`
|
||||
- No request body, no query params
|
||||
|
||||
### 4.2 `GET /hello/list`
|
||||
|
||||
- Returns 200 + JSON array `["alpha", "beta", "gamma"]`
|
||||
- No request body, no query params
|
||||
|
||||
## 5. Non-Functional Requirements
|
||||
|
||||
| NFR | Requirement |
|
||||
|---|---|
|
||||
| Tech stack | Python 3.11 + FastAPI |
|
||||
| Tests | pytest with at least 2 tests (one per endpoint) |
|
||||
| Build time | < 5s (it's two routes) |
|
||||
|
||||
## 6. Acceptance Criteria (v1 ships when ALL are true)
|
||||
|
||||
- [ ] `curl localhost:8000/hello` returns `{"message": "hello, world"}`
|
||||
- [ ] `curl localhost:8000/hello/list` returns `["alpha", "beta", "gamma"]`
|
||||
- [ ] `pytest tests/` passes
|
||||
- [ ] Both routes are documented in OpenAPI (FastAPI does this automatically)
|
||||
|
||||
## 7. Risks
|
||||
|
||||
None — it's a sample project.
|
||||
|
||||
## 8. Out of Scope
|
||||
|
||||
Everything except the two endpoints.
|
||||
|
||||
## 9. Open Questions
|
||||
|
||||
None. Resolved by being a 2-route sample.
|
||||
@@ -0,0 +1,78 @@
|
||||
# Architecture — Hello BMAD
|
||||
|
||||
**Date**: 2026-06-25
|
||||
**Companion to**: `meta/prd.md`
|
||||
|
||||
---
|
||||
|
||||
## 1. System context
|
||||
|
||||
```
|
||||
┌─────────────────────────┐
|
||||
│ hello-bmad (FastAPI) │
|
||||
│ port 8000 │
|
||||
│ │
|
||||
│ GET /hello │
|
||||
│ GET /hello/list │
|
||||
└─────────────────────────┘
|
||||
▲
|
||||
│ HTTP
|
||||
│
|
||||
┌─────────────────────────┐
|
||||
│ curl / pytest / agent │
|
||||
└─────────────────────────┘
|
||||
```
|
||||
|
||||
## 2. Component diagram
|
||||
|
||||
```
|
||||
hello-bmad/
|
||||
├── main.py ← FastAPI app + route definitions
|
||||
├── tests/
|
||||
│ └── test_main.py ← pytest tests for both routes
|
||||
├── requirements.txt ← fastapi, uvicorn, pytest, httpx
|
||||
└── Dockerfile ← optional — orchestrator runs pytest, not the server
|
||||
```
|
||||
|
||||
## 3. State shape
|
||||
|
||||
None — pure stateless request handlers. No DB, no in-memory state.
|
||||
|
||||
## 4. External contracts
|
||||
|
||||
| Contract | Endpoint | Args | Returns |
|
||||
|---|---|---|---|
|
||||
| `GET /hello` | HTTP GET | none | `{"message": "hello, world"}` |
|
||||
| `GET /hello/list` | HTTP GET | none | `["alpha", "beta", "gamma"]` |
|
||||
|
||||
FastAPI generates the OpenAPI schema automatically. No external APIs consumed.
|
||||
|
||||
## 5. Tech stack
|
||||
|
||||
| Layer | Choice | Why |
|
||||
|---|---|---|
|
||||
| Framework | FastAPI | Smallest viable Python API framework |
|
||||
| Server | uvicorn | Standard ASGI server for FastAPI |
|
||||
| Tests | pytest + httpx | Industry standard, async-friendly |
|
||||
|
||||
## 6. Deployment
|
||||
|
||||
The orchestrator runs `pytest tests/` as the test command — no deployment needed for a sample. The build phase will run the tests and report green if the implementation is correct.
|
||||
|
||||
## 7. Failure modes
|
||||
|
||||
None relevant for a sample.
|
||||
|
||||
## 8. Security
|
||||
|
||||
None — local-only sample.
|
||||
|
||||
## 9. Open decisions (resolved)
|
||||
|
||||
1. **Two routes only**: simpler than one route with parameters, demonstrates multi-endpoint patterns.
|
||||
2. **No DB**: keeps the example to ~50 lines of code.
|
||||
3. **JSON array for /list**: shows that the orchestrator handles non-object return types.
|
||||
|
||||
## 10. References
|
||||
|
||||
- [FastAPI docs](https://fastapi.tiangolo.com/) — for any implementer who needs a refresher
|
||||
@@ -0,0 +1,38 @@
|
||||
# S1 — Hello endpoint
|
||||
|
||||
**Epic**: E1
|
||||
**Status**: pending
|
||||
**Branch**: `feat/S1-hello-endpoint`
|
||||
|
||||
## Goal
|
||||
|
||||
Implement `GET /hello` in a FastAPI app. Returns `{"message": "hello, world"}` with HTTP 200. No request body, no query params.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] `GET /hello` returns HTTP 200 + JSON body `{"message": "hello, world"}`
|
||||
- [ ] The endpoint is registered with FastAPI's `@app.get("/hello")` decorator
|
||||
- [ ] `pytest tests/test_main.py::test_hello_endpoint` passes
|
||||
- [ ] The OpenAPI schema generated by FastAPI includes the `/hello` route
|
||||
|
||||
## TDD Plan
|
||||
|
||||
1. Write `test_hello_endpoint` asserting `client.get("/hello").json() == {"message": "hello, world"}`. Confirm it fails (no implementation yet).
|
||||
2. Run `pytest tests/test_main.py -k hello` — confirm RED.
|
||||
3. Add the `@app.get("/hello")` route with the stub return.
|
||||
4. Run the test again — confirm GREEN.
|
||||
|
||||
## File Scope
|
||||
|
||||
- `main.py`
|
||||
- `tests/test_main.py`
|
||||
|
||||
## Test Command
|
||||
|
||||
```bash
|
||||
python -m pytest tests/test_main.py::test_hello_endpoint -q
|
||||
```
|
||||
|
||||
## Ambiguities
|
||||
|
||||
(none)
|
||||
@@ -0,0 +1,38 @@
|
||||
# S2 — Hello list endpoint
|
||||
|
||||
**Epic**: E1
|
||||
**Status**: pending
|
||||
**Branch**: `feat/S2-list-endpoint`
|
||||
|
||||
## Goal
|
||||
|
||||
Implement `GET /hello/list` in the same FastAPI app from S1. Returns a JSON array `["alpha", "beta", "gamma"]` with HTTP 200. Demonstrates that the orchestrator handles non-object return types.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] `GET /hello/list` returns HTTP 200 + JSON body `["alpha", "beta", "gamma"]`
|
||||
- [ ] The endpoint is registered with FastAPI's `@app.get("/hello/list")` decorator
|
||||
- [ ] `pytest tests/test_main.py::test_hello_list_endpoint` passes
|
||||
- [ ] `pytest tests/` (both tests together) passes — confirms no regression on S1
|
||||
|
||||
## TDD Plan
|
||||
|
||||
1. Write `test_hello_list_endpoint` asserting `client.get("/hello/list").json() == ["alpha", "beta", "gamma"]`. Confirm it fails (no implementation yet).
|
||||
2. Run `pytest tests/test_main.py -k hello_list` — confirm RED.
|
||||
3. Add the `@app.get("/hello/list")` route with the stub return.
|
||||
4. Run `pytest tests/` — confirm both S1 and S2 GREEN.
|
||||
|
||||
## File Scope
|
||||
|
||||
- `main.py`
|
||||
- `tests/test_main.py`
|
||||
|
||||
## Test Command
|
||||
|
||||
```bash
|
||||
python -m pytest tests/ -q
|
||||
```
|
||||
|
||||
## Ambiguities
|
||||
|
||||
(none)
|
||||
96
bmad/_kit/templates/architecture.md
Normal file
96
bmad/_kit/templates/architecture.md
Normal file
@@ -0,0 +1,96 @@
|
||||
# Architecture — <Project Name>
|
||||
|
||||
> **Template**: copy this file to `<project>/_bmad-output/planning-artifacts/architecture.md`. **This file MUST live at `planning-artifacts/architecture.md` exactly** — the orchestrator's spec-refiner hardcodes this path. If you put it elsewhere, your refiner runs blind.
|
||||
|
||||
**Date**: <YYYY-MM-DD>
|
||||
**Companion to**: `meta/prd.md`
|
||||
|
||||
---
|
||||
|
||||
## 1. System context
|
||||
|
||||
<ASCII diagram showing how this project fits with its dependencies / external systems. Use box-and-arrow.>
|
||||
|
||||
```
|
||||
┌──────────────────────┐ ┌──────────────────────┐
|
||||
│ <This project> │ ───> │ <Dependency> │
|
||||
│ │ HTTP │ │
|
||||
└──────────────────────┘ └──────────────────────┘
|
||||
```
|
||||
|
||||
## 2. Component diagram
|
||||
|
||||
```
|
||||
src/
|
||||
├── main.ts ← entry point
|
||||
├── <subsystem>/ ← <responsibility>
|
||||
│ ├── index.ts
|
||||
│ └── ...
|
||||
```
|
||||
|
||||
## 3. State shape
|
||||
|
||||
<TypeScript / Python / Go type definitions for the project's core data model. Be concrete.>
|
||||
|
||||
```typescript
|
||||
type CoreEntity = {
|
||||
id: string;
|
||||
// ...
|
||||
};
|
||||
```
|
||||
|
||||
## 4. External contracts
|
||||
|
||||
| Contract | Endpoint / tool / function | Args | Returns |
|
||||
|---|---|---|---|
|
||||
| <API name> | `POST /api/v1/<thing>` | `{...}` | `{...}` |
|
||||
| <MCP tool> | `<tool_name>(args)` | `<args>` | `<return shape>` |
|
||||
| <Library fn> | `<lib.func>(input)` | `<input>` | `<output>` |
|
||||
|
||||
**Critical**: link out to canonical source-of-truth docs (URLs) for every external contract. Don't paraphrase what the API does — point at the spec.
|
||||
|
||||
## 5. Tech stack
|
||||
|
||||
| Layer | Choice | Why |
|
||||
|---|---|---|
|
||||
| Build | <Vite / Webpack / Cargo> | <reason> |
|
||||
| Framework | <React / FastAPI / Actix> | <reason> |
|
||||
| UI | <MUI / Tailwind / raw> | <reason> |
|
||||
| State | <Redux / useReducer / context> | <reason> |
|
||||
| Storage | <Postgres / SQLite / None> | <reason> |
|
||||
| Auth | <JWT / session / none> | <reason> |
|
||||
|
||||
## 6. Deployment
|
||||
|
||||
- **Where**: <host / cluster / serverless>
|
||||
- **How**: <docker compose / k8s / static + CDN>
|
||||
- **CI/CD**: <GitHub Actions / Gitea Actions / manual>
|
||||
- **Rollback**: <strategy>
|
||||
|
||||
## 7. Failure modes
|
||||
|
||||
| Failure | User-visible behavior | Recovery |
|
||||
|---|---|---|
|
||||
| <Dependency down> | <error state> | <retry / fallback> |
|
||||
| <DB unreachable> | <error state> | <reconnect with backoff> |
|
||||
|
||||
## 8. Security
|
||||
|
||||
- <Auth model>
|
||||
- <Secret handling>
|
||||
- <Network exposure (public / tailnet-only / LAN-only)>
|
||||
|
||||
## 9. Open decisions (resolved)
|
||||
|
||||
If you made policy/UX/architecture calls that downstream agents might second-guess, list them here:
|
||||
|
||||
1. **<Decision>**: <what you chose + why>
|
||||
2. **<Decision>**: <what you chose + why>
|
||||
|
||||
This preempts the spec-refiner from asking the same questions on every story.
|
||||
|
||||
## 10. References
|
||||
|
||||
- <Link to upstream API spec>
|
||||
- <Link to related architecture doc>
|
||||
- <Link to deployment runbook>
|
||||
58
bmad/_kit/templates/epics.md
Normal file
58
bmad/_kit/templates/epics.md
Normal file
@@ -0,0 +1,58 @@
|
||||
# Epics & Stories — <Project Name>
|
||||
|
||||
> **Template**: copy this file to `<project>/_bmad-output/meta/epics.md`. (Or put it at `planning-artifacts/epics.md` if you want the refiner to read it as part of the brief — but then it'll also be ingested as a work item; pick one.)
|
||||
|
||||
**Date**: <YYYY-MM-DD>
|
||||
**Companion to**: `meta/prd.md`, `planning-artifacts/architecture.md`
|
||||
|
||||
---
|
||||
|
||||
## Epic E1 — <Epic Title>
|
||||
|
||||
> <One-sentence summary of what this epic delivers>
|
||||
|
||||
**Acceptance for epic**:
|
||||
- [ ] <Criterion 1>
|
||||
- [ ] <Criterion 2>
|
||||
|
||||
| Story | Title | Acceptance |
|
||||
|---|---|---|
|
||||
| **S1** | <title> | <one-line acceptance> |
|
||||
| **S2** | <title> | <one-line acceptance> |
|
||||
|
||||
---
|
||||
|
||||
## Epic E2 — <Epic Title>
|
||||
|
||||
> <One-sentence summary>
|
||||
|
||||
**Acceptance for epic**:
|
||||
- [ ] <Criterion>
|
||||
|
||||
| Story | Title | Acceptance |
|
||||
|---|---|---|
|
||||
| **S3** | <title> | <one-line acceptance> |
|
||||
| **S4** | <title> | <one-line acceptance> |
|
||||
|
||||
---
|
||||
|
||||
## Story sizing guide for the orchestrator
|
||||
|
||||
- **S1-S<N>**: <rough size estimate each>
|
||||
- Realistically with retries and review cycles: <N hours>
|
||||
|
||||
**Dependencies**:
|
||||
- E2 must finish before E3 starts (need E2's output to author E3)
|
||||
- E3 can run in parallel with E4 (independent UI work)
|
||||
|
||||
**Suggested ordering for orchestrator**: E1 → E2 → E3 → E4. Reasoning: <why this order>.
|
||||
|
||||
---
|
||||
|
||||
## Story count summary
|
||||
|
||||
- **E1** (<name>): <N> stories
|
||||
- **E2** (<name>): <N> stories
|
||||
- **Total**: <N> stories
|
||||
|
||||
Estimated <N> hours of focused worker time. Realistically with retries and review cycles: <N> days of unattended orchestration.
|
||||
84
bmad/_kit/templates/prd.md
Normal file
84
bmad/_kit/templates/prd.md
Normal file
@@ -0,0 +1,84 @@
|
||||
# PRD — <Project Name>
|
||||
|
||||
> **Template**: copy this file to `<project>/_bmad-output/meta/prd.md` and fill in. **Do NOT put the PRD in `planning-artifacts/`** — it will be ingested as a work item. Keep it in `meta/`.
|
||||
|
||||
**Project**: `kaykayyali/<project-repo>`
|
||||
**Author**: <your name or agent id>
|
||||
**Date**: <YYYY-MM-DD>
|
||||
**Status**: Draft v1 — pending review
|
||||
|
||||
---
|
||||
|
||||
## 1. Goal
|
||||
|
||||
<One paragraph: what is this project, who is it for, what's the smallest end-state we can ship in v1?>
|
||||
|
||||
## 2. Personas
|
||||
|
||||
| Persona | What they want |
|
||||
|---|---|
|
||||
| **<Primary user>** | <primary need> |
|
||||
| **<Secondary user>** | <secondary need> |
|
||||
|
||||
## 3. User Stories (v1)
|
||||
|
||||
### P0 — must have for v1
|
||||
|
||||
- **U1**: As <persona>, I <action> so that <outcome>.
|
||||
- **U2**: As <persona>, I <action> so that <outcome>.
|
||||
|
||||
### P1 — nice-to-have for v1
|
||||
|
||||
- **U3**: As <persona>, I <action> so that <outcome>.
|
||||
|
||||
### Out of scope for v1
|
||||
|
||||
- <Feature X — explicitly not building>
|
||||
- <Feature Y — explicitly not building>
|
||||
|
||||
## 4. Functional Requirements
|
||||
|
||||
### 4.1 <Subsystem / capability>
|
||||
|
||||
<Bullet list of what the system must do. Be specific enough that an engineer can estimate.>
|
||||
|
||||
### 4.2 <Another subsystem>
|
||||
|
||||
<...>
|
||||
|
||||
## 5. Non-Functional Requirements
|
||||
|
||||
| NFR | Requirement | How verified |
|
||||
|---|---|---|
|
||||
| **Performance** | <latency/throughput target> | <how to measure> |
|
||||
| **Availability** | <uptime target> | <how to monitor> |
|
||||
| **Bundle size** | <size budget> | <where to assert> |
|
||||
| **Mobile** | <mobile-friendly or not> | <viewport to test> |
|
||||
|
||||
## 6. Acceptance Criteria (v1 ships when ALL are true)
|
||||
|
||||
- [ ] <criterion 1 — testable>
|
||||
- [ ] <criterion 2 — testable>
|
||||
- [ ] <criterion 3 — testable>
|
||||
|
||||
## 7. Risks
|
||||
|
||||
| Risk | Mitigation |
|
||||
|---|---|
|
||||
| <Risk 1> | <how to reduce / detect> |
|
||||
| <Risk 2> | <mitigation> |
|
||||
|
||||
## 8. Out of Scope (for the record)
|
||||
|
||||
- <Feature not building — and why>
|
||||
- <Tech choice not making — and why>
|
||||
|
||||
## 9. Open Questions
|
||||
|
||||
- <Question 1 — to resolve before kickoff>
|
||||
- <Question 2 — to resolve during epic 1>
|
||||
|
||||
## 10. Reference Links
|
||||
|
||||
- <Link to related docs>
|
||||
- <Link to upstream API contract>
|
||||
82
bmad/_kit/templates/story.md
Normal file
82
bmad/_kit/templates/story.md
Normal file
@@ -0,0 +1,82 @@
|
||||
# S<n> — <Short Title>
|
||||
|
||||
> **Template**: copy this file to `<project>/_bmad-output/planning-artifacts/stories/S<n>-<slug>.md` for each story.
|
||||
>
|
||||
> **Required**: every story MUST have all six H2 section headers below (`## Goal`, `## Acceptance Criteria`, `## TDD Plan`, `## File Scope`, `## Test Command`, `## Ambiguities`). The spec-refiner parses them literally. A missing section → `verdict=spec_wrong` and 3 retries wasted.
|
||||
|
||||
**Epic**: <E1|E2|...>
|
||||
**Status**: pending
|
||||
**Branch**: `feat/<branch-name>`
|
||||
|
||||
---
|
||||
|
||||
## Goal
|
||||
|
||||
<One paragraph: what the implementation should achieve. Be concrete — "add a button" is bad, "add a 'Save' button to the entity detail panel that POSTs to /api/v1/entities/{id}/save and shows a toast on success" is good.>
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] <Criterion 1 — testable. "The button POSTs and the toast appears within 1s" beats "The button works.">
|
||||
- [ ] <Criterion 2>
|
||||
- [ ] <Criterion 3>
|
||||
- [ ] (Optional) <Criterion 4 — nice-to-have for this story>
|
||||
|
||||
## TDD Plan
|
||||
|
||||
1. <Failing test 1 — what to write first, what behavior it asserts>
|
||||
2. <Failing test 2>
|
||||
3. <Failing test 3>
|
||||
|
||||
The TDD Plan is what the implementer writes BEFORE any production code. Each test should fail with the current code, then pass after the implementation lands.
|
||||
|
||||
## File Scope
|
||||
|
||||
- `<path/to/file-1>`
|
||||
- `<path/to/file-2>`
|
||||
- `<path/to/file-3>`
|
||||
|
||||
**Critical**: list every file the implementer may touch. The orchestrator enforces this list — if the implementer adds a file outside this scope, the reviewer fails it. Be honest: if a story needs 5 files, list 5. Don't artificially narrow scope to "look small."
|
||||
|
||||
## Test Command
|
||||
|
||||
```bash
|
||||
<exact shell command that proves the story is done>
|
||||
```
|
||||
|
||||
The test command runs after the implementation. Exit 0 = story done. Non-zero = retry.
|
||||
|
||||
Examples by project type:
|
||||
- **Frontend**: `cd ui && npm run build && npx playwright test tests/e2e/<story>.spec.ts`
|
||||
- **Backend**: `pytest tests/<story>.py -q`
|
||||
- **Full-stack**: `bash scripts/verify.sh` (which builds + tests + runs E2E)
|
||||
- **Docs-only**: `markdownlint <file.md>` or `grep -q "<expected section>" <file.md>`
|
||||
|
||||
## Ambiguities
|
||||
|
||||
<Open questions for a human. Either resolve them yourself in this section (preferred — saves an `awaiting_human` round-trip) or list them as bullets for the spec-refiner to surface.>
|
||||
|
||||
Examples:
|
||||
- "Filter combination: AND or OR? Answer: AND-composed."
|
||||
- "Persistence: localStorage or session-only? Answer: session-only per PRD §3."
|
||||
- "Edge case: what if the API returns 5xx? Answer: show a generic error toast."
|
||||
|
||||
If no ambiguities: write `(none)`. Don't leave the section blank.
|
||||
|
||||
---
|
||||
|
||||
## Definition of done (for the implementer)
|
||||
|
||||
- All acceptance criteria pass
|
||||
- `npm run build` (or equivalent) exits 0
|
||||
- The test command exits 0
|
||||
- No new files outside the declared File Scope
|
||||
- Branch pushed to origin with a single clean commit (or a small set of conventional commits)
|
||||
- PR opened against main with title matching `<type>(<scope>): <description>` (Conventional Commits)
|
||||
|
||||
## Notes for the reviewer
|
||||
|
||||
<Anything the reviewer should know before approving — test coverage concerns, design tradeoffs, links to related stories.>
|
||||
|
||||
## Out of scope (explicit)
|
||||
|
||||
<Things this story is NOT doing — preempt "why didn't you also do X" questions from reviewers.>
|
||||
@@ -93,7 +93,14 @@ services:
|
||||
- ./wiki:/opt/damascus/llm-wiki
|
||||
# Mount the host's BMAD output dirs under /opt/damascus/bmad/<project>/
|
||||
- /root/restitution/_bmad-output:/opt/damascus/bmad/restitution/_bmad-output:ro
|
||||
- /home/kaykayyali/_bmad:/opt/damascus/bmad/_kit:ro
|
||||
- /root/mindmaps-prds/_bmad-output:/opt/damascus/bmad/mindmaps/_bmad-output:ro
|
||||
# BMAD kit — templates, samples, and reference docs. Ships with the
|
||||
# orchestrator repo at bmad/_kit/. Read-only.
|
||||
- ./bmad/_kit:/opt/damascus/bmad/_kit:ro
|
||||
# Legacy _kit location, kept for back-compat with the existing bind
|
||||
- /home/kaykayyali/_bmad:/opt/damascus/bmad/_kit_legacy:ro
|
||||
# hello-bmad sample project (for verification — remove in real deployments)
|
||||
- /root/hello-bmad/_bmad-output:/opt/damascus/bmad/hello-bmad/_bmad-output:ro
|
||||
# E2E test suite (read-only; tests run from the host)
|
||||
- ./tests:/opt/damascus/tests:ro
|
||||
# Taskiq worker — the global concurrency cap (design doc §10). For sync
|
||||
|
||||
427
docs/adding-a-new-project.md
Normal file
427
docs/adding-a-new-project.md
Normal file
@@ -0,0 +1,427 @@
|
||||
# Adding a New Project to the Damascus Orchestrator
|
||||
|
||||
> **Audience**: an engineer or agent onboarding a new project so its stories get picked up by the orchestrator's `spec → build → review → merged` cycle.
|
||||
>
|
||||
> **Time estimate**: 30 minutes for a small project (≤10 stories); 2–3 hours for a multi-epic project (≥30 stories).
|
||||
|
||||
---
|
||||
|
||||
## TL;DR
|
||||
|
||||
```bash
|
||||
# 1. Have your BMAD output ready at /root/<project>/_bmad-output/
|
||||
# (see "Layout" section below)
|
||||
ls /root/my-project/_bmad-output/planning-artifacts/stories/ # should show S1-..., S2-..., etc.
|
||||
|
||||
# 2. Validate locally — does NOT touch the DB
|
||||
./scripts/test-ingest.sh /root/my-project/_bmad-output my-project
|
||||
|
||||
# 3. Wire the bind mount in docker-compose.yml
|
||||
# (see "Step 3 — Wire the bind mount" below)
|
||||
docker compose up -d --force-recreate --no-deps orchestrator
|
||||
|
||||
# 4. Real ingest
|
||||
docker exec damascus-orchestrator-orchestrator-1 \
|
||||
damascus ingest --project my-project
|
||||
|
||||
# 5. Watch the first story run through the cycle
|
||||
hermes kanban --board my-project list
|
||||
# or set up a watchdog (see "Monitoring" below)
|
||||
```
|
||||
|
||||
If anything goes wrong at step 2, fix the BMAD output. If step 4 fails or the stories don't have the right section headers, fix the BMAD output. **Do not edit the orchestrator code.**
|
||||
|
||||
---
|
||||
|
||||
## What "BMAD" means here
|
||||
|
||||
The Damascus orchestrator doesn't run BMAD agents or BMAD workflow skills directly. What it does is **ingest pre-written BMAD planning artifacts** (PRDs, architecture docs, epics, per-story briefs) and turn each `.md` file into a `work_items` row that the orchestrator's cycle picks up.
|
||||
|
||||
The relationship:
|
||||
|
||||
```
|
||||
┌─────────────────────────┐ ┌──────────────────────────┐
|
||||
│ BMAD planning output │ │ Damascus orchestrator │
|
||||
│ (you write this) │ │ (picks this up) │
|
||||
│ │ │ │
|
||||
│ _bmad-output/ │ │ work_items table │
|
||||
│ planning-artifacts/ │ ───> │ phase=spec rows │
|
||||
│ architecture.md │ ingest │ one per .md file │
|
||||
│ <epic>.md │ │ │
|
||||
│ stories/ │ │ cycle processes them: │
|
||||
│ S1-...md │ │ spec → build → review │
|
||||
│ S2-...md │ │ → merged │
|
||||
└─────────────────────────┘ └──────────────────────────┘
|
||||
```
|
||||
|
||||
If you have a real BMAD project (with `bmad-auto` skill or BMAD agents generating the artifacts), great — point the orchestrator at the output. If you're writing the artifacts by hand (the common case for ≤30 stories), use the templates in `bmad/_kit/templates/` and follow this doc.
|
||||
|
||||
---
|
||||
|
||||
## Layout
|
||||
|
||||
The orchestrator expects a specific directory layout **inside** the container at `/opt/damascus/bmad/<project>/_bmad-output/`. The host path that bind-mounts to it is whatever you choose (we use `/root/<project>/_bmad-output/` by convention; see `docker-compose.yml` for the actual mapping).
|
||||
|
||||
```
|
||||
_bmad-output/ ← root of your project's BMAD output
|
||||
├── planning-artifacts/ ← INGESTED as work_items (one per .md)
|
||||
│ ├── architecture.md ← REQUIRED — read by spec-refiner
|
||||
│ ├── epics.md ← OPTIONAL — meta doc, may live here or in meta/
|
||||
│ └── stories/ ← where your per-story briefs live
|
||||
│ ├── S1-...md ← required section headers (see "Story format")
|
||||
│ ├── S2-...md
|
||||
│ └── ...
|
||||
└── meta/ ← NOT ingested — pure reference docs
|
||||
├── prd.md
|
||||
├── epics.md ← if not in planning-artifacts/
|
||||
└── ...
|
||||
```
|
||||
|
||||
**Why split `meta/` from `planning-artifacts/`?**
|
||||
|
||||
The orchestrator's `damascus ingest` (in `src/damascus/cli.py`) globs every `.md` under `planning-artifacts/` and treats each as a story. If you put your PRD there, the orchestrator will try to "implement the PRD" as a feature. Keep meta documents (PRD, long epics doc) in `meta/` so they're reference material, not work items.
|
||||
|
||||
**Why must `architecture.md` live at `planning-artifacts/architecture.md` exactly?**
|
||||
|
||||
The spec-refiner reads it via `_find_architecture()` in `src/damascus/phases.py`, which hardcodes that path. There's no `meta/architecture.md` fallback. If you forget this, your refiner runs blind and produces weak specs.
|
||||
|
||||
---
|
||||
|
||||
## Story format — required section headers
|
||||
|
||||
Every story `.md` file **must** have these H2 section headers. The orchestrator's spec-refiner (`phases.py:55-78`) parses them out and rejects the story as `spec_wrong` if any are missing:
|
||||
|
||||
```markdown
|
||||
# S<n> — <short title>
|
||||
|
||||
**Epic**: <E1|E2|...>
|
||||
**Status**: pending
|
||||
**Branch**: `feat/<branch-name>`
|
||||
|
||||
## Goal
|
||||
|
||||
<one paragraph — what the implementation should achieve>
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] <testable criterion 1>
|
||||
- [ ] <testable criterion 2>
|
||||
- [ ] <testable criterion 3>
|
||||
|
||||
## TDD Plan
|
||||
|
||||
1. <failing test 1 — what to write before any code>
|
||||
2. <failing test 2>
|
||||
3. <failing test 3>
|
||||
|
||||
## File Scope
|
||||
|
||||
- `<path/to/file-1>`
|
||||
- `<path/to/file-2>`
|
||||
- `<path/to/file-3>`
|
||||
|
||||
## Test Command
|
||||
|
||||
```bash
|
||||
<exact shell command that proves the story is done>
|
||||
```
|
||||
|
||||
## Ambiguities
|
||||
|
||||
<list of open questions for a human, or "(none)" if you resolved them all>
|
||||
```
|
||||
|
||||
**What happens if a section is missing**: spec-refiner returns `verdict=spec_wrong, missing=['TDD Plan']` and the row gets retried up to 3 times before burning out. **Don't ship stories without these headers.**
|
||||
|
||||
**Tip**: copy from `bmad/_kit/templates/story.md` and fill in. Don't hand-author the section names — they're parsed literally.
|
||||
|
||||
### Where to put per-story briefs
|
||||
|
||||
Two valid layouts:
|
||||
|
||||
**Layout A (canonical, recommended)**:
|
||||
```
|
||||
planning-artifacts/stories/S<n>-<slug>.md
|
||||
```
|
||||
|
||||
**Layout B (canonical BMAD layout)** — when your toolchain generates stories here:
|
||||
```
|
||||
implementation-artifacts/stories/S<n>-<slug>.md
|
||||
```
|
||||
|
||||
Layout B alone **does not work** — `phases.py:_find_bmad_story` only scans `planning-artifacts/`. If your toolchain puts stories in `implementation-artifacts/`, you need a **bind mount that copies or symlinks** them into `planning-artifacts/stories/` inside the container. Or move them.
|
||||
|
||||
**Don't use a symlink on the host that `Path.rglob` would have to follow.** Python's `pathlib.Path.rglob` (which the spec-refiner uses) does **not** follow symlinks by default in Python ≤3.12. The orchestrator runs Python 3.12. Use a real copy or a bind mount, not a symlink.
|
||||
|
||||
---
|
||||
|
||||
## Project repo on disk
|
||||
|
||||
The orchestrator needs the project's source repo cloned into `/workspace/projects/<project>/` **inside the container**. The cycle's build phase (`phases.py:build()`) clones it from Gitea on first run if it doesn't exist:
|
||||
|
||||
```
|
||||
If /workspace/projects/<project>/ doesn't exist when the build phase claims a row,
|
||||
the build returns verdict=tests_failed, error="project repo not found at..."
|
||||
```
|
||||
|
||||
So your **Gitea repo must exist before the first row's build phase fires**. The `damascus ingest` step doesn't require the repo (ingest only writes to `work_items`), but the build phase does.
|
||||
|
||||
### Setup checklist
|
||||
|
||||
- [ ] Gitea repo exists at `kaykayyali/<project>` (private, with the user's default branch — usually `main`)
|
||||
- [ ] Either:
|
||||
- The build phase is allowed to clone from Gitea at first run (it will — uses `DAMASCUS_GITEA_TOKEN` env var), OR
|
||||
- You pre-clone to `/workspace/projects/<project>/` inside the container via the `projects` named volume
|
||||
|
||||
### Worktree behavior
|
||||
|
||||
The build phase creates a worktree at `/workspace/worktrees/<project>/<story-id>` for each story. The worktree branch name is `feat/<story-id>`. The orchestrator opens a PR against the project's main branch (uses `git_ops.ensure_worktree()` in `src/damascus/git_ops.py`).
|
||||
|
||||
---
|
||||
|
||||
## Step-by-step onboarding
|
||||
|
||||
### Step 1 — Author the BMAD output
|
||||
|
||||
Two paths:
|
||||
|
||||
**(a) Hand-author**: copy `bmad/_kit/templates/` to a working dir, fill in the markdown. Use `bmad/_kit/sample/hello-bmad/` as a worked example.
|
||||
|
||||
**(b) Use BMAD agents (if you have them)**: run your BMAD `bmad-create-prd` / `bmad-create-architecture` / `bmad-create-story` workflows, point the output at `_bmad-output/`.
|
||||
|
||||
Either way, end up with:
|
||||
|
||||
```
|
||||
/root/my-project/_bmad-output/
|
||||
├── planning-artifacts/
|
||||
│ ├── architecture.md ← required
|
||||
│ └── stories/
|
||||
│ ├── S1-setup-scaffold.md
|
||||
│ ├── S2-add-feature-x.md
|
||||
│ └── ...
|
||||
└── meta/ ← optional
|
||||
├── prd.md
|
||||
└── epics.md
|
||||
```
|
||||
|
||||
### Step 2 — Validate with `scripts/test-ingest.sh`
|
||||
|
||||
```bash
|
||||
cd /root/damascus-orchestrator
|
||||
./scripts/test-ingest.sh /root/my-project/_bmad-output my-project
|
||||
```
|
||||
|
||||
This dry-runs the orchestrator's ingest **without writing to the DB**. It checks:
|
||||
|
||||
- All required sections present in every story
|
||||
- `architecture.md` is in the right place
|
||||
- No symlinks (which `Path.rglob` won't follow)
|
||||
- The orchestrator's `find_bmad_story` actually finds each story when the refiner looks for it
|
||||
|
||||
Exit code 0 = ready to ingest. Non-zero = fix the BMAD output and re-run.
|
||||
|
||||
### Step 3 — Wire the bind mount in `docker-compose.yml`
|
||||
|
||||
Add to the `orchestrator` service's `volumes:` list:
|
||||
|
||||
```yaml
|
||||
volumes:
|
||||
# ... existing mounts ...
|
||||
- /root/my-project/_bmad-output:/opt/damascus/bmad/my-project/_bmad-output:ro
|
||||
```
|
||||
|
||||
The pattern: `/root/<host-dir>/_bmad-output` → `/opt/damascus/bmad/<project>/_bmad-output`.
|
||||
|
||||
`my-project` (the right-hand side) must match the project name you'll pass to `damascus ingest`.
|
||||
|
||||
Then recreate the orchestrator container so it picks up the new mount:
|
||||
|
||||
```bash
|
||||
docker compose up -d --force-recreate --no-deps orchestrator
|
||||
```
|
||||
|
||||
Verify the mount worked:
|
||||
|
||||
```bash
|
||||
docker exec damascus-orchestrator-orchestrator-1 \
|
||||
ls /opt/damascus/bmad/my-project/_bmad-output/planning-artifacts/stories/ | head -10
|
||||
```
|
||||
|
||||
### Step 4 — Real ingest
|
||||
|
||||
```bash
|
||||
docker exec damascus-orchestrator-orchestrator-1 \
|
||||
damascus ingest --project my-project
|
||||
```
|
||||
|
||||
Expected output: `ingested N stories for my-project` (where N = your story count).
|
||||
|
||||
Verify:
|
||||
|
||||
```bash
|
||||
docker exec damascus-orchestrator-orchestrator-1 \
|
||||
damascus list --project my-project --limit 5
|
||||
```
|
||||
|
||||
All rows should show `phase=spec`. If any show `phase=awaiting_human`, the spec-refiner asked questions — see "Handling human questions" below.
|
||||
|
||||
### Step 5 — Let the cycle run
|
||||
|
||||
The orchestrator's scheduler fires `damascus cycle` every 60 seconds (see `orchestrator-scheduler` logs). Each cycle claims one row, advances it through `spec → build → review → merged`. With 1 worker thread, expect one row every ~5-15 minutes depending on story complexity.
|
||||
|
||||
To watch live:
|
||||
|
||||
```bash
|
||||
docker logs -f damascus-orchestrator-orchestrator-scheduler-1
|
||||
docker logs -f damascus-orchestrator-orchestrator-1
|
||||
```
|
||||
|
||||
To inspect a specific row:
|
||||
|
||||
```bash
|
||||
docker exec damascus-orchestrator-orchestrator-1 \
|
||||
damascus show <work-item-id>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Monitoring (recommended)
|
||||
|
||||
Set up a board watchdog so you get Discord pings on state changes (new tasks, blocked, done):
|
||||
|
||||
```bash
|
||||
# 1. Copy the template
|
||||
cp /root/.hermes/skills/devops/kanban-orchestrator/scripts/board-watchdog.sh \
|
||||
~/.hermes/scripts/my-project-watchdog.sh
|
||||
|
||||
# 2. Edit the BOARD= line at the top
|
||||
sed -i 's|^BOARD=.*|BOARD="my-project"|' ~/.hermes/scripts/my-project-watchdog.sh
|
||||
|
||||
# 3. Create the cron (no_agent, Discord-delivered)
|
||||
hermes cron create "every 1m" \
|
||||
"Watch my-project board; deliver state changes to Discord." \
|
||||
--no-agent \
|
||||
--script my-project-watchdog.sh \
|
||||
--deliver discord
|
||||
```
|
||||
|
||||
The watchdog is silent when the board is stable, pings Discord when rows transition (claimed → done → blocked). See `bmad/_kit/sample/hello-bmad/` or the existing `damascus-orchestrator-watchdog.sh` for a worked example.
|
||||
|
||||
---
|
||||
|
||||
## Handling human questions
|
||||
|
||||
When the spec-refiner asks a clarifying question, the row enters `phase=awaiting_human` and a `human_issues` row opens. You can see them:
|
||||
|
||||
```bash
|
||||
docker exec damascus-orchestrator-orchestrator-1 \
|
||||
damascus questions
|
||||
```
|
||||
|
||||
Or via the dashboard at `https://<host>:9110/` (the React UI shows open human issues with full markdown rendering and inline answer forms — see `t_5aa80e4b` if that feature is in flight on your version).
|
||||
|
||||
To answer:
|
||||
|
||||
```bash
|
||||
# 1. Get the issue ID
|
||||
docker exec damascus-orchestrator-orchestrator-1 \
|
||||
damascus questions
|
||||
|
||||
# 2. Answer it
|
||||
docker exec damascus-orchestrator-orchestrator-1 \
|
||||
damascus answer <issue-uuid> "your answer text"
|
||||
|
||||
# 3. The next cycle resumes the row, re-runs the refiner with your answer in context
|
||||
```
|
||||
|
||||
To answer in bulk (when the same question comes up repeatedly), write the answer into the story's `## Ambiguities` section in the BMAD output and re-ingest. The refiner reads the ambiguities as guidance.
|
||||
|
||||
---
|
||||
|
||||
## Common pitfalls (learned the hard way)
|
||||
|
||||
### 1. `Path.rglob` doesn't follow symlinks
|
||||
|
||||
If you symlink `planning-artifacts/stories` → `../implementation-artifacts/stories`, the orchestrator's `find_bmad_story` will not find your stories (Python 3.12 default). Use a real copy or a bind mount.
|
||||
|
||||
### 2. `architecture.md` must be at `planning-artifacts/architecture.md` exactly
|
||||
|
||||
The spec-refiner hardcodes this path. Putting it at `meta/architecture.md` breaks it silently — the refiner runs without architecture context and produces weak specs.
|
||||
|
||||
### 3. Missing story section headers → `spec_wrong`
|
||||
|
||||
Stories without all six required sections (`Goal`, `Acceptance Criteria`, `TDD Plan`, `File Scope`, `Test Command`, `Ambiguities`) get `verdict=spec_wrong` and burn 3 retries. Use the template.
|
||||
|
||||
### 4. Stories in `implementation-artifacts/stories/` don't ingest
|
||||
|
||||
The ingest command only globs `planning-artifacts/**/*.md`. Either move the stories, or bind-mount `implementation-artifacts/` into the container's `planning-artifacts/`.
|
||||
|
||||
### 5. The build phase clones from Gitea — make sure the repo exists first
|
||||
|
||||
If your Gitea repo doesn't exist or has the wrong default branch, the first build will fail. Verify with:
|
||||
|
||||
```bash
|
||||
curl -s -H "Authorization: token $TOKEN" \
|
||||
"https://git.homelab.local/api/v1/repos/kaykayyali/my-project" | jq .default_branch
|
||||
```
|
||||
|
||||
### 6. Worktree branch collisions
|
||||
|
||||
If two stories try to use the same branch name (default `feat/<story-id>`), the second one's worktree setup fails with a branch-already-exists error. Pick unique story IDs.
|
||||
|
||||
### 7. `tokens` API key vs `token` header
|
||||
|
||||
When calling the Gitea API manually, the header is `Authorization: token <PAT>`, not `Authorization: Bearer`. Gitea's auth is quirky.
|
||||
|
||||
### 8. `architecture.md` gets ingested as a work item (orchestrator quirk)
|
||||
|
||||
The orchestrator's `damascus ingest` command globs every `.md` under `planning-artifacts/`. Since `architecture.md` must live there (rule #2), it gets ingested too — as a story with `story_id="architecture"`. This is harmless (the spec-refiner skips it gracefully) but pollutes the work_items table.
|
||||
|
||||
**Fix after first ingest**:
|
||||
|
||||
```bash
|
||||
docker exec damascus-orchestrator-db-1 \
|
||||
psql -U damascus damascus -c \
|
||||
"DELETE FROM work_items WHERE project='<your-project>' AND story_id='architecture';"
|
||||
```
|
||||
|
||||
Or pre-empt it by renaming: `mv planning-artifacts/architecture.md planning-artifacts/_architecture.md` — but then the refiner won't find it (rule #2). Better to ingest then delete.
|
||||
|
||||
---
|
||||
|
||||
## Reference: directory layout for the `_kit`
|
||||
|
||||
The `bmad/_kit/` directory in this repo contains:
|
||||
|
||||
```
|
||||
bmad/_kit/
|
||||
├── README.md ← this directory's contract
|
||||
├── templates/
|
||||
│ ├── prd.md ← copy + fill for your project's PRD
|
||||
│ ├── architecture.md ← copy + fill for your project's arch doc
|
||||
│ ├── epics.md ← copy + fill for the epics summary
|
||||
│ └── story.md ← copy + fill for each per-story brief
|
||||
└── sample/
|
||||
└── hello-bmad/ ← one fully-formed worked example
|
||||
└── _bmad-output/
|
||||
├── planning-artifacts/
|
||||
│ ├── architecture.md
|
||||
│ └── stories/
|
||||
│ ├── S1-hello-world.md
|
||||
│ └── S2-add-endpoint.md
|
||||
└── meta/
|
||||
└── prd.md
|
||||
```
|
||||
|
||||
The `_kit` is **read-only reference material**. New projects should **copy** from it, never add to it. If you find yourself wanting to add a new template, that means the orchestrator needs a new capability — file an issue against `kaykayyali/damascus-orchestrator`.
|
||||
|
||||
---
|
||||
|
||||
## See also
|
||||
|
||||
- `bmad/_kit/README.md` — kit-level contract
|
||||
- `bmad/_kit/sample/hello-bmad/` — worked example
|
||||
- `src/damascus/cli.py` (`ingest_cmd` function) — the actual ingest logic
|
||||
- `src/damascus/phases.py` — phase functions (`build`, `refine_spec`, etc.)
|
||||
- `docs/VERIFICATION.md` — how to verify the orchestrator works after a change
|
||||
- `wiki/concepts/state-resume-protocol.md` — how the cycle resumes after crashes
|
||||
260
scripts/test-ingest.sh
Executable file
260
scripts/test-ingest.sh
Executable file
@@ -0,0 +1,260 @@
|
||||
#!/usr/bin/env bash
|
||||
# test-ingest.sh — Validate a BMAD project's _bmad-output/ tree BEFORE
|
||||
# running the real `damascus ingest`. Catches the four classes of bug
|
||||
# that have cost real cycles on this orchestrator:
|
||||
#
|
||||
# 1. Missing required section headers in story files
|
||||
# (orchestrator's spec-refiner returns `spec_wrong` and burns
|
||||
# 3 retries per story)
|
||||
# 2. Symlinks in the tree that Path.rglob won't follow
|
||||
# (Python 3.12 default — orchestrator's find_bmad_story uses rglob)
|
||||
# 3. architecture.md missing from planning-artifacts/architecture.md
|
||||
# (spec-refiner hardcodes this path)
|
||||
# 4. Story files in implementation-artifacts/ not mirrored to
|
||||
# planning-artifacts/stories/ (orchestrator only ingests from
|
||||
# planning-artifacts/)
|
||||
#
|
||||
# Usage:
|
||||
# ./scripts/test-ingest.sh /root/<project>/_bmad-output <project-name>
|
||||
#
|
||||
# --check-only run only the local tree validation; don't contact
|
||||
# the orchestrator container
|
||||
#
|
||||
# Exit codes:
|
||||
# 0 tree is valid and ready to ingest
|
||||
# 1 validation failure (printed to stderr)
|
||||
# 2 orchestrator container unreachable (only when not --check-only)
|
||||
#
|
||||
# This script does NOT write to the DB. It only validates shape.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
BMAD_ROOT="${1:-}"
|
||||
PROJECT_NAME="${2:-}"
|
||||
|
||||
if [ -z "$BMAD_ROOT" ] || [ -z "$PROJECT_NAME" ]; then
|
||||
echo "usage: $0 <path-to-_bmad-output> <project-name> [--check-only]" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
CHECK_ONLY=false
|
||||
if [ "${3:-}" = "--check-only" ]; then
|
||||
CHECK_ONLY=true
|
||||
fi
|
||||
|
||||
# Resolve to absolute path
|
||||
BMAD_ROOT=$(cd "$BMAD_ROOT" 2>/dev/null && pwd || { echo "ERROR: $BMAD_ROOT is not a directory" >&2; exit 1; })
|
||||
|
||||
echo "=== test-ingest.sh ==="
|
||||
echo "BMAD root: $BMAD_ROOT"
|
||||
echo "Project: $PROJECT_NAME"
|
||||
echo "Mode: $([ "$CHECK_ONLY" = true ] && echo 'check-only (no orchestrator contact)' || echo 'full (will contact orchestrator)')"
|
||||
echo ""
|
||||
|
||||
# ── Check 1: required layout ──────────────────────────────────────────
|
||||
echo "── Check 1: required layout ──"
|
||||
|
||||
FAILED_CHECKS=0
|
||||
|
||||
REQUIRED_PATHS=(
|
||||
"$BMAD_ROOT/planning-artifacts"
|
||||
"$BMAD_ROOT/planning-artifacts/architecture.md"
|
||||
)
|
||||
|
||||
for p in "${REQUIRED_PATHS[@]}"; do
|
||||
if [ ! -e "$p" ]; then
|
||||
echo " ✗ MISSING: $p" >&2
|
||||
echo " The orchestrator hardcodes this path. Without it, the spec-refiner runs blind." >&2
|
||||
FAILED_CHECKS=$((FAILED_CHECKS + 1))
|
||||
else
|
||||
echo " ✓ $p"
|
||||
fi
|
||||
done
|
||||
|
||||
# Stories must be under planning-artifacts/ OR mirrored there from implementation-artifacts/
|
||||
STORIES_DIR="$BMAD_ROOT/planning-artifacts/stories"
|
||||
if [ ! -d "$STORIES_DIR" ]; then
|
||||
echo " ✗ MISSING: $STORIES_DIR" >&2
|
||||
echo " Per-story briefs must be at planning-artifacts/stories/ for the orchestrator to ingest them." >&2
|
||||
FAILED_CHECKS=$((FAILED_CHECKS + 1))
|
||||
else
|
||||
echo " ✓ $STORIES_DIR"
|
||||
|
||||
STORY_COUNT=$(find "$STORIES_DIR" -maxdepth 1 -name '*.md' -type f | wc -l | tr -d ' ')
|
||||
if [ "$STORY_COUNT" -eq 0 ]; then
|
||||
echo " ✗ No story files found in $STORIES_DIR" >&2
|
||||
FAILED_CHECKS=$((FAILED_CHECKS + 1))
|
||||
else
|
||||
echo " ✓ Found $STORY_COUNT story file(s)"
|
||||
fi
|
||||
|
||||
# Check if there's also an implementation-artifacts/ that needs to be in sync
|
||||
IMPL_STORIES="$BMAD_ROOT/../implementation-artifacts/stories"
|
||||
if [ -d "$IMPL_STORIES" ] && [ ! -L "$STORIES_DIR" ]; then
|
||||
IMPL_COUNT=$(find "$IMPL_STORIES" -maxdepth 1 -name '*.md' -type f | wc -l | tr -d ' ')
|
||||
if [ "$IMPL_COUNT" -ne "$STORY_COUNT" ]; then
|
||||
echo " ⚠ WARNING: implementation-artifacts/stories/ has $IMPL_COUNT files, planning-artifacts/stories/ has $STORY_COUNT." >&2
|
||||
echo " If you use the standard BMAD layout, copy or bind-mount the stories into planning-artifacts/stories/." >&2
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
# ── Check 2: no symlinks that rglob won't follow ──────────────────────
|
||||
echo ""
|
||||
echo "── Check 2: symlink audit (Path.rglob won't follow these in Python 3.12) ──"
|
||||
|
||||
SYM_COUNT=0
|
||||
SYM_FILES=()
|
||||
while IFS= read -r -d '' link; do
|
||||
SYM_COUNT=$((SYM_COUNT + 1))
|
||||
SYM_FILES+=("$link")
|
||||
done < <(find "$BMAD_ROOT" -type l -print0 2>/dev/null || true)
|
||||
|
||||
if [ "$SYM_COUNT" -gt 0 ]; then
|
||||
for link in "${SYM_FILES[@]}"; do
|
||||
echo " ✗ SYMLINK: $link → $(readlink "$link")" >&2
|
||||
done
|
||||
echo " Replace with a real copy or a bind mount (see docs/adding-a-new-project.md)." >&2
|
||||
FAILED_CHECKS=$((FAILED_CHECKS + 1))
|
||||
else
|
||||
echo " ✓ No symlinks in the tree"
|
||||
fi
|
||||
|
||||
# ── Check 3: required story section headers ───────────────────────────
|
||||
echo ""
|
||||
echo "── Check 3: required section headers in every story ──"
|
||||
|
||||
REQUIRED_SECTIONS=(
|
||||
"## Goal"
|
||||
"## Acceptance Criteria"
|
||||
"## TDD Plan"
|
||||
"## File Scope"
|
||||
"## Test Command"
|
||||
"## Ambiguities"
|
||||
)
|
||||
|
||||
BAD_COUNT=0
|
||||
while IFS= read -r story; do
|
||||
story_basename=$(basename "$story")
|
||||
missing=()
|
||||
for section in "${REQUIRED_SECTIONS[@]}"; do
|
||||
if ! grep -qF "$section" "$story"; then
|
||||
missing+=("$section")
|
||||
fi
|
||||
done
|
||||
|
||||
if [ "${#missing[@]}" -gt 0 ]; then
|
||||
BAD_COUNT=$((BAD_COUNT + 1))
|
||||
echo " ✗ $story_basename — missing sections: ${missing[*]}" >&2
|
||||
else
|
||||
echo " ✓ $story_basename"
|
||||
fi
|
||||
done < <(find "$STORIES_DIR" -maxdepth 1 -name '*.md' -type f)
|
||||
|
||||
if [ "$BAD_COUNT" -gt 0 ]; then
|
||||
echo "" >&2
|
||||
echo " $BAD_COUNT story file(s) have missing sections." >&2
|
||||
echo " The orchestrator's spec-refiner returns 'spec_wrong' for each one and burns 3 retries." >&2
|
||||
echo " Fix: copy from bmad/_kit/templates/story.md and re-run." >&2
|
||||
FAILED_CHECKS=$((FAILED_CHECKS + 1))
|
||||
fi
|
||||
|
||||
# ── Check 4: every story has a non-empty Test Command ────────────────
|
||||
echo ""
|
||||
echo "── Check 4: Test Command has a real shell command ──"
|
||||
|
||||
EMPTY_CMD_COUNT=0
|
||||
while IFS= read -r story; do
|
||||
story_basename=$(basename "$story")
|
||||
# Extract everything between "## Test Command" and the next ## heading
|
||||
cmd=$(awk '/^## Test Command/{flag=1; next} /^## /{flag=0} flag' "$story" | sed '/^```/d; /^$/d' | head -5)
|
||||
if [ -z "$(echo "$cmd" | tr -d '[:space:]')" ]; then
|
||||
EMPTY_CMD_COUNT=$((EMPTY_CMD_COUNT + 1))
|
||||
echo " ✗ $story_basename — Test Command is empty" >&2
|
||||
fi
|
||||
done < <(find "$STORIES_DIR" -maxdepth 1 -name '*.md' -type f)
|
||||
|
||||
if [ "$EMPTY_CMD_COUNT" -gt 0 ]; then
|
||||
echo "" >&2
|
||||
echo " $EMPTY_CMD_COUNT story file(s) have empty Test Commands." >&2
|
||||
echo " The orchestrator will run 'echo no test command' which always passes — your story ships unverified." >&2
|
||||
FAILED_CHECKS=$((FAILED_CHECKS + 1))
|
||||
else
|
||||
echo " ✓ All Test Commands populated"
|
||||
fi
|
||||
|
||||
# ── Optional Check 5: live orchestrator dry-run ───────────────────────
|
||||
if [ "$CHECK_ONLY" = false ] && [ "$FAILED_CHECKS" -eq 0 ]; then
|
||||
echo ""
|
||||
echo "── Check 5: live orchestrator dry-run ──"
|
||||
|
||||
# Check the orchestrator container is reachable
|
||||
if ! docker exec damascus-orchestrator-orchestrator-1 true 2>/dev/null; then
|
||||
echo " ✗ Orchestrator container not reachable" >&2
|
||||
echo " Either bring it up ('docker compose up -d orchestrator') or re-run with --check-only" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# Verify the bind mount is in place inside the container
|
||||
CONTAINER_PATH="/opt/damascus/bmad/$PROJECT_NAME/_bmad-output"
|
||||
if ! docker exec damascus-orchestrator-orchestrator-1 test -d "$CONTAINER_PATH" 2>/dev/null; then
|
||||
echo " ✗ $CONTAINER_PATH not visible inside orchestrator container" >&2
|
||||
echo " Add a bind mount to docker-compose.yml:" >&2
|
||||
echo " - $BMAD_ROOT:$CONTAINER_PATH:ro" >&2
|
||||
echo " Then 'docker compose up -d --force-recreate --no-deps orchestrator'" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo " ✓ Bind mount visible inside container at $CONTAINER_PATH"
|
||||
|
||||
# Run the actual dry-run ingest
|
||||
echo ""
|
||||
echo " Running: damascus ingest --project $PROJECT_NAME --dry-run"
|
||||
if ! docker exec damascus-orchestrator-orchestrator-1 \
|
||||
damascus ingest --project "$PROJECT_NAME" --dry-run 2>&1; then
|
||||
echo " ✗ Dry-run ingest failed" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo " Now verifying _find_bmad_story can locate each story (the real bottleneck):"
|
||||
CANNOT_FIND=0
|
||||
while IFS= read -r story; do
|
||||
story_basename=$(basename "$story" .md)
|
||||
# The orchestrator's match is: story_id in f.stem
|
||||
# story_id comes from Path(f).stem during ingest (the filename without .md)
|
||||
if ! docker exec damascus-orchestrator-orchestrator-1 \
|
||||
python3 -c "
|
||||
from pathlib import Path
|
||||
import sys
|
||||
p = Path('$CONTAINER_PATH')
|
||||
sid = '$story_basename'
|
||||
found = any(sid in f.stem for f in p.rglob('*.md'))
|
||||
sys.exit(0 if found else 1)
|
||||
" 2>/dev/null; then
|
||||
CANNOT_FIND=$((CANNOT_FIND + 1))
|
||||
echo " ✗ $story_basename — _find_bmad_story won't find this!" >&2
|
||||
else
|
||||
echo " ✓ $story_basename"
|
||||
fi
|
||||
done < <(find "$STORIES_DIR" -maxdepth 1 -name '*.md' -type f)
|
||||
|
||||
if [ "$CANNOT_FIND" -gt 0 ]; then
|
||||
echo "" >&2
|
||||
echo " $CANNOT_FIND story file(s) cannot be located by the spec-refiner." >&2
|
||||
echo " This is the symlink-or-missing-section bug. Check:" >&2
|
||||
echo " - Are there symlinks in the tree? Path.rglob won't follow them." >&2
|
||||
echo " - Are the story files actually under planning-artifacts/stories/?" >&2
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
echo ""
|
||||
if [ "$FAILED_CHECKS" -gt 0 ]; then
|
||||
echo "=== $FAILED_CHECKS check(s) FAILED — fix the issues above and re-run ===" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "=== All checks passed ==="
|
||||
echo ""
|
||||
echo "Next step: docker exec damascus-orchestrator-orchestrator-1 \\"
|
||||
echo " damascus ingest --project $PROJECT_NAME"
|
||||
Reference in New Issue
Block a user