damascus-frontier/PLAN.md

Damascus Frontier — Multi-Agent Startup Swarm

For Hermes: Use subagent-driven-development to implement this plan task-by-task. All code stored in Gitea at git.homelab.local/kaykayyali/damascus-frontier

Goal: Build a self-organizing swarm of AI agents that operate like a small startup — generating business ideas, validating product-market fit, and building/selling products autonomously. Kay is the CEO, giving high-level direction. Agents self-coordinate.

Architecture: Each agent is an independent Python process in its own Docker container. Agents communicate via Redis pub/sub (message bus). Each agent has a system prompt defining its role and personality. Agents use Hermes's delegate_task mechanism via the LiteLLM proxy for inference. Shared filesystem for reports and code outputs.

Tech Stack: Python 3.12, Redis (message bus), FastAPI (management API), Docker Compose, LiteLLM (model routing), Gitea API (code storage), SQLite (state/context)

v1 Team:

• CEO (Kay — human in the loop, provides vision/direction)
• VP (orchestrator — distributes work, manages team)
• Product Manager (validates ideas, writes PRDs)
• Sales Rep (go-to-market strategy, pricing, customer acquisition plans)
• Full Stack Engineer (builds the products)

---

Architecture Diagram

┌─────────────────────────────────────────────────────┐
│                    Kay (CEO / Human)                 │
│          Talks to system via CLI / web UI            │
└───────────────────────┬─────────────────────────────┘
                        │ Redis pub/sub or HTTP
                        ▼
┌─────────────────────────────────────────────────────┐
│              VP Agent (orchestrator)                 │
│  - Receives CEO directives                          │
│  - Delegates to PM → Sales → Engineer               │
│  - Coordinates multi-agent workflows                │
│  - Reports progress back to CEO                     │
└───────┬──────────────┬──────────────┬───────────────┘
        │              │              │
        ▼              ▼              ▼
┌──────────┐  ┌────────────┐  ┌──────────────────┐
│  PM Agent │  │Sales Agent │  │Engineer Agent    │
│           │  │            │  │                  │
│- Research │  │- Pricing   │  │- Write code      │
│- Validate │  │- GTM plans │  │- Deploy to Gitea │
│- Write PRD│  │- Customer  │  │- Test & iterate  │
│           │  │  outreach  │  │                  │
└──────────┘  └────────────┘  └──────────────────┘
        │              │              │
        └──────────────┼──────────────┘
                       │
                       ▼
            ┌──────────────────┐
            │  Shared Resources │
            │  - Gitea (code)   │
            │  - /home/kaykayyali│
            │    /damascusfrontier│
            │    (reports/plans) │
            │  - Redis (comm)    │
            │  - SQLite (state)  │
            └──────────────────┘

Communication Protocol

Agents communicate via Redis pub/sub channels:

damascus:ceo    → CEO directives / human input
damascus:vp     → VP coordination messages
damascus:pm     → Product Manager assignments & responses
damascus:sales  → Sales assignments & responses
damascus:eng    → Engineer assignments & responses
damascus:status → Status updates / progress reports
damascus:system → System-level events (startup, shutdown, errors)

Message format:

{
  "id": "msg_001",
  "from": "vp",
  "to": "pm",
  "type": "assignment",
  "parent": "msg_000",  // reference to CEO directive
  "content": "Research market for SaaS uptime monitoring...",
  "context": {...},
  "timestamp": "2026-05-22T10:00:00Z"
}

Agent Architecture (Per Container)

Each agent runs:

1. Persona loader — reads persona from shared config
2. Message listener — subscribes to Redis channels
3. LLM inference — calls LiteLLM proxy (same as Hermes config)
4. Tool executor — limited toolset per role:
   • Terminal (for Engineer, PM)
   • File I/O (all — write reports to shared volume)
   • Web search (PM, Sales)
   • Gitea API (Engineer — push code)
   • Redis publish (all — respond to other agents)
5. State manager — SQLite per-agent for context/memory

Tools Per Role

Tool	VP	PM	Sales	Engineer
Redis publish (communicate)	✅	✅	✅	✅
File read/write (reports/)	✅	✅	✅	✅
Terminal (shell)	❌	✅	❌	✅
Web search/research	✅	✅	✅	❌
Gitea API (push code, create repo)	✅	❌	❌	✅
delegate_task (spawn sub-tasks)	✅	✅	✅	✅
memory (persistent context)	✅	✅	✅	✅

---

Task 1: Project Scaffold & Docker Compose Infrastructure

Objective: Create the project directory, Docker Compose setup, Redis, shared volumes, and base configuration.

Files:

• Create: /root/damascus-frontier/docker-compose.yml
• Create: /root/damascus-frontier/.env
• Create: /root/damascus-frontier/config.yaml
• Create: /root/damascus-frontier/redis.conf
• Create: /root/damascus-frontier/shared/ (empty dir, mounted volume)
• Create: /root/damascus-frontier/agents/ (empty dir)

Step 1: Create directory structure

mkdir -p /root/damascus-frontier/{agents,shared/{reports,code,logs},config}

Step 2: Write docker-compose.yml

version: '3.8'

services:
  redis:
    image: redis:7-alpine
    container_name: damascus-redis
    command: redis-server /usr/local/etc/redis/redis.conf
    volumes:
      - ./redis.conf:/usr/local/etc/redis/redis.conf:ro
      - redis-data:/data
    networks:
      - damascus-net
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      timeout: 3s
      retries: 5

  vp:
    build:
      context: ./agents
      dockerfile: Dockerfile
    container_name: damascus-vp
    environment:
      - AGENT_ROLE=vp
      - AGENT_NAME="Alex Chen"
      - REDIS_HOST=redis
      - LITELLM_URL=${LITELLM_URL:-http://litellm:4000/v1}
      - LITELLM_MODEL=${LITELLM_MODEL:-ollama-cloud-pro}
    volumes:
      - ./agents/personas:/app/personas:ro
      - ./shared:/app/shared
      - ./config.yaml:/app/config.yaml:ro
      - agent-vp-data:/app/data
    depends_on:
      redis:
        condition: service_healthy
    networks:
      - damascus-net
    restart: unless-stopped

  pm:
    build:
      context: ./agents
      dockerfile: Dockerfile
    container_name: damascus-pm
    environment:
      - AGENT_ROLE=pm
      - AGENT_NAME="Sarah Okafor"
      - REDIS_HOST=redis
      - LITELLM_URL=${LITELLM_URL:-http://litellm:4000/v1}
      - LITELLM_MODEL=${LITELLM_MODEL:-ollama-cloud-pro}
    volumes:
      - ./agents/personas:/app/personas:ro
      - ./shared:/app/shared
      - ./config.yaml:/app/config.yaml:ro
      - agent-pm-data:/app/data
    depends_on:
      redis:
        condition: service_healthy
    networks:
      - damascus-net
    restart: unless-stopped

  sales:
    build:
      context: ./agents
      dockerfile: Dockerfile
    container_name: damascus-sales
    environment:
      - AGENT_ROLE=sales
      - AGENT_NAME="Marcus Rivera"
      - REDIS_HOST=redis
      - LITELLM_URL=${LITELLM_URL:-http://litellm:4000/v1}
      - LITELLM_MODEL=${LITELLM_MODEL:-ollama-cloud-pro}
    volumes:
      - ./agents/personas:/app/personas:ro
      - ./shared:/app/shared
      - ./config.yaml:/app/config.yaml:ro
      - agent-sales-data:/app/data
    depends_on:
      redis:
        condition: service_healthy
    networks:
      - damascus-net
    restart: unless-stopped

  engineer:
    build:
      context: ./agents
      dockerfile: Dockerfile
    container_name: damascus-engineer
    environment:
      - AGENT_ROLE=engineer
      - AGENT_NAME="Jordan Kim"
      - REDIS_HOST=redis
      - LITELLM_URL=${LITELLM_URL:-http://litellm:4000/v1}
      - LITELLM_MODEL=${LITELLM_MODEL:-ollama-cloud-pro}
    volumes:
      - ./agents/personas:/app/personas:ro
      - ./shared:/app/shared
      - ./config.yaml:/app/config.yaml:ro
      - agent-eng-data:/app/data
    depends_on:
      redis:
        condition: service_healthy
    networks:
      - damascus-net
    restart: unless-stopped

  # Web dashboard (future)
  # dashboard:
  #   build: ./dashboard
  #   container_name: damascus-dashboard
  #   ports:
  #     - "5180:3000"
  #   networks:
  #     - damascus-net

networks:
  damascus-net:
    driver: bridge

volumes:
  redis-data:
  agent-vp-data:
  agent-pm-data:
  agent-sales-data:
  agent-eng-data:

Step 3: Write .env

# LLM Configuration (reuse your LiteLLM proxy)
LITELLM_URL=http://litellm:4000/v1
LITELLM_MODEL=ollama-cloud-pro

# Gitea configuration
GITEA_URL=https://git.homelab.local
GITEA_TOKEN=885a9202a1bc8231f1eb9f22e6edb978b20a4345
GITEA_USER=kaykayyali

# Report directory (on host)
REPORT_DIR=/home/kaykayyali/damascusfrontier

Step 4: Write config.yaml

# Damascus Frontier — Agent Configuration

litellm:
  base_url: "${LITELLM_URL}"
  model: "${LITELLM_MODEL}"
  api_key: ""  # LiteLLM proxy doesn't require key if on same network

gitea:
  base_url: "${GITEA_URL}"
  token: "${GITEA_TOKEN}"
  owner: "${GITEA_USER}"
  ssl_verify: false  # self-signed cert

redis:
  host: "redis"
  port: 6379
  channels:
    ceo: "damascus:ceo"
    vp: "damascus:vp"
    pm: "damascus:pm"
    sales: "damascus:sales"
    eng: "damascus:engineer"
    status: "damascus:status"
    system: "damascus:system"

output:
  reports_dir: "/app/shared/reports"
  code_dir: "/app/shared/code"

team:
  ceo:
    name: "Kay Kayyali"
    role: "CEO — human-in-the-loop, provides vision and final decisions"
  vp:
    name: "Alex Chen"
    role: "VP of Operations — orchestrates team, delegates work, ensures execution"
  pm:
    name: "Sarah Okafor"
    role: "Product Manager — researches markets, validates ideas, writes PRDs"
  sales:
    name: "Marcus Rivera"
    role: "Sales Lead — develops GTM strategies, pricing, customer acquisition plans"
  engineer:
    name: "Jordan Kim"
    role: "Full Stack Engineer — builds products, writes code, deploys to Gitea"

workflow:
  idea_generation_count: 3
  target_monthly_revenue: [300, 600]
  phases:
    - generate_ideas
    - validate_market
    - create_business_plan
    - build_mvp
    - go_to_market

Step 5: Write redis.conf

# Redis configuration for Damascus Frontier
maxmemory 256mb
maxmemory-policy allkeys-lru
save 900 1
save 300 10
save 60 10000
appendonly yes

Verification:

cd /root/damascus-frontier
docker compose config  # validate compose file
ls -la shared/ agents/ config/  # verify structure

---

Task 2: Agent Base Class (shared library)

Objective: Create the shared Python library that all agents inherit from. Includes Redis communication, LLM inference via LiteLLM, persona loading, and state management.

Files:

• Create: agents/requirements.txt
• Create: agents/Dockerfile
• Create: agents/base_agent.py
• Create: agents/personas/vp.txt
• Create: agents/personas/pm.txt
• Create: agents/personas/sales.txt
• Create: agents/personas/engineer.txt
• Create: agents/main.py

Step 1: Write requirements.txt

openai>=1.55.0
redis>=5.2.0
pyyaml>=6.0
httpx>=0.28.0
python-dotenv>=1.0.0

Step 2: Write Dockerfile

FROM python:3.12-slim

WORKDIR /app

RUN apt-get update && apt-get install -y --no-install-recommends \
    curl git ca-certificates \
    && rm -rf /var/lib/apt/lists/*

# For Gitea self-signed cert
ENV GIT_SSL_NO_VERIFY=1
ENV GITEA_SSL_VERIFY=false

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY base_agent.py .
COPY main.py .
COPY personas/ /app/personas/

CMD ["python", "main.py"]

Step 3: Write base_agent.py

"""Base agent class for Damascus Frontier multi-agent system."""

import json
import os
import time
import yaml
import redis
import logging
from pathlib import Path
from datetime import datetime, timezone
from typing import Optional, Callable
from dataclasses import dataclass, field
from openai import OpenAI

logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s [%(name)s] %(levelname)s: %(message)s'
)

@dataclass
class Message:
    """Message passed between agents via Redis."""
    id: str
    from_agent: str
    to_agent: str
    msg_type: str
    parent_id: Optional[str]
    content: str
    context: dict = field(default_factory=dict)
    timestamp: str = field(default_factory=lambda: datetime.now(timezone.utc).isoformat())

    def to_dict(self) -> dict:
        return {
            "id": self.id,
            "from": self.from_agent,
            "to": self.to_agent,
            "type": self.msg_type,
            "parent": self.parent_id,
            "content": self.content,
            "context": self.context,
            "timestamp": self.timestamp
        }

    @classmethod
    def from_dict(cls, d: dict) -> "Message":
        return cls(
            id=d["id"],
            from_agent=d["from"],
            to_agent=d["to"],
            msg_type=d["type"],
            parent_id=d.get("parent"),
            content=d["content"],
            context=d.get("context", {}),
            timestamp=d.get("timestamp", "")
        )


class BaseAgent:
    """Base agent with Redis communication, LLM inference, and shared tools."""

    def __init__(self, role: str, name: str):
        self.role = role
        self.name = name
        self.agent_id = f"{role}-{name.lower().replace(' ', '-')}"
        self.logger = logging.getLogger(f"agent.{role}")

        # Load config
        self.config = self._load_config()

        # Redis connection
        redis_cfg = self.config.get("redis", {})
        self.redis = redis.Redis(
            host=redis_cfg.get("host", "redis"),
            port=redis_cfg.get("port", 6379),
            decode_responses=True
        )

        # LLM client
        litellm_cfg = self.config.get("litellm", {})
        self.llm = OpenAI(
            base_url=litellm_cfg.get("base_url", "http://litellm:4000/v1"),
            api_key=litellm_cfg.get("api_key", "not-needed"),
        )
        self.model = litellm_cfg.get("model", "ollama-cloud-pro")

        # Load persona
        self.persona = self._load_persona()

        # State directory
        self.data_dir = Path("/app/data")
        self.data_dir.mkdir(exist_ok=True)
        self.context_file = self.data_dir / "context.json"
        self.context = self._load_context()

        # Message counter
        self._msg_counter = 0

        self.logger.info(f"Agent {self.agent_id} initialized | model={self.model}")

    def _load_config(self) -> dict:
        config_path = Path("/app/config.yaml")
        if config_path.exists():
            with open(config_path) as f:
                return yaml.safe_load(f)
        return {}

    def _load_persona(self) -> str:
        persona_path = Path(f"/app/personas/{self.role}.txt")
        if persona_path.exists():
            return persona_path.read_text().strip()
        self.logger.warning(f"No persona file at {persona_path}")
        return f"You are {self.name}, the {self.role} at Damascus Frontier."

    def _load_context(self) -> dict:
        if self.context_file.exists():
            return json.loads(self.context_file.read_text())
        return {"messages": [], "projects": [], "decisions": []}

    def _save_context(self):
        self.context_file.write_text(json.dumps(self.context, indent=2))

    def _next_msg_id(self) -> str:
        self._msg_counter += 1
        return f"{self.role}_{self._msg_counter:06d}"

    # ─── Redis Communication ─────────────────────────────────

    @property
    def inbox_channel(self) -> str:
        channels = self.config.get("redis", {}).get("channels", {})
        return channels.get(self.role, f"damascus:{self.role}")

    def publish(self, channel: str, msg: Message):
        """Send a message to a channel."""
        self.redis.publish(channel, json.dumps(msg.to_dict()))
        self.logger.info(f"→ {channel} | {msg.msg_type}: {msg.content[:80]}...")

    def send_to(self, target_role: str, msg_type: str, content: str,
                parent_id: Optional[str] = None, context: dict = None):
        """Send a message to a specific agent."""
        channels = self.config.get("redis", {}).get("channels", {})
        channel = channels.get(target_role, f"damascus:{target_role}")
        msg = Message(
            id=self._next_msg_id(),
            from_agent=self.role,
            to_agent=target_role,
            msg_type=msg_type,
            parent_id=parent_id,
            content=content,
            context=context or {}
        )
        self.publish(channel, msg)
        return msg

    def send_status(self, content: str, context: dict = None):
        """Send a status update to the status channel."""
        channels = self.config.get("redis", {}).get("channels", {})
        channel = channels.get("status", "damascus:status")
        msg = Message(
            id=self._next_msg_id(),
            from_agent=self.role,
            to_agent="all",
            msg_type="status",
            parent_id=None,
            content=content,
            context=context or {}
        )
        self.publish(channel, msg)
        return msg

    def listen(self, handler: Callable[[Message], None], channel: Optional[str] = None):
        """Subscribe and listen for messages. Blocks until interrupted."""
        sub = self.redis.pubsub()
        ch = channel or self.inbox_channel
        sub.subscribe(ch)
        self.logger.info(f"👂 Listening on {ch}")

        try:
            for raw in sub.listen():
                if raw["type"] != "message":
                    continue
                try:
                    data = json.loads(raw["data"])
                    msg = Message.from_dict(data)
                    self.logger.info(f"← {msg.from_agent} | {msg.msg_type}: {msg.content[:80]}...")
                    handler(msg)
                except Exception as e:
                    self.logger.error(f"Failed to process message: {e}", exc_info=True)
        except KeyboardInterrupt:
            self.logger.info("Shutting down listener")
        finally:
            sub.close()

    # ─── LLM Inference ───────────────────────────────────────

    def think(self, prompt: str, system: Optional[str] = None,
              temperature: float = 0.7, max_tokens: int = 2048) -> str:
        """Call the LLM with system + user prompt."""
        messages = []
        if system:
            messages.append({"role": "system", "content": system})
        messages.append({"role": "user", "content": prompt})

        try:
            resp = self.llm.chat.completions.create(
                model=self.model,
                messages=messages,
                temperature=temperature,
                max_tokens=max_tokens,
            )
            return resp.choices[0].message.content or ""
        except Exception as e:
            self.logger.error(f"LLM call failed: {e}")
            return f"[ERROR: {e}]"

    def think_with_context(self, prompt: str, context_messages: list = None,
                           temperature: float = 0.7, max_tokens: int = 2048) -> str:
        """Call LLM with persona + context history + new prompt."""
        messages = [{"role": "system", "content": self.persona}]

        if context_messages:
            # Include recent context (last 10 messages max to stay within limits)
            for ctx in context_messages[-10:]:
                role = ctx.get("role", "user")
                content = ctx.get("content", "")
                messages.append({"role": role, "content": content})

        messages.append({"role": "user", "content": prompt})

        try:
            resp = self.llm.chat.completions.create(
                model=self.model,
                messages=messages,
                temperature=temperature,
                max_tokens=max_tokens,
            )
            return resp.choices[0].message.content or ""
        except Exception as e:
            self.logger.error(f"LLM call failed: {e}")
            return f"[ERROR: {e}]"

    # ─── File Operations ─────────────────────────────────────

    def write_report(self, filename: str, content: str, category: str = "reports"):
        """Write a report to the shared volume."""
        path = Path(f"/app/shared/{category}/{filename}")
        path.parent.mkdir(parents=True, exist_ok=True)
        path.write_text(content)
        self.logger.info(f"📄 Wrote {path}")
        return str(path)

    def read_report(self, filename: str, category: str = "reports") -> str:
        """Read a report from the shared volume."""
        path = Path(f"/app/shared/{category}/{filename}")
        if path.exists():
            return path.read_text()
        return ""

    # ─── Context Management ──────────────────────────────────

    def add_to_context(self, entry: dict):
        """Add an entry to the agent's persistent context."""
        self.context.setdefault("messages", []).append(entry)
        # Keep last 100 messages
        if len(self.context["messages"]) > 100:
            self.context["messages"] = self.context["messages"][-100:]
        self._save_context()

    def get_recent_context(self, n: int = 20) -> list:
        """Get the last n context messages."""
        msgs = self.context.get("messages", [])
        return msgs[-n:] if len(msgs) > n else msgs

    # ─── Lifecycle ───────────────────────────────────────────

    def startup(self):
        """Called when the agent starts. Override in subclasses."""
        self.send_status(f"🟢 {self.name} ({self.role}) is online and ready.")
        self.logger.info(f"Agent {self.agent_id} startup complete")

    def run(self):
        """Main loop — override in subclasses."""
        self.startup()
        self.logger.info(f"Agent {self.agent_id} entering main loop")

Verification:

cd /root/damascus-frontier/agents
python3 -c "from base_agent import BaseAgent, Message; print('BaseAgent OK')"

---

Task 3: Agent Personas & main.py Entrypoint

Objective: Write the persona files for each agent and the main.py entrypoint that starts the right agent.

Files:

• Modify: agents/main.py (full implementation)
• Write: agents/personas/vp.txt
• Write: agents/personas/pm.txt
• Write: agents/personas/sales.txt
• Write: agents/personas/engineer.txt

---

Task 4: VP Agent Implementation

Objective: Implement the VP orchestrator agent. The VP receives CEO directives, breaks them into work items, delegates to PM/Sales/Engineer, and coordinates the workflow.

Files:

• Create: agents/vp_agent.py

Task 5: PM Agent Implementation

Objective: Implement the Product Manager agent. Researches markets, validates ideas, writes PRDs.

Files:

• Create: agents/pm_agent.py

Task 6: Sales Agent Implementation

Objective: Implement the Sales agent. Creates GTM strategies, pricing models, customer acquisition plans.

Files:

• Create: agents/sales_agent.py

Task 7: Engineer Agent Implementation

Objective: Implement the Full Stack Engineer agent. Builds products, writes code, pushes to Gitea.

Files:

• Create: agents/engineer_agent.py

Task 8: CEO CLI Interface & E2E Test

Objective: Create the CLI tool Kay uses to interact with the CEO agent, plus end-to-end test.

Files:

• Create: ceo_cli.py
• Create: test_e2e.sh

---

Task 9: Gitea Integration & Report Pipeline

Objective: Wire up Gitea API for code storage and ensure reports land in /home/kaykayyali/damascusfrontier.

Task 10: Docker Build & Deploy

Objective: Build all Docker images, deploy the swarm, and verify inter-agent communication.