claude-projects/agent-builder/.claude/playbook_agent_builder_agent.md

# Playbook: Agent Builder Agent

## Purpose
Builds `claude_agent` and `script` type automations from the `automation_ideas` table. Uses Ollama (llama3.1:8b) as the primary code generator and `claude -p` as the overseer/validator. Deploys to sandbox, runs all 4 test levels, notifies user for promotion approval.

## Trigger
Scheduled or manual. Queries `automation_ideas` for the next row where:
- `type IN ('claude_agent', 'script')`
- `status = 'ready_to_build'`
- `builder_status = 'not_started'`
- Ordered by `priority ASC NULLS LAST, id ASC`

Only processes one automation per run.

## Infrastructure
- Runs on: server-01 (n8n-sandbox, port 5679)
- Ollama endpoint: http://localhost:11434 (server-01 local)
- Model: llama3.1:8b
- Claude overseer: `claude -p` (non-interactive, SDK credits — use sparingly)
- Vault: sandbox AppRole at /opt/appdata/docker/docker-compose/vault/approle/
- Database: production api_business (read automation_ideas, write agent_test_results)
- NTFY: production NTFY instance for notifications

## Step-by-Step

### Step 1 — Claim the automation
```sql
UPDATE automation_ideas
SET builder_status = 'queued'
WHERE id = <selected_id> AND builder_status = 'not_started';
```
If 0 rows updated: another builder claimed it — stop, notify, exit.

### Step 2 — Build the prompt for Ollama
Construct a generation prompt using all available fields from the automation row:
- `name`: what the automation is called
- `task_description`: full structured spec — this is the primary instruction
- `type`: claude_agent or script
- `infrastructure_requirement`: what infra it needs access to

Prompt structure:
```
You are an expert automation engineer. Build a {type} automation with the following specification.

Name: {name}
Infrastructure: {infrastructure_requirement}

Specification:
{task_description}

Requirements:
- If type is claude_agent: output a complete shell-executable claude -p command with full system prompt and all logic. The agent must be self-contained.
- If type is script: output a complete Python script. Include a Dockerfile if the script has dependencies beyond stdlib.
- Output ONLY the code. No explanation, no markdown fences, no commentary.
- The code must handle its own error cases and log to stdout.
- Secrets must be fetched from Vault via AppRole — never hardcoded. AppRole credentials at /opt/appdata/docker/docker-compose/vault/approle/role-id and secret-id.
```

### Step 3 — Generate with Ollama
```
POST http://localhost:11434/api/generate
{
  "model": "llama3.1:8b",
  "prompt": "<constructed prompt>",
  "stream": false
}
```
Set builder_status = 'building' before calling.

If Ollama call fails or times out (>120s): set builder_status = 'failed', log error, notify via NTFY, stop.

### Step 4 — Overseer validation with claude -p
Pass the generated code to `claude -p` for structural review. Keep the prompt minimal to conserve SDK credits:

```bash
claude -p "Review this {type} automation code for the following only:
1. Does it correctly fetch secrets from Vault via AppRole (never hardcoded)?
2. Are there any obvious syntax errors or missing imports?
3. Does the logic match this spec summary: {name} — {task_description[:200]}

Respond with: PASS or FAIL, then one sentence explaining why.
Do not rewrite the code."
```

If FAIL: log claude's reason, set builder_status = 'failed', notify via NTFY with the failure reason, stop.
If PASS: proceed.

### Step 5 — Deploy to sandbox
**For `script` type:**
1. Write the generated code to a temp directory on server-01
2. If a Dockerfile was generated, build the image: `docker build -t agent-{id}-{slug} .`
3. Run a test container: `docker run --rm agent-{id}-{slug}` (dry run, no side effects)

**For `claude_agent` type:**
1. Write the generated claude -p command to a shell script
2. Make it executable
3. Run it once with `--dry-run` flag if supported, or with a test input that produces no side effects

If deployment fails: set builder_status = 'failed', log error, notify via NTFY, stop.

### Step 6 — Run 4-level automated tests
Run each level in order. Stop and fail if any level fails. Log every result to `agent_test_results`.

**Level 1 — Structure**
Validate the generated artifact:
- For scripts: `python3 -m py_compile script.py` — must exit 0
- For claude agents: verify the shell script is syntactically valid bash
- For Dockerfiles: `docker build --check` if available, else verify FROM and key directives exist
- Insert result: `INSERT INTO agent_test_results (automation_id, test_level, status, execution_log) VALUES ({id}, 1, 'pass'/'fail', '{log}')`

**Level 2 — Deployment**
- Verify the artifact can be deployed cleanly (no missing dependencies, image builds successfully, script runs without import errors)
- Must complete without crashing
- Insert result to agent_test_results

**Level 3 — Smoke**
- Execute the automation with minimal/test inputs
- Must run to completion without an unhandled exception or non-zero exit
- Insert result to agent_test_results

**Level 4 — Assertion**
- Verify the correct side effect occurred (not string matching — check the actual system state)
- Examples: a file was created, a DB row was written, an API call returned 200, a container is running
- Insert result to agent_test_results

### Step 7 — Notify user for promotion approval
If all 4 levels pass:
1. Set builder_status = 'awaiting_approval'
2. Send NTFY notification:
   ```
   Title: Agent Ready for Promotion — {name}
   Body: All 4 test levels passed in sandbox. Automation id={id} ({type}) is ready for production promotion. Reply to approve or reject.
   ```

User must explicitly approve before any production deployment. No auto-promotion in v1.

### Step 8 — On approval
Set builder_status = 'approved', then 'deployed' after production deployment completes.
Update automation_ideas status = 'deployed'.

## Error handling
- Any unhandled exception: set builder_status = 'failed', log to agent_test_results with test_level=0 and status='fail', send NTFY alert
- Always release the claim (reset builder_status to 'not_started') if failing before Step 3 so another run can retry
- After Step 3: leave as 'failed' — requires manual review before retry

## NTFY notification patterns
- Build started: `[Agent Builder] Building {name} (id={id}, type={type})`
- Overseer FAIL: `[Agent Builder] FAIL — Overseer rejected {name}: {reason}`
- Test level fail: `[Agent Builder] FAIL — {name} failed Level {n}: {error}`
- Ready for approval: `[Agent Builder] READY — {name} passed all tests, awaiting your approval`
- Unhandled error: `[Agent Builder] ERROR — {name}: {exception}`

## SDK credit budget
`claude -p` is called once per automation (Step 4 only). Keep the overseer prompt under 500 tokens. Do not call claude -p for retries or debugging — only for the initial validation pass.