project_name: agent-builder
# Agent Builder — Session Context

## What this project does
Design, build, and test autonomous N8N agents on server-01 sandbox before any production promotion.
First two agents: Agent Builder Agent + N8N Builder Agent.

## Scheduled work (2026-06-16, running behind — started ~6:38 PM)
1. Vision Alignment Grill-Me — Agent Builder + N8N Builder vision + testing methodology
2. Agent Builder Agent — Deploy + Test (server-01 sandbox)
3. N8N Builder Agent — Deploy + Test (server-01 sandbox)

## Architecture
- Agents run as N8N workflows on server-01 (n8n-sandbox, port 5679)
- Sandbox-first: all agents tested in sandbox before any production promotion
- server-01 sandbox stack: n8n-sandbox, postgres-sandbox, vault-sandbox, bitwarden-bridge-sandbox, vaultwarden-sandbox
- Sandbox N8N API key: prod Vault at secret/sandbox/n8n
- Sandbox reachable at 192.168.1.90

## Key decisions (set during vision grill-me — 2026-06-16)
- Agent Builder Agent: builds `claude_agent` and `script` types — Ollama (llama3.1:8b) does the building, `claude -p` is overseer/validator
- N8N Builder Agent: builds `n8n_automation` types — Ollama generates workflow JSON, imports via N8N API, assigns credentials
- automation_ideas schema changes needed: rename `description` → `task_description` (full structured spec), add `type` (n8n_automation/claude_agent/script), add `builder_status`
- New `agent_test_results` table needed in api_business DB
- Sandbox must mirror production: AppRole, Vaultwarden, bridge all configured before any agent deploys
- Promotion = user approval required after all 4 test levels pass (not auto-promote in v1)
- Dedicated backfill session needed for all 48 existing automation_ideas rows (type + task_description)
- claude -p uses SDK credits (Pro = $20/month hard limit) — use sparingly, Ollama does the heavy lifting
- Local model: llama3.1:8b already pulled on server-01 (4.9GB, fits in RTX 2060 Super 8GB VRAM)

## Testing methodology
- Four levels: Structure → Deployment → Smoke → Assertion
- LLM outputs validated on structure/side-effects only, never exact string match
- All results logged to agent_test_results table
- NTFY notification on pass and fail
- Full methodology: .claude/playbook_testing_methodology.md

## Agents
### Agent Builder Agent
- Status: pending — prereqs not complete
- Purpose: Receives automation spec from automation_ideas DB, uses Ollama to build claude_agent or script type automations, deploys to sandbox, runs automated tests, notifies user for promotion approval
- Builds: claude agents (via claude -p) and Python scripts (Docker containers)

### N8N Builder Agent
- Status: pending — prereqs not complete
- Purpose: Receives automation spec from automation_ideas DB, uses Ollama to generate N8N workflow JSON using n8n_automations playbook as context, imports to sandbox N8N via API, assigns credentials, runs automated tests
- Will be used to build: id=12 (Media Pipeline Learning), id=7 (Friday Research Session Prep)

## Related personal_projects DB rows
- id=4: N8N Workflow Builder Script (pending, weekend_block1)

## Prereq checklist (must complete before any agent deployment)
- [x] Schema: rename automation_ideas.description → task_description, add type, add builder_status, add priority
- [x] Create agent_test_results table in api_business
- [x] Sandbox Vault: set up AppRole auth method (credentials at /opt/appdata/docker/docker-compose/vault/approle/ on server-01)
- [x] Sandbox Vault: store sandbox N8N API key at secret/sandbox/n8n (key name: claude-sandbox, verified working)
- [x] Verify sandbox Bitwarden bridge ↔ Vaultwarden sandbox end-to-end (bridge on port 8080, returns [] for empty vault — correct)
- [x] Write Agent Builder Agent playbook → .claude/playbook_agent_builder_agent.md
- [x] Write N8N Builder Agent playbook → .claude/playbook_n8n_builder_agent.md
- [x] Backfill session: **COMPLETE.** All pending rows reviewed across sessions 1-3. Blocked rows verified (32, 30, 13, 27, 23, 24, 22, 2, 21, 35, 36, 55, 56, 18 — all still blocked, no changes without proper review). Admin done: id=28 → p10, id=57 keeps p9, dummy row id=16 deleted. Session 3 (2026-06-18) final batch: ids 47, 48, 31, 26, 54.
  - id=25: type→n8n_automation, status→pending (8b model lifts hardware block), full rewrite with SDK credit model, model router + cost gate + subscription monitor, model_effort_routing_log training table
  - id=17: type→n8n_automation, expanded with token_waste_patterns DB table (evolution mechanism), JSONL-based detection, NTFY claude-audit gate, fan-out plan noted
  - id=42: expanded with hybrid trigger (Claude flags + asks), B+C output (DB + wrap-up surface), human gate permanent, hard boundary (business research = id=55 only), voice note
  - id=46: type→n8n_automation, Hermes ref removed, playbook_schema.md bootstrap design, three risk tiers, id=52 detector/id=46 executor boundary, fan-out plan noted
  - id=43: Vault AppRole required (youtube-oauth-script, new policy), vault_registry entries, port 8085 (8080 reserved for bridge), dynamic Vault IP, write verification, token refresh companion
  - id=45: hard dep on id=43 (Vault auth), niche_saturation_thresholds DB table (evolution), niche_check_results table, three-signal output (saturated/viable/unclear), id=18 stage 2 integration

## New automation_ideas rows added 2026-06-18 (ids 59–67)
- id=59: Claude Audit Research Prep (n8n_automation, p12) — Saturday briefing for 12:45 PM review
- id=60: Claude Audit Topic Detector (n8n_automation, p11) — detects new topics → claude_config.audit_topics; must build first
- id=61: Claude Config Dev Work Scheduler (n8n_automation, p13) — Sunday session prep from Saturday decisions
- id=62: Monthly Security Audit (n8n_automation, p7) — first Monday of month, 7-area audit → security_audit DB
- id=63: Security Patch Backlog Handler (claude_agent, p7) — works security_audit.findings; Vault/prod always human-gate
- id=64: Business Projects Worker (claude_agent, BLOCKED p29) — 12:45–4:30 PM daily; blocked pending business_projects schema
- id=65: Personal Projects Worker (claude_agent, BLOCKED p29) — 5:30–8:00 PM daily; blocked pending personal_projects schema
- id=66: Media Pipeline Project Worker (claude_agent, BLOCKED p29) — Wednesday 5:30–8:00 PM; blocked pending id=65 + scope
- id=67: N8N Builder Agent (claude_agent, p2, ready_to_build) — builds all n8n_automation rows; deploy Monday June 22 alongside id=24

## Readiness check — COMPLETE (2026-06-18 evening)
All 8 prereq checklist items verified live on server-01:
- Vault AppRole: token acquired, N8N secret readable
- N8N sandbox API: returns 200 with real key
- Bridge /items: returns [] (correct for empty sandbox vault)
- Ollama: llama3.1:8b loaded
- agent_test_results table: confirmed correct schema + FK

## June 19 — Manual Infrastructure Day (before any agent builds)
Two manual projects only:
- **id=51 Secrets Execution Proxy** — build FastAPI script that intercepts secret access, prevents raw credential exposure in agent stdout. Must be live before agents deploy.
- **id=57 Sandbox Environment Deployment Completion** — (1) Migrate production N8N from primary server to server-01. (2) Mirror production security in sandbox: equivalent Vault AppRole policies, Bitwarden Bridge populated with test credential entries matching production structure, N8N sandbox credentials matching production names (-sandbox suffix). Goal: sandbox = production mirror, so passing sandbox tests guarantees production works.
- Grill-me required at session start before any work begins.

## June 22 (Monday) — Builder Agents (extended session)
Build, test, and push both builder agents to production. Work as long as it takes.
- id=24 Agent Builder Agent (claude_agent + script types)
- id=67 N8N Builder Agent (n8n_automation types)
- Prerequisites: id=51 live, id=57 complete (sandbox mirrors production)

## Priority queue summary (as of 2026-06-18)
- p1: id=51 (manual), id=57 (manual) — infrastructure foundation
- p2: id=24, id=67 — builder agents (built in sessions, build everything else)
- p3: id=54 NTFY Provisioner, id=58 Human Action Gate
- p4: id=25 Cost Intelligence, id=38 Credential Emergency Rollout
- p5: id=3 Secrets Rotation, id=8 Vault Token Audit, id=52 Session Wrap-up
- p6: id=37 N8N Log Scanner, id=40 Coolify UUID Monitor
- p7: id=62 Monthly Security Audit, id=63 Security Patch Handler
- p8–p17: non-blocked pending/ready items
- p20–p29: all blocked items (builder agents skip these)

## Update instructions
Update at the end of every agent-builder session. Keep agent status, key decisions, and prereq checklist current.