e0ed41ffeb
All prereqs confirmed live on server-01. Full automation_ideas priority reorder (p1–p17 non-blocked, p20–p29 blocked). Added id=67 N8N Builder Agent. id=57 expanded to full production mirror scope. June 19: manual infrastructure day (id=51 + id=57). June 22: builder agents extended session. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
8.5 KiB
8.5 KiB
project_name: agent-builder
Agent Builder — Session Context
What this project does
Design, build, and test autonomous N8N agents on server-01 sandbox before any production promotion. First two agents: Agent Builder Agent + N8N Builder Agent.
Scheduled work (2026-06-16, running behind — started ~6:38 PM)
- Vision Alignment Grill-Me — Agent Builder + N8N Builder vision + testing methodology
- Agent Builder Agent — Deploy + Test (server-01 sandbox)
- N8N Builder Agent — Deploy + Test (server-01 sandbox)
Architecture
- Agents run as N8N workflows on server-01 (n8n-sandbox, port 5679)
- Sandbox-first: all agents tested in sandbox before any production promotion
- server-01 sandbox stack: n8n-sandbox, postgres-sandbox, vault-sandbox, bitwarden-bridge-sandbox, vaultwarden-sandbox
- Sandbox N8N API key: prod Vault at secret/sandbox/n8n
- Sandbox reachable at 192.168.1.90
Key decisions (set during vision grill-me — 2026-06-16)
- Agent Builder Agent: builds
claude_agentandscripttypes — Ollama (llama3.1:8b) does the building,claude -pis overseer/validator - N8N Builder Agent: builds
n8n_automationtypes — Ollama generates workflow JSON, imports via N8N API, assigns credentials - automation_ideas schema changes needed: rename
description→task_description(full structured spec), addtype(n8n_automation/claude_agent/script), addbuilder_status - New
agent_test_resultstable needed in api_business DB - Sandbox must mirror production: AppRole, Vaultwarden, bridge all configured before any agent deploys
- Promotion = user approval required after all 4 test levels pass (not auto-promote in v1)
- Dedicated backfill session needed for all 48 existing automation_ideas rows (type + task_description)
- claude -p uses SDK credits (Pro = $20/month hard limit) — use sparingly, Ollama does the heavy lifting
- Local model: llama3.1:8b already pulled on server-01 (4.9GB, fits in RTX 2060 Super 8GB VRAM)
Testing methodology
- Four levels: Structure → Deployment → Smoke → Assertion
- LLM outputs validated on structure/side-effects only, never exact string match
- All results logged to agent_test_results table
- NTFY notification on pass and fail
- Full methodology: .claude/playbook_testing_methodology.md
Agents
Agent Builder Agent
- Status: pending — prereqs not complete
- Purpose: Receives automation spec from automation_ideas DB, uses Ollama to build claude_agent or script type automations, deploys to sandbox, runs automated tests, notifies user for promotion approval
- Builds: claude agents (via claude -p) and Python scripts (Docker containers)
N8N Builder Agent
- Status: pending — prereqs not complete
- Purpose: Receives automation spec from automation_ideas DB, uses Ollama to generate N8N workflow JSON using n8n_automations playbook as context, imports to sandbox N8N via API, assigns credentials, runs automated tests
- Will be used to build: id=12 (Media Pipeline Learning), id=7 (Friday Research Session Prep)
Related personal_projects DB rows
- id=4: N8N Workflow Builder Script (pending, weekend_block1)
Prereq checklist (must complete before any agent deployment)
- Schema: rename automation_ideas.description → task_description, add type, add builder_status, add priority
- Create agent_test_results table in api_business
- Sandbox Vault: set up AppRole auth method (credentials at /opt/appdata/docker/docker-compose/vault/approle/ on server-01)
- Sandbox Vault: store sandbox N8N API key at secret/sandbox/n8n (key name: claude-sandbox, verified working)
- Verify sandbox Bitwarden bridge ↔ Vaultwarden sandbox end-to-end (bridge on port 8080, returns [] for empty vault — correct)
- Write Agent Builder Agent playbook → .claude/playbook_agent_builder_agent.md
- Write N8N Builder Agent playbook → .claude/playbook_n8n_builder_agent.md
- Backfill session: COMPLETE. All pending rows reviewed across sessions 1-3. Blocked rows verified (32, 30, 13, 27, 23, 24, 22, 2, 21, 35, 36, 55, 56, 18 — all still blocked, no changes without proper review). Admin done: id=28 → p10, id=57 keeps p9, dummy row id=16 deleted. Session 3 (2026-06-18) final batch: ids 47, 48, 31, 26, 54.
- id=25: type→n8n_automation, status→pending (8b model lifts hardware block), full rewrite with SDK credit model, model router + cost gate + subscription monitor, model_effort_routing_log training table
- id=17: type→n8n_automation, expanded with token_waste_patterns DB table (evolution mechanism), JSONL-based detection, NTFY claude-audit gate, fan-out plan noted
- id=42: expanded with hybrid trigger (Claude flags + asks), B+C output (DB + wrap-up surface), human gate permanent, hard boundary (business research = id=55 only), voice note
- id=46: type→n8n_automation, Hermes ref removed, playbook_schema.md bootstrap design, three risk tiers, id=52 detector/id=46 executor boundary, fan-out plan noted
- id=43: Vault AppRole required (youtube-oauth-script, new policy), vault_registry entries, port 8085 (8080 reserved for bridge), dynamic Vault IP, write verification, token refresh companion
- id=45: hard dep on id=43 (Vault auth), niche_saturation_thresholds DB table (evolution), niche_check_results table, three-signal output (saturated/viable/unclear), id=18 stage 2 integration
New automation_ideas rows added 2026-06-18 (ids 59–67)
- id=59: Claude Audit Research Prep (n8n_automation, p12) — Saturday briefing for 12:45 PM review
- id=60: Claude Audit Topic Detector (n8n_automation, p11) — detects new topics → claude_config.audit_topics; must build first
- id=61: Claude Config Dev Work Scheduler (n8n_automation, p13) — Sunday session prep from Saturday decisions
- id=62: Monthly Security Audit (n8n_automation, p7) — first Monday of month, 7-area audit → security_audit DB
- id=63: Security Patch Backlog Handler (claude_agent, p7) — works security_audit.findings; Vault/prod always human-gate
- id=64: Business Projects Worker (claude_agent, BLOCKED p29) — 12:45–4:30 PM daily; blocked pending business_projects schema
- id=65: Personal Projects Worker (claude_agent, BLOCKED p29) — 5:30–8:00 PM daily; blocked pending personal_projects schema
- id=66: Media Pipeline Project Worker (claude_agent, BLOCKED p29) — Wednesday 5:30–8:00 PM; blocked pending id=65 + scope
- id=67: N8N Builder Agent (claude_agent, p2, ready_to_build) — builds all n8n_automation rows; deploy Monday June 22 alongside id=24
Readiness check — COMPLETE (2026-06-18 evening)
All 8 prereq checklist items verified live on server-01:
- Vault AppRole: token acquired, N8N secret readable
- N8N sandbox API: returns 200 with real key
- Bridge /items: returns [] (correct for empty sandbox vault)
- Ollama: llama3.1:8b loaded
- agent_test_results table: confirmed correct schema + FK
June 19 — Manual Infrastructure Day (before any agent builds)
Two manual projects only:
- id=51 Secrets Execution Proxy — build FastAPI script that intercepts secret access, prevents raw credential exposure in agent stdout. Must be live before agents deploy.
- id=57 Sandbox Environment Deployment Completion — (1) Migrate production N8N from primary server to server-01. (2) Mirror production security in sandbox: equivalent Vault AppRole policies, Bitwarden Bridge populated with test credential entries matching production structure, N8N sandbox credentials matching production names (-sandbox suffix). Goal: sandbox = production mirror, so passing sandbox tests guarantees production works.
- Grill-me required at session start before any work begins.
June 22 (Monday) — Builder Agents (extended session)
Build, test, and push both builder agents to production. Work as long as it takes.
- id=24 Agent Builder Agent (claude_agent + script types)
- id=67 N8N Builder Agent (n8n_automation types)
- Prerequisites: id=51 live, id=57 complete (sandbox mirrors production)
Priority queue summary (as of 2026-06-18)
- p1: id=51 (manual), id=57 (manual) — infrastructure foundation
- p2: id=24, id=67 — builder agents (built in sessions, build everything else)
- p3: id=54 NTFY Provisioner, id=58 Human Action Gate
- p4: id=25 Cost Intelligence, id=38 Credential Emergency Rollout
- p5: id=3 Secrets Rotation, id=8 Vault Token Audit, id=52 Session Wrap-up
- p6: id=37 N8N Log Scanner, id=40 Coolify UUID Monitor
- p7: id=62 Monthly Security Audit, id=63 Security Patch Handler
- p8–p17: non-blocked pending/ready items
- p20–p29: all blocked items (builder agents skip these)
Update instructions
Update at the end of every agent-builder session. Keep agent status, key decisions, and prereq checklist current.