feat: add required_playbooks fetch + prompt review gate to both builder playbooks

Agent Builder: added Step 2 (fetch required_playbooks from Gitea URLs), added
mandatory claude_agent prompt design rules to Ollama prompt (--max-turns 15,
loop detection, credential map inline, defined output), added Step 6 synchronous
human NTFY gate for claude_agent prompt review before sandbox testing.

N8N Builder: same required_playbooks fetch step, added Step 8 synchronous human
NTFY gate for any workflow containing a claude -p node.

Both changes prevent runaway token loops in built automations by ensuring every
generated claude -p call has hard stops baked in and user-reviewed before deploy.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

agent-builder/.claude/playbook_agent_builder_agent.md

@@ -31,32 +31,55 @@ WHERE id = <selected_id> AND builder_status = 'not_started';
 ```
 If 0 rows updated: another builder claimed it — stop, notify, exit.
 
-### Step 2 — Build the prompt for Ollama
+### Step 2 — Fetch required playbooks
+Read the `required_playbooks` field from the automation row. It contains a comma-separated list of Gitea raw URLs. For each URL:
+```
+GET {gitea_raw_url}
+```
+Inject the full content of each playbook into the Ollama generation prompt as context. If a URL returns non-200, set builder_status = 'failed', notify via NTFY with the missing URL, stop.
+
+If `required_playbooks` is empty: proceed without playbook context.
+
+### Step 3 — Build the prompt for Ollama
 Construct a generation prompt using all available fields from the automation row:
 - `name`: what the automation is called
 - `task_description`: full structured spec — this is the primary instruction
 - `type`: claude_agent or script
 - `infrastructure_requirement`: what infra it needs access to
+- Playbook content fetched in Step 2
 
-Prompt structure:
+**For `claude_agent` type**, the generated output is a shell-executable `claude -p` command. The Ollama prompt must include these mandatory requirements for the generated command:
+
+```
+claude_agent MANDATORY PROMPT REQUIREMENTS (all must appear in the generated claude -p system prompt):
+1. Hard turn limit: the claude -p call must include --max-turns 15
+2. Loop detection instruction: "If you issue the same command twice with identical arguments, STOP immediately and report what you found so far."
+3. Credential map: all credentials and access patterns the agent will need must be provided inline in the system prompt — the agent must never search for auth at runtime
+4. Defined output: the system prompt must specify the exact output format expected
+5. Data pre-fetched: any static data the agent needs (DB rows, config values) must be included inline in the system prompt rather than fetched via tool calls where possible
+```
+
+Ollama prompt structure:
 ```
 You are an expert automation engineer. Build a {type} automation with the following specification.
 
 Name: {name}
 Infrastructure: {infrastructure_requirement}
 
+{playbook_context if any}
+
 Specification:
 {task_description}
 
 Requirements:
-- If type is claude_agent: output a complete shell-executable claude -p command with full system prompt and all logic. The agent must be self-contained.
+- If type is claude_agent: output a complete shell-executable claude -p command with full system prompt and all logic. The agent must be self-contained. MUST include --max-turns 15. MUST include loop detection instruction. MUST include credential map inline. MUST include defined output format.
 - If type is script: output a complete Python script. Include a Dockerfile if the script has dependencies beyond stdlib.
 - Output ONLY the code. No explanation, no markdown fences, no commentary.
 - The code must handle its own error cases and log to stdout.
 - Secrets must be fetched from Vault via AppRole — never hardcoded. AppRole credentials at /opt/appdata/docker/docker-compose/vault/approle/role-id and secret-id.
 ```
 
-### Step 3 — Generate with Ollama
+### Step 4 — Generate with Ollama
 ```
 POST http://localhost:11434/api/generate
 {
@@ -69,7 +92,7 @@ Set builder_status = 'building' before calling.
 
 If Ollama call fails or times out (>120s): set builder_status = 'failed', log error, notify via NTFY, stop.
 
-### Step 4 — Overseer validation with claude -p
+### Step 5 — Overseer validation with claude -p (for claude_agent type: also validates prompt design rules)
 Pass the generated code to `claude -p` for structural review. Keep the prompt minimal to conserve SDK credits:
 
 ```bash
@@ -85,7 +108,27 @@ Do not rewrite the code."
 If FAIL: log claude's reason, set builder_status = 'failed', notify via NTFY with the failure reason, stop.
 If PASS: proceed.
 
-### Step 5 — Deploy to sandbox
+### Step 6 — Human prompt review gate (claude_agent type only)
+For `claude_agent` type automations, send the full generated `claude -p` command to the user for review before any testing begins. This is synchronous — do not proceed until the user approves or rejects.
+
+Send NTFY notification:
+```
+Title: [Agent Builder] Prompt Review Required — {name}
+Body: Overseer passed. Review the generated claude -p prompt before sandbox testing begins.
+
+--- GENERATED COMMAND ---
+{generated_claude_p_command}
+
+Reply APPROVE to proceed to testing, or REJECT to cancel.
+```
+
+Wait for user response via NTFY reply or next session confirmation.
+- On APPROVE: proceed to Step 7
+- On REJECT: set builder_status = 'failed', log rejection, stop. User should update task_description and re-queue.
+
+Skip this step for `script` type automations.
+
+### Step 7 — Deploy to sandbox
 **For `script` type:**
 1. Write the generated code to a temp directory on server-01
 2. If a Dockerfile was generated, build the image: `docker build -t agent-{id}-{slug} .`
@@ -98,7 +141,7 @@ If PASS: proceed.
 
 If deployment fails: set builder_status = 'failed', log error, notify via NTFY, stop.
 
-### Step 6 — Run 4-level automated tests
+### Step 8 — Run 4-level automated tests
 Run each level in order. Stop and fail if any level fails. Log every result to `agent_test_results`.
 
 **Level 1 — Structure**
@@ -123,7 +166,7 @@ Validate the generated artifact:
 - Examples: a file was created, a DB row was written, an API call returned 200, a container is running
 - Insert result to agent_test_results
 
-### Step 7 — Notify user for promotion approval
+### Step 9 — Notify user for promotion approval
 If all 4 levels pass:
 1. Set builder_status = 'awaiting_approval'
 2. Send NTFY notification:
@@ -134,7 +177,7 @@ If all 4 levels pass:
 
 User must explicitly approve before any production deployment. No auto-promotion in v1.
 
-### Step 8 — On approval
+### Step 10 — On approval
 Set builder_status = 'approved', then 'deployed' after production deployment completes.
 Update automation_ideas status = 'deployed'.
 

agent-builder/.claude/playbook_n8n_builder_agent.md

@@ -48,11 +48,20 @@ WHERE id = <selected_id> AND builder_status = 'not_started';
 ```
 If 0 rows updated: another builder claimed it — stop, notify, exit.
 
-### Step 2 — Fetch sandbox N8N API key from Vault
+### Step 2 — Fetch required playbooks
+Read the `required_playbooks` field from the automation row. It contains a comma-separated list of Gitea raw URLs. For each URL:
+```
+GET {gitea_raw_url}
+```
+Inject the full content of each playbook into the Ollama generation prompt as context in Step 4. If a URL returns non-200, set builder_status = 'failed', notify via NTFY with the missing URL, stop.
+
+If `required_playbooks` is empty: proceed without playbook context.
+
+### Step 3 — Fetch sandbox N8N API key from Vault
 Use sandbox AppRole to read secret/sandbox/n8n. Extract `api_key` and `base_url`.
 Never log the key value. Pass it in memory only.
 
-### Step 3 — Discover available N8N credentials
+### Step 4 — Discover available N8N credentials
 Before generating, query the sandbox N8N for existing credentials so the generated workflow references them by name:
 ```
 GET {base_url}/api/v1/credentials
@@ -60,7 +69,7 @@ X-N8N-API-KEY: {api_key}
 ```
 Extract credential names and types. Pass this list to the Ollama prompt so the generated workflow uses real credential names.
 
-### Step 4 — Build the prompt for Ollama
+### Step 5 — Build the prompt for Ollama
 ```
 You are an expert N8N workflow engineer. Generate a valid N8N workflow JSON for the following automation.
 
@@ -81,7 +90,7 @@ Requirements:
 - The workflow must fully implement the specification — do not stub or placeholder any steps.
 ```
 
-### Step 5 — Generate with Ollama
+### Step 6 — Generate with Ollama
 ```
 POST http://localhost:11434/api/generate
 {
@@ -97,7 +106,7 @@ Validate it parses as JSON before proceeding. If invalid JSON: set builder_statu
 
 If Ollama call fails or times out (>120s): set builder_status = 'failed', log error, notify, stop.
 
-### Step 6 — Overseer validation with claude -p
+### Step 7 — Overseer validation with claude -p
 Pass the generated JSON to `claude -p` for structural review. Keep prompt minimal to conserve SDK credits:
 
 ```bash
@@ -114,7 +123,30 @@ Do not rewrite the workflow."
 If FAIL: log claude's reason, set builder_status = 'failed', notify via NTFY with reason, stop.
 If PASS: proceed.
 
-### Step 7 — Import to sandbox N8N
+### Step 8 — Human prompt review gate (claude -p workflows only)
+Detect if the workflow calls `claude -p` by scanning the generated JSON for any node containing `claude -p` or `claude_agent` in its parameters.
+
+If detected: send the full generated workflow JSON to the user for review before importing to sandbox. This is synchronous — do not proceed until the user approves or rejects.
+
+Send NTFY notification:
+```
+Title: [N8N Builder] Prompt Review Required — {name}
+Body: Workflow contains claude -p call. Review before sandbox import.
+
+--- CLAUDE -P NODE PARAMETERS ---
+{extracted claude -p node parameters}
+
+Required checks: --max-turns present, loop detection instruction present, credential map inline, defined output format.
+
+Reply APPROVE to proceed, or REJECT to cancel.
+```
+
+- On APPROVE: proceed to Step 9
+- On REJECT: set builder_status = 'failed', log rejection, stop. User should update task_description and re-queue.
+
+Skip this step for workflows with no claude -p nodes.
+
+### Step 9 — Import to sandbox N8N
 ```
 POST {base_url}/api/v1/workflows
 X-N8N-API-KEY: {api_key}
@@ -125,14 +157,14 @@ Body: {generated workflow JSON}
 On success: capture the returned workflow `id` from N8N. Store in notes or a temp variable.
 On failure (non-2xx): set builder_status = 'failed', log the N8N error response, notify, stop.
 
-### Step 8 — Assign credentials
+### Step 10 — Assign credentials
 For each node in the workflow that references a credential:
 ```
 GET {base_url}/api/v1/workflows/{workflow_id}
 ```
 Verify credential references resolved correctly. If any credential reference is broken (credential name not found), attempt to match by type — if unresolvable, set builder_status = 'failed', notify user with list of missing credentials, stop.
 
-### Step 9 — Run 4-level automated tests
+### Step 11 — Run 4-level automated tests
 Run each level in order. Stop and fail if any level fails. Log every result to `agent_test_results`.
 
 **Level 1 — Structure**
@@ -162,7 +194,7 @@ Validate the imported workflow via the N8N API:
 - The specific assertion depends on the automation — derive it from task_description
 - Insert result to agent_test_results (test_level=4)
 
-### Step 10 — Notify user for promotion approval
+### Step 12 — Notify user for promotion approval
 If all 4 levels pass:
 1. Set builder_status = 'awaiting_approval'
 2. Send NTFY notification:
@@ -173,7 +205,7 @@ If all 4 levels pass:
 
 User must explicitly approve before production import. No auto-promotion in v1.
 
-### Step 11 — On approval
+### Step 13 — On approval
 1. Import the same workflow JSON to production N8N (port 5678)
 2. Assign production credentials (different credential names from sandbox)
 3. Set builder_status = 'deployed'