feat: add required_playbooks fetch + prompt review gate to both builder playbooks

Agent Builder: added Step 2 (fetch required_playbooks from Gitea URLs), added mandatory claude_agent prompt design rules to Ollama prompt (--max-turns 15, loop detection, credential map inline, defined output), added Step 6 synchronous human NTFY gate for claude_agent prompt review before sandbox testing. N8N Builder: same required_playbooks fetch step, added Step 8 synchronous human NTFY gate for any workflow containing a claude -p node. Both changes prevent runaway token loops in built automations by ensuring every generated claude -p call has hard stops baked in and user-reviewed before deploy. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-26 01:31:58 -05:00
parent 924e7d5e8b
commit b99af3d0aa
2 changed files with 94 additions and 19 deletions
@@ -48,11 +48,20 @@ WHERE id = <selected_id> AND builder_status = 'not_started';
 ```
 If 0 rows updated: another builder claimed it — stop, notify, exit.

-### Step 2 — Fetch sandbox N8N API key from Vault
+### Step 2 — Fetch required playbooks
+Read the `required_playbooks` field from the automation row. It contains a comma-separated list of Gitea raw URLs. For each URL:
+```
+GET {gitea_raw_url}
+```
+Inject the full content of each playbook into the Ollama generation prompt as context in Step 4. If a URL returns non-200, set builder_status = 'failed', notify via NTFY with the missing URL, stop.
+
+If `required_playbooks` is empty: proceed without playbook context.
+
+### Step 3 — Fetch sandbox N8N API key from Vault
 Use sandbox AppRole to read secret/sandbox/n8n. Extract `api_key` and `base_url`.
 Never log the key value. Pass it in memory only.

-### Step 3 — Discover available N8N credentials
+### Step 4 — Discover available N8N credentials
 Before generating, query the sandbox N8N for existing credentials so the generated workflow references them by name:
 ```
 GET {base_url}/api/v1/credentials
@@ -60,7 +69,7 @@ X-N8N-API-KEY: {api_key}
 ```
 Extract credential names and types. Pass this list to the Ollama prompt so the generated workflow uses real credential names.

-### Step 4 — Build the prompt for Ollama
+### Step 5 — Build the prompt for Ollama
 ```
 You are an expert N8N workflow engineer. Generate a valid N8N workflow JSON for the following automation.

@@ -81,7 +90,7 @@ Requirements:
 - The workflow must fully implement the specification — do not stub or placeholder any steps.
 ```

-### Step 5 — Generate with Ollama
+### Step 6 — Generate with Ollama
 ```
 POST http://localhost:11434/api/generate
 {
@@ -97,7 +106,7 @@ Validate it parses as JSON before proceeding. If invalid JSON: set builder_statu

 If Ollama call fails or times out (>120s): set builder_status = 'failed', log error, notify, stop.

-### Step 6 — Overseer validation with claude -p
+### Step 7 — Overseer validation with claude -p
 Pass the generated JSON to `claude -p` for structural review. Keep prompt minimal to conserve SDK credits:

 ```bash
@@ -114,7 +123,30 @@ Do not rewrite the workflow."
 If FAIL: log claude's reason, set builder_status = 'failed', notify via NTFY with reason, stop.
 If PASS: proceed.

-### Step 7 — Import to sandbox N8N
+### Step 8 — Human prompt review gate (claude -p workflows only)
+Detect if the workflow calls `claude -p` by scanning the generated JSON for any node containing `claude -p` or `claude_agent` in its parameters.
+
+If detected: send the full generated workflow JSON to the user for review before importing to sandbox. This is synchronous — do not proceed until the user approves or rejects.
+
+Send NTFY notification:
+```
+Title: [N8N Builder] Prompt Review Required — {name}
+Body: Workflow contains claude -p call. Review before sandbox import.
+
+--- CLAUDE -P NODE PARAMETERS ---
+{extracted claude -p node parameters}
+
+Required checks: --max-turns present, loop detection instruction present, credential map inline, defined output format.
+
+Reply APPROVE to proceed, or REJECT to cancel.
+```
+
+- On APPROVE: proceed to Step 9
+- On REJECT: set builder_status = 'failed', log rejection, stop. User should update task_description and re-queue.
+
+Skip this step for workflows with no claude -p nodes.
+
+### Step 9 — Import to sandbox N8N
 ```
 POST {base_url}/api/v1/workflows
 X-N8N-API-KEY: {api_key}
@@ -125,14 +157,14 @@ Body: {generated workflow JSON}
 On success: capture the returned workflow `id` from N8N. Store in notes or a temp variable.
 On failure (non-2xx): set builder_status = 'failed', log the N8N error response, notify, stop.

-### Step 8 — Assign credentials
+### Step 10 — Assign credentials
 For each node in the workflow that references a credential:
 ```
 GET {base_url}/api/v1/workflows/{workflow_id}
 ```
 Verify credential references resolved correctly. If any credential reference is broken (credential name not found), attempt to match by type — if unresolvable, set builder_status = 'failed', notify user with list of missing credentials, stop.

-### Step 9 — Run 4-level automated tests
+### Step 11 — Run 4-level automated tests
 Run each level in order. Stop and fail if any level fails. Log every result to `agent_test_results`.

 **Level 1 — Structure**
@@ -162,7 +194,7 @@ Validate the imported workflow via the N8N API:
 - The specific assertion depends on the automation — derive it from task_description
 - Insert result to agent_test_results (test_level=4)

-### Step 10 — Notify user for promotion approval
+### Step 12 — Notify user for promotion approval
 If all 4 levels pass:
 1. Set builder_status = 'awaiting_approval'
 2. Send NTFY notification:
@@ -173,7 +205,7 @@ If all 4 levels pass:

 User must explicitly approve before production import. No auto-promotion in v1.

-### Step 11 — On approval
+### Step 13 — On approval
 1. Import the same workflow JSON to production N8N (port 5678)
 2. Assign production credentials (different credential names from sandbox)
 3. Set builder_status = 'deployed'