docs(netbird): grill-me decisions + consolidated design (#258)
12-Q grill-me folded in: not-CGNAT -> cloudflared front-door only + straight self-host; two-plane split (control=cloudflared, relay=scoped UDP direct); 5 nodes incl phone; admin/daily identities + ACL matrix; Authelia 6-svc audit; inventory corrected 16->21 (coolify + sudo-bridge-server01 retire); DNS-01 wildcard TLS req; NetBird bumped ahead of agent-sudo/secrets-proxy. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,82 @@
|
||||
# Cloudflared Tunnel — Public Endpoint Inventory (2026-09-17)
|
||||
|
||||
> Taken for the NetBird cutover (NetBird will replace cloudflared for the owner's access; #258).
|
||||
> Method: the tunnel is **dashboard/token-managed** (`tunnel run --protocol http2`, no local
|
||||
> `config.yml`; token in Vault `secret/cloudflare` → `CLOUDFLARE_TUNNEL_TOKEN`). **Every ingress rule
|
||||
> routes to one origin: `http://coolify-proxy:80` (Traefik).** So the public surface = the set of
|
||||
> `Host()` routers Traefik serves that also resolve publicly to Cloudflare. Enumerated from all
|
||||
> container Traefik labels + confirmed each against public DNS (all → Cloudflare `2606:4700::…`).
|
||||
> We hold NO Cloudflare **API** token (only the tunnel token + the `dash.cloudflare.com` dashboard
|
||||
> login in Bitwarden), so this was done without the CF API — re-verify against the dashboard tunnel
|
||||
> config at cutover.
|
||||
|
||||
## The 16 public hostnames (all `*.reverseproxyserver.net`, all Cloudflare-proxied → tunnel → Traefik)
|
||||
|
||||
| # | Hostname | Backing service | Category (for cutover) |
|
||||
|---|----------|-----------------|------------------------|
|
||||
| 1 | auth | authelia | **SSO gatekeeper** — special (see notes) |
|
||||
| 2 | vault | vault | Owner admin — **sensitive** → NetBird-only |
|
||||
| 3 | traefik | coolify-proxy (dashboard) | Owner admin — **sensitive** → NetBird-only |
|
||||
| 4 | sudo-bridge | sudo-bridge | Owner admin — **sensitive** → NetBird-only |
|
||||
| 5 | grafana | grafana | Owner admin → NetBird-only |
|
||||
| 6 | prometheus | prometheus | Owner admin → NetBird-only |
|
||||
| 7 | cadvisor | cadvisor | Owner admin → NetBird-only |
|
||||
| 8 | gitea | gitea | Owner admin → NetBird-only (also has internal `gitea.local`) |
|
||||
| 9 | qui | qui | Owner admin → NetBird-only |
|
||||
| 10 | autobrr | autobrr | Owner admin → NetBird-only |
|
||||
| 11 | qbittorrent | qbittorrent | Owner admin → NetBird-only |
|
||||
| 12 | qbittorrent-private | qbittorrent-private | Owner admin → NetBird-only |
|
||||
| 13 | privacyidea | privacyidea | Owner admin (2FA/OTP) → NetBird-only |
|
||||
| 14 | ntfy | ntfy | Mixed — push notifications; **check phone/topic reachability** |
|
||||
| 15 | nextcloud | nextcloud-nextcloud-1 + aio-talk | **Partner service → Twingate**; owner → NetBird; **public link-shares tension** |
|
||||
| 16 | jellyfin | jellyfin | **Partner service → Twingate** (Tyler + Hailee media) + owner → NetBird |
|
||||
|
||||
(`gitea.local` = internal LAN alias only, not tunneled — excluded.)
|
||||
|
||||
## ⚠️ CORRECTION 2026-09-17 (design phase) — 5 MORE public hostnames found → **21 total, not 16**
|
||||
The original enumeration only walked **container-label** routers. Traefik's **file provider**
|
||||
(`/traefik/dynamic/*.yaml`, host `/data/coolify/proxy/dynamic/`) defines 5 more routers, **all of which
|
||||
resolve publicly to Cloudflare** (verified `getent hosts` → `2606:4700:3035::…`):
|
||||
|
||||
| # | Hostname | Backing service | Category (cutover) | Note |
|
||||
|---|----------|-----------------|--------------------|------|
|
||||
| 17 | hermes | hermes | Owner admin → NetBird-only | native auth only |
|
||||
| 18 | n8n | n8n | Owner admin → NetBird-only | native auth only |
|
||||
| 19 | jenkins | jenkins | Owner admin → NetBird-only | native auth only |
|
||||
| 20 | coolify | coolify (+realtime/terminal ws) | **NetBird-only or RETIRE** | ⚠️ **NO auth middleware** (gzip+https only); Coolify orchestration is **retired** (`jenkins_deployment_transition`) yet the panel is still public |
|
||||
| 21 | sudo-bridge-server01 | sudo-bridge on server-01 | **NetBird-only or RETIRE** | ⚠️ **privileged** endpoint exposed public; sudo-bridge is being replaced by **agent-sudo #176** |
|
||||
|
||||
**None of these 5 use Authelia** (confirmed). **Cutover implication:** coolify (#20) and sudo-bridge-server01
|
||||
(#21) are strong **retire-don't-migrate** candidates — coolify is dead weight, sudo-bridge-server01 dies
|
||||
with agent-sudo. The other 3 (hermes/n8n/jenkins) → NetBird-only with the admin tier.
|
||||
**Revised cloudflared end-state = control-plane route + public-shares route only; all 21 service routes
|
||||
pulled or retired.**
|
||||
|
||||
## Cutover categorization
|
||||
- **→ NetBird only (pull off cloudflared entirely):** vault, traefik, sudo-bridge, grafana, prometheus,
|
||||
cadvisor, gitea, qui, autobrr, qbittorrent, qbittorrent-private, privacyidea. These are owner-only
|
||||
admin surfaces — exactly what principle #1 / P5 says shouldn't be internet-exposed. High-value target.
|
||||
- **→ Twingate (partners) + NetBird (owner):** nextcloud, jellyfin.
|
||||
- **Special / needs a decision:**
|
||||
- **auth (Authelia)** — the SSO gatekeeper. Many of the above likely sit behind Authelia forward-auth.
|
||||
If `auth.` leaves cloudflared, confirm the SSO redirect still works for mesh-only access (whoever
|
||||
authenticates must reach `auth.`). Sequencing-sensitive — likely migrate auth *with* or *after* the
|
||||
services that depend on it, never orphan them.
|
||||
- **ntfy** — the phone must still receive pushes (sudo-bridge + alerts depend on ntfy). If ntfy goes
|
||||
NetBird-only, the phone must be a mesh peer for pushes to arrive off-LAN. Verify.
|
||||
- **nextcloud public link-shares** (shares id6/id7, login-less public URLs for external people) —
|
||||
these fundamentally need a public path; NetBird/Twingate both require a client. Unresolved (grill-me).
|
||||
|
||||
## ⚠️ Blocking design tension discovered (headline for the NetBird grill-me)
|
||||
Cloudflared is an **outbound-only** tunnel — it's what lets these services be reachable **despite the
|
||||
owner's network almost certainly being behind CGNAT** (same reason Tyler needs Twingate). **NetBird's
|
||||
self-hosted control plane (management + signal + relay) must itself be publicly reachable** for roaming
|
||||
peers (phone on mobile data) to connect. If cloudflared is retired, **what gives NetBird's own control
|
||||
plane a public front behind CGNAT?** Options to resolve: (a) keep a thin tunnel just for NetBird's
|
||||
443/gRPC; (b) host the NetBird control plane on a cheap public-IP VPS; (c) accept a scoped inbound port
|
||||
if the network isn't actually CGNAT'd. **"NetBird replaces cloudflared" may be partly circular** — must
|
||||
be settled before design. See the grill-me doc.
|
||||
|
||||
## Update instructions
|
||||
Re-verify against the live Cloudflare dashboard tunnel config at cutover (we lacked a CF API token here).
|
||||
Update categories as owner decisions land in the grill-me.
|
||||
@@ -0,0 +1,145 @@
|
||||
# NetBird Design — Consolidated (2026-09-17)
|
||||
|
||||
> Design-phase output. Inputs: the 12 grill-me decisions (`netbird-selfhost-buildplan.md` → GRILL-ME
|
||||
> DECISIONS banner), the corrected cloudflared inventory (`cloudflared-tunnel-inventory_2026-09-17.md`,
|
||||
> now **21** hostnames), and the two read-only audits run this session (Authelia + ntfy).
|
||||
> Role (locked): **NetBird = owner's own device mesh; replaces cloudflared for ALL owner service access;
|
||||
> straight self-hosted (no cloud bootstrap); control plane on the PRIMARY server; server-01 = throwaway
|
||||
> sandbox.** Partners never join this mesh (that's Twingate).
|
||||
> Next artifact after sign-off: `playbook_netbird_phases.md` (the phase-prompt set) → server-01.
|
||||
|
||||
---
|
||||
|
||||
## 1. Audit results (read-only, this session)
|
||||
|
||||
### 1a. Authelia forward-auth dependency graph (Q8) — 6 leaf services
|
||||
Middleware `authelia@file` (`/traefik/dynamic/authelia.yaml` → `forwardAuth: http://authelia:9091/api/authz/forward-auth`,
|
||||
chained with `authelia-headers`). Protects **exactly 6 services**, all via container labels:
|
||||
- **prove-it tier:** grafana, prometheus, cadvisor
|
||||
- **admin tier:** qui, qbittorrent, qbittorrent-private
|
||||
|
||||
**Not behind Authelia (native auth):** vault, gitea, autobrr, privacyidea, nextcloud, jellyfin, ntfy,
|
||||
and all 5 file-provider routers (hermes, n8n, jenkins, coolify, sudo-bridge-server01). **No Authelia
|
||||
sequencing dependency for any of those.**
|
||||
|
||||
**The one constraint:** the 6 redirect the browser to `auth.reverseproxyserver.net` to log in. The
|
||||
internal forward-auth call (Traefik→`authelia:9091`) is container-to-container and works regardless of
|
||||
the mesh; only the **browser redirect to `auth.`** matters → over the mesh, `auth.` (and the 6) must
|
||||
resolve to Traefik via the routing peer. **Same split-DNS pattern already proven with Twingate.**
|
||||
**→ Migrate Authelia into the mesh WITH the prove-it tier (not last)**, since grafana/prometheus/cadvisor
|
||||
need it from step one. No lockout landmines.
|
||||
|
||||
### 1b. ntfy hardening (Q9) — already done
|
||||
`NTFY_AUTH_DEFAULT_ACCESS=deny-all` + `NTFY_AUTH_FILE=/var/lib/ntfy/auth.db` both live. Default-deny is
|
||||
the target posture. Only a cutover-time check that auth.db grants are correct remains. No build work.
|
||||
ntfy **stays public** on cloudflared (Q9 — wake-from-sleep FCM reliability).
|
||||
|
||||
### 1c. Inventory correction — 21 public hostnames, not 16
|
||||
File-provider routers hermes/n8n/jenkins/coolify/sudo-bridge-server01 all resolve to Cloudflare (public).
|
||||
Flags: **coolify** has NO auth middleware + orchestration is retired → **RETIRE**; **sudo-bridge-server01**
|
||||
is a privileged public endpoint + dies with agent-sudo #176 → **RETIRE**. Details in the inventory doc.
|
||||
|
||||
---
|
||||
|
||||
## 2. Identities, groups & ACL matrix (Q6)
|
||||
|
||||
**Two identities** (NetBird management authority) — `admin` and `daily`. Manual approval per device.
|
||||
Authelia via generic-OIDC is the IdP. **Key distinction:** identity role governs *NetBird administration*
|
||||
rights (approve peers, edit policies); **ACL policies** govern *service reachability* independently — so
|
||||
a `daily` device can still hold a narrow service grant (e.g. the phone's BW-unlock SSH) without any mesh-
|
||||
admin power.
|
||||
|
||||
### Groups
|
||||
| Group | Members | Notes |
|
||||
|---|---|---|
|
||||
| `admin-devices` | laptop | owner workstation; full access |
|
||||
| `daily-devices` | phone, tablet | user-facing services + narrow grants |
|
||||
| `servers` | primary, server-01 | enrolled via setup keys (short TTL) |
|
||||
| `routing` | primary | advertises LAN + docker subnet |
|
||||
|
||||
### ACL policy matrix (default-deny; only these allowed)
|
||||
| Source | Destination | Ports | Rationale |
|
||||
|---|---|---|---|
|
||||
| `admin-devices` (laptop) | all 21 Traefik hostnames (via routing peer → Traefik) | 443 | full admin |
|
||||
| `admin-devices` (laptop) | primary, server-01 | 22 | admin SSH + BW-unlock (redundant path) |
|
||||
| `daily-devices` (phone+tablet) | nextcloud, jellyfin, ntfy | 443 | user-facing only |
|
||||
| `daily-devices` **(phone only)** | primary | 22 | **JIT BW-unlock (Q5)** — narrow grant |
|
||||
| `routing` (primary) | advertises `192.168.1.0/24` + `172.16.16.0/24` | — | mesh reaches services w/o per-service exposure |
|
||||
| `servers` | each other | minimal, as needed | e.g. monitoring scrape |
|
||||
|
||||
**Blast-radius:** a compromised `daily` device reaches only nextcloud/jellyfin/ntfy (+ phone's SSH to
|
||||
primary) — never vault/traefik/agent-sudo/gitea/etc., and cannot administer the mesh.
|
||||
|
||||
---
|
||||
|
||||
## 3. Production deploy shape (primary server)
|
||||
|
||||
- **Control-plane containers** (bridged compose, official `netbirdio/*` images — no linuxserver variant):
|
||||
`management` + `signal` + `relay` + `dashboard` (+ `coturn` only if the sandbox test proves the relay
|
||||
needs it). DB = external Postgres.
|
||||
- **The two-plane split (Q10) — the core rule:**
|
||||
- **Control plane (management + signal, tiny):** published via **cloudflared** (outbound-only, no open
|
||||
port). This is the *only* NetBird traffic on cloudflared. Never video.
|
||||
- **Relay (data-plane fallback, could carry a stream):** self-hosted on primary, reachable via a
|
||||
**scoped, directly-reachable UDP 3478** (WireGuard-encrypted stealth port — silent to unauth'd
|
||||
packets). **NOT on cloudflared** → no re-throttling, no stutter even on fallback.
|
||||
- **Direct P2P** (hole-punching) is primary; with the public IP `66.164.11.87` it should nearly always
|
||||
succeed → most traffic never touches the relay at all.
|
||||
- **DNS (mesh split-DNS):** NetBird DNS maps `*.reverseproxyserver.net` → primary's LAN/mesh IP where
|
||||
Traefik listens (reached via the `routing` peer). Same override pattern as Twingate's Resource FQDN.
|
||||
This is what makes the Authelia `auth.` redirect (and all service hostnames) resolve correctly on-mesh.
|
||||
- **TLS (⚠ new requirement):** today Traefik gets certs via **http-01 through cloudflared**. Once
|
||||
hostnames leave cloudflared, http-01 breaks for them → **switch Traefik to a DNS-01 wildcard cert for
|
||||
`*.reverseproxyserver.net`.** DNS-01 needs a **Cloudflare DNS API token** (scoped `Zone:DNS:Edit`) —
|
||||
**we don't hold one today** (only the tunnel token + dashboard login). **New secret to create** →
|
||||
Vault `secret/cloudflare/dns-api`. One wildcard cert then serves every hostname, mesh or public.
|
||||
- **Agents** on all 5 nodes (`netbirdio/netbird`): documented host-mode exception (`network_mode: host`
|
||||
+ `cap_add: NET_ADMIN` + `/dev/net/tun`).
|
||||
- **Routing peer** = primary, advertises host LAN + docker `172.16.16.x` (until OPNsense takes the role).
|
||||
- **Postgres (Q11):** dedicated `netbird` DB + user on `postgres-lggkk0kcgwko440kk04wowgk`; DSN in Vault.
|
||||
- **Vault paths to CREATE:** `secret/netbird/{management,oidc,turn,setup-key,db}` + **`secret/cloudflare/dns-api`** (new, for DNS-01).
|
||||
- **Backup:** Postgres `netbird` DB (restic) + `/var/lib/netbird` config; management redeployable from
|
||||
Vault + backup; existing peer tunnels keep forwarding if the plane is down.
|
||||
|
||||
---
|
||||
|
||||
## 4. server-01 sandbox validation runbook (throwaway; never prod data)
|
||||
1. Deploy NetBird self-hosted stack on server-01 via official `docker-compose` + `setup.env`. **SQLite** store.
|
||||
2. Wire **OIDC to Authelia** (generic-OIDC: Client ID/Secret/Issuer). Confirm dashboard login (auth-code
|
||||
+ PKCE). **Test CLI/device-flow enrollment** → resolves buildplan item (b) caveat (setup keys = fallback).
|
||||
3. Enroll a throwaway peer; verify a policy (deny/allow), a **posture check** (OS/version), and a
|
||||
**routing peer** advertising a test subnet.
|
||||
4. **★ Prove streams go DIRECT (the Jellyfin proof):** enroll two peers, run a sustained transfer, confirm
|
||||
`netbird status` shows **P2P/direct, not relayed.** Then **force relay fallback** (block direct) and
|
||||
confirm the relay path works **via the UDP port, NOT via cloudflared** (validates Q10 / buildplan item c).
|
||||
5. Confirm **Postgres DSN** connectivity from a management container against a scratch DB (dry-run Q11).
|
||||
6. Confirm **DNS-01 wildcard cert** issuance with the new Cloudflare DNS API token (dry-run §3 TLS).
|
||||
7. **Tear down** — server-01 keeps only Ollama/GPU + Obsidian (`feedback_sandbox_isolation`).
|
||||
|
||||
---
|
||||
|
||||
## 5. Cutover plan (Q7 — conservative)
|
||||
Order, with Authelia pulled forward per §1a:
|
||||
1. **prove-it + Authelia:** cadvisor, prometheus, grafana **+ `auth.`** → prove mesh access + the
|
||||
forward-auth redirect works on-mesh from laptop & phone off-LAN.
|
||||
2. **admin:** vault, traefik, gitea, qui, autobrr, qbittorrent(+private), privacyidea, hermes, n8n,
|
||||
jenkins, agent-sudo endpoint.
|
||||
3. **retire (don't migrate):** coolify (#20), sudo-bridge-server01 (#21).
|
||||
4. **services (last):** nextcloud, jellyfin — stay dual-homed (Twingate for partners, NetBird for owner).
|
||||
- **Rollback:** everything dual-homed for a **1–2 week soak**, then pull ALL cloudflared service routes at
|
||||
once. Cloudflared end-state = **control-plane route + public-shares route only.**
|
||||
|
||||
---
|
||||
|
||||
## 6. Timing (Q12)
|
||||
**NetBird #258 → agent-sudo #176 → secrets-proxy #174.** agent-sudo right after NetBird sweeps up all
|
||||
deferred systemd timer registrations (Twingate connector self-heal + NetBird relay self-heal) in one pass.
|
||||
|
||||
## 7. Open items to confirm at build
|
||||
- Cloudflare DNS API token creation (for DNS-01) — owner action (dashboard).
|
||||
- Whether NetBird's built-in `relay` alone suffices or `coturn` is also needed (sandbox step 4 decides).
|
||||
- Authelia device-code flow vs setup keys for CLI/mobile enrollment (sandbox step 2 decides).
|
||||
|
||||
## Update instructions
|
||||
Fill in sandbox results for steps 2, 4, 6; mark buildplan items (b)/(c) RESOLVED; then spin §3–§5 into
|
||||
`playbook_netbird_phases.md` and start the build on server-01.
|
||||
@@ -10,6 +10,119 @@
|
||||
|
||||
---
|
||||
|
||||
## ✅ GRILL-ME DECISIONS (owner, 2026-09-17) — AUTHORITATIVE (override the research below on any conflict)
|
||||
|
||||
> Ran the 12-question grill-me (`netbird-grillme-questions_2026-09-17.md`) with the owner present.
|
||||
> These decisions **supersede** the agent-K recommendations below where they differ — most importantly
|
||||
> the **bootstrap recommendation is REVERSED** (now straight self-host) and **item (c) is DECIDED**
|
||||
> (open the scoped UDP relay port). Design phase turns these into `playbook_netbird_phases.md`.
|
||||
|
||||
### The headline finding that reshaped everything — NO CGNAT (Q1)
|
||||
- **Owner's WAN is a real routable public IP** (`66.164.11.87`, verified: `curl ifconfig.me` == the
|
||||
wireless-gateway WAN IP; no `100.64.x.x`). **The circular-dependency worry dissolves.** NetBird's
|
||||
control plane can be self-hosted behind Cloudflare with **no CGNAT workaround**.
|
||||
- **DECISION — cloudflared becomes NetBird's *front door only*:** ALL Docker service routes come off
|
||||
cloudflared (Jellyfin especially — Cloudflare's free-tunnel TOS throttles sustained video = the
|
||||
stutter; off-tunnel + direct P2P is the real fix). Cloudflared keeps exactly **two** routes:
|
||||
(1) NetBird's **control plane** (management + signal), (2) the **public Nextcloud shares** (Q2).
|
||||
- **Conscious tradeoff accepted (owner flagged it as a great catch):** direct exposure would reveal the
|
||||
home IP + require an open port; instead cloudflared stays as a thin outbound-only front for the
|
||||
control plane → **home IP stays hidden, near-zero attack surface preserved.** This chose **option (B)**
|
||||
over full-direct-exposure.
|
||||
|
||||
### Q2 — public Nextcloud link-shares → **option (a)**
|
||||
- Keep **one narrow cloudflared route** for the login-less prospect shares (id6/id7). Nearly free
|
||||
(cloudflared already up for NetBird), zero prospect friction, no new attack surface. Open detail
|
||||
(who receives them / passcode tolerance) confirmed at cutover — non-blocking.
|
||||
|
||||
### Q3 — **straight self-hosted, NO cloud bootstrap** (REVERSES the Path-A recommendation below)
|
||||
- The 09-20 Twingate deadline that justified bootstrapping is **done**; NetBird is now the owner's
|
||||
*primary* access and the security stance is self-hosted-paramount. **No NetBird Cloud tenant ever
|
||||
touches the mesh, even temporarily.** Prove on server-01 sandbox → deploy on primary. (The Path-A
|
||||
table + recommendation below are now HISTORICAL.)
|
||||
|
||||
### Q4 — node set = **5 peers** (⚠ phone constraint REVERSED)
|
||||
- primary server, server-01, laptop, **phone, tablet**. **The phone CAN run NetBird** — owner tested it,
|
||||
installed fine (NetBird's Android floor is lower than Twingate's, which is what actually failed
|
||||
before). Tablet newer → fine. This **undoes** the "phone too old for the client" constraint carried
|
||||
from the Twingate session.
|
||||
- **Routing peer = primary server** (advertises home LAN + docker `172.16.16.x`), until the **OPNsense
|
||||
router** takes that role later (see future roadmap).
|
||||
|
||||
### Q5 — phone as first-class peer = **YES to both**
|
||||
- **Phone = off-LAN BW-unlock peer** (SSH → primary over the mesh) **+ laptop keeps it too** → two
|
||||
redundant off-LAN unlock paths.
|
||||
- **Phone = ntfy push target** over the mesh. Justification **decoupled from sudo-bridge** (agent-sudo
|
||||
#176 replaces sudo-bridge) — ntfy-to-phone stands on its own for general alerts.
|
||||
|
||||
### Q6 — identities = **two (admin + daily), manual approval per device**
|
||||
- `admin` identity (mesh management authority) vs `daily` (no admin rights) → compromised daily device
|
||||
can't administer the mesh. Every new peer hand-approved. **Authelia via generic-OIDC** IdP holds.
|
||||
- **Design-phase task:** device→identity mapping (proposed default: laptop=`admin`; phone+tablet=`daily`;
|
||||
primary+server-01 = service peers under ACLs) + the ACL policy matrix → bring as a table for sign-off.
|
||||
|
||||
### Q7 — cutover order agreed + **conservative rollback**
|
||||
- Order: **prove-it** (cadvisor, prometheus, grafana) → **admin** (vault, traefik, gitea, qui, autobrr,
|
||||
qbittorrent(+private), privacyidea, agent-sudo endpoint) → **auth** (Authelia, with/after its
|
||||
dependents) → **services** (nextcloud, jellyfin) last.
|
||||
- **Rollback = conservative:** everything stays **dual-homed for a 1–2 week soak**, then pull ALL
|
||||
cloudflared service routes at once once the whole mesh is proven. Extra surface is temporary/accepted.
|
||||
|
||||
### Q8 — Authelia sequencing = **audit-first**
|
||||
- **Design phase's FIRST task:** map the Authelia forward-auth dependency graph (grep all Traefik
|
||||
labels) + confirm the `auth.` redirect works over the mesh **before** migrating anything behind it.
|
||||
No guessing — self-lockout risk.
|
||||
|
||||
### Q9 — ntfy = **stays public, hardened**
|
||||
- Mesh-only would break wake-from-sleep pushes (Android kills the mesh/persistent conn on sleep) →
|
||||
keep ntfy on cloudflared for reliable FCM delivery. **Design-phase task:** audit current ntfy
|
||||
hardening (token-protected topics, `auth-default-access: deny-all` ACLs per `playbook_ntfy_users_topics`)
|
||||
and implement if not already done.
|
||||
|
||||
### Q10 — relay = **direct P2P primary; self-hosted relay via scoped UDP 3478, NOT cloudflared** (DECIDES item (c))
|
||||
- **Owner's catch that settled it:** a relay running over cloudflared WSS would re-throttle any stream
|
||||
that fell back to it → **same stutter.** So the two planes are split:
|
||||
- **Control plane** (management+signal, tiny) → **through cloudflared** (outbound-only, never video).
|
||||
- **Relay** (data-plane fallback, could carry a stream) → **self-hosted on primary via a scoped,
|
||||
directly-reachable UDP 3478** — a **WireGuard-encrypted STEALTH port** (silent to unauthenticated
|
||||
packets, invisible to scans → least-exposing open port possible). Un-throttled, so even the rare
|
||||
fallback never stutters.
|
||||
- Direct P2P (hole-punching) is the primary path — with the public IP it should nearly always succeed.
|
||||
**Server-01 still validates streams go DIRECT (not relayed) before cutover** (empirical Jellyfin proof).
|
||||
- This **overrides item (c)'s "prefer WSS-relay-through-cloudflared"** — the anti-stutter goal makes the
|
||||
direct UDP relay the correct choice; the stealth+encryption makes the open port acceptable to the owner.
|
||||
|
||||
### Q11 — store = **confirmed** (matches research)
|
||||
- **Prod:** dedicated `netbird` DB + user on homelab Postgres (`postgres-lggkk0kcgwko440kk04wowgk`),
|
||||
**DSN in Vault `secret/netbird/db`**. **Sandbox:** throwaway SQLite.
|
||||
|
||||
### Q12 — timing = **NetBird bumped AHEAD** of secrets-proxy/agent-sudo
|
||||
- New order: **NetBird #258 → agent-sudo #176 → secrets-proxy #174.** No hard dependency blocks NetBird
|
||||
(it uses SecretSpec+Vault like the Twingate connector). Doing **agent-sudo right after** lets it sweep
|
||||
up ALL deferred systemd timer registrations at once — the **Twingate connector self-heal timer +
|
||||
NetBird relay self-heal timer** — in one clean pass. NetBird's own timer is deferred meanwhile
|
||||
(`restart:always` covers reboots, same pattern as the Twingate connector).
|
||||
|
||||
### 🔒 Design rule locked (protects the future roadmap)
|
||||
**Keep the CONTROL-PLANE role (management/signal on primary) cleanly separable from the ROUTING-PEER /
|
||||
EXIT role.** Then the OPNsense router (or a colo box) can take the routing-peer/exit role later **without
|
||||
rebuilding the control plane** — it's just another peer added.
|
||||
|
||||
### 🗺️ Future roadmap (captured from the privacy discussion — NOT this build)
|
||||
1. **Exit node + Mullvad chaining** for genuine ISP-blinding on untrusted networks (self-hosting can't
|
||||
replicate crowd-anonymity; Mullvad is the right tool for the exit leg — €5/mo beats a dedicated box).
|
||||
2. **OPNsense DIY router** as the mesh routing-peer/exit → edge-level encryption + zero-trust
|
||||
segmentation on owned hardware at home (covers non-client devices: IoT/TV/guests).
|
||||
3. **Nym mixnet** evaluation — the "real decentralized crowd" option, to augment privacy after the core
|
||||
is deployed/tested. Owner explicitly wants to revisit.
|
||||
4. **Torrenting privacy (concluded, not a build item):** self-hosting does NOT achieve it — the swarm
|
||||
sees the exit IP directly and a dedicated IP is fully attributable; a **shared-IP no-log commercial
|
||||
VPN (Mullvad) is the only tool that works.** Mesh for services, Mullvad for the swarm.
|
||||
5. **Offshore/colo on owned hardware — DECLINED at the "shell-corp to defeat legal process" level**
|
||||
(structuring for untraceability, not privacy); the legitimate privacy goals are met by 1–3 above.
|
||||
|
||||
---
|
||||
|
||||
## The 3 open items — RESOLVED
|
||||
|
||||
### (a) Management DB engine — SQLite default; CAN use the homelab Postgres ✅
|
||||
|
||||
Reference in New Issue
Block a user