feat(network): add network policy create (--stamp/--deny/--reply-stamp)

[brunodam]

Description

Implements solo-provisioner network policy create — Story 1.2 of epic TS_1 (#738), the second scope in the network command family after network firewall (#757). This is the command-driven authoring surface for the inet weaver classification/ACL plane (design §7.2.4, §8.4.2, §8.4.6).

create renders one named category's rule(s) into the inet weaver forward chain, ensures its nft set @<name> exists, writes a per-policy registry JSON under /etc/solo-provisioner/policies/, applies the full chain to the live kernel with nft -f, and atomically rewrites /etc/solo-provisioner/network-weaver.nft. The new internal/network/policy package mirrors the merged internal/network/firewall package (Manager, atomic temp+rename write, shared apply flock, shared boot oneshot) but targets the inet weaver table, which has the opposite lifecycle — its set membership churns continuously as the daemon reconciles it from statusz.

Key design decisions (so reviewers don't reverse-engineer them):

• --stamp priority lookup is in-code. The 6 stable QoS classes (publisher … reserve-egress) map to fixed skb->priority / ct-mark values from design §5, baked into class.go. Referencing an unknown class is a create-time error. This means Story 1.2 has no runtime dependency on Story 1.4 (network shape) — the name→priority encoding is static; shape only sets each class's bandwidth.
• No --direction flag. Every class in the §5 map has exactly one direction (§7.2.4's worked ruleset never mixes a class across both), so Validate derives Policy.Direction from --stamp's class instead of taking it as a separate flag that could contradict it. --reply-stamp is checked as the mirror direction of the forward --stamp class (egress forward → ingress reply) rather than requiring a matching --direction value. Design doc (§5, §8.4.2, §8.4.3) and issues Epic TS_1 — solo-provisioner network command family #738/Story 1.2 — Implement network policy create (--stamp/--deny/--reply-stamp) #758/Story 1.4 — Implement network shape scope (tc HTB classes + device roots) #771 updated to match.
• Full chain rendered in tier order (deny → reply-restore → specific stamp → fallthrough stamp → est/rel accept → drop), never creation order. A mis-ordered deny would let already-open connections survive a quarantine, so ordering is a correctness invariant, pinned by a golden file. This is the boundary the epic owner and I agreed on for 1.2; see "Scope boundary" below for what's deferred to Story 1.5 — Policy registry and tier-order chain re-render #772.
• Set membership is never persisted to network-weaver.nft (§8.3.1) — statusz is the source of truth and the daemon reconciles it. --cidrs seeds the live set only, via a separate nft add element.
• Kernel apply is direct nft -f, not a service restart (the shared boot oneshot does not load network-weaver.nft until Story 0.3 — inet host reboot persistence (network-host.nft + oneshot) #780 extends it). create still enables the shared unit for boot readiness.
• --pod-cidr is no longer required (or auto-detected) for --deny. POD_CIDR is only ever read by renderStampRule; a deny rule (ip saddr/daddr @<name> drop) never references it. Render now only requires it when the merged policy set has at least one --stamp policy (needsPodCIDR), and the CLI skips cluster-based auto-detection entirely for --deny. Caught by manually testing --deny in isolation — quarantining a CIDR shouldn't need a reachable cluster.
• When --deny's merged chain does need a pod CIDR (a --stamp sibling exists) and none was supplied, Create recovers it from the existing network-weaver.nft instead of erroring — ExtractPodCIDR (new, parse.go) regex-extracts the literal CIDR any rendered --stamp rule starts with. This mirrors internal/network/firewall's Parse(), which exists for the same reason ("element verbs use this to load prior state so they don't need the full flag set re-spec") — podCIDR is a deployment-wide constant, not a per-call argument, so an unrelated --deny create shouldn't need it re-supplied just to correctly re-render an unchanged --stamp sibling. No CLI change: --stamp's own auto-detect-or-hard-error path is untouched, since that policy's own correctness still depends on a fresh value, not a cached one. network-weaver.nft genuinely missing (not just the kernel table) still errors clearly.
• create is create-if-missing, mirroring network firewall create — not additive. An existing policy is left untouched (warn, no changes) unless --force is passed, which replaces (not merges) its config and membership from the given flags/--cidrs. This is a deliberate divergence from an earlier iteration of this PR that made create diff-and-reapply automatically; matching firewall's already-established convention is simpler and more predictable.
• Two related correctness issues surfaced during manual UAT while landing the above (neither was caught by the original unit tests — the original fake Runner didn't model Apply()'s destructive wipe, so it couldn't have; it does now):
  1. The rendered document always starts with delete table; add table (§8.3.1: membership is never part of it), so every Apply() destroys and recreates every set in the table, not just the policy being created — any other policy's live membership (permanently so, for operator-curated policies like bn-mgmt-in, which the daemon never touches) was lost on every subsequent create. Fixed by snapshotting every policy's live membership (Runner.ListElements, new) before Apply() and restoring it afterward (TestCreate_PreservesSiblingMembershipAcrossRerender).
  2. A policy's registry entry can outlive the live kernel table: nft tables don't survive a reboot, and the boot oneshot doesn't reload network-weaver.nft yet (Story 0.3 — inet host reboot persistence (network-host.nft + oneshot) #780) — so "the registry has this policy" doesn't imply "the live table has it". Without --force, create now checks Runner.Exists and self-heals a missing table from the already-registered config — never the caller's just-typed flags/--cidrs, since applying those requires --force (TestCreate_SelfHealsMissingTableWithoutForce). Membership itself can't be recovered this way (it was never persisted); it comes back empty until --force re-seeds it or the daemon's poll loop catches up.

Files changed

File	Change
internal/network/policy/paths.go	Package doc + inet weaver path/service/lock constants
internal/network/policy/class.go	Static §5 class→mark/priority map + lookupClass
internal/network/policy/policy.go	Policy registry model + Validate (all flag-combination rules)
internal/network/policy/render.go	Tier-order full-chain renderer + atomicWriteFile
internal/network/policy/parse.go	ExtractPodCIDR — recovers the pod CIDR from an existing network-weaver.nft (mirrors firewall.Parse)
internal/network/policy/registry.go	Per-policy registry JSON read/write/load-all
internal/network/policy/manager.go	Manager.Create (flock, idempotent, apply+persist)
internal/network/policy/nft.go	Exec nft Runner (Apply/AddElements/ListElements/List/Delete/Exists)
internal/network/policy/service_{linux,other}.go	Shared oneshot enable (Linux) / no-op (other)
internal/network/policy/*_test.go + testdata/network-weaver.golden.nft	Render golden, tier-order, validation, idempotency, registry tests
cmd/cli/commands/network/policy/{policy,create}.go	network policy create verb + flags
cmd/cli/commands/network/policy/policy_test.go	Command structure, flag, and execution tests
cmd/cli/commands/network/network.go	Wire policy.GetCmd() into the network group
docs/quickstart.md	Create a Traffic Policy section (examples + flags table)

Review guide

Code review checklist

• class.go — the 6 class priorities/marks match design §5 exactly (publisher 0x10/0x10010 … reserve-egress 0x60/0x10060); reply ct-mark = the reply class's mark.
• policy.go:Validate — --stamp⊕--deny; --stamp derives Direction from its class; --reply-stamp requires --stamp to resolve to an egress class and itself resolve to the mirror ingress class; --from-entity world ⊕ --cidrs and stamp-only; --reply-stamp CIDRs are ip:port; IPv6 rejected (ipv4_addr sets).
• render.go — tier order (deny → reply-restore → specific → fallthrough → est/rel → drop); membership set schemas carry no elements, only _ports sets do; compound ipv4_addr . inet_service set for reply-stamp; ip daddr . compound match on the egress forward rule; needsPodCIDR only requires podCIDR when the merged set has a --stamp policy, since only renderStampRule reads it.
• manager.go:Create — create-if-missing: an existing policy is a no-op without --force, regardless of whether the given flags/--cidrs differ; --force re-renders and replaces (not merges) config and membership. Render happens before any disk write (a failed apply leaves the registry untouched); created_at preserved across a --force replace; whole op under the shared flock. snapshotMembership runs before Apply() and its restore loop runs after, so siblings' membership survives the destructive re-render — a snapshot failure aborts before the kernel is touched. Without --force, a missing live table for an existing policy self-heals from the registry, ignoring the caller's new flags/--cidrs. An empty podCIDR is recovered from the existing network-weaver.nft via ExtractPodCIDR before falling through to Render's needsPodCIDR check.
• Port-set naming (@<policy-name>_ports) — the design examples are internally inconsistent (@bn_ports_publisher by class vs @bn_ports_subscriber shared across policies). I chose a mechanical per-policy set. Flag if you want shared class-grouped port sets instead.
• network-weaver.golden.nft — read it top-to-bottom against §7.2.4; it is the behavioral contract.

Scope boundary (epic #738) — this PR is create only. Deferred:

• add/remove/set/show/delete element verbs → Story 1.3 — Implement network policy element verbs (add/remove/set/show/delete) #759 (Story 1.3)
• overlap rejection, the created_at intra-tier tiebreaker, delete + last-policy service teardown → Story 1.5 — Policy registry and tier-order chain re-render #772 (Story 1.5)
• extending the boot oneshot to load network-weaver.nft → Story 0.3 — inet host reboot persistence (network-host.nft + oneshot) #780
• network shape class declaration → Story 1.4 — Implement network shape scope (tc HTB classes + device roots) #771 (Story 1.4)

Test commands

# Unit tests (macOS OK — internal/network/policy has no Linux-only deps)
go test -race -cover -tags='!integration' ./internal/network/policy/...

# CLI command package (Linux only — pulls in internal/mount; run in the VM)
task vm:test:unit

# Linux cross-compile of the touched packages
GOOS=linux GOARCH=amd64 go build ./internal/network/policy/... ./cmd/cli/commands/network/...

# Lint (errorx/forbidigo) + license headers
task lint:check
task license:check

Manual UAT (UTM VM — Debian/Ubuntu with nftables + systemd)

1. Build and copy the binary to the VM:

   task build:cli GOOS=linux GOARCH=amd64
   scp bin/solo-provisioner-linux-amd64 <vm>:/usr/local/bin/solo-provisioner

2. Create a stamp policy (pod CIDR passed explicitly so no cluster is needed):

   sudo solo-provisioner network policy create --name bn-publisher \
     --ports 40840 --stamp publisher \
     --cidrs 10.1.0.1/32 --pod-cidr 10.4.0.0/24

   Verify the live chain:

   sudo nft list table inet weaver

   Expect a chain forward with policy drop, the set bn-publisher (type ipv4_addr; flags interval), the port set bn-publisher_ports with elements = { 40840 }, and the rule:

   ip daddr 10.4.0.0/24 ip saddr @bn-publisher tcp dport @bn-publisher_ports meta priority set 1:10 accept
   

   (nft list table reformats meta priority values that decode as a valid tc classid into major:minor notation — 0x10010 → 1:10. This is nft's own display choice on read-back, not what we write: network-weaver.nft and the golden test both correctly show the literal 0x10010 we author.)
   The live @bn-publisher set should contain 10.1.0.1/32, but the on-disk file must not:

   sudo nft list set inet weaver bn-publisher        # shows 10.1.0.1/32 (live)
   grep 10.1.0.1 /etc/solo-provisioner/network-weaver.nft   # no match (membership not persisted)
   cat /etc/solo-provisioner/policies/bn-publisher.json     # registry entry present

3. Add a quarantine (deny) policy and confirm the drops render above ct state established,related accept. Deliberately omit --pod-cidr here — --deny never references POD_CIDR itself, so it must not try to auto-detect it. Note that the merged chain does still include bn-publisher (a --stamp policy, from step 2) — this call succeeding with no --pod-cidr and no cluster reachable demonstrates the recovery path: the CIDR is pulled back out of the existing network-weaver.nft (10.4.0.0/24, from step 2), not re-detected or required:

   sudo solo-provisioner network policy create --name bn-restricted \
     --deny --cidrs 10.99.0.0/16
   sudo nft -a list table inet weaver | grep -nE 'drop|established|10.4.0.0'

   Expect ip saddr @bn-restricted drop / ip daddr @bn-restricted drop to appear before the ct state established,related accept line, no pod-CIDR detection log line, no failure even if this VM has no cluster configured, and bn-publisher's rule still correctly showing ip daddr 10.4.0.0/24 ... (recovered, not lost).

4. Add a backfill reply-stamp policy and confirm the compound set + auto-rendered restore rule:

   sudo solo-provisioner network policy create --name bn-backfill \
     --stamp reserve-egress --reply-stamp backfill-response \
     --cidrs 10.30.5.7:43473 --pod-cidr 10.4.0.0/24
   sudo nft list table inet weaver | grep -E 'ct direction reply|ct mark set 0x20|bn-backfill'

   Expect the set bn-backfill { type ipv4_addr . inet_service; }, the egress forward rule with ct mark set 0x20 meta priority set 1:60, and the ingress restore ct direction reply ct mark 0x20 meta priority set 1:20 accept. (ct mark stays raw hex on read-back; only meta priority gets the classid reformat — see the note in step 2.)

5. Create-if-missing and --force:

   • Re-run step 2 verbatim → warns network policy already exists — supplied flags/cidrs were not applied; pass --force to replace; nft list set inet weaver bn-publisher still shows exactly 10.1.0.1/32.
   • Re-run step 2 with a different --ports and a different --cidrs, still without --force → same warning, same no-op. Differing flags don't matter without --force.
   • Re-run with --force:

     sudo solo-provisioner network policy create --name bn-publisher \
       --ports 40840,40841 --stamp publisher \
       --cidrs 10.1.0.2/32 --pod-cidr 10.4.0.0/24 --force

     Expect the port set to now show { 40840, 40841 }, and nft list set inet weaver bn-publisher to show only 10.1.0.2/32 — --force replaces membership, it does not merge with 10.1.0.1/32.

6. Missing-table self-heal without --force (this is what actually surfaced the bugs above). Note: nft delete table only deletes the kernel table — it does not touch /etc/solo-provisioner/policies/*.json (the registry). If you want a true clean-slate reset for testing (not this step, which specifically wants the registry to still think the policy exists), delete both: sudo nft delete table inet weaver && sudo rm -f /etc/solo-provisioner/policies/*.json.

   sudo nft delete table inet weaver
   sudo solo-provisioner network policy create --name bn-publisher \
     --ports 40840,40841 --stamp publisher \
     --cidrs 10.1.0.99/32 --pod-cidr 10.4.0.0/24

   No --force, and deliberately yet another different --cidrs. The registry still says bn-publisher is configured, but the kernel table is gone — this is also what a reboot looks like before Story 0.3 — inet host reboot persistence (network-host.nft + oneshot) #780 extends the boot oneshot. Expect success (not nft add element ... failed: No such file or directory): the table is recreated from the last registered config (ports 40840,40841, from step 5's --force call), but 10.1.0.99/32 must not appear anywhere and nft list set inet weaver bn-publisher comes back empty — membership was never persisted, so it can't be recovered without --force re-seeding it.

   Re-seed known membership with --force (the table exists again now, so this is a normal replace) so the next step has something to verify:

   sudo solo-provisioner network policy create --name bn-publisher \
     --ports 40840,40841 --stamp publisher \
     --cidrs 10.1.0.1/32 --pod-cidr 10.4.0.0/24 --force

7. --from-entity world fallthrough, and creation-order independence. Create the fallthrough policy first, then the specific one that shares its direction and ports, and confirm the chain still orders specific-before-fallthrough despite the reverse creation order:

   sudo solo-provisioner network policy create --name bn-public-out \
     --stamp public --from-entity world --ports 40980,40981 --pod-cidr 10.4.0.0/24
   sudo solo-provisioner network policy create --name bn-partner-out \
     --stamp partner --ports 40980,40981 --cidrs 10.20.0.0/16 --pod-cidr 10.4.0.0/24
   sudo solo-provisioner network policy create --name bn-subscriber-in \
     --stamp reserve-ingress --from-entity world --ports 40980,40981 --pod-cidr 10.4.0.0/24
   sudo nft list table inet weaver | grep -nE 'bn-partner-out|bn_ports_subscriber|meta priority set 1:(4|5|3)0'

   Expect, in this order: the bn-partner-out rule (ip saddr @bn-partner-out ... meta priority set 1:40) before the bn-public-out rule (... meta priority set 1:50, no ip daddr @set clause — --from-entity world suppresses it) — even though bn-public-out was created first. bn-subscriber-in's rule (ip daddr 10.4.0.0/24 tcp dport ... meta priority set 1:30, no ip saddr @set) confirms the same on the ingress side.

8. Confirm bn-publisher's membership (re-seeded at the end of step 6) survived the three brand-new-policy re-renders just triggered in step 7:

   sudo nft list set inet weaver bn-publisher

   Must still show exactly 10.1.0.1/32 — this is the sibling-membership bug fixed above: each create in step 7 forces a full chain re-render (delete table; add table), which previously wiped every other policy's live membership.

9. Error paths:

   • --stamp bogus exits non-zero with unknown class "bogus": must be one of … and leaves the table unchanged.
   • --stamp publisher --reply-stamp backfill-response (an ingress class with a reply-stamp) → --reply-stamp is only valid when --stamp resolves to an egress class.
   • --stamp reserve-egress --reply-stamp partner (both egress — partner is not the mirror ingress class) → --reply-stamp class "partner" must resolve to an ingress class.
   • Recovery genuinely has nothing to recover from: sudo rm /etc/solo-provisioner/network-weaver.nft (the file itself, independent of the kernel table or the JSON registry), then re-run the step-3 --deny command again. bn-publisher's registry entry is still there, so the merged chain still needs a pod CIDR, and there's no network-weaver.nft left to recover it from → expect the same pod CIDR is required to render a --stamp policy error, not a silent/incorrect success.

10. --pod-cidr hard-error path (no cluster reachable from this VM): omit --pod-cidr entirely, for a new policy name so create-if-missing doesn't short-circuit first —

    sudo solo-provisioner network policy create --name bn-test --stamp publisher --ports 40840

    Expect a hard error ("could not auto-detect the pod CIDR; pass --pod-cidr explicitly"), not a warning-and-continue. This is the deliberate difference from network firewall create, which falls back best-effort when no cluster is reachable — network policy create always requires a resolvable pod CIDR.

11. Boot readiness: systemctl is-enabled solo-provisioner-network-nft.service → enabled.

Risks / rollback

• Wrong rule order = security/QoS bug; mitigated by the golden-file test pinning tier order.
• Membership snapshot/restore adds one nft list set per existing policy before every re-render; bounded by the shared flock (no concurrent mutator, including the future daemon poll loop, can interleave with it), so the snapshot can't go stale mid-Create.
• Shared coupling with firewall: the boot unit (Story 0.3 — inet host reboot persistence (network-host.nft + oneshot) #780) and the apply lock (Story 1.3 — Implement network policy element verbs (add/remove/set/show/delete) #759) are shared; path/service constants are duplicated by value with a matching-values note (hoisting to a shared package is a follow-up).
• The verb is create-if-missing (--force to replace an existing policy). Rollback: nft delete table inet weaver + remove /etc/solo-provisioner/policies/ and /etc/solo-provisioner/network-weaver.nft. No behavior change to firewall or existing commands beyond the one AddCommand line.

Related Issues

• Closes Story 1.2 — Implement network policy create (--stamp/--deny/--reply-stamp) #758

🤖 Generated with Claude Code

[swirlds-automation]

✅ Snyk checks have passed. No issues have been found so far.

Status	Scan Engine	Critical	High	Medium	Low	Total (0)
✅	Open Source Security	0	0	0	0	0 issues
✅	Licenses	0	0	0	0	0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

[Copilot]

Pull request overview

Adds the solo-provisioner network policy create command and a new internal/network/policy implementation for authoring and applying tier-ordered inet weaver nftables policies (stamp/deny/reply-stamp), including on-disk persistence (network-weaver.nft) and a per-policy registry.

Changes:

• Introduce internal/network/policy with policy model + validation, deterministic full-chain renderer (golden-file pinned), registry IO, and a Manager.Create implementation that applies via nft -f and seeds live set membership.
• Wire a new network policy create Cobra verb (flags, pod-CIDR auto-detect, tests) and register it under the network command group.
• Document the new command in docs/quickstart.md with examples and a flags table.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
internal/network/policy/paths.go	Defines inet weaver constants (paths/service/lock) and package-level scope docs
internal/network/policy/class.go	Static QoS class → mark/priority lookup used by --stamp/--reply-stamp
internal/network/policy/policy.go	Policy registry model + flag-combination and input validation
internal/network/policy/render.go	Tier-ordered full-chain renderer + atomic file writer
internal/network/policy/registry.go	Read/write/load-all for per-policy registry JSON files
internal/network/policy/manager.go	Manager.Create orchestration (lock, merge, apply, persist, enable service)
internal/network/policy/nft.go	nft runner implementation (Apply, AddElements, List, Delete, Exists)
internal/network/policy/service_linux.go	Linux systemd unit install/enable logic for boot readiness
internal/network/policy/service_other.go	Non-Linux no-op service enabler for cross-platform unit tests
internal/network/policy/validate_test.go	Unit tests for Policy.Validate and lookupClass
internal/network/policy/policy_test.go	Golden render + tier invariants + create/registry behavior tests
internal/network/policy/testdata/network-weaver.golden.nft	Golden expected nft document contract for render output
cmd/cli/commands/network/policy/policy.go	Adds network policy command group and wires create
cmd/cli/commands/network/policy/create.go	Implements network policy create flag parsing, CIDR resolution, podCIDR detection
cmd/cli/commands/network/policy/policy_test.go	CLI structure/flags tests + end-to-end command execution tests (stubbed manager/detect)
cmd/cli/commands/network/network.go	Registers policy under the network command group
docs/quickstart.md	Adds quickstart docs for creating traffic policies and flag semantics

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated 2 comments.

[alex-au]

Nice!