feat(ec2): per-subnet docker networks for L3 isolation (#1745 phase 2)#1755
Merged
Conversation
Backing containers all shared the default bridge, so instances in different
VPCs could reach each other and there was no L3 segmentation. Attach each
instance's container to a per-subnet daemon network instead:
- RunInstances computes an `InstanceNetwork { subnet_id, internal }` from the
resolved subnet (internal = the subnet has no `0.0.0.0/0 -> igw` route) and
passes it to the runtime.
- The Docker backend ensures `fakecloud-subnet-<id>` exists (idempotent;
`--internal` for private subnets), labels it `fakecloud-subnet=<id>` plus the
shared `fakecloud-instance` ownership label so the startup reaper prunes it,
and attaches the container with `--network`.
- Same-subnet instances share a bridge and can talk; different VPCs/subnets get
different bridges and cannot route to each other. Network creation is
best-effort: on failure the instance still boots on the default bridge (no
regression vs metadata-only).
- k8s pods keep their flat network (isolation there is a NetworkPolicy concern,
phase 4). Subnet placement is captured in the runtime record so persisted
instances recover onto the same network after a restart, and so phase-5
introspection can report the backing network.
Tests: e2e (Docker-gated, hard-fails in CI) proving same-subnet reachability,
cross-VPC isolation (ping passes/fails accordingly), and that private subnets
back onto `--internal` networks while public/default subnets do not.
The per-subnet network arg added for phase-2 changed run_instance's signature; the feature-gated k8s integration test (only compiled in the kind CI job) still called the 3-arg form and would fail to compile there.
ee40f92 to
5b8c1af
Compare
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 2 of EC2 real network isolation (#1745). Stacked on #1754 (phase 1) — base will retarget to
mainonce that merges.Backing containers all shared the default Docker bridge, so instances in different VPCs could reach each other and there was no L3 segmentation. This attaches each instance's container to a per-subnet daemon network:
RunInstancescomputes anInstanceNetwork { subnet_id, internal }from the resolved subnet (internal= the subnet has no0.0.0.0/0 -> igwroute) and passes it to the runtime.fakecloud-subnet-<id>exists (idempotent;--internalfor private subnets), labels itfakecloud-subnet=<id>+ the sharedfakecloud-instanceownership label so the startup reaper prunes it, and attaches with--network.Test plan
crates/fakecloud-e2e/tests/ec2_network_isolation.rs(Docker-gated, hard-fails in CI): same-subnet reachability, cross-VPC isolation (ping passes/fails accordingly), private subnet ->--internalnetwork, public/default subnet -> not internal.cargo test -p fakecloud-ec2,cargo clippy -p fakecloud-ec2 --all-targets -- -D warnings,cargo fmt --all --checkclean.Summary by cubic
Add per-subnet Docker networks for EC2 instances to enforce L3 isolation across VPCs/subnets. Phase 2 of #1745; persists subnet placement so restarts keep the same network; k8s remains flat.
New Features
RunInstancesderivesInstanceNetwork { subnet_id, internal }(private if no0.0.0.0/0 -> igw) and passes it to the runtime.fakecloud-subnet-<id>exists (idempotent), labels itfakecloud-subnet=<id>andfakecloud-instance=<daemon>, attaches with--network; uses--internalfor private subnets. Falls back to the default bridge if creation fails.RunningInstance.network; restarts re-attach to the same network. k8s returnsnetwork: None.--internalon private vs public/default subnets.Bug Fixes
run_instancesignature.clippy -D warnings.Written for commit 800114f. Summary will update on new commits.