vyos-benchmarking

Reproducible performance benchmarking for VyOS routers using the Soucy iperf3 methodology. Ansible playbooks orchestrate test node setup, benchmark runs, and teardown. Results are stored as markdown files for version-controlled comparison.

flowchart LR
    subgraph sender["Sender node"]
        c["iperf3 client\n8 cores × 8 streams"]
    end

    subgraph dut["VyOS DUT"]
        router["routing / firewall\n/ NAT"]
    end

    subgraph receiver["Receiver node"]
        s["iperf3 server\n8 cores × 8 streams"]
    end

    c -->|"unidirectional\ntest traffic"| router --> s

Loading

---

Quick start

# 1. Configure inventory — set IPs for sender, receiver, DUT
$EDITOR inventory/hosts.yml

# 2. Set SSH key for test nodes
# ansible_ssh_private_key_file in inventory/hosts.yml

# 3. Prepare test nodes (install iperf3, apply sysctl tuning)
ansible-playbook playbooks/setup.yml

# 4. Run the full benchmark suite (all 6 packet sizes)
ansible-playbook playbooks/benchmark.yml

# 5. Save results
ansible-playbook playbooks/benchmark.yml \
  -e "save_results=true" \
  -e "result_label='VyOS 1.5 forwarding-only'"

# 6. Teardown
ansible-playbook playbooks/teardown.yml

Routed and VPN-managed test nodes

If the sender or receiver is managed through a different router, VPN overlay, or control-plane VLAN than the test path, make the return path explicit before trusting benchmark or reachability results.

Example: a node's default gateway is an upstream firewall, but DNS, package mirrors, VPN clients, or the opposite benchmark subnet must return through the VyOS DUT:

# inventory/group_vars/sender.yml or receiver.yml
bench_static_routes:
  - destination: 192.0.2.0/24      # management / DNS / package services
    gateway: 198.51.100.1          # DUT address on this node's VLAN
    interface: eth1
  - destination: 100.64.0.0/10     # VPN overlay client range
    gateway: 198.51.100.1
    interface: eth1

playbooks/setup.yml applies those routes live. Set benchmark_persist_routes=true for Debian/Proxmox hosts using /etc/network/interfaces.

Packet sizes tested

Size	Category	Multiplier (iperf3 → L3)
128-byte	Small packets	1.6842×
256-byte	Small packets	1.2549×
512-byte	"Typical"	1.1130×
1024-byte	Large	1.0535×
1500-byte	Max Ethernet	1.0359×
9000-byte	"Maximum" (jumbo)	1.0058×

Results are unidirectional L3 Mbps. Double for comparison with vendor aggregate (TX+RX) specifications.

Playbooks

Playbook	Purpose
playbooks/setup.yml	Install iperf3, apply sysctl, copy scripts, set MTU
playbooks/benchmark.yml	Run benchmark suite, collect and display results
playbooks/teardown.yml	Kill iperf3, remove scripts, reset MTU

Repository layout

vyos-benchmarking/
├── ansible.cfg
├── inventory/
│   ├── hosts.yml                 # sender, receiver, dut
│   └── group_vars/
│       ├── all.yml               # shared benchmark parameters
│       ├── sender.yml            # interface, MTU config
│       └── receiver.yml
├── playbooks/
│   ├── setup.yml
│   ├── benchmark.yml
│   └── teardown.yml
├── scripts/
│   ├── perf-client.sh            # iperf3 client (8-core, overhead-adjusted)
│   └── perf-server.sh            # iperf3 server (8-core)
├── results/
│   ├── README.md                 # naming convention, how to commit
│   └── template.md.j2            # Jinja2 template for result files
└── docs/
    ├── hardware-setup.md         # cabling, NIC selection, OS prep
    ├── methodology.md            # Soucy methodology, multiplier table
    ├── interpreting-results.md   # reading output, vendor comparison
    ├── industry-standards.md     # RFC 2544, ITU-T Y.1564, RFC 3511
    ├── advanced-tools.md         # TRex, Pktgen-DPDK, Flent, FD.io CSIT
    └── modern-stack.md           # 4-tier E2E CI benchmarking pipeline

Beyond iperf3

This repo uses iperf3 for practical commodity-hardware benchmarking. For formal RFC 2544 testing, application-layer benchmarking, or CI-integrated performance regression testing:

Need	Tool	Doc
RFC 2544 NDR/PDR	Cisco TRex (DPDK)	advanced-tools.md
Bufferbloat / real TCP	Flent	advanced-tools.md
Formal standards context	RFC 2544, Y.1564	industry-standards.md
CI regression pipeline	4-tier stack	modern-stack.md
FD.io CSIT suite	Robot Framework + TRex	advanced-tools.md

Related

• vyos-ansible — Ansible playbooks for managing the VyOS cluster under test
• Soucy methodology paper

License

MIT — TWN Systems