Implement adaptive attacks: stat-opt, dny-opt, min-max, min-sum#4
Open
Implement adaptive attacks: stat-opt, dny-opt, min-max, min-sum#4
Conversation
Co-authored-by: self1am <117443069+self1am@users.noreply.github.com>
Co-authored-by: self1am <117443069+self1am@users.noreply.github.com>
Co-authored-by: self1am <117443069+self1am@users.noreply.github.com>
Co-authored-by: self1am <117443069+self1am@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Add implementation for adaptive attacks documentation
Implement adaptive attacks: stat-opt, dny-opt, min-max, min-sum
Feb 3, 2026
self1am
approved these changes
Feb 3, 2026
Add comprehensive documentation and tooling to run production FL experiments: documentation index, architecture diagrams, execution checklist, anomaly scoring explanation, quick reference, setup summary, VM troubleshooting, and server logs analysis. Add experiment automation and helper scripts (optimize_gcp_instance.sh, run_production_experiments.sh, test_connection.sh), experiment configs for 100-client runs, analysis/visualization scripts, design assets, and supporting HTML/PNG files. Also update multiple client training log JSONs to include recent run data. These changes provide the docs, configs and scripts needed to run, monitor and analyze large-scale experiments on a GCP instance.
Switch server startup from a background thread to a multiprocessing.Process so the Flower server can register signal handlers from its own main thread. Add process-alive check, reduce server startup wait (5s -> 3s), and implement cleanup/terminate/join logic to ensure graceful shutdown. Relax resource spawning logic in client_orchestrator: lower the minimum available memory threshold to 300MB (from 500MB) and remove the blocking CPU percent check so clients can spawn more aggressively under memory pressure. Add a small test config (experiments/configs/test_multiprocess_fix.yaml) and a MULTIPROCESS_FIX.md summary documenting the fixes and testing instructions. Files changed: src/orchestration/experiment_runner.py, src/orchestration/client_orchestrator.py, experiments/configs/test_multiprocess_fix.yaml, MULTIPROCESS_FIX.md.
Switch orchestrator synchronization from waiting for client processes to waiting for the server process. Add server_process parameter to ClientOrchestrator.run_experiment and ExperimentRunner to block on server_process.join(), then terminate clients after server completes. Redirect client stdout/stderr to logs/client_<id>.log and improve monitoring to surface recent client output on failures (last 500 chars). Update wait_for_completion to support indefinite waiting, longer polling interval, KeyboardInterrupt handling, and ensure clients are cleaned up. These changes fix an infinite "Waiting for N clients" loop by using server-driven completion and provide better client diagnostics.
Replace multiprocessing-based in-process server with an external subprocess to improve signal handling and capture logs. experiment_runner.py: spawn server via subprocess.Popen running run_server_with_eval.py, create logs/ directory and write server output to a per-experiment log file, switch liveness checks from is_alive() to poll(), and use terminate()/wait()/kill() with timeout handling (subprocess.TimeoutExpired). Also add sys import and remove multiprocessing usage. client_orchestrator.py: replace server_process.join() with server_process.wait() to block on subprocess termination. These changes centralize server logs and provide more robust lifecycle handling for the server process.
…r-only mode support
Introduce a new SimulationRunner (src/orchestration/simulation_runner.py) that runs Flower simulations with Ray-backed client execution. Includes an attack factory, centralized evaluation function, multiple aggregation strategy selection, and a client_fn that constructs EnhancedFLClient instances. Add a baseline experiment config for 100 clients (experiments/configs/baseline_100_clients.yaml) and update requirements to include ray>=2.0.0 to support Ray integration.
This reverts commit aa9e311.
Add eight YAML experiment configurations for federated learning poisoning experiments: adaptive_attacks_{cognitive,horizontal,vertical,no_defence}.yaml and static_attacks_{cognitive,horizontal,vertical,no_defence}.yaml. Each file defines experiment metadata (seed, rounds, client counts), defence strategy parameters (cognitive/horizontal/vertical/none), attack settings (static/adaptive poison specifics), orchestration (num_clients, batch sizes, epochs), simulation/Ray arguments, and evaluation sample sizes. These configs provide baselines and defended scenarios to run comparative robustness evaluations.
Add a new CONFIG_SUMMARY.md documenting all 8 experiment configs and expected behavior. Normalize and simplify all experiment YAMLs: unify attack schema (use attack_type, intensity, target_clients, etc.), replace legacy 'static_poison'/'adaptive_poison' fields with explicit attack types (label_flip, stat_opt, dny_opt, min_max, min_sum), and remove obsolete defense-specific parameters (DP/krum/aggregation/extra cognitive tuning) in favor of consistent anomaly_threshold, reputation_decay, and history_size settings. Ensure orchestration.num_clients remains consistent across configs. These changes make configs compatible with the simulation runner and simplify comparative experiments.
Rename the experiment to reflect 40% malicious clients and expand the label-flip target_clients list from 10 (0–9) to 40 (0–39). This config now runs a static label-flip attack with intensity 0.5 affecting 40 out of 100 clients; other orchestration and experiment settings remain unchanged.
Enable Krum aggregation and broaden attack scenarios: update experiment YAML to use strategy=krum, set num_byzantine=30, add multi_krum flag, expand label_flip targets to clients 0-19 and add a gaussian_noise attack for clients 20-29. Extend defenceConfig dataclass with parameters for multiple defense strategies (Krum, Trimmed Mean, VERT) while keeping existing cognitive defense fields and update the docstring to reflect multi-strategy support. These changes add configurability needed to run and tune different robust aggregation methods and attack mixes.
Raise label-flip intensity to 1.0 and expand target clients to first 40 in static_attacks_horizontal_defence.yaml (removed the gaussian_noise attack) to simulate full poisoning and stress-test the defence. Add a new static_attacks_no_defence_STRONG.yaml config that defines a no-defence scenario with 40% malicious clients (clients 0–39), full label-flip intensity, orchestration/simulation tuning, and evaluation settings to provide a strong baseline for comparison.
Add cloud_vm_test.md documenting steps to set up a cloud VM, create a Python venv, use tmux, and run experiment simulations. Update experiment configs (adaptive_attacks_cognitive_defence.yaml and static_attacks_cognitive_defence.yaml) to change defence.strategy from "cognitive" to "cognitive_defence" to match the updated defence naming/implementation.
Add a comprehensive set of experiment YAMLs to standardize baseline and adaptive evaluations: static label-flip, DnyOpt, StatOpt, Min-Max across defence strategies (cognitive_defence, krum, trimmed_mean, vert, none). Include a clean baseline (no attack), unified settings (seed 123, 30 rounds, 100 clients) and target client lists to make comparisons reproducible. Fix a bug in adaptive_attacks_vertical_defence.yaml where strategy was "vertical" (fell through to NoDefence) by changing it to "vert" and adding parameters (kappa, history_size, projection_dim, learning_rate, min_history_rounds). Add copilot_analysis.md containing an extensive analysis and roadmap for improving the cognitive defence and benchmarking. Also add a helper script experiments/scripts/run_baseline_experiments.sh and update a few existing attack/attack-config files to align with the new experiments. These changes prepare the repo for systematic, reproducible experiments and further development of the cognitive defence.
Clamp predictor weight updates to avoid exploding gradients/NaNs. vert_defence.py: compute grad_W = outer(error, p_input) then clip by L2 norm (threshold 1.0) before applying learning_rate step. Added test_vert.py to reproduce and monitor predictor weight norms and NaNs during aggregation with synthetic client updates. Added two small repro scripts (fix_vert.py, fix_vert2.py) that demonstrate input normalization and gradient clipping behavior during iterative updates.
Introduce a POSG-based reinforcement-learning defence and related utilities: add CognitiveDefencePOSG (sac + GRU tracker), SACAgent, and ClientTracker (src/defences/cognitive_defence_posg.py, src/defences/sac_agent.py, src/defences/client_tracker.py) and export it from defences.__init__.yaml. Update baseline experiment configs to use the new cognitive_defence_posg strategy and include SAC/tracker hyperparameters. Fix device handling to avoid MPS/CUDA tensor mismatches in enhanced_client and simulation_runner by moving tensors to the model/device and set PYTORCH_ENABLE_MPS_FALLBACK early. Add CUDA/MPS checks and deterministic CuDNN setup in utils/config. Also add a cognitive_defence_v2 branch in experiment_runner to wire a new aggregation strategy. These changes enable a learned aggregation defence, improve reproducibility, and address Apple Silicon / GPU device issues.
- Add POSG/SAC parameters to defenceConfig dataclass (max_clients, obs_dim, belief_hidden_dim, sac_hidden_dims, lr, gamma, reward_*, buffer_capacity, batch_size, device) - Create src/server/cognitive_defence_posg_server.py that wraps CognitiveDefencePOSG as a Flower FedAvg strategy - Export POSGAggregationStrategy from src/server/__init__.py - Add cognitive_defence_posg branch in simulation_runner._create_strategy() so the runner can instantiate it from YAML config
- Remove extra 'config' positional argument from evaluate() override (Flower calls it as strategy.evaluate(round, parameters) with no config) - Use self.evaluate_fn (FedAvg attribute) instead of self._evaluate_fn - Call evaluate_fn with NDArrays directly, matching FedAvg parent behaviour - Store centralized_accuracy as float for SAC reward computation
Change experiment seed from 123 to 321 in two baseline cognitive defence config files to alter randomness/reproducibility for runs. Affected files: experiments/configs/baseline/01_static_label_flip_cognitive_defence.yaml and experiments/configs/baseline/02_adaptive_dny_opt_cognitive_defence.yaml.
…pact 2xH SAC state, FLTrust pairwise cosine warmup
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds four adaptive attack strategies that learn from defense responses and optimize evasion. Each attack implements a distinct optimization approach based on published adversarial FL research.
Implementations
Base Infrastructure (
adaptive_base.py)AdaptiveAttackclass with feedback collection, detection/acceptance rate tracking, and adaptation hooksAttack Strategies
Integration
Client Runner (
client_runner.py)Extended
create_attack()to instantiate adaptive attacks from YAML configs with attack-specific parameters.Configuration (
config.py)AttackConfigdataclass now includes parameters for all four attacks (constraint_factor, learning_rate, defense_models, distance_weight, etc.). Default initialization ensures uniform threat model weights.Usage
YAML Configuration
Documentation
docs/ADAPTIVE_ATTACKS.md: Algorithm descriptions, mathematical formulations, defense evasion strategies, academic references (Fang et al. USENIX'20, Baruch et al. NeurIPS'19, Shejwalkar & Houmansadr NDSS'21, Bhagoji et al. ICML'19)docs/IMPLEMENTATION_SUMMARY.md: Implementation details, architecture, integration pointsexperiments/configs/{stat_opt,dny_opt,min_max,min_sum}_attack_test.yamlFiles Changed
💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.