Refactor sync test#137
Open
OlufemiAdeOlusile wants to merge 31 commits into
Open
Conversation
…lt, aligning with IOG's current defaults. Fixes broken Preview/Preprod tests caused by hardcoded Mainnet peer snapshot. ## Changes - Removed hardcoded Mainnet peer snapshot file (`sync_tests/data/peersnapshotfile.json`) - Added `remove_json_keys()` helper and `disable_genesis_mode()` function - Added `normalize_peer_snapshot()` to convert IOG's `domain` keys to `address` - Downloads environment-specific peer snapshot from IOG automatically - Replaced `--use-genesis-mode` flag with `--disable-genesis-mode` ## Usage # Genesis mode (default) python sync_tests/tests/node_sync_test.py -e preview -r1 master -t1 my_test # Praos mode (fallback) python sync_tests/tests/node_sync_test.py -e preview -r1 master -t1 my_test --disable-genesis-mode ## Breaking Changes - `-g` / `--use-genesis-mode` flag removed - Genesis mode now ON by default (previously OFF)
Move export_env_var, make_tarfile, write_json_to_file, manage_directory, get_file_sha256_sum, and print_last_n_lines from db_sync.py to helpers.py to eliminate code duplication. Update all call sites in db_sync.py to use helpers functions.
- Add `LOGGER = logging.getLogger(__name__)` - Replace `logging.info()` with `LOGGER.info()` - Replace `logging.error()` with `LOGGER.error()` - Remove `logging.basicConfig()` call
- Remove `sh_colors` class - Remove `print_color_log()` function - Replace calls with `helpers.print_message()` or `LOGGER`
- Create `DbSyncTip` dataclass - Create `PerfStats` dataclass - Update functions to return dataclasses instead of tuples - Update callers to use dataclass fields
- Replace global variables with DbSyncConfig dataclass. All functions now - accept config parameter instead of relying on globals. Updated all test - files to create and pass config explicitly. Matches node_sync_test.py - patterns for better maintainability and thread-safety.
- Replace all os.chdir() calls with subprocess cwd parameter and absolute paths derived from DbSyncConfig.
Extract helper functions from wait_for_db_to_sync() to improve maintainability: - _check_for_rollback() - rollback detection - _log_sync_progress() - progress logging - _collect_perf_stats() - stats collection Also fixes all linting issues and mypy type errors.
- Split db_sync.py into postgres.py, artifacts.py, charts.py, and snapshots.py for better separation of concerns. - Updated aws_db.py to use DbSyncConfig instead of global variables. All functions re-exported from db_sync.py for backward compatibility.
- Add wait_for_shelley_era() function to optimize db-sync start time. - Instead of waiting for full node sync or starting immediately, the test - now waits for the node to reach Shelley era before starting db-sync.
Extract db sync data helper; keep APIs stable
saratomaz
reviewed
Feb 2, 2026
saratomaz
requested changes
Feb 2, 2026
saratomaz
left a comment
saratomaz
left a comment
Contributor
There was a problem hiding this comment.
sync_tests/tests/node_sync_test.py was removed and it's used by buildkite.
saratomaz
approved these changes
Feb 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refactors node sync test configuration to make Genesis mode the default, aligning with IOG's current defaults. Fixes broken Preview/Preprod tests caused by hardcoded Mainnet peer snapshot.
Changes
sync_tests/data/peersnapshotfile.json)remove_json_keys()helper anddisable_genesis_mode()functionnormalize_peer_snapshot()to convert IOG'sdomainkeys toaddress--use-genesis-modeflag with--disable-genesis-modeGenesis mode (default)
python sync_tests/tests/node_sync_test.py -e preview -r1 master -t1 my_test
Praos mode (fallback)
python sync_tests/tests/node_sync_test.py -e preview -r1 master -t1 my_test --disable-genesis-mode
Breaking Changes
-g/--use-genesis-modeflag removed