Skip to content

Pytest sync test#139

Merged
mkoura merged 1 commit into
mainfrom
pytest-sync-test
Jun 23, 2026
Merged

Pytest sync test#139
mkoura merged 1 commit into
mainfrom
pytest-sync-test

Conversation

@OlufemiAdeOlusile

@OlufemiAdeOlusile OlufemiAdeOlusile commented Feb 17, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR migrates the sync-test flow toward direct pytest-based GitHub Actions execution on self-hosted NixOS runners, replacing the old script-heavy / Buildkite-oriented path for node and db-sync validation.

Key Changes

  • Added GitHub Actions workflows for node sync and full db-sync runs, including inputs for environment, node/db-sync revisions, db-sync start options, cleanup, and artifact upload.
  • Introduced pytest session fixtures for node sync, db-sync setup, Postgres lifecycle, marker writing, and artifact generation.
  • Added phase-aware CI heartbeats so long sync runs show periodic status without flooding the Actions log.
  • Split CI logs into focused files:
    • ci_step.log for nix develop, shell setup, and pytest output.
    • nix_build.log for execute_command() Nix build output.
    • node_sync.log and db_sync.log for process logs.
  • Reduced Actions log noise by lowering routine progress logs, quieting pip in CI, and printing a concise pytest summary at the end.
  • Improved artifact bundling by separating monitor/OOM/Postgres logs into monitor.zip while keeping sync logs/results in their own bundles.
  • Added tests for CI log capture, artifact generation, and disk cleanup behavior.
  • Fixed low-risk cleanup issues:
    • Guarded db-sync epoch export process cleanup.
    • Used captured node base directory during teardown.
    • Closed matplotlib figures after chart generation.
    • Normalized failed-sync logging format.

Test Plan

  • Ran focused pytest checks for disk cleanup and CI log capture.
  • Verified node/db-sync artifact tests collect successfully.
  • Ran Ruff checks/formatting on affected Python files.
  • Manually validated GitHub Actions heartbeat/log behavior through workflow runs.

@OlufemiAdeOlusile OlufemiAdeOlusile force-pushed the pytest-sync-test branch 2 times, most recently from 401ff36 to b60eee9 Compare February 19, 2026 15:40
@OlufemiAdeOlusile OlufemiAdeOlusile force-pushed the pytest-sync-test branch 2 times, most recently from 9046444 to b954623 Compare March 13, 2026 02:43
Comment thread .github/workflows/node_sync_test.yaml Fixed
Comment thread .github/workflows/node_sync_test.yaml Fixed
Comment thread .github/workflows/node_sync_test.yaml Fixed
Comment thread .github/workflows/node_sync_test.yaml Fixed
Comment thread .github/workflows/node_sync_test.yaml Fixed
Comment thread sync_tests/utils/node/__init__.py Fixed
Add pytest fixtures and GitHub Actions for node/db-sync runs, phase-aware
heartbeats, split CI/nix/process logs, artifact bundling, cleanup handling,
and focused tests for disk cleanup, artifact creation, and CI log capture.
Reduce Actions log noise while preserving full logs in artifacts.

Fix repo_lint CI failures: add markdownlint config, fix mypy INTERNAL ERROR
by running as single process, fix generator return type on nix_build_log
fixture, fix CodeQL clear-text logging alert.
@mkoura mkoura merged commit 789073a into main Jun 23, 2026
6 checks passed
@mkoura mkoura deleted the pytest-sync-test branch June 23, 2026 11:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants