Skip to content

CMP-3795, CMP-3796: Enhance TestScheduledSuiteUpdate#1185

Open
taimurhafeez wants to merge 4 commits into
ComplianceAsCode:masterfrom
taimurhafeez:CMP-3795-96-enhance-TestScheduledSuiteUpdate
Open

CMP-3795, CMP-3796: Enhance TestScheduledSuiteUpdate#1185
taimurhafeez wants to merge 4 commits into
ComplianceAsCode:masterfrom
taimurhafeez:CMP-3795-96-enhance-TestScheduledSuiteUpdate

Conversation

@taimurhafeez
Copy link
Copy Markdown
Collaborator

@taimurhafeez taimurhafeez commented Apr 22, 2026

Supersedes PR-1021

  1. Enhance TestScheduledSuiteUpdate by adding resource verification for suitererunner jobs
  2. Enhance TestScheduledSuiteUpdate by adding compliancesuite rerun check after schedule update

Tested on OCP 4.22:
make e2e-parallel E2E_GO_TEST_FLAGS="-v -test.timeout 120m -run TestScheduledSuiteUpdate"

=== RUN   TestScheduledSuiteUpdate
=== PAUSE TestScheduledSuiteUpdate
=== CONT  TestScheduledSuiteUpdate
2026/05/08 11:02:06 waiting until suite test-scheduled-suite-update reaches target status 'DONE'. Current status: RUNNING
2026/05/08 11:02:11 waiting until suite test-scheduled-suite-update reaches target status 'DONE'. Current status: RUNNING
2026/05/08 11:02:16 waiting until suite test-scheduled-suite-update reaches target status 'DONE'. Current status: RUNNING
2026/05/08 11:02:21 waiting until suite test-scheduled-suite-update reaches target status 'DONE'. Current status: RUNNING
2026/05/08 11:02:26 waiting until suite test-scheduled-suite-update reaches target status 'DONE'. Current status: RUNNING
2026/05/08 11:02:31 waiting until suite test-scheduled-suite-update reaches target status 'DONE'. Current status: RUNNING
2026/05/08 11:02:36 waiting until suite test-scheduled-suite-update reaches target status 'DONE'. Current status: RUNNING
2026/05/08 11:02:41 waiting until suite test-scheduled-suite-update reaches target status 'DONE'. Current status: RUNNING
2026/05/08 11:02:46 waiting until suite test-scheduled-suite-update reaches target status 'DONE'. Current status: RUNNING
2026/05/08 11:02:51 waiting until suite test-scheduled-suite-update reaches target status 'DONE'. Current status: RUNNING
2026/05/08 11:02:56 waiting until suite test-scheduled-suite-update reaches target status 'DONE'. Current status: AGGREGATING
2026/05/08 11:03:01 waiting until suite test-scheduled-suite-update reaches target status 'DONE'. Current status: AGGREGATING
2026/05/08 11:03:11 ComplianceScan ready (DONE)
2026/05/08 11:03:11 All scans in ComplianceSuite have finished (test-scheduled-suite-update)
2026/05/08 11:03:16 Found test-scheduled-suite-update-rerunner CronJob
2026/05/08 11:03:22 Found test-scheduled-suite-update-rerunner CronJob
2026/05/08 11:03:32 ComplianceScan ready (DONE)
2026/05/08 11:03:32 All scans in ComplianceSuite have finished (test-scheduled-suite-update)
2026/05/08 11:03:38 Initial scan index set to 0. Waiting for re-scan
2026/05/08 11:03:43 re-scan hasn't taken place. CurrentIndex 0. Waiting for re-scan
2026/05/08 11:03:48 re-scan hasn't taken place. CurrentIndex 0. Waiting for re-scan
2026/05/08 11:03:53 re-scan hasn't taken place. CurrentIndex 0. Waiting for re-scan
2026/05/08 11:03:58 re-scan hasn't taken place. CurrentIndex 0. Waiting for re-scan
2026/05/08 11:04:03 re-scan hasn't taken place. CurrentIndex 0. Waiting for re-scan
2026/05/08 11:04:08 Waiting for run of test-scheduled-suite-update-workers-scan compliancescan (RUNNING)
2026/05/08 11:04:13 Waiting for run of test-scheduled-suite-update-workers-scan compliancescan (RUNNING)
2026/05/08 11:04:18 Waiting for run of test-scheduled-suite-update-workers-scan compliancescan (RUNNING)
2026/05/08 11:04:23 Waiting for run of test-scheduled-suite-update-workers-scan compliancescan (RUNNING)
2026/05/08 11:04:28 Waiting for run of test-scheduled-suite-update-workers-scan compliancescan (RUNNING)
2026/05/08 11:04:33 Waiting for run of test-scheduled-suite-update-workers-scan compliancescan (RUNNING)
2026/05/08 11:04:38 Waiting for run of test-scheduled-suite-update-workers-scan compliancescan (RUNNING)
2026/05/08 11:04:43 Waiting for run of test-scheduled-suite-update-workers-scan compliancescan (RUNNING)
2026/05/08 11:04:48 Waiting for run of test-scheduled-suite-update-workers-scan compliancescan (RUNNING)
2026/05/08 11:04:53 Waiting for run of test-scheduled-suite-update-workers-scan compliancescan (RUNNING)
2026/05/08 11:04:58 Waiting for run of test-scheduled-suite-update-workers-scan compliancescan (AGGREGATING)
2026/05/08 11:05:03 Waiting for run of test-scheduled-suite-update-workers-scan compliancescan (AGGREGATING)
2026/05/08 11:05:08 ComplianceScan ready (DONE)
2026/05/08 11:05:08 Suite rerunner job test-scheduled-suite-update-rerunner-29637244 has correct resource limits and requests
--- PASS: TestScheduledSuiteUpdate (188.28s)
PASS

xiaojiey and others added 2 commits March 23, 2026 14:05
…suitererunner Jobs

Ensuring the suite schedule works after the schedule change
@openshift-ci-robot
Copy link
Copy Markdown
Collaborator

@taimurhafeez: This pull request references CMP-3795 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "5.0.0" version, but no target version was set.

This pull request references CMP-3796 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Supersedes PR-1021

  1. Enhance TestScheduledSuiteUpdate by adding resource verification for suitererunner jobs
  2. Enhance TestScheduledSuiteUpdate by adding compliancesuite rerun check after schedule update

Tested on OCP 4.22:
make e2e-parallel E2E_GO_TEST_FLAGS="-v -run TestScheduledSuiteUpdate"

--- PASS: TestScheduledSuiteUpdate (248.03s)

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci
Copy link
Copy Markdown

openshift-ci Bot commented Apr 22, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: taimurhafeez

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@github-actions
Copy link
Copy Markdown

🤖 To deploy this PR, run the following command:

make catalog-deploy CATALOG_IMG=ghcr.io/complianceascode/compliance-operator-catalog:1185-2c6c54351e19bf295d5bd64b7e0c5268015b8628

@taimurhafeez
Copy link
Copy Markdown
Collaborator Author

/test e2e-aws-parallel-arm

1 similar comment
@taimurhafeez
Copy link
Copy Markdown
Collaborator Author

/test e2e-aws-parallel-arm

@taimurhafeez
Copy link
Copy Markdown
Collaborator Author

/test e2e-aws-serial-arm

Copy link
Copy Markdown
Collaborator

@rhmdnd rhmdnd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could make this a bit more strict (assuming we don't have more than one job per suite, which I don't think we do?) - so it's more clear what the expectation and contract is.

Otherwise, this looks great.

Comment thread tests/e2e/framework/common.go Outdated
Comment thread tests/e2e/framework/common.go
Comment thread tests/e2e/parallel/main_test.go Outdated
@taimurhafeez taimurhafeez requested a review from rhmdnd May 8, 2026 10:11
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 8, 2026

🤖 To deploy this PR, run the following command:

make catalog-deploy CATALOG_IMG=ghcr.io/complianceascode/compliance-operator-catalog:1185-c4f7d1185d246c65f2b909b0059cc6cbd761fbb1

@openshift-ci
Copy link
Copy Markdown

openshift-ci Bot commented May 8, 2026

@taimurhafeez: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-rosa c4f7d11 link true /test e2e-rosa
ci/prow/e2e-aws-serial-arm c4f7d11 link true /test e2e-aws-serial-arm
ci/prow/e2e-aws-serial c4f7d11 link true /test e2e-aws-serial
ci/prow/e2e-aws-parallel-arm c4f7d11 link true /test e2e-aws-parallel-arm
ci/prow/e2e-aws-parallel c4f7d11 link true /test e2e-aws-parallel

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants