Skip to content

manager/prow: fix mce release image access#625

Open
AlexNPavel wants to merge 2 commits into
openshift:mainfrom
AlexNPavel:rebuild-release-mce
Open

manager/prow: fix mce release image access#625
AlexNPavel wants to merge 2 commits into
openshift:mainfrom
AlexNPavel:rebuild-release-mce

Conversation

@AlexNPavel

@AlexNPavel AlexNPavel commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Fix access to custom release images for MCE clusters. When ci-operator is used to create a custom release, it will import the release image specified by release-controller and then import the images that the release image references. The release image itself continues to point to the app.ci registry (or the app.ci based quay-proxy), which MCE based clusters cannot use as they don't have the proper pull secret.

This PR fixes the issue by adding a new step after ci-operator runs to create a new release image from the imported images in the stable imagestream created by ci-operator to replace the original release image. It also removes the original permission script used by build and changes build commands to follow the same pattern of using a multistage test that only includes ipi-install-rbac. The original permission script would not work with MCE images due to a permissions issue, so the ipi-install-rbac route was the best way to allow Build and MCECustomImageset jobs to share the same logic.

I have tested this with reference: false releases (5.0.0-0.ci) and reference: true releases (4.22.0-0.ci) and have tested builds to make sure they still work as expected.

Summary by CodeRabbit

  • New Features

    • Added support for MCE custom image jobs to automatically resolve and rewrite release images to use stable image streams.
    • Enhanced job execution so release/version information is consistently provided across both ref-based and non-ref runs.
  • Bug Fixes

    • Fixed job setup so RBAC-related multistage steps apply correctly to both build and MCE custom image workflows.
    • Improved failure behavior by stopping early when the required release version can’t be determined, preventing confusing downstream errors.

@openshift-ci openshift-ci Bot requested review from bradmwilliams and hoxhaeris July 2, 2026 22:06
@openshift-ci

openshift-ci Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: AlexNPavel

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 2, 2026
Fix access to custom release images for MCE clusters. When ci-operator
is used to create a custom release, it will import the release image
specified by release-controller and then import the images that the
release image references. The release image itself continues to point to
the app.ci registry (or the app.ci based quay-proxy), which MCE based
clusters cannot use as they don't have the proper pull secret.

This PR fixes the issue by adding a new step after ci-operator runs to
create a new release image from the imported images in the `stable`
imagestream created by ci-operator to replace the original release
image. It also removes the original permission script used by `build`
and changes `build` commands to follow the same pattern of using a
multistage test that only includes `ipi-install-rbac`. The original
permission script would not work with MCE images due to a permissions
issue, so the `ipi-install-rbac` route was the best way to allow `Build`
and `MCECustomImageset` jobs to share the same logic.

I have tested this with reference: false releases (5.0.0-0.ci) and
reference: true releases (4.22.0-0.ci) and have tested builds to make
sure they still work as expected.
@AlexNPavel AlexNPavel force-pushed the rebuild-release-mce branch from fe7c79d to af7aaac Compare July 2, 2026 22:07
@coderabbitai

coderabbitai Bot commented Jul 2, 2026

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: a58d98d7-fb0a-4d1f-8ca2-893d2eeaee24

📥 Commits

Reviewing files that changed from the base of the PR and between af7aaac and 1024f01.

📒 Files selected for processing (1)
  • pkg/manager/prow.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • pkg/manager/prow.go

📝 Walkthrough

Walkthrough

This PR modifies pkg/manager/prow.go to derive MCE release versions, replace the old permissions helper with a release rewrite script, and update job command assembly for refs and non-refs MCE custom image execution paths.

Changes

MCE custom image release rewrite

Layer / File(s) Summary
Release helper and rewrite script
pkg/manager/prow.go
Adds mceReleaseName for release derivation and replaces permissionsScript with mceReleaseRewriteScript that rewrites the release image using oc adm release info and oc adm release new.
Job setup and refs command path
pkg/manager/prow.go
Expands RBAC step selection to build jobs, computes mceReleaseVersion early for MCE custom image jobs, and changes refs-based command construction for build and MCE custom image modes.
Non-refs container rewrite
pkg/manager/prow.go
Rewrites the non-refs MCE custom image container command and args to pass registry host, release name, oc client version, namespace handling, and the release rewrite script.

Estimated code review effort: 4 (Complex) | ~45 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately reflects the main change: fixing MCE release image access in manager/prow.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@AlexNPavel

Copy link
Copy Markdown
Contributor Author

/hold

@openshift-ci openshift-ci Bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 2, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
pkg/manager/prow.go (1)

1624-1629: 🩺 Stability & Availability | 🔵 Trivial | 💤 Low value

Minor: mkdir /tmp/bin can fail if the dir already exists; prefer mkdir -p.

Under the hasRefs path this script runs with set -e, so a pre-existing /tmp/bin would abort the step. Using mkdir -p makes the rewrite idempotent and robust to future changes in the surrounding command composition.

♻️ Proposed tweak
-mkdir /tmp/bin
+mkdir -p /tmp/bin
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/manager/prow.go` around lines 1624 - 1629, The mceReleaseRewriteScript
setup is not idempotent because `mkdir /tmp/bin` can fail if the directory
already exists. Update the script in the `mceReleaseRewriteScript` constant to
use `mkdir -p` before adding `oc` to `/tmp/bin`, so the `hasRefs` path remains
safe under `set -e` and the rewrite logic is robust if the directory is
pre-created.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/manager/prow.go`:
- Around line 898-923: The MCE custom image branch in prow.go builds a shell
command for the ci-operator path without fail-fast handling, so a failed
ci-operator invocation can still fall through into the release rewrite step.
Update the JobTypeMCECustomImage / !hasRefs block in the relevant prow command
assembly to include the same error-exit semantics used by the hasRefs path (via
the shared script or equivalent shell flags), ensuring mceReleaseRewriteScript
only runs after a successful ci-operator execution. Verify the command
construction around container.Command, container.Args, and the ci-operator $@
invocation preserves early exit behavior.

---

Nitpick comments:
In `@pkg/manager/prow.go`:
- Around line 1624-1629: The mceReleaseRewriteScript setup is not idempotent
because `mkdir /tmp/bin` can fail if the directory already exists. Update the
script in the `mceReleaseRewriteScript` constant to use `mkdir -p` before adding
`oc` to `/tmp/bin`, so the `hasRefs` path remains safe under `set -e` and the
rewrite logic is robust if the directory is pre-created.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: d256b69b-f59a-4827-b0d0-d8d32e91da15

📥 Commits

Reviewing files that changed from the base of the PR and between 33dfc66 and af7aaac.

📒 Files selected for processing (1)
  • pkg/manager/prow.go

Comment thread pkg/manager/prow.go
@AlexNPavel AlexNPavel force-pushed the rebuild-release-mce branch from 7d5ce9a to 1024f01 Compare July 2, 2026 23:02
@openshift-ci

openshift-ci Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

@AlexNPavel: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant