Skip to content

MCO-1906: Add Extra Validations - #6357

Open
anandram2 wants to merge 1 commit into
openshift:mainfrom
anandram2:add-extra-validations
Open

MCO-1906: Add Extra Validations#6357
anandram2 wants to merge 1 commit into
openshift:mainfrom
anandram2:add-extra-validations

Conversation

@anandram2

@anandram2 anandram2 commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

- What I did

  • validates the OCP version by fetching the os release file(through inspectorFactory), then checking for a digest comparison against the clusters resolved base OS image when thats not available
  • Verifies registry accessibility as a side effect if it is not able to fetch the image at both steps
  • Added unit tests covering version match/mismatch, digest fallback, missing-label warn-through, and registry-failure cases.

- How to verify it

-go test ./pkg/controller/bootstrap/...

or

  • Seed a MachineOSConfig with the machineconfiguration.openshift.io/pre-built-image annotation pointing at a digested image whose /etc/os-release OPENSHIFT_VERSION doesn't match the cluster's release version, bootstrap should fail with an error like pre-built image "..." OCP version "X.Y" does not match the cluster's OCP version "A.B".
  • Point the annotation at an unreachable/nonexistent registry, bootstrap should fail with could not access pre-built image "..." (registry unreachable or image not found): ....
  • Point the annotation at a correctly-versioned pre-built image, bootstrap should succeed as before.

- Description for the changelog
Add OCP-version-match and registry-accessibility validation for hybrid-OCL pre-built images at bootstrap time
MCO-1906

Summary by CodeRabbit

  • Bug Fixes
    • Improved validation of pre-built images during installation.
    • Checks image compatibility with the cluster’s OpenShift major and minor versions.
    • Supports registry inspection and digest-based fallback validation when image metadata is unavailable.
    • Handles image inspection failures more clearly.
    • Improves error handling when creating machine configurations.
    • Expanded coverage for image parsing, version comparison, fallback validation, and related error scenarios.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jul 31, 2026
@openshift-ci-robot

openshift-ci-robot commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

@anandram2: This pull request references MCO-1906 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "5.0.0" version, but no target version was set.

Details

In response to this:

- What I did

  • validates the OCP version by fetching the os release file(through inspectorFactory), then checking for a digest comparison against the clusters resolved base OS image when thats not available
  • Verifies registry accessibility as a side effect if it is not able to fetch the image at both steps
  • Added unit tests covering version match/mismatch, digest fallback, missing-label warn-through, and registry-failure cases.

- How to verify it

-go test ./pkg/controller/bootstrap/...

or

  • Seed a MachineOSConfig with the machineconfiguration.openshift.io/pre-built-image annotation pointing at a digested image whose /etc/os-release OPENSHIFT_VERSION doesn't match the cluster's release version, bootstrap should fail with an error like pre-built image "..." OCP version "X.Y" does not match the cluster's OCP version "A.B".
  • Point the annotation at an unreachable/nonexistent registry, bootstrap should fail with could not access pre-built image "..." (registry unreachable or image not found): ....
  • Point the annotation at a correctly-versioned pre-built image, bootstrap should succeed as before.

- Description for the changelog
Add OCP-version-match and registry-accessibility validation for hybrid-OCL pre-built images at bootstrap time
MCO-1906

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci

openshift-ci Bot commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: anandram2
Once this PR has been reviewed and has the lgtm label, please assign yuqi-zhang for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@coderabbitai

coderabbitai Bot commented Jul 31, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@anandram2, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 52 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 815e06a8-40bd-49f4-ad86-f5df8809da7b

📥 Commits

Reviewing files that changed from the base of the PR and between aac0a16 and 67aa42a.

📒 Files selected for processing (2)
  • pkg/controller/bootstrap/bootstrap.go
  • pkg/controller/bootstrap/bootstrap_test.go

Walkthrough

Bootstrap now validates install-time pre-built images with a timeout-bound image inspector. Validation checks registry access, OpenShift version metadata, and base-OS digest compatibility before creating component MachineConfigs.

Changes

Pre-built image validation

Layer / File(s) Summary
Inspector wiring and bootstrap integration
pkg/controller/bootstrap/bootstrap.go, pkg/controller/bootstrap/bootstrap_test.go
Bootstrap creates a timeout-bound system context and passes a context-bound image inspector into pre-built image processing. Tests add configurable inspection fakes and verify factory use.
Image compatibility and fallback validation
pkg/controller/bootstrap/bootstrap.go
Pre-built images are checked for registry access and OpenShift major/minor compatibility. Validation reads /etc/os-release metadata and uses base-OS digest fallback checks.
Validation fixtures and MachineConfig coverage
pkg/controller/bootstrap/bootstrap_test.go
Tests cover digest parsing, version comparison, metadata extraction, digest fallback, inspection errors, per-image validation, and MachineConfig creation paths.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Bootstrap
  participant SystemContextFactory
  participant ImagesInspector
  participant ClusterVersion
  participant MachineOSConfig
  Bootstrap->>SystemContextFactory: create timeout-bound system context
  Bootstrap->>ImagesInspector: inspect pre-built image
  ImagesInspector->>ImagesInspector: read registry metadata and /etc/os-release
  Bootstrap->>ClusterVersion: compare OpenShift major/minor version
  Bootstrap->>MachineOSConfig: append validated component MachineConfigs
Loading

Suggested reviewers: pablintino, harshwardhanpatil07


Important

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

❌ Failed checks (1 error, 1 inconclusive)

Check name Status Explanation Resolution
No-Sensitive-Data-In-Logs ❌ Error New klog warnings include full pre-built and cluster base-OS image references, which can expose private registry hostnames; fallback errors also include image details. Redact registry hosts and other image-reference details before logging or returning validation errors. Log only a safe identifier such as a digest suffix.
Title check ❓ Inconclusive The title relates to the validation changes but does not identify the pre-built image or bootstrap scope. Specify that the pull request adds bootstrap validation for hybrid-OCL pre-built images.
✅ Passed checks (13 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed The PR uses standard Go tests, not Ginkgo. New t.Run labels are literals or static table names; no runtime-derived names, generated identifiers, or Ginkgo title calls were found.
Test Structure And Quality ✅ Passed The changed file uses Go's testing package, not Ginkgo; it has no cluster resources or Eventually/Consistently waits, and temporary directories use t.TempDir cleanup.
Microshift Test Compatibility ✅ Passed The PR adds Go unit tests in pkg/controller/bootstrap/bootstrap_test.go using testing/testify; no new Ginkgo e2e tests or MicroShift-sensitive API references were added.
Single Node Openshift (Sno) Test Compatibility ✅ Passed The PR adds only standard Go unit tests in pkg/controller/bootstrap/bootstrap_test.go; it adds no Ginkgo e2e tests or multi-node assumptions requiring SNO review.
Topology-Aware Scheduling Compatibility ✅ Passed The bootstrap changes validate pre-built images and create MachineConfigs with OSImageURL only; they add no pod scheduling constraints, replicas, affinities, selectors, tolerations, or PDBs.
Ote Binary Stdout Contract ✅ Passed The PR changes only bootstrap code. The OTE main and BeforeAll setup contain no stdout writes, and the added klog warning is outside OTE process-level code.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed No new Ginkgo e2e tests were added. The bootstrap tests use Go testing with fake image inspectors and no IPv4 literals or external network access.
No-Weak-Crypto ✅ Passed PR code uses SHA-256 image digests only; no MD5, SHA-1, DES, RC4, 3DES, Blowfish, ECB, custom crypto, or secret comparisons were added.
Container-Privileges ✅ Passed The PR changes only two Go files. No container or Kubernetes manifest changes, and no added privilege settings such as privileged, hostPID, hostNetwork, hostIPC, SYS_ADMIN, or allowPrivilegeEscalat...
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (1)
pkg/controller/bootstrap/bootstrap.go (1)

371-376: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Consider a per-image deadline for pre-built image validation.

One time.Minute budget covers validation of every MachineOSConfig. Each validation may pull and unpack image layers to read /etc/os-release, then also call Inspect. With several MachineOSConfigs or a slow registry, the deadline can expire and createPreBuiltImageMachineConfigs returns a hard bootstrap error. Scale the budget with the number of images, or derive a per-image context inside the loop.

defer preBuiltImageCancel() also runs only when Run returns, not when the block ends. Moving the context creation into createPreBuiltImageMachineConfigs scopes both concerns.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/controller/bootstrap/bootstrap.go` around lines 371 - 376, Update the
pre-built image validation flow around createPreBuiltImageMachineConfigs so the
timeout budget is scoped per image or scales with the number of
MachineOSConfigs, rather than applying one fixed minute to the entire batch.
Move context creation and cancellation into createPreBuiltImageMachineConfigs or
otherwise ensure each validation receives its own deadline and cancellation
occurs when that validation/block completes; preserve the existing bootstrap
error behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/controller/bootstrap/bootstrap.go`:
- Around line 841-851: In the pre-built image validation flow around
validatePreBuiltImageDigestFallback, add a guard after the results[0].Error
check to detect nil results[0].InspectInfo and return a descriptive error
instead of dereferencing it. Extend TestValidatePreBuiltImageVersion with a case
where both Error and InspectInfo are nil, preserving the existing validation
path for valid inspection results.
- Around line 809-812: Update the bootstrap validation around imageDigest and
expectedBaseOSImage so tag-form cluster base OS images do not return a hard
error when digest resolution fails. Log a warning and skip verification,
matching the existing missing-OPENSHIFT_VERSION handling, while preserving the
error path for other invalid image conditions.
- Around line 828-838: Update the bootstrap setup in runBootstrapCmd to bind the
cluster release version into the bootstrap container’s RELEASE_VERSION
environment variable before validatePreBuiltImageVersion compares it with the
pre-built image version; preserve the existing --release-image handling and
ensure the value comes from the cluster release version already available to
bootstrap.

---

Nitpick comments:
In `@pkg/controller/bootstrap/bootstrap.go`:
- Around line 371-376: Update the pre-built image validation flow around
createPreBuiltImageMachineConfigs so the timeout budget is scoped per image or
scales with the number of MachineOSConfigs, rather than applying one fixed
minute to the entire batch. Move context creation and cancellation into
createPreBuiltImageMachineConfigs or otherwise ensure each validation receives
its own deadline and cancellation occurs when that validation/block completes;
preserve the existing bootstrap error behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 868c8848-2ad8-413a-a63a-49dbf40ecbb9

📥 Commits

Reviewing files that changed from the base of the PR and between 75bdcd8 and fe24879.

📒 Files selected for processing (2)
  • pkg/controller/bootstrap/bootstrap.go
  • pkg/controller/bootstrap/bootstrap_test.go

Comment thread pkg/controller/bootstrap/bootstrap.go
Comment on lines +828 to +838
imgVersion, versionErr := openshiftVersionFromImage(ctx, inspector, imageSpec)
if versionErr == nil {
matches, err := sameMajorMinor(imgVersion, version.ReleaseVersion)
if err != nil {
return fmt.Errorf("could not compare OCP versions for pre-built image %q: %w", imageSpec, err)
}
if !matches {
return fmt.Errorf("pre-built image %q OCP version %q does not match the cluster's OCP version %q", imageSpec, imgVersion, version.ReleaseVersion)
}
return nil
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Inspect how ReleaseVersion is defined and populated.
fd -t f 'version.go' -p pkg/version --exec cat -n {}

# Find any injection of the release version (env var, ldflags, flags).
rg -n 'RELEASE_VERSION|ReleaseVersion' --type go -g '!vendor/**' -C 3
rg -n 'ReleaseVersion' --iglob '*.yaml' --iglob 'Makefile' --iglob 'Dockerfile*' -C 3

Repository: openshift/machine-config-operator

Length of output: 2499


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== files =="
git ls-files | rg '(^|/)(Dockerfile|.*Dockerfile.*|.*\.ya?ml$|Makefile|.*Makefile.*)$' || true

echo "== bootstrap outline =="
ast-grep outline pkg/controller/bootstrap/bootstrap.go --match main --view expanded || true
ast-grep outline pkg/controller/bootstrap/bootstrap.go --match controllerStart --view expanded || true

echo "== main/start references =="
rg -n 'bootstrap|bootstrap-|MachineConfigBootstrap|controller-start|bootstrapController|ReleaseVersion|RELEASE_VERSION' --type go -g '!vendor/**' -C 3

echo "== Docker/build manifests references =="
rg -n 'RELEASE_VERSION|ReleaseVersion|bootstrap|machine-config|make' -g 'Dockerfile*' -g '*.yaml' -g 'Makefile' -g 'Makefile*' -g '*.mk' -C 4 || true

Repository: openshift/machine-config-operator

Length of output: 50389


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== bootstrap.go package/main outline small slices =="
sed -n '1,140p' pkg/controller/bootstrap/bootstrap.go
printf '\n--- bootstrap.go around prebuilt check ---\n'
sed -n '808,848p' pkg/controller/bootstrap/bootstrap.go

echo "== Go sources mentioning bootstrap command/env injection =="
rg -n 'bootstrap[- ]|bootstrap-|MachineConfigBootstrap|mco-bootstrap|ReleaseVersion|RELEASE_VERSION|os.Setenv|environmentFrom|env' \
  --type go --glob 'cmd/**' --glob 'pkg/**' -C 2 || true

echo "== focused build env var refs =="
rg -n 'RELEASE_VERSION|ReleaseVersion|LD_FLAGS|GOFLAGS|make binary|machine-config-operator|bootstrap' \
  Dockerfile Dockerfile.rhel7 Makefile .ci-operator.yaml --C 4 || true

Repository: openshift/machine-config-operator

Length of output: 7738


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== command entrypoints =="
git ls-files cmd pkg/controller | rg '(^cmd/|bootstrap|machine|controller)' | sed -n '1,200p'

echo "== find command files by path =="
git ls-files cmd | sed -n '1,200p'
git ls-files 'pkg/cmd*' 'pkg/controller/**/**' | sed -n '1,200p'

echo "== release version exact refs outside tests/vendor =="
rg -n 'RELEASE_VERSION|ReleaseVersion' --glob '!vendor/**' --glob '!**/testdata/**' --glob '!**/*test*.go' --glob '!pkg/version/version.go' -C 3 || true

echo "== build manifest exact refs outside tests/vendor =="
rg -n 'RELEASE_VERSION|ReleaseVersion' Dockerfile Dockerfile.rhel7 Makefile .ci-operator.yaml --C 4 || true

Repository: openshift/machine-config-operator

Length of output: 22290


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== bootstrap command file =="
sed -n '1,220p' cmd/machine-config-operator/bootstrap.go
printf '\n--- version.go for command ---\n'
sed -n '1,220p' cmd/machine-config-operator/version.go

echo "== bootstrap bootstrap.go helper implementation section =="
rg -n 'func (b \*Bootstrap) Run|validatePreBuiltImageVersion|openshiftVersionFromImage|sameMajorMinor|ReleaseVersion|manifestDir|pullSecretFile|destDir' pkg/controller/bootstrap/bootstrap.go -C 3

echo "== full bootstrap.go helper lines with validate/openshift version =="
python3 - <<'PY'
from pathlib import Path
p=Path('pkg/controller/bootstrap/bootstrap.go')
text=p.read_text()
lines=text.splitlines()
for name in ['validatePreBuiltImageVersion','openshiftVersionFromImage','sameMajorMinor']:
    try:
        i=next(lines.index(f'{line} {name}') for line in ['func','func(') for start in range(len(lines)) if lines[start]==f'func {name}')
    except StopIteration:
        continue
PY

Repository: openshift/machine-config-operator

Length of output: 17318


Bind the cluster release version for bootstrap before comparing pre-built images.

mco-bootstrap/runBootstrapCmd only receives --release-image, while validatePreBuiltImageVersion compares OPENSHIFT_VERSION against version.ReleaseVersion, whose only initialization is from RELEASE_VERSION. Without binding RELEASE_VERSION for the bootstrap container, images reporting 4.x fail the comparison and manifest rendering can stop.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/controller/bootstrap/bootstrap.go` around lines 828 - 838, Update the
bootstrap setup in runBootstrapCmd to bind the cluster release version into the
bootstrap container’s RELEASE_VERSION environment variable before
validatePreBuiltImageVersion compares it with the pre-built image version;
preserve the existing --release-image handling and ensure the value comes from
the cluster release version already available to bootstrap.

Comment thread pkg/controller/bootstrap/bootstrap.go
@anandram2
anandram2 force-pushed the add-extra-validations branch from aac0a16 to 67aa42a Compare August 3, 2026 19:14
@anandram2

Copy link
Copy Markdown
Contributor Author

/retest

@openshift-ci

openshift-ci Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

@anandram2: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/unit 67aa42a link true /test unit
ci/prow/bootstrap-unit 67aa42a link true /test bootstrap-unit

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants