Skip to content

fix: treat OCM4001 inflight check failure as non-terminal during cluster create - #6463

Open
Cliff Schomburg (cssjr) wants to merge 2 commits into
Azure:mainfrom
cssjr:fix/aro-28755-ocm4001-non-terminal-during-create
Open

fix: treat OCM4001 inflight check failure as non-terminal during cluster create#6463
Cliff Schomburg (cssjr) wants to merge 2 commits into
Azure:mainfrom
cssjr:fix/aro-28755-ocm4001-non-terminal-during-create

Conversation

@cssjr

@cssjr Cliff Schomburg (cssjr) commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • When Cluster Service (CS) reports ClusterStateError with error code OCM4001 (inflight MI permission checks failed) during a CREATE operation, the backend now keeps the operation in Provisioning state instead of marking it as terminal Failed
  • This allows the operation controller to continue polling CS, giving time for delayed role assignments to propagate through Azure RBAC
  • The 60-minute CreateOperationCompletionDeadline provides a hard backstop if CS never recovers
  • Scoped to CREATE only — UPDATE and DELETE operations continue to treat OCM4001 as terminal Failed

Root Cause (ARO-28755)

The E2E test cluster_delayed_role_assignments.go fails ~7% of the time because of a timing race: when Managed Identity role assignments are deployed after cluster creation begins, CS's inflight MI permission checks may exhaust their retry window before the role assignments propagate through Azure RBAC. Once CS reports ClusterStateError, the backend unconditionally mapped this to terminal ProvisioningStateFailed, preventing any recovery even after role assignments were in place.

Test plan

  • Unit tests: 3 new test cases in TestConvertClusterStatus covering OCM4001 during CREATE (stays Provisioning), OCM4001 during UPDATE (stays Failed), and non-OCM4001 during CREATE (stays Failed)
  • All existing backend tests pass with no regressions
  • CI E2E prod/parallel suite validates the delayed role assignment test passes

Resolves: https://redhat.atlassian.net/browse/ARO-28755

🤖 Generated with Claude Code

…ter create

When Cluster Service reports ClusterStateError with error code OCM4001
(inflight MI permission checks failed), the backend now keeps the
operation in Provisioning state instead of marking it as terminal Failed
during CREATE operations. This allows the operation controller to
continue polling CS, giving time for delayed role assignments to
propagate through Azure RBAC.

The 60-minute CreateOperationCompletionDeadline provides a hard backstop
if CS never recovers. The inflight check error details are still captured
in the operation message for observability.

Scoped to CREATE only — UPDATE and DELETE operations continue to treat
OCM4001 as terminal Failed.

Resolves: ARO-28755

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings August 6, 2026 17:54
@openshift-ci

openshift-ci Bot commented Aug 6, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: cssjr
Once this PR has been reviewed and has the lgtm label, please assign geoberle for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@cssjr

Copy link
Copy Markdown
Collaborator Author

/test lint

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 6, 2026 21:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@cssjr

Copy link
Copy Markdown
Collaborator Author

/test e2e-parallel

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants