Skip to content

NO-JIRA: General payload-monitor improvements and improved UX - #248

Open
vimauro wants to merge 7 commits into
openshift-eng:mainfrom
vimauro:fix/dead-ci-skill-refs-and-legend-overflow
Open

NO-JIRA: General payload-monitor improvements and improved UX#248
vimauro wants to merge 7 commits into
openshift-eng:mainfrom
vimauro:fix/dead-ci-skill-refs-and-legend-overflow

Conversation

@vimauro

@vimauro vimauro commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Summary by CodeRabbit

  • New Features
    • Refreshed dashboard edge failure analysis with precomputed stream views and an updated “nightly failure analysis” layout.
    • AI analysis cards now optionally display job names; code cells support one-click copy with a “Copied!” tooltip.
  • Bug Fixes
    • Nightly stream selection now uses configured version streams (no live discovery).
    • Improved report HTML/JSON loading and patching, including better handling of missing assets and encoded job links.
    • Escalation-risk now tracks the latest failing job’s Prow URL.
  • Documentation
    • Updated deep-analysis guidance to use ci:prow-job-analysis.
  • Chores
    • Bumped the edge-ocp-ci plugin version to 1.2.1.

@openshift-ci

openshift-ci Bot commented Jul 29, 2026

Copy link
Copy Markdown

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 29, 2026
@openshift-ci

openshift-ci Bot commented Jul 29, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: vimauro

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 29, 2026
@coderabbitai

coderabbitai Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Changes

Payload Monitor reporting flow

Layer / File(s) Summary
Configured release streams and shared collection
payload-monitor/payload_monitor/__main__.py, payload-monitor/payload_monitor/collectors/release_controller.py, payload-monitor/payload_monitor/collectors/timing.py, payload-monitor/tests/test_collectors_release_controller.py
Stream names now derive from configured versions, while edge-job retrieval and topology filtering delegate to the Sippy collector.
Analysis model contracts
payload-monitor/payload_monitor/models.py, payload-monitor/payload_monitor/analyzer.py, payload-monitor/tests/test_analyzer.py, payload-monitor/tests/test_models_timing.py
Escalation risks now carry the latest failing job’s Prow URL, and removed timing properties are reflected in the models and tests.
Report context and analysis patching
payload-monitor/payload_monitor/report/generator.py, payload-monitor/payload_monitor/report/templates/_analysis_card.html, payload-monitor/tests/test_report_generator.py
JSON loading errors are centralized, report context includes stream and payload summaries, and analysis cards and HTML patching support job names and escaped URLs.
Dashboard failure-analysis rendering
payload-monitor/payload_monitor/report/templates/dashboard.html, plugins/edge-ocp-ci/skills/generate-dashboard/SKILL.md
The dashboard consumes precomputed summaries, restructures failure details, and uses the unified job-analysis command in rendered guidance.
Dashboard interactions and timing presentation
payload-monitor/payload_monitor/report/templates/scripts.js, payload-monitor/payload_monitor/report/templates/styles.css, payload-monitor/payload_monitor/report/timing_section.py
Filtering targets detail rows, action commands support click-to-copy, AI highlights display job names, and timing statistics use shared formatting with dynamic legends.
Plugin release metadata
plugins/edge-ocp-ci/.claude-plugin/plugin.json, .claude-plugin/marketplace.json
The edge-ocp-ci plugin version is updated from 1.2.0 to 1.2.1.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 10 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 21.21% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (10 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is broad, but it accurately reflects the payload-monitor refactor and UX/reporting improvements in the changeset.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
No-Weak-Crypto ✅ Passed No weak-crypto APIs or custom secret comparisons were introduced in the touched code; repo search only found a docs reference and a bearer-token test string.
Container-Privileges ✅ Passed No changed Kubernetes/container manifests or privilege flags (privileged, hostPID/Network/IPC, SYS_ADMIN, allowPrivilegeEscalation) were found in the PR diff.
No-Sensitive-Data-In-Logs ✅ Passed Changed logging is operational only (versions, paths, job/prow URLs); I found no passwords, tokens, PII, session IDs, or customer data.
No-Hardcoded-Secrets ✅ Passed Scans of all modified files found no API keys, tokens, passwords, private keys, credentialed URLs, or long base64-like literals; added strings are public URLs/UI text.
No-Injection-Vectors ✅ Passed No flagged sinks in changed files; dynamic HTML uses Jinja autoescape/html.escape, and searches found no shell/eval/pickle/yaml/os.system/dangerouslySetInnerHTML.
Ai-Attribution ✅ Passed PR-range commits show no Assisted-by/Generated-by or Co-Authored-By trailers; the only Claude co-author line is in an older pre-PR commit.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@vimauro

vimauro commented Jul 29, 2026

Copy link
Copy Markdown
Contributor Author

/label tide/merge-method-squash

@openshift-ci openshift-ci Bot added the tide/merge-method-squash Denotes a PR that should be squashed by tide when it merges. label Jul 29, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
payload-monitor/payload_monitor/report/templates/scripts.js (1)

226-241: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Add a failure handler to the clipboard copy
navigator.clipboard.writeText(text) can reject in insecure contexts or when permissions are denied, leaving the click with no user feedback. Add a .catch() so failures are handled explicitly.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@payload-monitor/payload_monitor/report/templates/scripts.js` around lines 226
- 241, Add a rejection handler to the promise returned by
navigator.clipboard.writeText in the action-cell click listener. Handle copy
failures explicitly with appropriate user feedback, while preserving the
existing success tooltip behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@payload-monitor/payload_monitor/report/templates/scripts.js`:
- Around line 226-241: Add a rejection handler to the promise returned by
navigator.clipboard.writeText in the action-cell click listener. Handle copy
failures explicitly with appropriate user feedback, while preserving the
existing success tooltip behavior.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: eea58109-60f5-4a27-be7b-1c2190bb0456

📥 Commits

Reviewing files that changed from the base of the PR and between 33fef01 and 221a8d7.

📒 Files selected for processing (12)
  • payload-monitor/payload_monitor/__main__.py
  • payload-monitor/payload_monitor/collectors/release_controller.py
  • payload-monitor/payload_monitor/collectors/timing.py
  • payload-monitor/payload_monitor/report/generator.py
  • payload-monitor/payload_monitor/report/templates/_analysis_card.html
  • payload-monitor/payload_monitor/report/templates/dashboard.html
  • payload-monitor/payload_monitor/report/templates/scripts.js
  • payload-monitor/payload_monitor/report/templates/styles.css
  • payload-monitor/payload_monitor/report/timing_section.py
  • payload-monitor/tests/test_collectors_release_controller.py
  • payload-monitor/tests/test_report_generator.py
  • plugins/edge-ocp-ci/skills/generate-dashboard/SKILL.md

@openshift-ci openshift-ci Bot added the ready-for-human-review Indicates a PR has been reviewed by automated tools and is ready for human review label Jul 29, 2026
@vimauro
vimauro marked this pull request as ready for review July 29, 2026 14:41
@openshift-ci openshift-ci Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 29, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@payload-monitor/payload_monitor/analyzer.py`:
- Line 83: Update the failure-selection logic around latest_prow_url to track
whether the newest failure has been seen independently via a latest_failure_seen
flag, so an empty newest prow_url prevents older failure URLs from being
selected; add a regression test using distinct URLs with an empty URL on the
newest failure.

In `@payload-monitor/payload_monitor/report/templates/dashboard.html`:
- Line 15: Fix or remove the broken documentation link in the dashboard
template’s meta section. Update the anchor URL to a valid Edge CI Payload
Manager responsibilities document if the correct destination is known; otherwise
remove the link while preserving the surrounding metadata.

In `@plugins/edge-ocp-ci/.claude-plugin/plugin.json`:
- Line 4: Update the corresponding plugin entry in
.claude-plugin/marketplace.json to version 1.2.1, matching the version declared
by the plugin.json change. Ensure the marketplace metadata for the edge-ocp-ci
plugin uses the same version.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: b8f6416e-5d0f-4760-9fe3-eb92cb160dd4

📥 Commits

Reviewing files that changed from the base of the PR and between 221a8d7 and bf0fdd9.

📒 Files selected for processing (8)
  • payload-monitor/payload_monitor/analyzer.py
  • payload-monitor/payload_monitor/models.py
  • payload-monitor/payload_monitor/report/templates/dashboard.html
  • payload-monitor/payload_monitor/report/timing_section.py
  • payload-monitor/tests/test_analyzer.py
  • payload-monitor/tests/test_models_timing.py
  • payload-monitor/tests/test_report_generator.py
  • plugins/edge-ocp-ci/.claude-plugin/plugin.json
💤 Files with no reviewable changes (1)
  • payload-monitor/tests/test_models_timing.py
🚧 Files skipped from review as they are similar to previous changes (2)
  • payload-monitor/payload_monitor/report/timing_section.py
  • payload-monitor/tests/test_report_generator.py

Comment thread payload-monitor/payload_monitor/analyzer.py
Comment thread payload-monitor/payload_monitor/report/templates/dashboard.html
Comment thread plugins/edge-ocp-ci/.claude-plugin/plugin.json

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@payload-monitor/tests/test_analyzer.py`:
- Line 292: Wrap the test docstring in the relevant analyzer test so no line
exceeds Ruff’s 88-character limit, preserving its existing wording and meaning.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 1ce12250-0ddb-4f9b-9e2e-c95844729948

📥 Commits

Reviewing files that changed from the base of the PR and between 4781ef1 and 0b512da.

📒 Files selected for processing (2)
  • payload-monitor/payload_monitor/analyzer.py
  • payload-monitor/tests/test_analyzer.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • payload-monitor/payload_monitor/analyzer.py

Comment thread payload-monitor/tests/test_analyzer.py Outdated

@Neilhamza Neilhamza left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this PR — there's a lot of genuinely good work here. A few things I especially appreciated while reading through it:

  • The Sippy link fix (_job_analysis_url building jobs/{version}/analysis?filters=…) resolves the "no rows" issue I hit on the last dashboard — and the URL-shape test locks it in.
  • The patch_analysis_html rewrite is a real robustness win: the explicit <!-- /claude-suggestion --> terminator, the HTML-unescape handling with the ampersand test, and idempotent badge insertion all close failure modes the old .*?</div>\s*</div> matching had.
  • Moving the findings-summary counting and trend math out of Jinja namespace() logic into _build_template_context() makes it testable — much nicer.
  • Adding job_name to the analysis cards directly addresses the "hard to tell which job a summary belongs to" feedback. Thank you for that.

I've left a handful of inline comments — all questions/suggestions, nothing I'd consider blocking. The one I'd most like your thoughts on is the removal of the failures table (see the dashboard.html comment): with the payload tag/date now hidden inside the collapsed rows, I'm finding cross-payload comparison — which is most of what we do on payload-manager duty — a bit harder than before, and I've suggested a small summary-line tweak that I think recovers it. Happy to pair on any of these if useful!

Comment thread payload-monitor/payload_monitor/report/templates/dashboard.html
Comment thread payload-monitor/payload_monitor/report/templates/dashboard.html
Comment thread payload-monitor/payload_monitor/analyzer.py
Comment thread payload-monitor/payload_monitor/collectors/sippy.py
Comment thread payload-monitor/payload_monitor/report/generator.py
Comment thread payload-monitor/payload_monitor/report/templates/scripts.js
Comment thread plugins/edge-ocp-ci/skills/generate-dashboard/SKILL.md
Comment thread payload-monitor/payload_monitor/report/generator.py
@Neilhamza

Copy link
Copy Markdown
Contributor

General UI/readability suggestions — not blocking at all, just "ease of the eyes" thoughts while using the dashboard daily. Take or leave any of them:

  1. Badge density. A summary line can currently carry topology + type + recurrence + Unstable + AI-Analyzed + cross-topo badges before the job name even starts, in ~7 different hues. Since type and recurrence are already filterable/derivable, would demoting some to subtler treatments help — e.g. a colored left border for blocking/informing (already used for .regression-row), keeping full badges only for the "act on this" signals (Persistent, Unstable)? Fewer colors competing = faster scanning.

  2. Light-mode option. The dark palette is easy on the eyes at 7 AM, but some of us read it in bright rooms or project it in meetings. Since colors are already centralized in :root, a @media (prefers-color-scheme: light) override block (or a manual toggle persisted per session) would be cheap to add. If there's appetite, I'm happy to draft the light palette.

  3. Muted-value contrast. --text-muted: #8b949e on #0d1117 sits around ~4.6:1 — fine for large text, borderline for the 11–12 px labels it's used on (filter labels are 11 px). Bumping small muted text to ~13 px or lightening the color one step would help tired eyes.

  4. Guided reading order. Building on Chad's "guided UX" idea: the sections already mirror the role doc's procedure. Numbering the section headers to match ("1. Health Overview → 2. Findings → 3. Failure Analysis → 4. Act") — or a tiny sticky progress rail — would encode the recommended path for new payload managers at near-zero cost.

  5. Semantic color duplication. TNF (orange) and Unstable (orange) share a hue, as do informing (yellow) and recurring (gold) — so color alone can't disambiguate topology vs. status at a glance. If badges stay, maybe reserve warm colors for status and cool colors for topology?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. ready-for-human-review Indicates a PR has been reviewed by automated tools and is ready for human review tide/merge-method-squash Denotes a PR that should be squashed by tide when it merges.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants