Skip to content

docs: add hardware benchmark dashboard template#172

Merged
Flink-ddd merged 2 commits into
RL-Align:mainfrom
smarslou:docs/hardware-benchmark-dashboard
Jun 24, 2026
Merged

docs: add hardware benchmark dashboard template#172
Flink-ddd merged 2 commits into
RL-Align:mainfrom
smarslou:docs/hardware-benchmark-dashboard

Conversation

@smarslou

@smarslou smarslou commented Jun 22, 2026

Copy link
Copy Markdown

Summary

  • Add a hardware benchmark dashboard template.
  • Define environment, workload, backend, metric, status, and reproduction fields.
  • Document that fallback results must not be presented as fused-kernel benchmarks.

Scope

This PR establishes the documentation template and reporting rules only.
It does not add measured H100 or MI300 benchmark results.

Validation

  • git diff --cached --check
  • mkdocs build --strict -f mkdocs.yaml

Part of #20

Summary by CodeRabbit

  • Documentation
    • Added a new “Hardware Benchmark Dashboard” page with standardized, cross-hardware reporting templates, required benchmark fields, and a status taxonomy (pass/blocked/oom/pending).
    • Included guidance and an example reproduction command for generating benchmark outputs.
    • Updated the documentation navigation and benchmarking README to reference the new dashboard.

@coderabbitai

coderabbitai Bot commented Jun 22, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: ef1ea9d5-6052-4d24-8ad9-5f40b7da14af

📥 Commits

Reviewing files that changed from the base of the PR and between ea6c770 and 16dec93.

📒 Files selected for processing (1)
  • docs/benchmarking/hardware-dashboard.md
✅ Files skipped from review due to trivial changes (1)
  • docs/benchmarking/hardware-dashboard.md

📝 Walkthrough

Walkthrough

Adds a new documentation page docs/benchmarking/hardware-dashboard.md that defines reporting fields, status taxonomy (pass, blocked, oom, pending), environment matrix and benchmark result table templates, and a reproduction command. The benchmarking README is updated to reference it, and the docs navigation registers the new page.

Changes

Hardware Benchmark Dashboard Documentation

Layer / File(s) Summary
Hardware dashboard page, README pointer, and nav entry
docs/benchmarking/hardware-dashboard.md, docs/benchmarking/README.md, docs/.nav.yml
New hardware-dashboard.md establishes required reporting fields, fallback vs. fused-kernel rules, a four-value status taxonomy, environment matrix and LogP/sampling result table templates with pending placeholders, and a concrete scripts/run_profile_suite.py reproduction command. The benchmarking README adds a sentence linking to the dashboard, and .nav.yml registers the page under the Benchmarking section.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Poem

A rabbit hops in with a dashboard in paw,
"Here's pending, here's pass, here's the benchmark law!"
With tables of latency, VRAM, and throughput so neat,
The nav now points onward — the docs are complete.
🐇📊✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'docs: add hardware benchmark dashboard template' directly and clearly describes the main change—adding a hardware benchmark dashboard template to the documentation.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
docs/benchmarking/hardware-dashboard.md (1)

54-63: 🧹 Nitpick | 🔵 Trivial | ⚡ Quick win

Align code fence indentation with opening fence.

The closing triple-backticks on line 63 has leading whitespace (2 spaces), while the opening fence on line 54 is flush-left. While technically valid Markdown, this is inconsistent and reduces readability. Move the closing fence to column 0 to match the opening.

📝 Proposed fix for fence alignment
 python scripts/run_profile_suite.py \
   --device cuda \
   --dtype float16 \
   --batch-sizes 8,16,32 \
   --seq-lens 128,512 \
   --vocab-sizes 4096,128256 \
   --workloads logp-native,logp-fused \
   --output reports/logp_profile.csv
-  ```
+```
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/benchmarking/hardware-dashboard.md` around lines 54 - 63, The closing
code fence (triple-backticks) has two leading spaces of indentation while the
opening code fence is flush-left at column 0. Remove the leading whitespace from
the closing fence to align it with the opening fence, ensuring both are at
column 0 for consistent Markdown formatting.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@docs/benchmarking/hardware-dashboard.md`:
- Around line 54-63: The closing code fence (triple-backticks) has two leading
spaces of indentation while the opening code fence is flush-left at column 0.
Remove the leading whitespace from the closing fence to align it with the
opening fence, ensuring both are at column 0 for consistent Markdown formatting.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 9a6de270-b75e-4081-82c7-da13f20f0397

📥 Commits

Reviewing files that changed from the base of the PR and between b667425 and ea6c770.

📒 Files selected for processing (3)
  • docs/.nav.yml
  • docs/benchmarking/README.md
  • docs/benchmarking/hardware-dashboard.md

@Flink-ddd Flink-ddd requested review from a-kaa and z1ying June 22, 2026 12:44
@Flink-ddd

Copy link
Copy Markdown
Collaborator

cc @inaniloquentee @z1ying @a-kaa PTAL

@inaniloquentee

Copy link
Copy Markdown
Collaborator

LGTM. Thank you for contribution!

@a-kaa a-kaa left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! THANK YOU!

@Flink-ddd Flink-ddd merged commit 77b620e into RL-Align:main Jun 24, 2026
4 checks passed
@Flink-ddd

Copy link
Copy Markdown
Collaborator

@smarslou Thanks for the PR, just a heads-up that your local Git email isn't linked to your GitHub account, so please add it to your GitHub Settings to link these commits and claim your Contributor badge!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants