Skip to content

Integrate embench-iot as benchmark suite#724

Merged
jserv merged 2 commits into
sysprog21:masterfrom
yc199911:master
May 17, 2026
Merged

Integrate embench-iot as benchmark suite#724
jserv merged 2 commits into
sysprog21:masterfrom
yc199911:master

Conversation

@yc199911

@yc199911 yc199911 commented Mar 19, 2026

Copy link
Copy Markdown
Contributor

embench-iot provides 19 benchmarks designed for embedded systems, offering more representative coverage than the existing dhrystone. This complements the current benchmark infrastructure and enables more thorough performance evaluation of rv32emu.

  • Add embench-iot as a git submodule under tests/embench-iot
  • Add rv32emu board support (boardsupport.c/h) in the fork
  • Add wrapper Makefile to bridge rv32emu build system with scons
  • Register embench-iot in TEST_SUITES in mk/artifact.mk
  • Add scons to CI install dependencies
  • Update CI to detect changes under tests/embench-iot

Note: This integration requires sysprog21 to create an official fork of embench-iot (similar to sysprog21/ansibench) and update the submodule URL accordingly before merging.

Close #28


Summary by cubic

Integrates embench-iot as a new benchmark suite for rv32emu, adding 19 embedded-focused tests to improve coverage beyond Dhrystone. Hooks the suite into the build, CI, and artifact pipeline for repeatable runs.

  • New Features
    • Add tests/embench-iot submodule pointing to sysprog21/embench-iot.
    • Add rv32emu board support and a wrapper Makefile in the fork to build via scons.
    • Register the suite in TEST_SUITES in mk/artifact.mk.
    • CI: install scons and watch tests/embench-iot/** for changes.
    • Normalize submodule URL (drop .git suffix) to match GitHub format.

Written for commit fadf9a9. Summary will update on new commits. Review in cubic

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 4 files


Since this is your first cubic review, here's how it works:

  • cubic automatically reviews your code and comments on bugs and improvements
  • Teach cubic by replying to its comments. cubic learns from your replies and gets better over time
  • Add one-off context when rerunning by tagging @cubic-dev-ai with guidance or docs links (including llms.txt)
  • Ask questions if you need clarification on any suggestion

@jserv

jserv commented Mar 26, 2026

Copy link
Copy Markdown
Contributor

Note: This integration requires sysprog21 to create an official fork of embench-iot (similar to sysprog21/ansibench) and update the submodule URL accordingly before merging.

This approach should be avoided initially. The only valid reason to fork an existing benchmark suite is if it has not been maintained for some time. If you encounter build or runtime errors in embench-iot, please submit a pull request upstream first.

@jserv jserv requested review from vacantron March 26, 2026 02:03
@yc199911

yc199911 commented Mar 26, 2026

Copy link
Copy Markdown
Contributor Author

Submitted two PRs to upstream embench-iot to avoid maintaining a separate fork:

  1. fix(wikisort): replace custom bool typedef with stdbool.h
    fix(wikisort): replace custom bool typedef with stdbool.h embench/embench-iot#215
  2. Add rv32emu board support
    Add rv32emu board support embench/embench-iot#216

While testing embench-iot with a GCC 15 toolchain (which defaults to C23),
I encountered a build failure in wikisort caused by typedef uint8_t bool
since bool is now a language keyword in C23.
embench-iot#215 addresses
this by replacing the custom bool definitions with #include <stdbool.h>.

Once these are merged upstream, I will update this PR to point the
submodule directly at the upstream repo, removing the need for a fork.

@jserv

jserv commented May 15, 2026

Copy link
Copy Markdown
Contributor

It is pity that upstream embench-iot does not review the proposed changes. I fork it to sysprog21/embench-iot. @yc199911, please push commits to enable the integeration.

@jserv jserv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid changing DTS file.

embench-iot provides 19 benchmarks designed for embedded systems,
offering more representative coverage than the existing dhrystone.
This complements the current benchmark infrastructure and enables
more thorough performance evaluation of rv32emu.

- Add embench-iot as a git submodule under tests/embench-iot
- Add rv32emu board support (boardsupport.c/h) in the fork
- Add wrapper Makefile to bridge rv32emu build system with scons
- Register embench-iot in TEST_SUITES in mk/artifact.mk
- Add scons to CI install dependencies
- Update CI to detect changes under tests/embench-iot

Note: This integration requires sysprog21 to create an official
fork of embench-iot (similar to sysprog21/ansibench) and update
the submodule URL accordingly before merging.

Close sysprog21#28
@yc199911

Copy link
Copy Markdown
Contributor Author

Avoid changing DTS file.

Removed the DTS change in the latest push. The src/devices/minimal.dts modification is no longer included.

@jserv jserv added this to the release-2026.2 milestone May 17, 2026
Comment thread .gitmodules Outdated
@jserv jserv merged commit 7ada7d0 into sysprog21:master May 17, 2026
18 checks passed
@jserv

jserv commented May 17, 2026

Copy link
Copy Markdown
Contributor

Thank @yc199911 for contributing!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Integrate embench-iot for benchmarking

2 participants