Skip to content

Conversation

@Jack-Khuu
Copy link
Contributor

KernelAgent currently hardcodes the test time limit when evaluating a kernel candidate to 30s.

In simpler cases this is fine, but for more complex models, this starts to timeout for valid kernels or more comprehensive tests.

  • For example, custom tests or tests with multiple input example (i.e GPUMode's test inputs)

This PR adds an optional arg test-timeout-s that configures this

Note: This happens more often in the kernelagent path than Fuser, since the subgraphs are typically smaller with quicker tests


Test

python -m Fuser.auto_agent --problem /home/jackkhuu/KernelAgent/problem.py --ka-model gpt-5 --router-model gpt-5 --extract-model gpt-5 --dispatch-model gpt-5 --compose-model gpt-5 --verify --ignore-router-config --no-router-cache --test-timeout-s 120

where problem.py is an edited version of gpumode trimul

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Feb 6, 2026
Copy link
Member

@msaroufim msaroufim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bit of a code smell that you have to pipe a timeout argument through so many data structures but otherwise lgtm

@Jack-Khuu
Copy link
Contributor Author

A bit of a code smell that you have to pipe a timeout argument through so many data structures but otherwise lgtm

Yeaaaa, I had a hydra/yaml config that makes the repo much easy to play with, that I'll revive once the opt modules come in

@Jack-Khuu Jack-Khuu merged commit 9df5916 into main Feb 6, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants