-
Notifications
You must be signed in to change notification settings - Fork 0
Issues
is:issue state:open
is:issue state:open
Issue creation is restricted in this repository
Search results
[Feature]: Notice when the M3 docker "helper boxes" die mid-test, instead of quietly writing down wrong answers
enhancementNew feature or requestNew feature or requestevaluation: vakraVakra (m3) evaluationVakra (m3) evaluationpriority: mediumNormal priorityNormal priorityStatus: Open.#119 In cuga-project/cuga-eval;[Evaluation experiment]: CUGA-lite Appworld baselines on frontier models
evaluation: appworldAppworld EvalutionAppworld EvalutionStatus: Open.#118 In cuga-project/cuga-eval;- Status: Open.#116 In cuga-project/cuga-eval;
[Bug]: Hardcoded shared /tmp paths collide between concurrent benchmark runs (console log, summary, registry log)
bugSomething isn't workingSomething isn't workingStatus: Open.#115 In cuga-project/cuga-eval;[Bug]: bpo/oak/appworld hardcode app ports into MCP server YAML, blocks custom ports
bugSomething isn't workingSomething isn't workingStatus: Open.#113 In cuga-project/cuga-eval;- Status: Open.#112 In cuga-project/cuga-eval;
[Bug]: compare.sh bundle mispairs trajectories/logs with wrong run's results when a mid-sequence run is interrupted
bugSomething isn't workingSomething isn't workingcomplexity: mediumModerate scope — multiple files or non-trivial logicModerate scope — multiple files or non-trivial logicevaluation: vakraVakra (m3) evaluationVakra (m3) evaluationneeds-triageNewly filed, not yet reviewedNewly filed, not yet reviewedpriority: mediumNormal priorityNormal priorityStatus: Open.#103 In cuga-project/cuga-eval;[Feature]: curated --eval-key subsets for reliably-passing M3 capability 2/3 tasks
complexity: lowSmall, focused change — easy to reviewSmall, focused change — easy to reviewenhancementNew feature or requestNew feature or requestevaluation: vakraVakra (m3) evaluationVakra (m3) evaluationpriority: lowNice to have, no urgencyNice to have, no urgencyStatus: Open.#102 In cuga-project/cuga-eval;[Feature]: Consistent LLM token tracking via callbacks (cache-aware) with tests
enhancementNew feature or requestNew feature or requestevaluation: appworldAppworld EvalutionAppworld Evalutionpriority: mediumNormal priorityNormal priorityStatus: Open.#95 In cuga-project/cuga-eval;[Bug]: Agent crashes treating an already-parsed dict tool response as a JSON string
bugSomething isn't workingSomething isn't workingcomplexity: mediumModerate scope — multiple files or non-trivial logicModerate scope — multiple files or non-trivial logicevaluation: vakraVakra (m3) evaluationVakra (m3) evaluationpriority: mediumNormal priorityNormal priorityStatus: Open.#91 In cuga-project/cuga-eval;[Benchmark]: Include Tau-2 Bench into evaluation repository
priority: mediumNormal priorityNormal priorityStatus: Open.#84 In cuga-project/cuga-eval;[Feature]: Enable running of different CUGA configurations
enhancementNew feature or requestNew feature or requestevaluation: appworldAppworld EvalutionAppworld Evalutionpriority: mediumNormal priorityNormal priorityStatus: Open.#83 In cuga-project/cuga-eval;