fix(ADWs): routines that fail still exit 0 — scheduler logs them as success - #128
Open
mt-alarcon wants to merge 1 commit into
Open
Conversation
`summary()` computes `failed`, prints "N failure(s)" and returns nothing. Every routine ends with a bare `summary(...)` call and no `sys.exit()`, so the process exits 0 regardless of outcome. The scheduler decides health by return code, and only a non-zero exit triggers the failure alert path. A routine whose only step failed is therefore logged as a success and never alerts — an active false green, which is worse than no signal at all. `summary()` now returns `failed` (additive — no existing caller consumed the return value) and each routine's footer becomes `sys.exit(1 if summary(results, "...") else 0)`. Measured on a live install (61 days of run logs): under this change the daily routines would have exited non-zero on 1 of 23-28 runs per month each. It surfaces real failures that are currently logged as successes, and creates no alert noise — no routine shows chronic partial failure. Watchdog-style scripts that intentionally always exit 0 are untouched. Note: tests/test_routines_exit_code.py is not picked up by the current CI workflow, which runs a named list of files under tests/backend/. Wiring it in is left out of this change on purpose. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Reviewer's GuideMake ADW core routines exit non-zero when steps fail by having runner.summary() return the failure count and ensuring all routine footers propagate that value via sys.exit; add a regression test that guards both behaviors via direct execution and static AST analysis. Sequence diagram for updated routine exit code and scheduler loggingsequenceDiagram
actor Scheduler
participant Routine_good_morning as good_morning.py
participant runner_summary as summary
Scheduler->>Routine_good_morning: subprocess_run(good_morning.py)
Routine_good_morning->>runner_summary: summary(results, "Good Morning")
runner_summary-->>Routine_good_morning: failed_count
alt [failed_count > 0]
Routine_good_morning->>Routine_good_morning: sys.exit(1)
else [failed_count == 0]
Routine_good_morning->>Routine_good_morning: sys.exit(0)
end
Routine_good_morning-->>Scheduler: returncode
Scheduler->>Scheduler: status = "✓" if returncode == 0 else "✗"
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
There was a problem hiding this comment.
Hey - I've left some high level feedback:
- In the routine modules you now have both
import sysandimport sys, osat the top; consolidate these imports to avoid duplication and potential confusion. - Since
summary()now serves dual purposes (UI output and exit-code source), consider splitting responsibility (e.g., a purecompute_failure_count()helper) or making the return type explicitly boolean to make its semantics clearer and reduce the coupling between presentation and process control.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- In the routine modules you now have both `import sys` and `import sys, os` at the top; consolidate these imports to avoid duplication and potential confusion.
- Since `summary()` now serves dual purposes (UI output and exit-code source), consider splitting responsibility (e.g., a pure `compute_failure_count()` helper) or making the return type explicitly boolean to make its semantics clearer and reduce the coupling between presentation and process control.Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The defect
ADWs/runner.py::summary(results, title)computesfailed = len(results) - success, prints⚠ N failure(s)— and returns nothing. Every core routine ends with a baresummary(...)call and nosys.exit(), so the process exits 0 regardless of outcome.The scheduler decides health by return code:
and only a non-zero exit reaches the failure-alert path. So a routine whose only step failed is logged as
✓and never alerts. That is an active false green — worse than no signal, because it reads as a healthy run.Affected routines (all end in
summary()with no exit):good_morning.py,end_of_day.py,memory_sync.py,weekly_review.py,memory_lint.py,backup.py.Reproduce
Impact, measured
On a live install with 61 days of run logs: under this change the daily routines would have exited non-zero on 1 of 23–28 runs per month each (4 routines had exactly one failed run in the worst month; the rest zero).
So it surfaces roughly 1–4 real failures per month that are currently logged as successes, and it does not create alert noise — no routine shows chronic partial failure. That was checked before proposing, because a uniform
failed > 0 → exit 1would be a bad trade if some routine failed partially by design.The fix
summary()returnsfailed(additive — no existing caller consumed the return value), and each footer becomes:Watchdog-style scripts that intentionally always exit 0 are untouched.
Note on CI
tests/test_routines_exit_code.pyis not picked up by the current workflow, which runs a named list of files undertests/backend/. Wiring it in would be a second, unrelated change, so it is deliberately left out — flagging it so the test is not assumed to be running.🤖 Generated with Claude Code