Skip to content

fix(ADWs): routines that fail still exit 0 — scheduler logs them as success - #128

Open
mt-alarcon wants to merge 1 commit into
evolution-foundation:developfrom
mt-alarcon:fix/adw-routines-propagate-exit-code
Open

fix(ADWs): routines that fail still exit 0 — scheduler logs them as success#128
mt-alarcon wants to merge 1 commit into
evolution-foundation:developfrom
mt-alarcon:fix/adw-routines-propagate-exit-code

Conversation

@mt-alarcon

@mt-alarcon mt-alarcon commented Jul 31, 2026

Copy link
Copy Markdown

The defect

ADWs/runner.py::summary(results, title) computes failed = len(results) - success, prints ⚠ N failure(s) — and returns nothing. Every core routine ends with a bare summary(...) call and no sys.exit(), so the process exits 0 regardless of outcome.

The scheduler decides health by return code:

status = "✓" if result.returncode == 0 else "✗"

and only a non-zero exit reaches the failure-alert path. So a routine whose only step failed is logged as and never alerts. That is an active false green — worse than no signal, because it reads as a healthy run.

Affected routines (all end in summary() with no exit): good_morning.py, end_of_day.py, memory_sync.py, weekly_review.py, memory_lint.py, backup.py.

Reproduce

$ python3 ADWs/routines/good_morning.py     # with a failing step
  ✗ good-morning (0.2s | ...)
  ⚠ 1 failure(s)  Steps: 0/1
$ echo $?
0        # scheduler will log "✓ good-morning" and send no alert

Impact, measured

On a live install with 61 days of run logs: under this change the daily routines would have exited non-zero on 1 of 23–28 runs per month each (4 routines had exactly one failed run in the worst month; the rest zero).

So it surfaces roughly 1–4 real failures per month that are currently logged as successes, and it does not create alert noise — no routine shows chronic partial failure. That was checked before proposing, because a uniform failed > 0 → exit 1 would be a bad trade if some routine failed partially by design.

The fix

summary() returns failed (additive — no existing caller consumed the return value), and each footer becomes:

sys.exit(1 if summary(results, "...") else 0)

Watchdog-style scripts that intentionally always exit 0 are untouched.

Note on CI

tests/test_routines_exit_code.py is not picked up by the current workflow, which runs a named list of files under tests/backend/. Wiring it in would be a second, unrelated change, so it is deliberately left out — flagging it so the test is not assumed to be running.

🤖 Generated with Claude Code

`summary()` computes `failed`, prints "N failure(s)" and returns nothing.
Every routine ends with a bare `summary(...)` call and no `sys.exit()`, so
the process exits 0 regardless of outcome.

The scheduler decides health by return code, and only a non-zero exit
triggers the failure alert path. A routine whose only step failed is
therefore logged as a success and never alerts — an active false green,
which is worse than no signal at all.

`summary()` now returns `failed` (additive — no existing caller consumed the
return value) and each routine's footer becomes
`sys.exit(1 if summary(results, "...") else 0)`.

Measured on a live install (61 days of run logs): under this change the daily
routines would have exited non-zero on 1 of 23-28 runs per month each. It
surfaces real failures that are currently logged as successes, and creates no
alert noise — no routine shows chronic partial failure.

Watchdog-style scripts that intentionally always exit 0 are untouched.

Note: tests/test_routines_exit_code.py is not picked up by the current CI
workflow, which runs a named list of files under tests/backend/. Wiring it in
is left out of this change on purpose.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@sourcery-ai

sourcery-ai Bot commented Jul 31, 2026

Copy link
Copy Markdown

Reviewer's Guide

Make ADW core routines exit non-zero when steps fail by having runner.summary() return the failure count and ensuring all routine footers propagate that value via sys.exit; add a regression test that guards both behaviors via direct execution and static AST analysis.

Sequence diagram for updated routine exit code and scheduler logging

sequenceDiagram
    actor Scheduler
    participant Routine_good_morning as good_morning.py
    participant runner_summary as summary

    Scheduler->>Routine_good_morning: subprocess_run(good_morning.py)
    Routine_good_morning->>runner_summary: summary(results, "Good Morning")
    runner_summary-->>Routine_good_morning: failed_count
    alt [failed_count > 0]
        Routine_good_morning->>Routine_good_morning: sys.exit(1)
    else [failed_count == 0]
        Routine_good_morning->>Routine_good_morning: sys.exit(0)
    end
    Routine_good_morning-->>Scheduler: returncode
    Scheduler->>Scheduler: status = "✓" if returncode == 0 else "✗"
Loading

File-Level Changes

Change Details Files
Make runner.summary() return the failure count so callers can use it as the basis for the process exit code.
  • Add an int return type annotation to summary and document that it returns the failure count for use as an exit code.
  • Compute failed count as before but now return it at the end of summary.
  • Preserve existing banner/printing behavior so the terminal output is unchanged apart from the new return value.
ADWs/runner.py
Update all shipped ADW core routines to exit non-zero when a step fails by wiring summary() into sys.exit().
  • Import sys in each routine module that previously only called summary() as a bare statement.
  • Wrap each summary(results, ...) call in sys.exit(1 if summary(...) else 0) so any failure leads to a non-zero exit code.
  • Ensure the weekly_review variant that adds a '(Team)' suffix also uses the same exit propagation pattern.
ADWs/routines/backup.py
ADWs/routines/end_of_day.py
ADWs/routines/good_morning.py
ADWs/routines/memory_lint.py
ADWs/routines/memory_sync.py
ADWs/routines/weekly_review.py
Add a regression test suite that enforces both the runtime and structural guarantees around routine exit codes.
  • Introduce tests that load runner.py and assert summary() returns the correct failure count for various combinations of success/failure results.
  • Add AST-based tests that locate all summary() calls in the core routines and assert each is used inside a sys.exit(...) call, failing if any footer discards the return value.
  • Document within the test file the two guarded failure modes and the rationale tied to scheduler behavior and false-green runs.
tests/test_routines_exit_code.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • In the routine modules you now have both import sys and import sys, os at the top; consolidate these imports to avoid duplication and potential confusion.
  • Since summary() now serves dual purposes (UI output and exit-code source), consider splitting responsibility (e.g., a pure compute_failure_count() helper) or making the return type explicitly boolean to make its semantics clearer and reduce the coupling between presentation and process control.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- In the routine modules you now have both `import sys` and `import sys, os` at the top; consolidate these imports to avoid duplication and potential confusion.
- Since `summary()` now serves dual purposes (UI output and exit-code source), consider splitting responsibility (e.g., a pure `compute_failure_count()` helper) or making the return type explicitly boolean to make its semantics clearer and reduce the coupling between presentation and process control.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant