Skip to content

refactor: detect async modules at discovery time instead of per-request#41

Closed
isaacbmiller wants to merge 1 commit intomainfrom
isaac/perf-async-detection
Closed

refactor: detect async modules at discovery time instead of per-request#41
isaacbmiller wants to merge 1 commit intomainfrom
isaac/perf-async-detection

Conversation

@isaacbmiller
Copy link
Copy Markdown
Contributor

Split from #38 (2/5)

Changes

Check for user-implemented aforward() once during module discovery using cls.__dict__ (avoids false positives from inherited base-class default methods). The result is stored as has_native_async on DiscoveredModule and used in execute_pipeline instead of a per-request hasattr() check.

Why cls.dict instead of hasattr

dspy.Module provides a default aforward that just delegates to forward(). Using hasattr would incorrectly route all modules through the async path. Checking cls.__dict__ ensures only user-written aforward methods are detected.

Files changed

  • src/dspy_cli/discovery/module_finder.py -- new field + helper function
  • src/dspy_cli/server/execution.py -- use module.has_native_async
  • tests/test_execution.py -- updated mocks

Testing

All 123 existing tests pass.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c078488854

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/dspy_cli/discovery/module_finder.py Outdated
Comment on lines +49 to +50
if 'aforward' not in cls.__dict__:
return False
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Detect inherited async overrides during discovery

The new cls.__dict__ check only marks modules as async when aforward is defined directly on the concrete class, so subclasses that inherit a user-defined aforward from an intermediate base class are now misclassified as sync. In that case execute_pipeline uses instance(**inputs) instead of await instance.acall(**inputs), which can bypass intended async behavior or fail for async-only module hierarchies even though they previously worked with the per-request hasattr check.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@codex fix

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

  • Fixed async capability detection to walk the class MRO and treat aforward implemented on any user-defined base class as native async, while still stopping at dspy.Module to avoid false positives from DSPy’s default method. This addresses inherited async module hierarchies called out in review feedback. src/dspy_cli/discovery/module_finder.pyL47-L59
  • Added targeted unit tests covering: (1) default-only dspy.Module behavior, (2) direct aforward override, and (3) inherited user-defined aforward override from an intermediate base class. tests/discovery/test_module_finder.pyL253-L290
  • Committed the fix on the current branch as 6967a04 and created a follow-up PR via the make_pr tool with a title/body describing this delta on top of the original PR.

Testing

  • ⚠️ pytest tests/discovery/test_module_finder.py tests/test_execution.py (fails during collection in this environment: missing dspy dependency).
  • python -m compileall src/dspy_cli/discovery/module_finder.py tests/discovery/test_module_finder.py

View task →

Check for user-implemented aforward() once during module discovery.
Walks the MRO up to (but not including) dspy.Module, so aforward
inherited from an intermediate base class is still detected while
the default base-class method is ignored. Store result as
has_native_async on DiscoveredModule and use it in execute_pipeline.

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
@isaacbmiller isaacbmiller force-pushed the isaac/perf-async-detection branch from c078488 to 4165260 Compare February 28, 2026 20:38
@isaacbmiller
Copy link
Copy Markdown
Contributor Author

Dropping this -- hasattr per-request is fine for a nanosecond check against second-long LLM calls, and the cls.dict/MRO approach adds complexity + inheritance edge cases for no measurable gain. PR #42 (executor) will keep using hasattr.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant