Skip to content

fix(anthropic): warn when stop_reason='max_tokens' silently truncates output#5717

Open
Beandon13 wants to merge 1 commit intocrewAIInc:mainfrom
Beandon13:fix/crewai-5148-anthropic-stop-reason
Open

fix(anthropic): warn when stop_reason='max_tokens' silently truncates output#5717
Beandon13 wants to merge 1 commit intocrewAIInc:mainfrom
Beandon13:fix/crewai-5148-anthropic-stop-reason

Conversation

@Beandon13
Copy link
Copy Markdown

Summary

  • AnthropicCompletion previously discarded response.stop_reason after extracting token usage, so callers had no way to detect that a Claude response was cut off by max_tokens. The follow-up call inside _handle_tool_use_conversation is the most painful site because the truncated synthesis answer is what downstream code parses.
  • This PR adds a small module-level helper, _warn_on_max_tokens_truncation, and calls it at all four completion sites (sync/async × plain completion / tool-use follow-up). When stop_reason == "max_tokens" we emit logging.warning(...) including the agent role (when available) and the configured max_tokens so users can act on it.
  • Behavior is otherwise unchanged. No new public API; no extra LLM calls; no events touched.

Closes #5148

Testing

cd lib/crewai
uv pip install -e ".[anthropic]"
uv pip install pytest pytest-asyncio pytest-xdist pytest-timeout pytest-recording vcrpy
python -m pytest \
  tests/llms/anthropic/test_anthropic.py::test_warn_on_max_tokens_truncation_emits_warning \
  tests/llms/anthropic/test_anthropic.py::test_warn_on_max_tokens_truncation_silent_for_normal_stop \
  tests/llms/anthropic/test_anthropic.py::test_warn_on_max_tokens_truncation_includes_agent_role \
  tests/llms/anthropic/test_anthropic.py::test_handle_completion_logs_truncation_warning
``` ============================== 4 passed in 7.31s =============================== ```

Existing token-usage tests in the same file (test_anthropic_token_usage_tracking, test_anthropic_cache_creation_tokens_extraction, test_anthropic_missing_cache_fields_default_to_zero, test_anthropic_completion_is_used_when_anthropic_provider) also pass unchanged.

The Anthropic Message API returns stop_reason='max_tokens' when output is
cut off before generation finished, but AnthropicCompletion previously
discarded the field after extracting token usage. Silent truncation in
the final synthesis response (after tool use) is especially harmful
because partial JSON / answers are passed downstream as "complete".

Surface the signal as a logging.warning at all four completion sites
(sync/async, plain completion + tool-use follow-up). The warning
includes the agent role and current max_tokens for actionable triage.

Closes crewAIInc#5148
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AnthropicCompletion._handle_completion silently discards stop_reason; no way to detect truncation via hooks or events

1 participant