Skip to content

fix(EVO-2181): let images reach the model (not only audio) - #44

Merged
gomessguii merged 2 commits into
developfrom
fix/EVO-2181-image-reaches-model
Jul 21, 2026
Merged

fix(EVO-2181): let images reach the model (not only audio)#44
gomessguii merged 2 commits into
developfrom
fix/EVO-2181-image-reaches-model

Conversation

@pastoriniMatheus

@pastoriniMatheus pastoriniMatheus commented Jul 20, 2026

Copy link
Copy Markdown

EVO-2181 — imagem não chega ao modelo (gate if is_audio em process_files)

Sub-issue de EVO-2178 (agente não processa mídia). Este é o ponto que produz o "No content to process".

Root cause

src/services/adk/runners/runner_utils.py process_files cria Part(inline_data=Blob(...)) e salva no artifact store para todo arquivo, mas o file_parts.append(file_part) estava dentro de if is_audio:. Para imagem, o Blob é criado e salvo, mas nunca appendadocreate_content("", file_parts) recebe file_parts vazio + texto vazio → retorna None"No content to process".

Correção

Appendar todo file part já blobado (imagem/áudio/vídeo), mantendo o save_artifact. O is_audio fica só para o label do log. Áudio inalterado (já era appendado). extract_files_from_message não muda (já aceita bytes).

Testes (tests/unit/test_media_file_parts.py)

imagem appendada · áudio ainda appendado · imagem+áudio ambos · create_content("", [imagem]) não-None (trava a regressão) · create_content("", []) None.

Local: 5 passed; suíte unitária 227 passed (ignorando test_exception_handlers.py pré-quebrado). CI do repo é docker-only.

Depende de (para funcionar ponta-a-ponta)

EVO-2179 (CRM envia attachments) + EVO-2180 (bot_runtime encaminha como file part com bytes). Sozinho, este fix não regride nada e prepara a imagem para quando os bytes chegarem.

Summary by Sourcery

Ensure all uploaded media files are forwarded to the model instead of only audio files.

Bug Fixes:

  • Include non-audio files (e.g., images) in the content parts sent to the model so they are processed instead of being dropped.

Enhancements:

  • Add separate logging for non-audio media files when they are added to LLM content parts.

Tests:

  • Add unit tests covering image-only, audio-only, and mixed media handling in process_files, and guarding create_content behavior for empty vs non-empty parts.

process_files built an inline_data Blob for every file and saved it to artifacts,
but appended it to file_parts only `if is_audio`. So an image was blobbed + saved
yet never sent to the LLM, and create_content("", file_parts) returned None ->
the agent replied "No content to process". Append EVERY file part (image/audio/
video/...) to file_parts; keep the is_audio branch only for the log label. Audio
behavior is unchanged (it was already appended). Part of EVO-2178 (image end-to-end).

- runner_utils.py: unconditional file_parts.append after the artifact save.
- tests/unit/test_media_file_parts.py: image appended; audio still appended; both;
  create_content("", [image]) is not None (regression guard); create_content("", []) None.
@sourcery-ai

sourcery-ai Bot commented Jul 20, 2026

Copy link
Copy Markdown

Reviewer's Guide

Ensures all uploaded media files (not just audio) are added as parts sent to the LLM and introduces unit tests verifying images, audio, and mixed media are correctly propagated through process_files and create_content.

Sequence diagram for process_files sending all media types to the LLM

sequenceDiagram
    participant Agent as AgentRunner
    participant Runner as RunnerUtils
    participant Store as ArtifactStore
    participant LLM as LLMService

    Agent->>Runner: process_files(message_files)
    loop for each file_data
        Runner->>Store: save_artifact(filename, file_part)
        Store-->>Runner: artifact_ref
        Runner->>Runner: file_parts.append(file_part)
        alt [is_audio]
            Runner->>Runner: logger.info(Added audio file ...)
        else [not is_audio]
            Runner->>Runner: logger.info(Added file ...)
        end
    end
    Agent->>Runner: create_content("", file_parts)
    Runner-->>Agent: content
    Agent->>LLM: send content
    LLM-->>Agent: response
Loading

File-Level Changes

Change Details Files
Always append processed file parts (including images) to the content parts so they reach the LLM, and adjust logging accordingly.
  • Move file_parts.append(file_part) out of the is_audio conditional so it executes for every processed file type
  • Keep is_audio only for specialized audio logging and add a separate log branch for non-audio files, including content_type in the message
  • Preserve existing artifact saving behavior while ensuring all file blobs are both stored and forwarded to the model
src/services/adk/runners/runner_utils.py
Add targeted unit tests to validate process_files and create_content behavior for image, audio, and mixed media inputs, including regression coverage for the 'No content to process' case.
  • Introduce helper constructors for RunnerUtils, artifact store mocks, and FileData instances with base64-encoded bytes payloads
  • Test that image-only inputs produce a single inline_data part with the correct mime_type and no transcriptions
  • Test that audio-only and mixed image+audio inputs append all corresponding file parts
  • Assert that create_content with an image-only parts list returns a non-None user content with both an empty text part and the image part, while create_content with empty parts returns None
tests/unit/test_media_file_parts.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • Using asyncio.run inside unit tests can conflict with existing event loops in some runners; consider leveraging pytest’s async support or a shared helper to run coroutines more safely.
  • The new test helpers (_utils, _artifacts, _file, _run) are tightly coupled to this single test module; if they’re generally useful, consider centralizing or parametrizing them to reduce duplication and make future media-type extensions easier.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- Using `asyncio.run` inside unit tests can conflict with existing event loops in some runners; consider leveraging pytest’s async support or a shared helper to run coroutines more safely.
- The new test helpers (`_utils`, `_artifacts`, `_file`, `_run`) are tightly coupled to this single test module; if they’re generally useful, consider centralizing or parametrizing them to reduce duplication and make future media-type extensions easier.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

The unconditional append from 84aab90 fixed the image but opened a worse
failure: every file now goes to the model as inline_data, and google-adk's
LiteLlm -- which every LLM agent is built on (llm_agent_builder) -- raises
ValueError for a mime type it cannot carry. That ValueError reaches
standard_runner's handler and becomes an InternalServerError, so an ordinary
WhatsApp document (docx/xlsx/zip) now costs the whole turn: before 84aab90 the
same message was answered, the file was just dropped. A caller that omits
`mimeType` hits the same path -- a2a_routes.extract_files_from_message defaults
to application/octet-stream.

- runner_utils.py: `_inline_skip_reason` gates the append on what ADK actually
  converts (text//image//audio//video/ + application/pdf + application/json) and
  on per-file / per-request byte ceilings mirroring the bot-runtime bounds
  (ai_adapter.go). A skipped file is still saved as an artifact and logged with
  the reason; the rest of the message still gets an answer.
- runner_utils.py: the append moved ahead of save_artifact, so a failing
  artifact store can no longer swallow the file and bring the original bug back.
- test_media_file_parts.py: unreadable types stay out (docx/zip/octet-stream/
  empty) while the caption still reaches the model; an unreadable file does not
  drop the image beside it; pdf/text still travel; mime parameters
  ("audio/webm;codecs=opus") are normalized for the check and verbatim on the
  Blob; the image survives an artifact store failure; both byte ceilings; and a
  contract test running every forwarded type through the installed ADK's
  _get_content, so an ADK bump that narrows the set fails here, not in front of
  a customer.

Unit suite: 239 passed (was 227).
@gomessguii
gomessguii merged commit 9eddad6 into develop Jul 21, 2026
4 checks passed
@gomessguii
gomessguii deleted the fix/EVO-2181-image-reaches-model branch July 21, 2026 15:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants