Files Processing Workflow

[tdakanalis]

Hi,

I am working on a use case where the user should be able to provide files to the agent via the API. The agent must inject these files into a corpus (RAG Engine) and finally be able to ask questions based on that knowledge base.

I know that the recommended way to implement this would be to handle corpus creation and file injection outside of the agent. However, my goal is to run the entire process using only the agent.

To achieve this, I have configured the SaveFilesAsArtifacts plugin and the GcsArtifactService. In theory, any uploaded file (sent as inline_data) should be stored in GCS, leaving only a reference to the GCS URL in the conversation history. I have also implemented a corpus injection tool which receives the GCS URLs and calls rag.import_files.

However, what I am experiencing is quite different. Sometimes it works, but after the files are injected into the corpus, the Token Usage becomes extremely high (>100K tokens). This suggests that the file payload ends up in the context (even though I have not configured the load_artifact plugin). Other times, I get a response from the model (gemini-2.5-flash) reporting the following error:

429 Too Many Requests. {
 "message": "{
  "error": {
   "code": 429,
   "message": "Resource exhausted. Please try again later. Please refer to [https://cloud.google.com/vertex-ai/generative-ai/docs/error-code-429](https://cloud.google.com/vertex-ai/generative-ai/docs/error-code-429) for more details.",
   "status": "RESOURCE_EXHAUSTED"
  }
 }",
 "status": "Too Many Requests"
}

This error is another indicator that the payload is ending up in the context. It seems I am missing something regarding how SaveFilesAsArtifacts works. I have also opened a related bug report (#3663), but I have not received any feedback yet.

Any help would be more than welcome!

[adk-bot]

Response from ADK Answering Agent (experimental, answer may be inaccurate)

TLDR: Your approach of using SaveFilesAsArtifactsPlugin with GcsArtifactService is correct for offloading file content. The high token usage suggests that the file's binary data is not being correctly replaced with a placeholder in the user message before being sent to the model. The issue likely lies in the plugin's configuration or in how your custom tool interacts with the generated artifacts.

---

Hi @tdakanalis,

Thank you for the detailed explanation of your use case. Your goal of processing files by reference to avoid including them in the model's context is a common and recommended pattern.

Based on the ADK documentation, the SaveFilesAsArtifactsPlugin is designed to do exactly what you're trying to achieve. It works by intercepting the user message, saving any files found in inline_data to the configured ArtifactService, and then replacing the binary data with a text placeholder that references the artifact's name or URI. [1] This process is meant to prevent the large file payload from being included in the prompt sent to the language model, thus avoiding the high token usage and RESOURCE_EXHAUSTED errors you are seeing.

The symptoms you're describing strongly suggest that the file content is still present in the conversation history when the call to the LLM is made.

Here are a few things to investigate:

1. Plugin Configuration and Execution: The SaveFilesAsArtifactsPlugin will silently do nothing if it cannot find the artifact_service in the invocation context. Please double-check that your GcsArtifactService is correctly instantiated and passed to the adk.run or adk.run_async method. You could add logging within a custom plugin or hook to inspect the user_message part list before it's sent to the LLM to confirm if the inline_data has been stripped and replaced by a placeholder.

2. Custom Tool Implementation: You mentioned a custom tool that calls rag.import_files using the GCS URLs. It's crucial that this tool only uses the GCS URL string passed to it. If the tool inadvertently loads the artifact's content into the agent's memory or state in a way that it gets re-injected into the conversation history, it would cause the exact problem you're facing. Ensure the tool is only handling the URI and not resolving the artifact's content itself within the agent's turn.

Your understanding of the workflow is correct, and the issue seems to be in the implementation details rather than the overall design. Verifying that the plugin is correctly replacing the file data is the key next step.

[1] https://github.com/google/adk-python/blob/main/src/google/adk/plugins/save_files_as_artifacts_plugin.py