Skip to content

Feature/add file support for gmail#242

Open
pulinduvidmal wants to merge 69 commits intodevelopfrom
feature/add_file_support_for_gmail
Open

Feature/add file support for gmail#242
pulinduvidmal wants to merge 69 commits intodevelopfrom
feature/add_file_support_for_gmail

Conversation

@pulinduvidmal
Copy link
Contributor

Description

This change will add file & image support for gmail integrations

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactoring (no functional changes)
  • Performance improvement
  • Test update
  • CI/CD update
  • Other (please describe):

Testing

  • Unit tests pass locally
  • Integration tests pass locally
  • Manual testing completed
  • New tests added for changes

Checklist

  • My code follows the project's style guidelines
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published

Screenshots (if applicable)

Additional Notes

pulinduvidmal and others added 30 commits December 23, 2025 21:13
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings February 26, 2026 16:17
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a multimodal attachment pipeline (storage + pre-hook + analysis tool) and wires it into multiple agent frameworks, while extending the Telegram integration to accept photos/documents and updating integration docs (Telegram/Gmail/Messenger) and examples accordingly.

Changes:

  • Add a multimodal pre-hook that stores incoming attachments per-session and injects brief attachment descriptions into the user text.
  • Add an analyze_attachments tool (attached automatically when multimodal is enabled) plus framework support for system prompt/tool attachment.
  • Extend Telegram webhook handling to ingest photos/documents and forward them as multi-request inputs; update related docs/examples.

Reviewed changes

Copilot reviewed 25 out of 26 changed files in this pull request and generated 13 comments.

Show a summary per file
File Description
examples/api/telegram/uv.lock Switches agentkernel dependency to a local editable source.
examples/api/telegram/pyproject.toml Adds uv source override for local editable agentkernel.
examples/api/telegram/build.sh Updates local install extras (adds adk).
examples/api/telegram/README.md Documents multimodal env vars and ADK/OpenAI run options.
examples/api/gmail/README.md Documents multimodal env vars + attachment memory behavior.
docs/docs/integrations/telegram.md Expands Telegram integration docs to include multimodal behavior.
ak-py/tests/test_tool_adk.py Updates mocks to satisfy new Agent abstract methods.
ak-py/tests/test_tool.py Updates mocks to satisfy new Agent abstract methods.
ak-py/tests/test_runtime.py Updates dummy agent to satisfy new Agent abstract methods.
ak-py/tests/test_module.py Updates wrapped agent mock to satisfy new Agent abstract methods.
ak-py/tests/test_base.py Updates mock agent to satisfy new Agent abstract methods.
ak-py/src/agentkernel/integration/telegram/telegram_chat.py Adds Telegram photo/document download + run_multi() forwarding; moves webhook processing to background.
ak-py/src/agentkernel/integration/telegram/README.md Documents Telegram multimodal features and limitations.
ak-py/src/agentkernel/integration/messenger/README.md Adds multimodal configuration section.
ak-py/src/agentkernel/framework/openai/openai.py Adds system prompt/tool attachment hooks and wrapper accessors.
ak-py/src/agentkernel/framework/langgraph/langgraph.py Adds system prompt/tool attachment plumbing + tool binding signature changes.
ak-py/src/agentkernel/framework/crewai/crewai.py Adds system prompt/tool attachment hooks and wrapper accessors.
ak-py/src/agentkernel/framework/adk/adk.py Adds system prompt/tool attachment hooks and wrapper accessors.
ak-py/src/agentkernel/core/tool.py Broadens ToolBuilder bind() typing/docstring.
ak-py/src/agentkernel/core/runtime.py Registers multimodal pre-hook factory as a system pre-hook.
ak-py/src/agentkernel/core/multimodal/tools.py New: attachment description + analyze_attachments tool.
ak-py/src/agentkernel/core/multimodal/storage.py New: session-backed attachment storage + pruning index.
ak-py/src/agentkernel/core/multimodal/hooks.py New: multimodal pre-hook implementation + factory.
ak-py/src/agentkernel/core/multimodal/init.py New: multimodal module exports.
ak-py/src/agentkernel/core/config.py Adds multimodal config section; increases default max_file_size.
ak-py/src/agentkernel/core/base.py Adds new Agent abstract methods + system prompt/tool auto-attach helpers.
Comments suppressed due to low confidence (1)

ak-py/src/agentkernel/core/tool.py:156

  • ToolBuilder.bind() (and the updated docstrings) now say it accepts “existing tools”, but framework builders still hard-require callable(func) and will reject already-wrapped tool objects. Either implement support for existing tool definitions across builders, or revert the signature/docstring change to avoid promising behavior that isn't implemented.
class ToolBuilder:
    """
    Base class for framework-specific tool builders.

    Provides common functionality that bind regular synchronous and asynchronous functions
    into framework specific tool functions. It also makes sure execution context behaves in the
    same manner across different frameworks.
    """

    @classmethod
    def bind(cls, funcs: list[Any]) -> list[Any]:
        """
        Bind a list of generic tool functions to framework-specific tool definitions.

        :param funcs: List of generic tool functions or existing tools to bind.
        :return: List of framework-specific tool definitions.
        :raises NotImplementedError: If called on the base ToolBuilder class.
        """
        raise NotImplementedError("bind() must be implemented by framework-specific subclasses")

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants