Skip to content

Feat/preserve thought signature for gemini3 #1227

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
dpolistwm wants to merge 17 commits into
base: main
Choose a base branch
from
+41 −32
Open

Feat/preserve thought signature for gemini3 #1227

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
fd3eda8
feat: preserve thoughtSignature for Gemini 3 Pro function calling
dpolistwm Nov 21, 2025
9730a99
feat: add Gemini thoughtSignature capture and handling
dpolistwm Nov 21, 2025
73f9bc3
fix: resolve mypy type checking errors
dpolistwm Nov 21, 2025
944b0e1
style: apply code formatting (remove trailing whitespace)
dpolistwm Nov 21, 2025
5424372
Unit test implementation
dpolistwm Nov 21, 2025
a20b7ea
chore: revert pyproject.toml to use dynamic versioning with hatch-vcs
dpolistwm Nov 21, 2025
a999c77
test: update test_stream_request_with_tool_spec to expect thinking_co…
dpolistwm Nov 21, 2025
5928342
style: remove trailing whitespace and unused imports
dpolistwm Nov 21, 2025
a80c602
refactor: rename type test files to be more specific
dpolistwm Nov 21, 2025
3e23297
Remove unnecessary TypedDict unit tests
dpolistwm Nov 21, 2025
82f8701
Remove redundant thoughtSignature test function
dpolistwm Nov 21, 2025
fa9c74c
test: add coverage for thoughtSignature edge cases in gemini.py
dpolistwm Nov 24, 2025
c42c0fc
refactor: keep thoughtSignature handling internal to GeminiModel
dpolistwm Dec 5, 2025
6f71ec2
revert: restore original SDK files, keep thoughtSignature only in gem…
dpolistwm Dec 5, 2025
8f0251a
fix: update test to expect thinking_config in tool_spec request
dpolistwm Dec 5, 2025
eff6015
Dealing with agents without tools
dpolistwm Dec 5, 2025
4184ab4
Removed hard-coded thinking config
dpolistwm Dec 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 33 additions & 10 deletions src/strands/models/gemini.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,10 @@ def __init__(

self.client_args = client_args or {}

# Store the last thought_signature from Gemini responses for multi-turn conversations
# See: https://ai.google.dev/gemini-api/docs/thought-signatures
self.last_thought_signature: Optional[bytes] = None
Copy link
Contributor

@mkmeral mkmeral Dec 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we default to skip? This way, if developers change model provider (e.g. anthropic to gemini), the agent would still work

Copy link
Author

@dpolistwm dpolistwm Dec 8, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is contained within the Gemini model implementation so it doesn't affect other providers. Could you please ellaborate? (the absence of thought signatures is also the root cause of the current incompatibility with Gemini 3 Pro)

Copy link
Contributor

@mkmeral mkmeral Dec 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually 2 use cases:

First, customers can switch model providers. So imagine they start with Bedrock, and then they want to change the agent's model provider to gemini 3

# Written is semi-psuedocode, don't focus on code too much 😅 
agent = Agent(model=BedrockModel(), tools=[tool1, tool2])

# agent is being used with tools
agent(...) 

# later (for some reason) devs might try to update the model providers to Gemini 3
agent.model = GeminiModel(modelId="3-pro")

# agent is being invoked. At this point, it has the original message history with tool use, but it has a clean gemini model provider. If we do not skip the 
agent(...)

Second, session managers can be problematic. If agent's context is loaded from session managers, the thought signature will not exist. We need to ensure for these cases, we do not throw exception.

# Create a session manager with a unique session ID
session_manager = FileSessionManager(session_id="test-session")
agent = Agent(session_manager=session_manager)
agent("Hello!")  # This conversation is persisted


# Couple hours later, in another runtime instance, running the same code again
session_manager = FileSessionManager(session_id="test-session") # Loads the messages from file system
agent = Agent(session_manager=session_manager)
agent("Hello!")  # Throws error due to though signatures

Copy link
Author

@dpolistwm dpolistwm Dec 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From what I understood reading Google's docs, the requirement for the preservation of tought signatures needs to happen within a turn/cycle of the agent loop. So, if a session/memory was created using another model, during the next turn execution with Gemini 3, a first/new thought signature would be introduced and be used. I can try to run both scenarios to try and reproduce.

Copy link
Author

@dpolistwm dpolistwm Dec 8, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's a quick script I've used:

#!/usr/bin/env python3
"""Example script using strands agents with FileSystem session manager."""

import os
from uuid import uuid4
from dotenv import load_dotenv

load_dotenv()
from strands import Agent, tool
from strands.models.gemini import GeminiModel
from strands.session import FileSessionManager
from strands.models.bedrock import BedrockModel

@tool
def get_weather(city: str) -> str:
    return f"The weather in {city} is sunny."

@tool
def greet_user(user_name: str) -> str:
    return f"Hello, {user_name}! How can I help you today?"


def main():
    """Main function demonstrating strands agents with FileSessionManager."""
    # Create a storage directory for sessions
    storage_dir = os.path.join(os.getcwd(), "sessions")
    
    # Generate a session ID (or use a fixed one for persistence)
    session_id = str(uuid4())
    print(f"Session ID: {session_id}")
    
    # Create FileSessionManager
    session_manager = FileSessionManager(
        session_id=session_id,
        storage_dir=storage_dir
    )
    
    # Create GeminiModel with gemini-3-pro-preview
    gemini_model = GeminiModel(
        model_id="gemini-3-pro-preview",
    )

    bedrock_model = BedrockModel(
        model_id="us.anthropic.claude-3-7-sonnet-20250219-v1:0",
    )
    
    # Create an agent with the session manager and Bedrock model
    agent = Agent(
        system_prompt="You are a helpful assistant that can answer questions and help with tasks. When the user provides their name, greet them using the greet_user tool.",
        model=bedrock_model, 
        tools=[greet_user, get_weather], 
        session_manager=session_manager,
        agent_id="my_agent"
    )
    
    agent("Hello! My name is Daniel.")

    agent.model = gemini_model
    agent ("What is the weather in Tokyo?")


if __name__ == "__main__":
    main()

Worked fine :) Here's the output:

➜  test-gemini3 uv run python main.py
Session ID: 8721956e-28db-4ee0-abe1-68a39ad64d08

Tool #1: greet_user
Hello Daniel! It's nice to meet you. How can I assist you today? I'm here to help answer questions or assist with tasks you might have.
Tool #2: get_weather
The weather in Tokyo is currently sunny


@override
def update_config(self, **model_config: Unpack[GeminiConfig]) -> None: # type: ignore[override]
"""Update the Gemini model configuration with the provided arguments.
Expand Down Expand Up @@ -141,12 +145,15 @@ def _format_request_content_part(self, content: ContentBlock) -> genai.types.Par
)

if "toolUse" in content:
# Use the last thought_signature stored from previous Gemini responses
# This is required for Gemini models that use thought signatures in multi-turn conversations
return genai.types.Part(
function_call=genai.types.FunctionCall(
args=content["toolUse"]["input"],
id=content["toolUse"]["toolUseId"],
name=content["toolUse"]["name"],
),
thought_signature=self.last_thought_signature,
Copy link
Contributor

@mkmeral mkmeral Dec 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would apply it to all tool uses right? Is there a way we can make it more specific?

Copy link
Author

@dpolistwm dpolistwm Dec 8, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is required per Gemini docs:

"As a general rule, if you receive a thought signature in a model response, you should pass it back exactly as received when sending the conversation history in the next turn. When using Gemini 3 Pro, you must pass back thought signatures during function calling, otherwise you will get a validation error (4xx status code)."

Copy link
Contributor

@mkmeral mkmeral Dec 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant more of this line, where we don't actually need to set this for all tool calls, just the first one. That said, this is more of a nit comment

The first functionCall part in each step of the current turn must include its thought_signature

https://ai.google.dev/gemini-api/docs/thought-signatures

)

raise TypeError(f"content_type=<{next(iter(content))}> | unsupported type")
Expand All @@ -170,7 +177,7 @@ def _format_request_content(self, messages: Messages) -> list[genai.types.Conten
for message in messages
]

def _format_request_tools(self, tool_specs: Optional[list[ToolSpec]]) -> list[genai.types.Tool | Any]:
def _format_request_tools(self, tool_specs: Optional[list[ToolSpec]]) -> Optional[list[genai.types.Tool | Any]]:
"""Format tool specs into Gemini tools.

- Docs: https://googleapis.github.io/python-genai/genai.html#genai.types.Tool
Expand All @@ -179,8 +186,11 @@ def _format_request_tools(self, tool_specs: Optional[list[ToolSpec]]) -> list[ge
tool_specs: List of tool specifications to make available to the model.

Return:
Gemini tool list.
Gemini tool list, or None if no tools are provided.
"""
if not tool_specs:
return None

return [
genai.types.Tool(
function_declarations=[
Expand All @@ -189,7 +199,7 @@ def _format_request_tools(self, tool_specs: Optional[list[ToolSpec]]) -> list[ge
name=tool_spec["name"],
parameters_json_schema=tool_spec["inputSchema"]["json"],
)
for tool_spec in tool_specs or []
for tool_spec in tool_specs
],
),
]
Expand Down Expand Up @@ -268,14 +278,14 @@ def _format_chunk(self, event: dict[str, Any]) -> StreamEvent:
# that name be set in the equivalent FunctionResponse type. Consequently, we assign
# function name to toolUseId in our tool use block. And another reason, function_call is
# not guaranteed to have id populated.
tool_use: dict[str, Any] = {
"name": event["data"].function_call.name,
"toolUseId": event["data"].function_call.name,
}

return {
"contentBlockStart": {
"start": {
"toolUse": {
"name": event["data"].function_call.name,
"toolUseId": event["data"].function_call.name,
},
},
"start": {"toolUse": cast(Any, tool_use)},
},
}

Expand Down Expand Up @@ -373,15 +383,28 @@ async def stream(
yield self._format_chunk({"chunk_type": "content_start", "data_type": "text"})

tool_used = False

async for event in response:
candidates = event.candidates
candidate = candidates[0] if candidates else None
content = candidate.content if candidate else None
parts = content.parts if content and content.parts else []

for part in parts:
# Capture thought_signature and store it for use in subsequent requests
# According to Gemini docs, thought_signature can be on any part
# See: https://ai.google.dev/gemini-api/docs/thought-signatures
if hasattr(part, "thought_signature") and part.thought_signature:
self.last_thought_signature = part.thought_signature

if part.function_call:
yield self._format_chunk({"chunk_type": "content_start", "data_type": "tool", "data": part})
yield self._format_chunk(
{
"chunk_type": "content_start",
"data_type": "tool",
"data": part,
}
)
yield self._format_chunk({"chunk_type": "content_delta", "data_type": "tool", "data": part})
yield self._format_chunk({"chunk_type": "content_stop", "data_type": "tool", "data": part})
tool_used = True
Expand Down
30 changes: 8 additions & 22 deletions tests/strands/models/test_gemini.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ async def test_stream_request_default(gemini_client, model, messages, model_id):
await anext(model.stream(messages))

exp_request = {
"config": {"tools": [{"function_declarations": []}]},
"config": {},
"contents": [{"parts": [{"text": "test"}], "role": "user"}],
"model": model_id,
}
Expand All @@ -99,7 +99,6 @@ async def test_stream_request_with_params(gemini_client, model, messages, model_

exp_request = {
"config": {
"tools": [{"function_declarations": []}],
"temperature": 1,
},
"contents": [{"parts": [{"text": "test"}], "role": "user"}],
Expand All @@ -113,7 +112,7 @@ async def test_stream_request_with_system_prompt(gemini_client, model, messages,
await anext(model.stream(messages, system_prompt=system_prompt))

exp_request = {
"config": {"system_instruction": system_prompt, "tools": [{"function_declarations": []}]},
"config": {"system_instruction": system_prompt},
"contents": [{"parts": [{"text": "test"}], "role": "user"}],
"model": model_id,
}
Expand Down Expand Up @@ -146,9 +145,7 @@ async def test_stream_request_with_document(content, formatted_part, gemini_clie
await anext(model.stream(messages))

exp_request = {
"config": {
"tools": [{"function_declarations": []}],
},
"config": {},
"contents": [{"parts": [formatted_part], "role": "user"}],
"model": model_id,
}
Expand All @@ -173,9 +170,7 @@ async def test_stream_request_with_image(gemini_client, model, model_id):
await anext(model.stream(messages))

exp_request = {
"config": {
"tools": [{"function_declarations": []}],
},
"config": {},
"contents": [
{
"parts": [
Expand Down Expand Up @@ -214,9 +209,7 @@ async def test_stream_request_with_reasoning(gemini_client, model, model_id):
await anext(model.stream(messages))

exp_request = {
"config": {
"tools": [{"function_declarations": []}],
},
"config": {},
"contents": [
{
"parts": [
Expand Down Expand Up @@ -277,9 +270,7 @@ async def test_stream_request_with_tool_use(gemini_client, model, model_id):
await anext(model.stream(messages))

exp_request = {
"config": {
"tools": [{"function_declarations": []}],
},
"config": {},
"contents": [
{
"parts": [
Expand Down Expand Up @@ -327,9 +318,7 @@ async def test_stream_request_with_tool_results(gemini_client, model, model_id):
await anext(model.stream(messages))

exp_request = {
"config": {
"tools": [{"function_declarations": []}],
},
"config": {},
"contents": [
{
"parts": [
Expand Down Expand Up @@ -371,9 +360,7 @@ async def test_stream_request_with_empty_content(gemini_client, model, model_id)
await anext(model.stream(messages))

exp_request = {
"config": {
"tools": [{"function_declarations": []}],
},
"config": {},
"contents": [{"parts": [], "role": "user"}],
"model": model_id,
}
Expand Down Expand Up @@ -614,7 +601,6 @@ async def test_structured_output(gemini_client, model, messages, model_id, weath

exp_request = {
"config": {
"tools": [{"function_declarations": []}],
"response_mime_type": "application/json",
"response_schema": weather_output.model_json_schema(),
},
Expand Down