Skip to content

Conversation

@jeromewynne-jpg
Copy link

@jeromewynne-jpg jeromewynne-jpg commented Dec 8, 2025

Hiya! I found agent participants weren't completing stages as intended and had their responses incorrectly parsed in group chats so have vibe-coded a fix. Code was written by Claude - I did my best to guide it towards making the minimum changes necessary and not introduce any breaking changes.

As part of the PR I've included a way to view agent participant API logs in the UI and have expanded the agent participant config UI component. Here's a video showing the result:

screen-recording-2025-12-08

Summary

This PR enhances agent participant functionality with three main improvements:

  1. Agent Log Viewer - New section in participant stats page to view agent model call history
  2. Bug Fixes - Debugged and fixed several issues preventing agent participants from working correctly
  3. Expanded Configuration - New dialog for quick-adding agents with model selection, web search, and chat settings

Changes

Agent Log Viewer

  • Added "Agent log" section to participant stats page
  • Displays model call history with prompts, responses, and timestamps
  • Helps debug agent behavior and track API interactions

Agent Participant Bug Fixes

  • Fixed chat settings not being applied for quick-add agents (settings were not merged into prompt config)
  • Fixed Claude API not capturing responses when web search is enabled (text was in last content block, not first)
  • Fixed OpenAI structured output compatibility issues
  • Fixed empty response handling for Gemini API
  • Added debug logging for troubleshooting agent behavior

Expanded Agent Configuration

  • New agent-participant-configuration-dialog component
  • Support for multiple LLM providers: Gemini, OpenAI, Claude, and Ollama
  • Web search capability for Gemini (Google Search grounding) and Claude (web_search tool)
  • Chat settings: typing speed, min messages before responding, self-triggering, max responses
  • Fixed dialog z-index to appear above cohort-list header

Backend Changes

  • Added useWebSearch to AgentModelSettings interface
  • Gemini API: Google Search grounding via googleSearch tool
  • Claude API: web_search_20250305 tool integration
  • Fixed Claude API text extraction for multi-block responses

Testing

I've created a few experiments with 'group chat', 'set your profile', and 'survey' stages, and have tested them against each combination of (Google/Anthropic/OpenAI, w/wo web search), with between one and four agent participants. The parsing errors have disappeared, the API log viewer seems to work, and the new agent participant settings worked as intended.

I haven't included automated tests as the contrib guidelines didn't ask for them, however if you have something in mind please say.

  • View agent logs in participant stats page
  • Add agent participant via cohort panel dialog
  • Test with Gemini model (with and without web search)
  • Test with Claude model (with and without web search)
  • Verify web search option is hidden for OpenAI and Ollama
  • Verify chat settings are applied correctly

Known Limitations

  • OpenAI web search not supported - the code's using the Chat Completions API rather than the Responses API, ideally it would use the Responses API
  • Ollama web search not supported

jeromewynne-jpg and others added 3 commits December 8, 2025 07:16
Display agent-specific information when viewing agent participant stats:
- Agent configuration (API type, model, prompt context)
- Summary of API calls (total, successful, failed counts)
- Detailed log entries for each LLM call with:
  - Status, timestamp, and duration
  - Stage where the call was made
  - Expandable prompt and response details

This helps experimenters debug and monitor agent participant behavior
in real-time as logs stream in via Firestore.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix empty agentId causing Firestore path error in getAgentParticipantPrompt
  When agent participants are created without a persona, agentId is empty string
  which caused .doc('') to crash. Now returns null to use default prompts.

- Fix Gemini API not parsing JSON responses for agent participants
  Added missing addParsedModelResponse() call that OpenAI/Claude APIs had.
  This was preventing profile and survey stage responses from being processed.

- Add debug logging to agent participant and chat flows
  Logs prefixed with [AgentParticipant], [AgentChat], [ChatTrigger] to help
  trace issues in Firebase Functions console.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add useWebSearch option to agent model settings for Gemini and Claude
- Hide web search option in UI for OpenAI/Ollama (not supported)
- Fix Claude API to extract text from correct content block when web search is used
- Fix dialog overlay z-index to appear above cohort-list header
- Add chat settings support for quick-add agent participants

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@google-cla
Copy link

google-cla bot commented Dec 8, 2025

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@jeromewynne-jpg
Copy link
Author

I'd signed it but needed to resync, if you check now you should you should see I've signed

@jeromewynne-jpg
Copy link
Author

I see that the CLA check is failing for Claude 😆

@vivtsai vivtsai self-requested a review December 9, 2025 18:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants