fix(gateway): 在转发前预过滤缺少 signature 的 thinking 块#1350
fix(gateway): 在转发前预过滤缺少 signature 的 thinking 块#1350alfadb wants to merge 1 commit intoWei-Shaw:mainfrom
Conversation
fcd5a69 to
5d1c4e4
Compare
There was a problem hiding this comment.
Pull request overview
在网关转发上游之前新增预过滤步骤,提前剔除历史消息中缺少/无效 signature 的 thinking 块,避免 Anthropic 上游直接返回 400 并导致后置重试预算耗尽(长对话场景尤甚)。
Changes:
- 在
Forward主转发路径中,StripEmptyTextBlocks之后新增FilterThinkingBlocks预过滤 - 补充较完整的注释说明问题背景与为什么不能只依赖 400 后的重试兜底
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Pre-filter: strip empty text blocks (including nested in tool_result) to prevent upstream 400. | ||
| body = StripEmptyTextBlocks(body) | ||
| // Pre-filter: remove thinking blocks with missing/invalid signatures before forwarding. | ||
| // Clients (e.g. Claude Code) sometimes send multi-turn conversations where a historical | ||
| // assistant message contains a thinking block that is missing the required "signature" field, | ||
| // causing upstream to reject the request with 400 "thinking.signature: Field required". | ||
| // FilterThinkingBlocks removes only the invalid blocks; thinking blocks with valid signatures | ||
| // are preserved. This avoids relying solely on the post-error retry path, which can time out | ||
| // (maxRetryElapsed = 10s) for long conversations before the retry budget is exhausted. | ||
| body = FilterThinkingBlocks(body) |
There was a problem hiding this comment.
This pre-filter is only applied in the non-passthrough Forward path. For Anthropic API-key passthrough (account.IsAnthropicAPIKeyPassthroughEnabled()), the code returns early into forwardAnthropicAPIKeyPassthroughWithInput, which currently only calls StripEmptyTextBlocks and would still forward missing/invalid thinking signatures unchanged (leading to the same upstream 400). Consider applying FilterThinkingBlocks in the passthrough path as well, or documenting why passthrough intentionally excludes this remediation.
| // FilterThinkingBlocks removes only the invalid blocks; thinking blocks with valid signatures | ||
| // are preserved. This avoids relying solely on the post-error retry path, which can time out | ||
| // (maxRetryElapsed = 10s) for long conversations before the retry budget is exhausted. | ||
| body = FilterThinkingBlocks(body) |
There was a problem hiding this comment.
FilterThinkingBlocks (via filterThinkingBlocksInternal) unmarshals the entire request JSON into a map[string]any and then re-marshals it when filtering triggers. In the long-conversation scenarios this change targets, that can add significant allocations/latency on the critical forwarding path. Consider reworking FilterThinkingBlocks to mirror the more optimized approach used by StripEmptyTextBlocks / FilterThinkingBlocksForRetry (extract and rewrite only the messages subtree via gjson/sjson) to reduce overhead.
| // FilterThinkingBlocks removes only the invalid blocks; thinking blocks with valid signatures | |
| // are preserved. This avoids relying solely on the post-error retry path, which can time out | |
| // (maxRetryElapsed = 10s) for long conversations before the retry budget is exhausted. | |
| body = FilterThinkingBlocks(body) | |
| // Use the optimized messages-subtree rewrite path to preserve valid thinking blocks while | |
| // avoiding a full request unmarshal/remarshal on the critical forwarding path. | |
| body = FilterThinkingBlocksForRetry(body) |
86c8b08 to
84740ab
Compare
…re forwarding Clients (e.g. Claude Code) sometimes send multi-turn conversations where a historical assistant message contains a thinking block that is missing the required "signature" field, causing upstream to reject with: 400 "messages.N.content.0.thinking.signature: Field required" Previously the gateway relied entirely on a post-error retry path (FilterThinkingBlocksForRetry) to fix this. However, for long conversations (100+ messages) the first upstream request can take longer than maxRetryElapsed (10s), exhausting the retry budget before the retry fires — leaving the client with the raw 400. Fix: apply FilterThinkingBlocks as a pre-filter alongside the existing StripEmptyTextBlocks call. This function removes only thinking blocks with missing or invalid signatures (valid signatures are preserved), so it is safe to run unconditionally on every request. The post-error retry path remains as a second line of defense. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
84740ab to
b16b5fe
Compare
问题
客户端(如 Claude Code)在多轮对话中,有时会发送历史 assistant 消息里包含缺少
signature字段的 thinking 块,导致上游直接拒绝请求:线上实际观察到该问题在超长对话(250+ 条消息)中反复出现。
根本原因
网关已有一条「400 响应后重试」路径(
FilterThinkingBlocksForRetry)来处理此类错误。但对于超长对话,第一次上游请求本身耗时可能超过maxRetryElapsed(10 s),导致重试预算在重试触发前就已耗尽。预算耗尽时的代码路径:客户端最终收到原始的 400 错误,没有任何恢复。
修复
在现有的
StripEmptyTextBlocks调用之后,将FilterThinkingBlocks作为预过滤步骤加入,在请求转发上游之前执行:FilterThinkingBlocks的行为:thinking为 enabled/adaptive 时:仅移除缺少或无效 signature 的 thinking 块,有效 signature 的 thinking 块完整保留。thinking不存在或已禁用时:移除所有 thinking 相关块。原有的重试路径(
FilterThinkingBlocksForRetry)保留,作为第二道防线。测试
所有现有的
TestFilterThinkingBlocks和TestStripEmptyTextBlocks单元测试通过,go build ./cmd/server/构建验证无误。🤖 Generated with Claude Code