Skip to content

fix(gateway): 在转发前预过滤缺少 signature 的 thinking 块#1350

Open
alfadb wants to merge 1 commit intoWei-Shaw:mainfrom
alfadb:fix/prefilter-thinking-signature
Open

fix(gateway): 在转发前预过滤缺少 signature 的 thinking 块#1350
alfadb wants to merge 1 commit intoWei-Shaw:mainfrom
alfadb:fix/prefilter-thinking-signature

Conversation

@alfadb
Copy link
Copy Markdown
Contributor

@alfadb alfadb commented Mar 27, 2026

问题

客户端(如 Claude Code)在多轮对话中,有时会发送历史 assistant 消息里包含缺少 signature 字段的 thinking 块,导致上游直接拒绝请求:

400 "messages.N.content.0.thinking.signature: Field required"

线上实际观察到该问题在超长对话(250+ 条消息)中反复出现。

根本原因

网关已有一条「400 响应后重试」路径(FilterThinkingBlocksForRetry)来处理此类错误。但对于超长对话,第一次上游请求本身耗时可能超过 maxRetryElapsed(10 s),导致重试预算在重试触发前就已耗尽。预算耗尽时的代码路径:

if time.Since(retryStart) >= maxRetryElapsed {
    resp.Body = io.NopCloser(bytes.NewReader(respBody)) // 恢复原始错误体
    break
}

客户端最终收到原始的 400 错误,没有任何恢复。

修复

在现有的 StripEmptyTextBlocks 调用之后,将 FilterThinkingBlocks 作为预过滤步骤加入,在请求转发上游之前执行:

body = StripEmptyTextBlocks(body)
body = FilterThinkingBlocks(body)  // ← 新增

FilterThinkingBlocks 的行为:

  • 顶层 thinkingenabled/adaptive 时:仅移除缺少或无效 signature 的 thinking 块,有效 signature 的 thinking 块完整保留
  • 顶层 thinking 不存在或已禁用时:移除所有 thinking 相关块。
  • 有 fast-path:请求体不含 thinking 相关字节时直接返回,常规请求零开销

原有的重试路径(FilterThinkingBlocksForRetry)保留,作为第二道防线。

测试

所有现有的 TestFilterThinkingBlocksTestStripEmptyTextBlocks 单元测试通过,go build ./cmd/server/ 构建验证无误。

🤖 Generated with Claude Code

@alfadb alfadb changed the title fix(gateway): pre-filter thinking blocks with missing signatures before forwarding fix(gateway): 在转发前预过滤缺少 signature 的 thinking 块 Mar 27, 2026
Copilot AI review requested due to automatic review settings April 9, 2026 09:49
@alfadb alfadb force-pushed the fix/prefilter-thinking-signature branch from fcd5a69 to 5d1c4e4 Compare April 9, 2026 09:49
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

在网关转发上游之前新增预过滤步骤,提前剔除历史消息中缺少/无效 signaturethinking 块,避免 Anthropic 上游直接返回 400 并导致后置重试预算耗尽(长对话场景尤甚)。

Changes:

  • Forward 主转发路径中,StripEmptyTextBlocks 之后新增 FilterThinkingBlocks 预过滤
  • 补充较完整的注释说明问题背景与为什么不能只依赖 400 后的重试兜底

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines 4071 to +4080
// Pre-filter: strip empty text blocks (including nested in tool_result) to prevent upstream 400.
body = StripEmptyTextBlocks(body)
// Pre-filter: remove thinking blocks with missing/invalid signatures before forwarding.
// Clients (e.g. Claude Code) sometimes send multi-turn conversations where a historical
// assistant message contains a thinking block that is missing the required "signature" field,
// causing upstream to reject the request with 400 "thinking.signature: Field required".
// FilterThinkingBlocks removes only the invalid blocks; thinking blocks with valid signatures
// are preserved. This avoids relying solely on the post-error retry path, which can time out
// (maxRetryElapsed = 10s) for long conversations before the retry budget is exhausted.
body = FilterThinkingBlocks(body)
Copy link

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pre-filter is only applied in the non-passthrough Forward path. For Anthropic API-key passthrough (account.IsAnthropicAPIKeyPassthroughEnabled()), the code returns early into forwardAnthropicAPIKeyPassthroughWithInput, which currently only calls StripEmptyTextBlocks and would still forward missing/invalid thinking signatures unchanged (leading to the same upstream 400). Consider applying FilterThinkingBlocks in the passthrough path as well, or documenting why passthrough intentionally excludes this remediation.

Copilot uses AI. Check for mistakes.
Comment on lines +4077 to +4080
// FilterThinkingBlocks removes only the invalid blocks; thinking blocks with valid signatures
// are preserved. This avoids relying solely on the post-error retry path, which can time out
// (maxRetryElapsed = 10s) for long conversations before the retry budget is exhausted.
body = FilterThinkingBlocks(body)
Copy link

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FilterThinkingBlocks (via filterThinkingBlocksInternal) unmarshals the entire request JSON into a map[string]any and then re-marshals it when filtering triggers. In the long-conversation scenarios this change targets, that can add significant allocations/latency on the critical forwarding path. Consider reworking FilterThinkingBlocks to mirror the more optimized approach used by StripEmptyTextBlocks / FilterThinkingBlocksForRetry (extract and rewrite only the messages subtree via gjson/sjson) to reduce overhead.

Suggested change
// FilterThinkingBlocks removes only the invalid blocks; thinking blocks with valid signatures
// are preserved. This avoids relying solely on the post-error retry path, which can time out
// (maxRetryElapsed = 10s) for long conversations before the retry budget is exhausted.
body = FilterThinkingBlocks(body)
// Use the optimized messages-subtree rewrite path to preserve valid thinking blocks while
// avoiding a full request unmarshal/remarshal on the critical forwarding path.
body = FilterThinkingBlocksForRetry(body)

Copilot uses AI. Check for mistakes.
@alfadb alfadb force-pushed the fix/prefilter-thinking-signature branch 2 times, most recently from 86c8b08 to 84740ab Compare April 12, 2026 11:03
…re forwarding

Clients (e.g. Claude Code) sometimes send multi-turn conversations where
a historical assistant message contains a thinking block that is missing
the required "signature" field, causing upstream to reject with:
  400 "messages.N.content.0.thinking.signature: Field required"

Previously the gateway relied entirely on a post-error retry path
(FilterThinkingBlocksForRetry) to fix this. However, for long
conversations (100+ messages) the first upstream request can take
longer than maxRetryElapsed (10s), exhausting the retry budget before
the retry fires — leaving the client with the raw 400.

Fix: apply FilterThinkingBlocks as a pre-filter alongside the existing
StripEmptyTextBlocks call. This function removes only thinking blocks
with missing or invalid signatures (valid signatures are preserved), so
it is safe to run unconditionally on every request. The post-error retry
path remains as a second line of defense.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@alfadb alfadb force-pushed the fix/prefilter-thinking-signature branch from 84740ab to b16b5fe Compare April 14, 2026 08:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants