Skip to content

docs: clarify AI request protocol handling - #13731

Open
kayx23 wants to merge 6 commits into
apache:masterfrom
kayx23:agent/document-ai-protocol-detection
Open

docs: clarify AI request protocol handling#13731
kayx23 wants to merge 6 commits into
apache:masterfrom
kayx23:agent/document-ai-protocol-detection

Conversation

@kayx23

@kayx23 kayx23 commented Jul 23, 2026

Copy link
Copy Markdown
Member

Description

Several AI request formats share the same body fields: Bedrock Converse, Anthropic Messages, and OpenAI Chat Completions can contain messages, while OpenAI Responses and Embeddings both use input. APISIX resolves these overlaps by checking URI-specific formats before body-only formats, but the plugin documentation did not explain the detection order, required URI suffixes, or downstream endpoint selection consistently.

The related AI plugins also handle the detected formats differently. Some inspect or modify each protocol's native content structure, while ai-prompt-template replaces the complete request body and ai-request-rewrite creates a separate provider request. The previous prompt-template description incorrectly limited templates to Chat Completions even though the plugin accepts any JSON template supported by the downstream plugin.

This PR updates the English and Chinese plugin references to:

  • document the complete request detection order for Bedrock Converse, Anthropic Messages, OpenAI Responses, Chat Completions, Embeddings, and passthrough JSON
  • explain which URI suffixes must be preserved and that custom URI prefixes remain valid
  • clarify automatic OpenAI endpoint selection and remove redundant Embeddings endpoint overrides
  • document the separate Anthropic Chat Completions and native Messages endpoints
  • add Responses API route and request examples for ai-proxy and ai-proxy-multi
  • document format-specific behavior for prompt decoration, prompt guarding, RAG, caching, Lakera Guard, and AWS and Aliyun content moderation
  • clarify that ai-prompt-template replaces the incoming body with an arbitrary rendered JSON template
  • add a Responses API prompt-template example that performs an OpenAI web search restricted to apisix.apache.org and shows the resolved request and cited response
  • clarify that ai-request-rewrite creates a separate non-streaming provider request instead of classifying and proxying the client's request as an AI protocol

Which issue(s) this PR fixes:

N/A

Validation

  • verified request detection, protocol adapters, provider endpoint selection, cache behavior, moderation behavior, prompt rewriting, and template replacement against the implementation and protocol tests
  • validated the Responses API web-search template end to end with APISIX and OpenAI; the request returned HTTP 200 with a completed web search call and URL citation
  • ran the repository-pinned markdownlint on all changed pages
  • ran utils/fix-zh-doc-segment.py and utils/check-category.py
  • parsed the new Responses request and response JSON examples
  • ran git diff --check

Checklist

  • I have explained the need for this PR and the problem it solves
  • I have explained the changes or the new features added to this PR
  • I have added tests corresponding to this change (documentation-only; existing protocol tests cover this behavior)
  • I have updated the documentation to reflect this change
  • I have verified that this change is backward compatible

@dosubot dosubot Bot added size:L This PR changes 100-499 lines, ignoring generated files. doc Documentation things labels Jul 23, 2026
@kayx23 kayx23 changed the title docs(ai-proxy): document OpenAI request protocols docs: document OpenAI request protocols in AI plugins Jul 23, 2026
@kayx23 kayx23 changed the title docs: document OpenAI request protocols in AI plugins docs: clarify AI request protocol handling Jul 23, 2026
@dosubot dosubot Bot added size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Jul 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc Documentation things size:XL This PR changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants