Skip to content

feat: add python-sdk as a built-in SDK runner target#329

Merged
pcarleton merged 4 commits into
modelcontextprotocol:mainfrom
waprin:add-python-sdk-runner
Jun 8, 2026
Merged

feat: add python-sdk as a built-in SDK runner target#329
pcarleton merged 4 commits into
modelcontextprotocol:mainfrom
waprin:add-python-sdk-runner

Conversation

@waprin

@waprin waprin commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Adds python-sdk to KNOWN_SDKS so conformance sdk python-sdk can clone, build, and run it like typescript-sdk and go-sdk. python-sdk already ships conformance fixtures; this is just the runner wiring.

  • build: uv sync --frozen --all-extras --all-packages — one sync over the uv workspace covers both the mcp (client) and mcp-everything-server (server) packages.
  • client/server: reuse the SDK's own fixtures (.github/actions/conformance/client.py and the everything-server console script). --port 3000 matches the url and the convention used by the other entries.

Validated: --mode server --scenario server-initialize (2/2) and --mode client --scenario initialize (1/1).

🤖 Generated with Claude Code

waprin and others added 2 commits June 3, 2026 10:56
Wire modelcontextprotocol/python-sdk into the `conformance sdk` runner
alongside typescript-sdk and go-sdk, using the SDK's existing conformance
fixtures for both client and server modes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@pkg-pr-new

pkg-pr-new Bot commented Jun 4, 2026

Copy link
Copy Markdown

Open in StackBlitz

npx https://pkg.pr.new/@modelcontextprotocol/conformance@329

commit: 44561d5

@pcarleton

Copy link
Copy Markdown
Member

Thanks @waprin!

Can you check the full output? (from discord, seems like there are some failures).

python on the v1 branch should pass 100% at 2025-11-25 (that's what was submitted for tiering). That might mean a few things:

  • i added a "v1" typescript sdk runner, which should pass there
  • if you do run on the v1 branch, and it still fails, let's investigate the failures because it may be a harness regression due to some of the transport refactors

Convert the built-in python-sdk entry from v2/main to python-sdk-v1,
mirroring typescript-sdk-v1: clone the v1.x branch, default specVersion
to 2025-11-25 so draft-only scenarios/checks are excluded, and point at
the SDK's own expected-failures baseline. v2/main is mid-refactor and
noisy; v1.x is the stable published line.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@waprin

waprin commented Jun 4, 2026

Copy link
Copy Markdown
Contributor Author

@pcarleton ok Im targeting Python v1 now. Following issues:

  • json-schema-2020-12 (server) — json-schema-2020-12-tool-found FAILURE: no
    json_schema_2020_12_tool advertised.
  • server-sse-polling (server) — server-sse-priming-event + server-sse-retry-field
    WARNINGs: no SSE priming event / retry field.
  • auth/scope-step-up (client) — sep-2350-scope-union-on-reauth WARNING: drops
    previously-granted scope on re-auth.
  • For json-schema tool and server-sse-polling found, looks likes its its cause the python sdk's own confrmance suite runs active instead of all . The server doesnt implement it.

So, should we be running active or all?

  • for auth/scope-step-up seems the SEP-2350 check is newer than what the python-sdk pins , but it's missing on SDK side. Should we try to add it there or is it out of scope?

@pcarleton

Copy link
Copy Markdown
Member

awesome, ty!

I'll approve this to merge in, however we should change it to run just active for servers (if you want to do that in this one, i'm happy to re-review). That's the suite we run for tiering, and so is the most reasonable default here to avoid confusion.

Here's where we set that in tiering:
https://github.com/modelcontextprotocol/conformance/blob/main/src/tier-check/checks/test-conformance-results.ts#L214

You're right about step-up... we should probably add a spec version branch in the step-up behavior so we don't fail on the "non accumulating" behavior from 2025-11-25 (we added "set-wise union" as a requirement in 2026-07-28).

pcarleton
pcarleton previously approved these changes Jun 5, 2026
Server runs previously fell through to the underlying `server` command's
default; make it explicit and consistent with tiering, which runs the
`active` suite (excludes pending/draft). Avoids surfacing intentionally
-deferred `pending` scenarios (e.g. json-schema-2020-12, server-sse
-polling) as failures. Runner-wide default for all SDKs; callers can
still override with --suite.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@waprin

waprin commented Jun 8, 2026

Copy link
Copy Markdown
Contributor Author

@pcarleton ok I changed it to suite active please re-review . I will look into spec-version branch thing.

@pcarleton

Copy link
Copy Markdown
Member

thanks @waprin !

@pcarleton pcarleton enabled auto-merge (squash) June 8, 2026 17:37
@pcarleton pcarleton merged commit 28f8338 into modelcontextprotocol:main Jun 8, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants