Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .changeset/response-cache-document-codec.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
'@modelcontextprotocol/client': minor
---

The response cache now stores results as JSON-serialized documents (serialize on write, parse on read) instead of live object graphs isolated with `structuredClone`. Same mutation isolation, but no dependency on the `structuredClone` global — whose absence (jest+jsdom, Node < 17) previously made every cache write throw into the store-error swallow, silently disabling caching and output-schema lookups for the session. A value without a JSON representation now fails the write loudly to the error sink, and an undecodable document in an external store is reported, dropped, and read as a miss.

Migration for custom `ResponseCacheStore` implementations: `CacheEntry.value` (and the `set()` entry value) is now `string` — persist and return it verbatim, `JSON.parse` to inspect. Entries persisted by a previous SDK version fail decode once (reported, dropped) and are rewritten on the next fetch.
2 changes: 1 addition & 1 deletion docs/clients/caching.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ const store = new InMemoryResponseCacheStore({ maxEntries: 2048 });
const client = new Client({ name: 'my-client', version: '1.0.0' }, { responseCacheStore: store });
```

Every method on the `ResponseCacheStore` interface may return a promise, so a Redis-style store implements the same five methods. Entries are keyed by connected-server identity, so one store can back many clients: connections to different servers never collide.
Every method on the `ResponseCacheStore` interface may return a promise, so a Redis-style store implements the same five methods. `CacheEntry.value` is the JSON-serialized result document — a persistent store persists the string verbatim, no extra serialization step, and behaves identically to the in-memory default. Entries are keyed by connected-server identity, so one store can back many clients: connections to different servers never collide.

## Partition the store per user

Expand Down
2 changes: 1 addition & 1 deletion docs/clients/calling.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ A failed tool call is still a result: check `isError` on it before trusting `con

## Let the SDK walk the pages

That `listTools()` already walked every page: when a server splits its list, the SDK follows `nextCursor` page by page and returns one aggregated list with `nextCursor: undefined`. `listPrompts()`, `listResources()`, and `listResourceTemplates()` aggregate the same way.
That `listTools()` already walked every page: when a server splits its list, the SDK follows `nextCursor` page by page and returns one aggregated list with no `nextCursor`. `listPrompts()`, `listResources()`, and `listResourceTemplates()` aggregate the same way.

Pass a `cursor` — a page's `nextCursor` your application held on to — and `listTools` returns exactly that page, raw.

Expand Down
5 changes: 4 additions & 1 deletion docs/migration/upgrade-to-v2.md
Original file line number Diff line number Diff line change
Expand Up @@ -1587,7 +1587,7 @@ rewrite required unless noted.
instead of sending the request. Set `enforceStrictCapabilities: true` in `ClientOptions`
to restore the v1 throw.
- Called **without a `cursor`**, the same methods now **auto-aggregate every page** and
return `nextCursor: undefined`. Passing `{ cursor }` still fetches one page. Manual
return an aggregate with no `nextCursor`. Passing `{ cursor }` still fetches one page. Manual
pagination loops keep working (the first iteration returns everything); replace them
with the bare no-arg call. The walk is capped at `ClientOptions.listMaxPages` (default
64); overrun throws `SdkError(ListPaginationExceeded)`. There is no way to fetch only
Expand All @@ -1605,6 +1605,9 @@ rewrite required unless noted.
trip. Per-call override: `{ cacheMode: 'refresh' | 'bypass' }`. New `ClientOptions`:
`cachePartition`, `defaultCacheTtlMs`. `ResponseCacheStore` gained `delete(key)`;
`InMemoryResponseCacheStore` is now bounded (`{ maxEntries }`, default 512).
`CacheEntry.value` (and the `set()` entry value) is the JSON-serialized result
document (`string`, not `unknown`): persist and return it verbatim, `JSON.parse`
it to inspect.

#### Server (Streamable HTTP transport)

Expand Down
5 changes: 3 additions & 2 deletions examples/caching/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,9 @@ class MyStore implements ResponseCacheStore {
async get(key: CacheKey): Promise<CacheEntry | undefined> {
/* read {value, stamp, expiresAt, scope} from your backend */
}
async set(key: CacheKey, entry: { value: unknown; expiresAt?: number; scope?: CacheScope }): Promise<number> {
/* write entry under key; return a monotonically-increasing stamp */
async set(key: CacheKey, entry: { value: string; expiresAt?: number; scope?: CacheScope }): Promise<number> {
/* write entry under key (value is the JSON-serialized result document —
persist the string verbatim); return a monotonically-increasing stamp */
}
async delete(key: CacheKey): Promise<void> {
/* drop the single entry under key (no-op if absent) */
Expand Down
36 changes: 18 additions & 18 deletions packages/client/src/client/client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1475,7 +1475,7 @@ export class Client extends Protocol<ClientContext> {
* Lists available prompts.
*
* Called without a `cursor` (the common case), this walks every page and
* returns the complete aggregated list with `nextCursor: undefined`; the
* returns the complete aggregated list with no `nextCursor`; the
* aggregate is also written to the {@linkcode ResponseCacheStore}. Pass an
* explicit `{ cursor }` to fetch a single page and walk pagination
* yourself — the per-page path returns the server's raw page (with
Expand Down Expand Up @@ -1515,7 +1515,7 @@ export class Client extends Protocol<ClientContext> {
* Lists available resources.
*
* Called without a `cursor` (the common case), this walks every page and
* returns the complete aggregated list with `nextCursor: undefined`; the
* returns the complete aggregated list with no `nextCursor`; the
* aggregate is also written to the {@linkcode ResponseCacheStore}. Pass an
* explicit `{ cursor }` to fetch a single page and walk pagination
* yourself — the per-page path returns the server's raw page (with
Expand Down Expand Up @@ -1557,7 +1557,7 @@ export class Client extends Protocol<ClientContext> {
* Lists available resource URI templates for dynamic resources.
*
* Called without a `cursor`, this walks every page and returns the
* complete aggregated list with `nextCursor: undefined`; the aggregate is
* complete aggregated list with no `nextCursor`; the aggregate is
* also written to the {@linkcode ResponseCacheStore}. Pass an explicit
* `{ cursor }` to fetch a single page — see
* {@linkcode listResources | listResources()} for the per-page contract.
Expand Down Expand Up @@ -1642,7 +1642,10 @@ export class Client extends Protocol<ClientContext> {
cursor = page.nextCursor;
pages++;
}
acc.nextCursor = undefined;
// delete, not `= undefined`: the cache's JSON codec drops
// explicit-undefined properties, so only absence keeps
// `'nextCursor' in result` identical between wire and cache hit.
delete acc.nextCursor;
finalize?.(acc);
if (bypass) return acc;
// The aggregate is ALWAYS written: even when the resolved TTL is ≤0
Expand Down Expand Up @@ -1681,21 +1684,22 @@ export class Client extends Protocol<ClientContext> {

/**
* The cache-serving front of every cacheable verb (mcp.d's `cachedFetch`
* read half): under `cacheMode: 'use'` (the default), a held entry whose
* `expiresAt` is in the future is returned as a `structuredClone` and the
* round trip is skipped. `'refresh'` and `'bypass'` always fetch (the
* caller decides whether to write). A custom store whose `get()` rejects
* is routed to `onerror` and treated as a miss — cache bookkeeping never
* blocks a request from reaching the wire.
* read half): under `cacheMode: 'use'` (the default), a fresh held entry
* is served and the round trip is skipped. `'refresh'` and `'bypass'`
* always fetch (the caller decides whether to write). Freshness and
* decoding live in {@linkcode ClientResponseCache.read}; every hit is
* freshly parsed, so the caller owns it outright. A custom store
* whose `get()` rejects is routed to `onerror` and treated as a miss —
* cache bookkeeping never blocks a request from reaching the wire.
*/
private async _serveFromCache<R>(
method: string,
params: string | undefined,
options: CacheableRequestOptions | undefined
): Promise<R | undefined> {
if (options?.cacheMode === 'bypass' || options?.cacheMode === 'refresh') return undefined;
const entry = await this._cache.read(method, params).catch(error => void this._reportStoreError(error));
if (entry?.expiresAt !== undefined && entry.expiresAt > this._cache.now()) {
const hit = await this._cache.read(method, params).catch(error => void this._reportStoreError(error));
if (hit !== undefined) {
// A pre-aborted caller signal must reject the same way it would on
// the wire path (`Protocol.request()` wraps an already-aborted
// signal as `SdkError(RequestTimeout, reason)`); without this guard
Expand All @@ -1705,11 +1709,7 @@ export class Client extends Protocol<ClientContext> {
const reason = options.signal.reason;
throw reason instanceof SdkError ? reason : new SdkError(SdkErrorCode.RequestTimeout, String(reason));
}
// Clone on the way out so a caller mutating the returned aggregate
// (e.g. `result.tools.sort(...)`) cannot reach the cache or the
// stamp-memoized indices derived from it — the same invariant
// `_cache.write` upholds on the way in.
return structuredClone(entry.value) as R;
return hit.value as R;
}
return undefined;
}
Expand Down Expand Up @@ -2369,7 +2369,7 @@ export class Client extends Protocol<ClientContext> {
* Lists available tools.
*
* Called without a `cursor` (the common case), this walks every page and
* returns the complete aggregated list with `nextCursor: undefined`; the
* returns the complete aggregated list with no `nextCursor`; the
* aggregate is also written to the {@linkcode ResponseCacheStore} (the
* source for {@linkcode callTool | callTool()}'s output-schema validation
* and SEP-2243 `Mcp-Param-*` header mirroring). Pass an explicit
Expand Down
Loading
Loading