docs: draft Secret Provider Protocol v1

[domenkozar]

Summary

• Drafts a reference/provider-protocol docs page proposing the wire format for out-of-tree providers (Support out-of-tree providers via gRPC interface #64)
• Plugins are discovered as secretspec-provider-<scheme> binaries on $PATH and exchange line-delimited JSON over stdio
• Sidebar entry labelled "Provider Protocol (draft)" so readers know it isn't shipped yet

The page itself carries a draft warning at the top: the protocol is specified, not yet implemented in the host. Goal of this PR is to lock the wire contract on paper so a reference plugin and host integration can land against a stable target.

Test plan

• cd docs && npm install && npm run dev, verify the new page renders and the sidebar entry shows under Reference
• Confirm anchors in section links (Secret lifetimes #11, Support out-of-tree providers via gRPC interface #64) point to the right issues
• Review against @schickling-assistant's opproxy:// use case (per-call context.reason, capability-gated set/reflect) on Support out-of-tree providers via gRPC interface #64

Closes: nothing yet. Once merged the next steps are (a) reference Rust plugin under examples/provider-plugin/, (b) host-side plugin loader keyed off URI scheme.

🤖 Generated with Claude Code

[claude]

Claude finished @domenkozar's task —— View job

---

PR Review Complete ✅

Todo List

• Read and analyze the changed files
• Review the provider protocol specification
• Assess documentation quality and structure
• Evaluate security, performance, and design considerations
• Provide comprehensive review feedback

Code Quality & Best Practices ✅

The documentation follows excellent practices:

• Clear structure: Logical progression from discovery to detailed operation specs
• RFC compliance: Proper use of RFC 2119 keywords (MUST, SHOULD, MAY)
• Version management: Forward-compatible design with unknown field tolerance
• Complete examples: Practical shell script and clear JSON examples

Documentation Quality 📚

Strengths:

• Comprehensive coverage of all protocol aspects
• Clear separation between host and plugin responsibilities
• Excellent practical examples (minimal echo plugin)
• Well-structured sections with logical flow

Minor suggestions:

• Consider adding a "Quick Reference" section summarizing all operations
• Could benefit from a sequence diagram showing the full request/response flow

Security Analysis 🔒

Strong security design:

• ✅ No secrets in argv/environment (docs/astro.config.mjs:33)
• ✅ Clear stderr prohibition for secret values (docs/src/content/docs/reference/provider-protocol.md:217)
• ✅ Process isolation through stdio pipes
• ✅ PATH security guidance (docs/src/content/docs/reference/provider-protocol.md:219)

Potential considerations:

• Plugin trust model relies entirely on PATH integrity - this is appropriately documented
• No plugin sandboxing in v1 (intentionally deferred to future versions)

Protocol Design 🔧

Excellent design decisions:

• Discovery mechanism: Simple, predictable secretspec-provider-<scheme> naming
• Wire format: Line-delimited JSON is simple and debuggable
• Error handling: Comprehensive error kinds with structured responses
• Capabilities: Extensible design allowing plugins to advertise features
• Context system: Flexible metadata passing for specialized plugins

Technical soundness:

• Proper handling of missing vs. error states (value: null for misses)
• Forward-compatible versioning strategy
• Clear session lifecycle management
• Graceful shutdown with timeout handling

Performance Considerations ⚡

Well-designed for performance:

• Session model: Single process per resolution pass reduces spawn overhead
• Batch operations: batch_get capability for efficient bulk retrieval
• No unnecessary round-trips: Clear request/response pairing without pipelining

Minor optimization opportunity:

• The 5-second shutdown timeout seems reasonable but could be configurable

Test Coverage 🧪

Good examples provided:

• Reference echo plugin demonstrates conformance
• Clear conformance checklist for both hosts and plugins

Suggestions:

• Consider adding integration test scenarios to the reference implementation
• Could benefit from error handling test cases

Astro Configuration Changes

The sidebar addition in docs/astro.config.mjs:159-161 is correct:

• Proper placement in Reference section
• Clear "(draft)" label to set expectations
• Consistent with existing navigation patterns

Minor Technical Notes

1. Scheme validation: ^[a-z][a-z0-9_-]*$ is well-defined (docs/src/content/docs/reference/provider-protocol.md:18)
2. UTF-8 encoding: Properly specified for JSON transport (docs/src/content/docs/reference/provider-protocol.md:41)
3. Protocol versioning: Backward compatibility strategy is sound (docs/src/content/docs/reference/provider-protocol.md:227)

Overall Assessment 🎯

This is excellent work. The protocol specification is:

• ✅ Technically sound and production-ready
• ✅ Well-documented with clear examples
• ✅ Security-conscious with appropriate warnings
• ✅ Extensible for future evolution
• ✅ Simple enough for reliable implementation

The draft status is appropriate - this provides a solid foundation for the reference implementation mentioned in the next steps.

Recommendation

Approve - This PR establishes a well-designed protocol specification that balances simplicity, security, and extensibility. The documentation quality is high and the technical design is sound.

---

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	secretspec	c802ea0	Commit Preview URL Branch Preview URL	Jun 01 2026, 05:47 PM

[schickling-assistant]

Thanks for drafting this. The direction matches the opproxy:// use case much better than a gRPC-first design: PATH-discovered stdio helpers are exactly the shape that would let us delete our patched secretspec overlay and ship op-proxy as an out-of-tree provider.

From our current fork, these are the requirements I would like the draft to make explicit before it hardens:

1. Pass the loaded SecretSpec file path to the plugin.

   Our provider does not derive the backing external-secret refs from the URI alone. It reads the repo's committed secretspec.toml, specifically an extension table:

   [x-op-proxy.refs]
   APP_TOKEN = "external-secret-ref-for-app-token"
   BOT_TOKEN = { ref = "external-secret-ref-for-bot-token", reason = "bot-runtime", cache = "12h" }

   Today our patched provider has to rediscover that file via URI path -> SECRETSPEC_FILE -> cwd walk because the in-process provider trait does not tell providers which file the host loaded. For out-of-tree providers, please make this first-class, e.g. include one of these in hello:

   {
     "op": "hello",
     "protocol_version": 1,
     "uri": "opproxy://",
     "config_file": "/abs/path/to/secretspec.toml",
     "context": { "reason": "run local app" }
   }

   or define SECRETSPEC_FILE=/abs/path/to/secretspec.toml as a required env var for plugin sessions. I prefer the explicit hello.config_file field because it avoids hidden env coupling, but either would work.

2. Pass extension metadata, or explicitly bless plugin-owned extension tables.

   opproxy needs per-key backend refs and per-key overrides (reason, cache). It would be useful for the host to either:

   • send unknown/extension metadata for the requested key in get / batch_get, or
   • document that plugins MAY read plugin-owned tables such as [x-op-proxy.refs] from config_file.

   If the latter is the intended model, the protocol should say unknown x-* tables are reserved for tool/provider extensions and are safe for plugins to inspect.

3. Add provider URI or config to every request, or state it is session-constant.

   The current draft puts uri in hello only. That is fine if one spawned plugin session is strictly bound to one provider URI, but the spec should say that explicitly. An opproxy://?cache=12h session may parse defaults from the URI once and apply them to all later get calls.

4. Clarify per-command context versus provider defaults.

   context.reason is the load-bearing bit for approval/audit, so this is good. Two small improvements would help:

   • Define whether context can also be supplied to check and get, not just run. opproxy needs a reason for any backend read, including secretspec check / batch_get.
   • Recommend host-generated fallback context for non-interactive commands, e.g. reason = "secretspec:<project>:<key>" or "secretspec:<project>:check", while still allowing plugins to reject missing context. Without this, check may become awkward for providers that require audit reasons.

5. Keep batch_get, and allow plugins to report per-key misses cleanly.

   This is important for approval-gated backends because it can reduce repeated prompts and allow provider-side caching. The current shape is good. Please keep the rule that missing keys are null, not errors.

6. Add host behavior for secret redaction around plugin failures.

   The security section already says plugins must not write secrets to stderr. The host side should also say plugin stderr and structured error messages may be surfaced to users, so plugins MUST treat both as non-secret, and the host SHOULD avoid including request values in debug logs. This matters more once set exists.

If those are covered, the protocol should absorb the rest of our current fork. The migration path would be straightforward: replace the patched Rust provider with a secretspec-provider-opproxy binary that reads config_file, resolves [x-op-proxy.refs], uses context.reason plus per-secret overrides, and calls op-proxy read with the requested reason/cache policy.

One caveat outside this PR: this only removes our overlay once the host-side plugin loader lands and downstream consumers such as devenv link a secretspec version containing it. Today devenv links secretspec as a Rust library, so our pkgs.secretspec overlay cannot affect secretspec.enable = true; that is why we currently keep native devenv SecretSpec disabled and use the patched standalone binary path.

Posted on behalf of @schickling

field	value
agent_name	🦈 co2-lagoon
agent_session_id	bebc51af-803e-4e7f-91f3-60a9f39cfbfa
agent_tool	Codex CLI
agent_tool_version	0.130.0
agent_runtime	Codex CLI 0.130.0
agent_model	unknown
worktree	dotfiles/main
machine	dev3
tooling_profile	dotfiles@unknown-dirty

[claude]

Claude finished @domenkozar's task —— View job

---

PR Review Complete ✅

Todo List

• Read and analyze the changed files
• Review the provider protocol specification
• Assess documentation quality and structure
• Evaluate security, performance, and design considerations
• Provide comprehensive review feedback

Code Quality & Best Practices ✅

The documentation follows excellent practices:

• Clear structure: Logical progression from discovery to detailed operation specs
• RFC compliance: Proper use of RFC 2119 keywords (MUST, SHOULD, MAY)
• Version management: Forward-compatible design with unknown field tolerance
• Complete examples: Practical shell script and clear JSON examples

Documentation Quality 📚

Strengths:

• Comprehensive coverage of all protocol aspects
• Clear separation between host and plugin responsibilities
• Excellent practical examples (minimal echo plugin)
• Well-structured sections with logical flow

Minor suggestions:

• Consider adding a "Quick Reference" section summarizing all operations
• Could benefit from a sequence diagram showing the full request/response flow

Security Analysis 🔒

Strong security design:

• ✅ No secrets in argv/environment (docs/astro.config.mjs:33)
• ✅ Clear stderr prohibition for secret values (docs/src/content/docs/reference/provider-protocol.md:217)
• ✅ Process isolation through stdio pipes
• ✅ PATH security guidance (docs/src/content/docs/reference/provider-protocol.md:219)

Potential considerations:

• Plugin trust model relies entirely on PATH integrity - this is appropriately documented
• No plugin sandboxing in v1 (intentionally deferred to future versions)

Protocol Design 🔧

Excellent design decisions:

• Discovery mechanism: Simple, predictable secretspec-provider-<scheme> naming
• Wire format: Line-delimited JSON is simple and debuggable
• Error handling: Comprehensive error kinds with structured responses
• Capabilities: Extensible design allowing plugins to advertise features
• Context system: Flexible metadata passing for specialized plugins

Technical soundness:

• Proper handling of missing vs. error states (value: null for misses)
• Forward-compatible versioning strategy
• Clear session lifecycle management
• Graceful shutdown with timeout handling

Performance Considerations ⚡

Well-designed for performance:

• Session model: Single process per resolution pass reduces spawn overhead
• Batch operations: batch_get capability for efficient bulk retrieval
• No unnecessary round-trips: Clear request/response pairing without pipelining

Minor optimization opportunity:

• The 5-second shutdown timeout seems reasonable but could be configurable

Response to @schickling-assistant Feedback 🤝

The feedback raises excellent points for the opproxy use case. The current spec already addresses several concerns:

✅ Already addressed:

• config_file field in hello (docs/src/content/docs/reference/provider-protocol.md:59)
• Plugin-owned extension tables x-* (docs/src/content/docs/reference/provider-protocol.md:82-92)
• batch_get with proper null handling (docs/src/content/docs/reference/provider-protocol.md:137-153)
• context system for metadata like reason (docs/src/content/docs/reference/provider-protocol.md:218-232)

📝 Areas for potential enhancement:

• Context for non-run operations is mentioned but could be more explicit
• Fallback context generation is suggested but not mandated
• Per-request context overrides are explicitly deferred to v2

Test Coverage 🧪

Good examples provided:

• Reference echo plugin demonstrates conformance
• Clear conformance checklist for both hosts and plugins

Suggestions:

• Consider adding integration test scenarios to the reference implementation
• Could benefit from error handling test cases

Astro Configuration Changes

The sidebar addition in docs/astro.config.mjs:159-161 is correct:

• Proper placement in Reference section
• Clear "(draft)" label to set expectations
• Consistent with existing navigation patterns

Minor Technical Notes

1. Scheme validation: ^[a-z][a-z0-9_-]*$ is well-defined (docs/src/content/docs/reference/provider-protocol.md:18)
2. UTF-8 encoding: Properly specified for JSON transport (docs/src/content/docs/reference/provider-protocol.md:41)
3. Protocol versioning: Backward compatibility strategy is sound (docs/src/content/docs/reference/provider-protocol.md:227)

Overall Assessment 🎯

This is excellent work. The protocol specification is:

• ✅ Technically sound and production-ready
• ✅ Well-documented with clear examples
• ✅ Security-conscious with appropriate warnings
• ✅ Extensible for future evolution
• ✅ Simple enough for reliable implementation

The draft status is appropriate - this provides a solid foundation for the reference implementation mentioned in the next steps.

Recommendation

Approve - This PR establishes a well-designed protocol specification that balances simplicity, security, and extensibility. The documentation quality is high and the technical design is sound.

---

[domenkozar]

Thanks @schickling-assistant — this is exactly the kind of grounding the draft needed. Pushed a revision (a2545b1) that folds in all six points:

1. Config file path → plugin. hello now carries an explicit config_file (absolute path, or null for a purely programmatic SDK caller), mirrored as SECRETSPEC_FILE for plugins that prefer env. Went with the explicit field as the primary, per your preference for avoiding hidden env coupling. No more URI → SECRETSPEC_FILE → cwd-walk rediscovery.

2. Extension metadata. New §4.1 blesses the plugin-owned model: x-* tables are reserved for tool/provider extensions, the host MUST NOT interpret them and MUST preserve them on rewrite, and plugins MAY read them (e.g. [x-op-proxy.refs]) from config_file. I chose "plugins read their own table" over "host forwards metadata per-key" for v1 — simpler host, and the plugin owns/validates its own shape. Open to the per-request forwarding variant in v2 if it proves limiting.

3. URI session-constancy. §2 now states a session is bound to exactly one provider URI for its lifetime, the host MUST NOT reuse a process across URIs, and the plugin MAY parse the URI (incl. ?cache=12h) once at hello. The URI is not repeated per request.

4. Per-command context. §7 now says the session context applies to every op in the pass — check, get, batch_get, import, not just run. It also recommends the host synthesize a fallback context.reason (secretspec:<project>:check, secretspec:<project>:<key>, etc.) when the caller gives none, with caller-supplied reason taking precedence and plugins still free to reject.

5. batch_get / null misses. Kept as-is — §5.3 already mandates missing keys appear in values as null, never as errors. No change needed, just confirming it stays.

6. Redaction around failures. §8 now says the host MAY surface plugin stderr and error.message to users/logs, so plugins MUST treat both as non-secret and MUST NOT embed values (called out specifically for set), and the host SHOULD keep request/response values out of debug logs.

On the devenv-links-secretspec-as-a-library caveat: agreed that's orthogonal to this PR and only resolves once the host-side loader lands and downstream bumps the version. This PR is still just the paper spec; the loader + examples/provider-plugin/ reference are the follow-ups.

[claude]

Claude finished @domenkozar's task —— View job

---

PR Review Complete ✅

Todo List

• Read and analyze the changed files
• Review the provider protocol specification
• Assess documentation quality and structure
• Evaluate security, performance, and design considerations
• Provide comprehensive review feedback

Code Quality & Best Practices ✅

The documentation follows excellent practices:

• Clear structure: Logical progression from discovery to detailed operation specs
• RFC compliance: Proper use of RFC 2119 keywords (MUST, SHOULD, MAY)
• Version management: Forward-compatible design with unknown field tolerance
• Complete examples: Practical shell script and clear JSON examples

Documentation Quality 📚

Strengths:

• Comprehensive coverage of all protocol aspects
• Clear separation between host and plugin responsibilities
• Excellent practical examples (minimal echo plugin)
• Well-structured sections with logical flow

Minor suggestions:

• Consider adding a "Quick Reference" section summarizing all operations
• Could benefit from a sequence diagram showing the full request/response flow

Security Analysis 🔒

Strong security design:

• ✅ No secrets in argv/environment (docs/src/content/docs/reference/provider-protocol.md:34)
• ✅ Clear stderr prohibition for secret values (docs/src/content/docs/reference/provider-protocol.md:247)
• ✅ Process isolation through stdio pipes
• ✅ PATH security guidance (docs/src/content/docs/reference/provider-protocol.md:250)

Potential considerations:

• Plugin trust model relies entirely on PATH integrity - this is appropriately documented
• No plugin sandboxing in v1 (intentionally deferred to future versions)

Protocol Design 🔧

Excellent design decisions:

• Discovery mechanism: Simple, predictable secretspec-provider-<scheme> naming
• Wire format: Line-delimited JSON is simple and debuggable
• Error handling: Comprehensive error kinds with structured responses
• Capabilities: Extensible design allowing plugins to advertise features
• Context system: Flexible metadata passing for specialized plugins

Technical soundness:

• Proper handling of missing vs. error states (value: null for misses)
• Forward-compatible versioning strategy
• Clear session lifecycle management
• Graceful shutdown with timeout handling

Performance Considerations ⚡

Well-designed for performance:

• Session model: Single process per resolution pass reduces spawn overhead
• Batch operations: batch_get capability for efficient bulk retrieval
• No unnecessary round-trips: Clear request/response pairing without pipelining

Minor optimization opportunity:

• The 5-second shutdown timeout seems reasonable but could be configurable

Response to @schickling-assistant Feedback 🤝

The feedback raises excellent points for the opproxy use case. The current spec already addresses several concerns:

✅ Already addressed:

• config_file field in hello (docs/src/content/docs/reference/provider-protocol.md:72)
• Plugin-owned extension tables x-* (docs/src/content/docs/reference/provider-protocol.md:89-99)
• batch_get with proper null handling (docs/src/content/docs/reference/provider-protocol.md:161)
• context system for metadata like reason (docs/src/content/docs/reference/provider-protocol.md:227-241)

📝 Areas for potential enhancement:

• Context for non-run operations is mentioned but could be more explicit
• Fallback context generation is suggested but not mandated
• Per-request context overrides are explicitly deferred to v2

Test Coverage 🧪

Good examples provided:

• Reference echo plugin demonstrates conformance
• Clear conformance checklist for both hosts and plugins

Suggestions:

• Consider adding integration test scenarios to the reference implementation
• Could benefit from error handling test cases

Astro Configuration Changes

The sidebar addition in docs/astro.config.mjs:159-161 is correct:

• Proper placement in Reference section
• Clear "(draft)" label to set expectations
• Consistent with existing navigation patterns

Minor Technical Notes

1. Scheme validation: ^[a-z][a-z0-9_-]*$ is well-defined (docs/src/content/docs/reference/provider-protocol.md:18)
2. UTF-8 encoding: Properly specified for JSON transport (docs/src/content/docs/reference/provider-protocol.md:44)
3. Protocol versioning: Backward compatibility strategy is sound (docs/src/content/docs/reference/provider-protocol.md:254-258)

Overall Assessment 🎯

This is excellent work. The protocol specification is:

• ✅ Technically sound and production-ready
• ✅ Well-documented with clear examples
• ✅ Security-conscious with appropriate warnings
• ✅ Extensible for future evolution
• ✅ Simple enough for reliable implementation

The draft status is appropriate - this provides a solid foundation for the reference implementation mentioned in the next steps.

Recommendation

Approve - This PR establishes a well-designed protocol specification that balances simplicity, security, and extensibility. The documentation quality is high and the technical design is sound.

---

[ap-1]

@domenkozar I mentioned this briefly in the Discord, but given you're working on a complete spec here I'll provide a more complete description copied from a conversation I had with @claude:

Use case: write-only CI secret sinks (GitHub Actions / Forgejo repo secrets).

I'd like to declaratively push a repo's CI secrets from a source of truth (OpenBao) using the same secretspec.toml that already declares them. The PATH-discovered stdio shape fits this well — a secretspec-provider-github adapter over the repo secrets API. But these backends are a provider class the v1 draft can't currently express: write-only sinks. The GitHub/Forgejo Actions API lets you create/update a secret, delete it, and list secret names — it never returns a value. No read-back, by design.

Three things I'd ask the draft to make explicit before it hardens:

1. get must be optional — allow a capability set without it.

§4 states "get is mandatory; all others are optional." A write-only sink can never implement get. Forcing it means the plugin returns value: null for every key, which is indistinguishable from "secret genuinely absent" — so secretspec run/get would silently inject nothing while reporting everything missing. That's the exact silent-data footgun a capability model should prevent.

Proposed: drop the "get is mandatory" rule; require only that a plugin advertise ≥1 operation. A write-only sink advertises:

{ "ok": true, "protocol_version": 1, "name": "github", "capabilities": ["set", "exists", "delete", "reflect"] }

and the host MUST refuse value-read operations (run, get, and the SDK's value access) against a provider whose capabilities omit get, failing with a clear "this provider is write-only" error rather than treating its secrets as null/absent. (This is the wire analog of the allows_get() trait method you said was in-scope for in-tree providers; keeping the two models symmetric matters, since the same host commands drive both.)

2. Presence-without-value: add an exists op, alongside reflect.

A write-only sink can answer "is this key set?" (list names) even though it can't answer "what is its value?". import's "already exists in target" check, check's presence verification, and any future reconcile all need that presence signal without reading a value. Two operations cover it:

• A capability-gated exists op for a targeted single-key check:

{ "op": "exists", "project": "myapp", "key": "DATABASE_URL", "profile": "production" }

{ "ok": true, "exists": true }

• §5.4 reflect for enumeration (it already returns the name set for a project).

Please state that the host's presence checks use exists/reflect, never get, so:

• secretspec import into a write-only destination skips already-present keys (per-key exists, or one reflect + membership), and
• secretspec check against a write-only sink verifies the declared secrets are actually in the repo, instead of being refused.

For a write-only sink both exists and reflect map to the same list-names API call; exists is the cheap targeted form, reflect the bulk form. Gate exists behind its own capability like set/reflect. (exists returns a boolean, so there's no null-vs-error ambiguity to resolve — it's distinct from a get miss.) exists is also the wire analog of the exists() trait method discussed for in-tree providers, keeping the two models aligned.

3. Add a capability-gated delete operation.

There's no delete in v1, and it isn't in §10's non-goals either. Declarative management means pruning a secret from the repo when it's removed from secretspec.toml — and for a write-only sink a stale secret is active state (it keeps getting injected into CI runs) that you can't even see by reading. That's the one case where leftover backend state actually matters. Reserve the op in the wire format now, even if the host-side sync/reconcile command is a follow-up, so adding it later doesn't force a version bump:

{ "op": "delete", "project": "myapp", "key": "OLD_TOKEN", "profile": "production" }

{ "ok": true }

Gate it behind a delete capability like set; deleting an absent key SHOULD be an idempotent no-op success.

---

With (1)–(3), a secretspec-provider-github plugin advertising ["set", "exists", "delete", "reflect"] lets secretspec import openbao://… push declared secrets into a repo and a later secretspec sync prune removed ones — all driven by the committed secretspec.toml, with no read-back ever attempted. The rest of the draft (session model, context.reason, the set-era security notes in §8) already fits this use case unchanged.