Skip to content

Conversation

@moosebay
Copy link
Contributor

@moosebay moosebay commented Feb 9, 2026

Covers architecture, file changes, testing strategy, and phasing for
adding {{#gcp:...}}, {{#aws:...}}, {{#azure:...}} secret references
to the expression/interpolation engine.

Closes #23

Implements infrastructure for resolving cloud secrets in {{}} template
expressions (Issue #23). Adds GCP Secret Manager support with #gcp:
prefix and optional #fragment JSON field extraction.

New packages:
- secretresolver: SecretResolver interface, ParseSecretRef, ExtractFragment
- secretresolver/gcpsecret: GCP Secret Manager implementation with caching

Expression package changes:
- Add #gcp:, #aws:, #azure: prefix recognition to resolveVar() dispatch
- Thread context.Context through interpolation for network calls
- Add SecretReferenceError type with provider/ref/fragment fields
- Add WithSecretResolver() builder on UnifiedEnv
- Mask secret values as "***" in variable tracking

Wiring:
- Add SecretResolver field to FlowNodeRequest
- Pass resolver through FlowLocalRunner -> FlowNodeRequest -> nodes
- Wire into request node (sync + async) and AI node
- Use variadic parameter in PrepareHTTPRequestWithTracking for
  backward-compatible secret resolver injection

Tests:
- Unit tests for fragment extraction, reference parsing, multi-resolver
- Expression tests with mock SecretResolver (GCP, AWS, Azure prefixes)
- Integration test skeleton behind gcp_integration build tag

Note: go.mod adds cloud.google.com/go/secretmanager dependency.
@ElecTwix
Copy link
Contributor

ElecTwix commented Feb 9, 2026

the SecretResolver interface, MultiResolver dispatcher, fragment extraction, and the GCP client are all well-designed. The test coverage is solid and the plan doc is thorough. A few things I'd like us to address before merging:

1. Secret values leak to DB and UI (critical)

In resolveSecretVar() (interpolate.go), the real secret value is stored in readVars:

readVars[varRef] = value          // ← plaintext secret
if e.tracker != nil {
    e.tracker.TrackRead(varRef, "***")
}

readVars flows through a chain that ultimately persists the plaintext secret to SQLite and streams it to the frontend:

ReadVarsnrequest copies into VariableTrackerInputDatanode_execution table → NodeExecutionSync / NodeExecutionCollection RPCs → UI

The tracker masking with "***" is also ineffective in the request path because PrepareHTTPRequestWithTracking (request.go:74) creates UnifiedEnv without a tracker — so the masking branch never executes there.

The fix is straightforward: readVars is metadata for the variable inspector, not the functional data path. The real value is already returned via the function's return value and written into the interpolated output string. Change to:

readVars[varRef] = "***"

I'd also recommend a defense-in-depth check in nrequest.go where ReadVars are copied into the tracker.

2. Consider using #secret:name instead of #gcp:, #aws:, #azure: prefixes

We already have a unified, workspace-scoped credential system (scredential + credvault) that handles encrypted storage, decrypt-on-demand, and keeps secrets out of the tracking pipeline. The PR builds a parallel system that bypasses all of that.

Provider-specific prefixes in templates have a few downsides:

  • Vendor lock-in — templates are coupled to the cloud provider. Migrating from GCP to AWS means rewriting every reference.
  • Prefix proliferation — every new provider (HashiCorp Vault, 1Password, Doppler, etc.) requires new prefixes, new constants, and new cases in IsSecretReference() / ParseSecretReference().
  • No workspace scoping — existing credentials are scoped to workspaces. Cloud secret refs in this PR have no access control.

Suggested alternative: Extend scredential with a CREDENTIAL_KIND_CLOUD_SECRET that stores the provider + path + fragment as a reference. Then use a single {{ #secret:my-credential-name }} prefix in templates. The expression engine stays simple (one prefix), templates are provider-agnostic, and we inherit the existing workspace permissions and credential management UI.

The secretresolver package (interface, MultiResolver, GCP client, caching) is solid — it could serve as the internal implementation behind scredential rather than being exposed directly in the expression engine.

3. LogPreparedRequest doesn't redact secrets (moderate)

request.go:492-506 logs the fully-interpolated HTTP request. Only the Authorization header is redacted. If a secret is interpolated into the URL, query params, other headers, or the body, it appears in plaintext in server logs.

4. Unbounded GCP cache (minor)

The sync.Map cache in gcpsecret has TTL-based expiry but no max-size bound. Under heavy usage with many distinct secrets, this could grow indefinitely. Consider an LRU or a max-entry limit.


The main ask is to route this through our existing credential system rather than creating a parallel path, and to fix the readVars leakage. Happy to discuss the #secret: approach further if it'd help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Support loading secrets from cloud providers (GCP, AWS, etc.)

2 participants