feat: in-process embedded permissions and per-schema-version caching#3166
Open
josephschorr wants to merge 4 commits into
Open
feat: in-process embedded permissions and per-schema-version caching#3166josephschorr wants to merge 4 commits into
josephschorr wants to merge 4 commits into
Conversation
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
e01a253 to
96c22be
Compare
The datalayer's stored-schema cache shares a single instance per schema version (keyed by schema hash). Bundle lazily-built, schema-derived caches (e.g. compiled caveats, type systems) into that instance via a new CachedSchema type, so they share the schema's lifetime and are discarded, together, when the schema changes. Register a cache kind once with RegisterDerivedCache and retrieve the per-schema instance with GetDerivedCache. The stored-schema cache now holds *CachedSchema instead of *datastore.ReadOnlyStoredSchema; pkg/datastore is unchanged.
NewDatastore's bootstrap always wrote schema in legacy per-definition form regardless of the configured schema mode, so a server reading from the unified schema would find none. Add a BootstrapSchemaMode option so the bootstrap datalayer writes schema in the requested mode. The zero value preserves the prior (legacy) behavior.
ComputeCheck constructs a fresh CaveatRunner per call, so its per-runner deserialized-caveat cache never persists across checks. Deserializing a caveat rebuilds its CEL environment, which is expensive and, with a rich caveat type set, dominated check cost. Cache compiled caveats on the schema-derived cache (CachedSchema), keyed by caveat name, so the CEL environment is built once per schema version rather than once per check. Falls back to per-runner caching when the reader is not backed by a unified stored schema.
Add pkg/embedded, a focused library for running permission checks in-process against a datastore via the dispatch engine, without standing up a gRPC server. Caveat context is passed as native Go values. Includes a README and tests covering both legacy and unified schema modes.
96c22be to
7fc3209
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the ability to embed SpiceDB's permission engine in-process (no gRPC server) and makes per-schema-version caching of derived artifacts possible, eliminating repeated work on the check hot path.
Changes
datalayer: cache schema-derived artifacts on the stored-schema cache. The stored-schema cache already shares one instance per schema version (keyed by schema hash). Bundle lazily-built, schema-derived caches (compiled caveats today; type systems / reachability later) into that instance via a newCachedSchematype, so they share the schema's lifetime and are discarded, together, when the schema changes. Register a kind withRegisterDerivedCache; retrieve the per-schema instance withGetDerivedCache. The stored-schema cache now holds*datalayer.CachedSchema;pkg/datastoreis unchanged.datastore: honor schema mode when writing bootstrap data.NewDatastore's bootstrap always wrote schema in legacy per-definition form regardless of the configured schema mode, so a server reading from the unified schema would find none. Adds aBootstrapSchemaModeoption; the zero value preserves prior behavior.caveats: cache compiled caveats per schema version.ComputeCheckconstructs a freshCaveatRunnerper call, so its per-runner deserialized-caveat cache never persists across checks. Deserializing a caveat rebuilds its CEL environment, which is expensive and — with a rich caveat type set — dominated check cost. This caches compiled caveats on the schema-derived cache, keyed by caveat name, so the CEL environment is built once per schema version rather than once per check. Falls back to per-runner caching when the reader is not backed by a unified stored schema.embedded: add in-process permissions library. Addspkg/embedded, a focused library for running permission checks in-process against a datastore via the dispatch engine — no gRPC, caveat context passed as native Go values. Includes a README and tests.Testing
mage lint:allpasses; unit suite passes.pkg/datalayer(derived-cache registry: lazy build, per-schema isolation, duplicate/unregistered handling),internal/caveats(compiled-caveat cache),pkg/embedded(checks across both legacy and unified schema modes, incl. caveated/conditional results).