Add offline purge of orphaned received shares to the jsoncs3 share manager#656
Add offline purge of orphaned received shares to the jsoncs3 share manager#656kobergj wants to merge 1 commit into
Conversation
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
There was a problem hiding this comment.
Pull request overview
This PR adds an offline, batched purge path for orphaned per-user received-share state in the jsoncs3 share manager, avoiding costly inline cleanup during ListReceivedShares when many orphan entries exist.
Changes:
- Add
Manager.PurgeOrphanedReceivedShares(ctx, spaceID, dryRun)to scan users’ received-share state and remove entries confirmed missing from the provider cache (after a successfulListSpaceprime). - Add
receivedsharecache.Cache.RemoveSpaceShares(...)to remove multiple received-share entries for a given user+space in a single persisted write (with the existing retry/backoff behavior). - Add unit tests for
RemoveSpaceSharesand a changelog entry describing the new functionality.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| pkg/share/manager/jsoncs3/receivedsharecache/receivedsharecache.go | Adds RemoveSpaceShares batched deletion method with retry/sync loop. |
| pkg/share/manager/jsoncs3/receivedsharecache/receivedsharecache_test.go | Adds test coverage for batched removal behavior and persistence. |
| pkg/share/manager/jsoncs3/jsoncs3.go | Adds PurgeOrphanedReceivedShares manager method to perform offline orphan cleanup. |
| changelog/unreleased/purge-orphaned-received-shares.md | Documents the enhancement and links to the PR. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
4fede7b to
05041b5
Compare
|
Thanks for the review. Addressed the feedback in the (amended) commit:
The span-attributes-after- |
| ctx, span = appctx.GetTracerProvider(ctx).Tracer(tracerName).Start(ctx, "RemoveSpaceShares") | ||
| defer span.End() | ||
| span.SetAttributes(attribute.String("cs3.userid", userID), attribute.String("cs3.spaceid", spaceID)) |
There was a problem hiding this comment.
I think this should be moved to the beginning of the function, before the if len(shareIDs) == 0 check, otherwise, calls without "shareIDs" won't be registered.
While it won't add very useful data (the function will return right away), knowing that the method has been called might still be useful.
| c.initializeIfNeeded(userID, spaceID) | ||
|
|
||
| rss, _ := c.ReceivedSpaces.Load(userID) | ||
| receivedSpace := rss.Spaces[spaceID] |
There was a problem hiding this comment.
Do we have guarantees that the spaceID always exists? otherwise, the receivedSpace might be nil, and it could crash the app.
| log.Error().Err(err).Msg("persisting removed received shares failed") | ||
| return err | ||
| } | ||
| timer := time.NewTimer(expBackoff(attempt)) |
There was a problem hiding this comment.
It might be easier to use time.After(...).
Alternatively, you might want to create the timer once (before the attempt loop) and reset the duration here. Docs seems a bit confusing though, you might want to double-check for potential problems.
| SpacesScanned int | ||
| OrphansFound int | ||
| OrphansRemoved int | ||
| Errors int |
There was a problem hiding this comment.
Any additional information we can extract from the errors? Right now, I don't think it matters that there have been 3 errors or 300, what matters is if there have been any error or none at all. A boolean type provides the same value.
Maybe including the first 25 or 50 error messages (if there are more) might be interesting. We don't want to have all the error messages, but some of them might be useful to have an idea of what's going on.
Add Manager.PurgeOrphanedReceivedShares to the jsoncs3 share manager to drain received-share entries whose share no longer exists in the provider cache (orphans left behind when a space is deleted without cleaning up the per-user received.json). Removal is batched per user and space via the new receivedsharecache.Cache.RemoveSpaceShares, supports a single-space scope and dry-run, and only removes entries confirmed absent after a successful space listing so live shares are never dropped on a transient failure. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
05041b5 to
6e23241
Compare
|
Thanks @jvillafanez — addressed most of it in the amended commit:
On Added manager-level tests for |
|
According to the docs, the timer-GC leak should be fixed with go 1.23 (https://pkg.go.dev/time#After). Anyway, the docs aren't very clear with some aspects of the behavior, and I don't think this is a time-critical path. |
Summary
The jsoncs3 share manager can accumulate orphaned received-share entries: a shareID stored in a user's
received.jsonwhose share no longer exists in the provider cache. These are left behind when a space is deleted but the per-user received-state blocks are not cleaned up (e.g. after a missedSpaceDeletedevent).They are removed inline on
ListReceivedShares, but that happens on the request hot path — with many orphans and/or HA replicas this can monopolize the call and race a client deadline (see OCISDEV-859 / the desktop-client disconnect symptom in BYCS school-2638, where ~47 orphans on one deleted space kept affected users in a permanent reconnect loop).This adds an offline, batched way to drain them.
Changes
Manager.PurgeOrphanedReceivedShares(ctx, spaceID, dryRun)— enumerates users (metadata.Storage.ReadDir("/users")), and for each received-share entry checks whether the share still exists in the provider cache. Confirmed orphans are removed in a single batched persist per user+space. Supports scoping to one space and a dry-run mode.receivedsharecache.Cache.RemoveSpaceShares— removes a set of shareIDs for one user's space in one persist (batched counterpart ofRemove), reusing the existing retry + full-jitter backoff +ctxcancellation loop.Safety
An entry is only removed when the share is confirmed absent from the provider cache after a successful
ListSpaceprime — exactly the predicate the inline cleanup already uses. If a space's share list cannot be read, the whole space is skipped (counted as an error), so live shares are never removed on a transient failure.Tests
RemoveSpaceShares(batched removal, selective removal, persistence, empty-list no-op).pkg/share/manager/jsoncs3/...suite passes.Relates to OCISDEV-859. Paired oCIS CLI command (
ocis shares purge-orphaned-received-shares) to follow.