Skip to content

feat: persist media preview backfill failures#4197

Merged
marinofaggiana merged 7 commits into
masterfrom
preview-process
Jul 21, 2026
Merged

feat: persist media preview backfill failures#4197
marinofaggiana merged 7 commits into
masterfrom
preview-process

Conversation

@marinofaggiana

@marinofaggiana marinofaggiana commented Jul 21, 2026

Copy link
Copy Markdown
Member

Summary

This PR introduces a new background media preview backfill processor that automatically generates missing previews for media files during background processing.

Changes

  • Added NCMediaPreviewBackfillProcessor to generate missing media previews in the background.
  • Integrated the preview backfill into the background processing pipeline after media metadata placeholder hydration.
  • Limited concurrent preview downloads to avoid excessive resource usage.
  • Skipped previews that already exist locally.
  • Added a lightweight blacklist to avoid repeatedly retrying previews that previously failed during the same background processing cycle.
  • Automatically clears the blacklist when there are no remaining previews to process, allowing previously failed items to be retried in future processing cycles.
  • Added dedicated logging for preview backfill progress and completion.

Background processing flow

The background processing pipeline now performs the following steps:

  1. Auto Upload
  2. Weekly maintenance (when scheduled)
  3. Media metadata backfill
  4. Media metadata placeholder hydration
  5. Media preview backfill

This keeps media metadata synchronized first, then hydrates missing placeholders, and finally generates previews only for media that still require them.

🤖 AI (if applicable)

  • The content of this PR was partly or fully generated using AI

Add Realm-backed APIs to track and clear media preview backfill failures, and bump the database schema to version 413.

Signed-off-by: Marino Faggiana <marino.faggiana@nextcloud.com>
@marinofaggiana
marinofaggiana marked this pull request as draft July 21, 2026 06:37
@marinofaggiana marinofaggiana self-assigned this Jul 21, 2026
Extract metadata backfill, placeholder hydration, and preview backfill into dedicated processors and organize processing and refresh sources.

Signed-off-by: Marino Faggiana <marino.faggiana@nextcloud.com>
Select a limited batch of missing previews inside the processor and skip previously failed items. Simplify failure tracking and clear it when no previews remain or an account is removed.

Signed-off-by: Marino Faggiana <marino.faggiana@nextcloud.com>
Add dedicated logging for media preview backfill progress and completion.

Signed-off-by: Marino Faggiana <marino.faggiana@nextcloud.com>
Signed-off-by: Marino Faggiana <marino.faggiana@nextcloud.com>
@marinofaggiana
marinofaggiana marked this pull request as ready for review July 21, 2026 08:00
@marinofaggiana
marinofaggiana merged commit 309f2c5 into master Jul 21, 2026
5 checks passed
@marinofaggiana
marinofaggiana deleted the preview-process branch July 21, 2026 12:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant