A screenshot blocklist with an admin editor keeps consent-walled pages out of link previews (v7.208.0) [cold-deploy] - #1260
Merged
Conversation
…s out of link previews (v7.208.0) [cold-deploy] Popular sites that members link to often answer a headless capture with a cookie-consent banner, a login wall or a bot check, so the "preview" we store is a picture of a dialog rather than of the page. heise.de is the case that prompted this; reddit.com was already skipped by a config-only host list. The list is now per-installation data with a GUI, the way the legal pages are: a screenshot_blocklist_entries table, an editor at /admin/screenshots?tab= blocklist (add, remove, a grammar cheat sheet and a "try a URL" box that answers what an entry really covers), and a persistent_term cache reloaded over PubSub so an edit takes effect at once on every node. Which sites need this changes the day one adds a consent layer, so it cannot live in a source or .env edit. :screenshot_blocklist in config.exs is only the seed the creating migration copies in: the two shipped entries plus anything an operator had in SCREENSHOT_BLOCKLIST / SCREENSHOT_BLOCKED_HOSTS, so no existing installation's list is dropped and the shipped ones land even where the variable overrode them. Either is one click from removal. An entry is a domain or a URL. Hosts match at the label boundary, so heise.de covers www. and m. but never notheise.de; paths match whole segments, so /news does not cover /newsroom; a leading *. or www. is dropped (an entry matching fewer URLs than intended is the worse mistake); scheme, port, query and fragment are ignored, so an appended ?utm_source cannot defeat an entry. An entry naming no host is refused by the changeset rather than stored as a line that silently matches nothing, or everything. Not capturing a page is now a skip, not a failure. A blocklisted profile link is no longer flagged broken? and logs nothing (that poisoned a perfectly good link and made the bulk task skip it for the wrong reason), and no task is spawned for it at all. What a link without a screenshot looks like moved into one component, VutuvWeb.UI.link_thumb/1: a stored capture renders as the thumbnail, a blocklisted page as a calm tile naming the site, and only a capture still on its way keeps the placeholder image, since "a screenshot has not been created yet" is a promise that never comes true for a blocklisted page. Posts already degraded to the plain link. Adding an entry only stops new captures, because a link is re-captured solely when its URL changes, so the admin page has a cleanup button for the screenshots taken earlier (purge_blocklisted/0 on both owners, also reachable as Vutuv.Release.purge_blocklisted_screenshots/0). Cold deploy: a new migration, a new supervised child and a runtime.exs change. The migration only adds a table, so the previous release keeps serving through the switch window (N-1). An AI agent wrote this text in my name, unreviewed by me. The work behind it is mine; I only delegated the writing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR heise.de and friends answer a headless capture with a cookie banner, so the stored "preview" is a picture of a dialog. There is now a screenshot blocklist of domains and URLs, edited by admins at
/admin/screenshots?tab=blocklist, and nothing downstream depends on a screenshot existing.Why
Popular sites members link to often meet a headless browser with a consent banner, a login wall or a bot check.
reddit.comwas already skipped through a config-only host list, but that list could only be changed by editing source or.env, and which sites need it changes the day one adds a consent layer. So the list became per-installation data with a GUI, the way the legal pages are.What
Vutuv.ScreenshotBlocklist— entries are domains or URLs:heise.de*.heise.deexample.com/news/newsroom)example.com/*/private*= exactly one path segmenthttps://example.com/story-1Hosts match at the label boundary (
notheise.deis notheise.de), paths by whole segments, and scheme/port/?query/#fragmentare ignored so an appendedutm_sourcecannot defeat an entry. An entry naming no host is refused by the changeset rather than stored as a line that silently matches nothing, or everything.Admin editor at
/admin/screenshots?tab=blocklist(third tab beside Queue and Gallery): add with an optional note, remove, a grammar cheat sheet, and a "Try a URL" box that answers whether the list covers a real link. Edits take effect at once on every node (:persistent_termcache + PubSub reload, theVutuv.Prefs.Cacheshape).Seeding.
:screenshot_blocklistinconfig.exsis only the seed the creating migration copies in: the two shipped entries (reddit.com,heise.de) plus anything an operator had inSCREENSHOT_BLOCKLIST/SCREENSHOT_BLOCKED_HOSTS, so no existing list is dropped and the shipped entries land even where the variable replaced them. Either is one click from removal.A skip, not a failure. A blocklisted profile link is no longer flagged
broken?and logs nothing, and no task is spawned for it.VutuvWeb.UI.link_thumb/1is now the one place deciding what a link without a screenshot looks like: stored capture → thumbnail, blocklisted → a calm tile naming the site, still pending → the placeholder. "A screenshot has not been created yet" is a promise that never comes true for a blocklisted page. Posts already degraded to the plain link.Captures taken earlier are not replaced by anything (a link is re-captured only when its URL changes), so the tab has a cleanup button —
purge_blocklisted/0on both owners, alsoVutuv.Release.purge_blocklisted_screenshots/0.Docs:
docs/ADMINS.mdgets a "Screenshot blocklist" section, plusimages.md,posts-and-feed.mdand the design rule for the new component. German strings written by hand (gettext.extract --mergefuzzy-filled "Blocklist" as "Blockieren"), pinned by a German-render test.Deploy
Cold deploy — new migration, new supervised child (
ScreenshotBlocklist.Cache),config/runtime.exschange. The migration only adds a table, so the previous release keeps serving through the switch window (N-1).Version 7.208.0.
mix precommitgreen locally (6290 tests).An AI agent wrote this text in my name, unreviewed by me. The work behind it is mine; I only delegated the writing.