fix(columns): Reddit RSS, GitHub stars/discussions token guards, remove Bluesky + Bing#102
Merged
Conversation
…N API Reddit now returns 403 + an anti-bot HTML page to unauthenticated JSON API requests (/r/<sub>/<sort>.json, /search.json) from all IPs — residential and datacenter alike. Its Atom RSS feeds (/.rss) are still served keyless to a descriptive User-Agent, so read those instead. Trade-off: the feed has no score / comment counts (degrade to 0, rendered with a ?? 0 fallback) and no `after` cursor (single ~25-item page). Fixes both fetchSubredditPage (reddit column) and searchReddit (feeds github-backlinks).
GitHub now auth-gates the endpoints these two columns depend on: - REST /repos/<repo>/stargazers returns 401 for unauthenticated requests (the plain /repos/<repo> endpoint still works keyless). There is no keyless way to list recent stargazers, so fetchStargazers throws a clear "requires a token" message instead of a raw 401, and uses GraphQL when a token is present. Removes the now-dead REST stargazer helpers (fetchStargazersREST, ghFetchStargazersPageREST, parseLastPage, GHStargazerEdgeREST). - The GraphQL API gives unauthenticated requests ~0 quota (immediate 403), and Discussions have no REST surface, so fetchDiscussions now fails fast with a clear message (the old "never fails without a token" comment was false). Updates both plugins' rateLimitHints and the GITHUB_TOKEN env description to list search / stars / discussions as token-required.
Both are unreliable to fetch keyless: - Bluesky search mode 403s (the public AppView's searchPosts needs auth). - Bing News RSS 307-redirects ~half of server-side requests to a consent interstitial regardless of headers (undici follows the redirect → HTML → 0 items); it also overlaps the reliable Google News column. Removes both plugins plus Bluesky's integration, and drops them from the manifest, client + server registries, and docs (49 → 47 columns). github-backlinks keeps Bing News as one of its four fan-out backlink sources, where it degrades gracefully when Bing fails.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes and cleanup from running all columns locally (follow-up to the hosting work).
Fixes
/r/…/….json,/search.json) now returns 403 + an anti-bot page to unauthenticated clients from all IPs. Switched to Reddit's keyless Atom RSS feed (/.rss). Trade-off: no score/comment counts (→ 0) and no cursor (single ~25-item page). Fixes both theredditcolumn andsearchReddit(feedsgithub-backlinks)./stargazers(401 keyless; the plain repo endpoint still works). Now throws a clear "requires a token" instead of a raw 401, uses GraphQL when a token is set, and drops the now-dead REST helpers.Removals
github-backlinkskeeps Bing as one of four fan-out sources (degrades gracefully).Column count 49 → 47.
Verify
next buildclean (TypeScript passes), registry parity at 47/47/47./api/columns/[type]; the github guards confirmed to return the clear token message.Note for the deployed instance
GITHUB_TOKENis unset on Railway, sogithub-stars/github-discussionswill show the new "requires a token" message until it's set (railway variables --service minitor --set 'GITHUB_TOKEN=…').