Skip to content

fix(sitemaps): reduce sitemap cache TTL to 2 hours#3680

Merged
nimrodkra merged 3 commits intomainfrom
seo/pr5-cache-reduction
Mar 9, 2026
Merged

fix(sitemaps): reduce sitemap cache TTL to 2 hours#3680
nimrodkra merged 3 commits intomainfrom
seo/pr5-cache-reduction

Conversation

@nimrodkra
Copy link
Member

@nimrodkra nimrodkra commented Mar 9, 2026

Summary

  • reduce sitemap cache-control max-age/s-maxage from 14400 seconds to 7200 seconds
  • keep behavior unchanged otherwise while making sitemap updates propagate faster
  • update sitemap tests that assert explicit cache-control values

Why this matters

A shorter sitemap cache TTL means deleted or changed URLs are reflected sooner in sitemap responses. That reduces stale crawl targets and helps search engines converge on the current site state faster.

Test plan

  • pnpm run lint
  • pnpm test -- sitemaps (fails in this environment because local Postgres on 127.0.0.1:5432 is not available)

Made with Cursor

Lower sitemap cache-control TTL from 4h to 2h so removed or updated URLs are reflected faster, and update sitemap header assertions accordingly.

Made-with: Cursor
@nimrodkra
Copy link
Member Author

@claude please rview this pr

@claude
Copy link

claude bot commented Mar 9, 2026

Claude finished @nimrodkra's task in 43s —— View job


PR Review

  • Read changed files
  • Review code changes
  • Post review feedback

The change is minimal and correct — the shared SITEMAP_CACHE_CONTROL constant in src/routes/sitemaps.ts:22 is updated in a single place and the two tests that assert an explicit cache-control value are updated to match.

One issue to address:

The project's CLAUDE.md explicitly requires avoiding magic numbers for time durations and using constants from src/common/constants.ts instead. 7200 should be expressed as 2 * ONE_HOUR_IN_SECONDS for readability and to make the intent clear.

- const SITEMAP_CACHE_CONTROL = 'public, max-age=7200, s-maxage=7200';
+ const SITEMAP_CACHE_CONTROL = `public, max-age=${2 * ONE_HOUR_IN_SECONDS}, s-maxage=${2 * ONE_HOUR_IN_SECONDS}`;

with the import:

import { ONE_HOUR_IN_SECONDS } from '../common/constants';

Fix this →

Minor observation (pre-existing): Most endpoint tests only check expect(res.header['cache-control']).toBeTruthy(), while agents-digest and squads assert the exact string. This inconsistency predates this PR and isn't a blocker, but it does mean the new TTL value is only explicitly verified in 2 of the 7+ endpoint tests.

Otherwise the change is clean and achieves the stated goal.

@pulumi
Copy link

pulumi bot commented Mar 9, 2026

🍹 The Update (preview) for dailydotdev/api/prod (at 09557e0) was successful.

✨ Neo Explanation

This is a routine application deployment rolling a new build across all 7 services and 33 cron jobs in production, with accompanying database and ClickHouse schema migration jobs being swapped in for the new version.

Root Cause Analysis

A new version of the application has been built and is being deployed to production. All workloads are being updated to point to the new container image, replacing the previous build across every Deployment and CronJob in the cluster.

Dependency Chain

The new application version cascades uniformly across the entire fleet:

  • 7 Deployments (API, background workers, WebSocket, private, temporal, personalized digest, worker jobs) are all rolling out the new image.
  • 33 CronJobs are updated to reference the new image so future scheduled runs use the latest code.
  • 2 migration Jobs (database and ClickHouse) tied to the previous build are deleted, and 2 new migration Jobs for the new build are created — these will run the schema migrations against the DB and ClickHouse before (or alongside) the new app pods coming up.

Risk analysis

The database and ClickHouse migration Jobs are the only area to watch. If either migration fails or introduces a breaking schema change, the newly deployed application pods could malfunction. The migrations run as one-shot Jobs with restartPolicy: Never, so failures won't auto-retry. All other changes are standard rolling Deployment updates with no stateful resources being replaced or deleted.

Resource Changes

    Name                                                       Type                           Operation
~   vpc-native-generate-search-invites-cron                    kubernetes:batch/v1:CronJob    update
~   vpc-native-calculate-top-readers-cron                      kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-zombie-users-cron                         kubernetes:batch/v1:CronJob    update
~   vpc-native-sync-subscription-with-cio-cron                 kubernetes:batch/v1:CronJob    update
+   vpc-native-api-clickhouse-migration-f4a9fc17               kubernetes:batch/v1:Job        create
~   vpc-native-clean-gifted-plus-cron                          kubernetes:batch/v1:CronJob    update
~   vpc-native-ws-deployment                                   kubernetes:apps/v1:Deployment  update
~   vpc-native-user-profile-updated-sync-cron                  kubernetes:batch/v1:CronJob    update
~   vpc-native-update-source-tag-view-cron                     kubernetes:batch/v1:CronJob    update
+   vpc-native-api-db-migration-f4a9fc17                       kubernetes:batch/v1:Job        create
-   vpc-native-api-clickhouse-migration-2f59a47c               kubernetes:batch/v1:Job        delete
~   vpc-native-worker-job-deployment                           kubernetes:apps/v1:Deployment  update
~   vpc-native-personalized-digest-deployment                  kubernetes:apps/v1:Deployment  update
~   vpc-native-update-current-streak-cron                      kubernetes:batch/v1:CronJob    update
~   vpc-native-post-analytics-history-day-clickhouse-cron      kubernetes:batch/v1:CronJob    update
~   vpc-native-temporal-deployment                             kubernetes:apps/v1:Deployment  update
~   vpc-native-update-tag-recommendations-cron                 kubernetes:batch/v1:CronJob    update
~   vpc-native-hourly-notification-cron                        kubernetes:batch/v1:CronJob    update
~   vpc-native-validate-active-users-cron                      kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-zombie-opportunities-cron                 kubernetes:batch/v1:CronJob    update
~   vpc-native-private-deployment                              kubernetes:apps/v1:Deployment  update
~   vpc-native-update-tags-str-cron                            kubernetes:batch/v1:CronJob    update
~   vpc-native-daily-digest-cron                               kubernetes:batch/v1:CronJob    update
~   vpc-native-update-trending-cron                            kubernetes:batch/v1:CronJob    update
~   vpc-native-check-analytics-report-cron                     kubernetes:batch/v1:CronJob    update
~   vpc-native-user-posts-analytics-refresh-cron               kubernetes:batch/v1:CronJob    update
~   vpc-native-squad-posts-analytics-refresh-cron              kubernetes:batch/v1:CronJob    update
~   vpc-native-generic-referral-reminder-cron                  kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-zombie-user-companies-cron                kubernetes:batch/v1:CronJob    update
~   vpc-native-update-highlighted-views-cron                   kubernetes:batch/v1:CronJob    update
~   vpc-native-update-achievement-rarity-cron                  kubernetes:batch/v1:CronJob    update
~   vpc-native-personalized-digest-cron                        kubernetes:batch/v1:CronJob    update
~   vpc-native-user-profile-analytics-history-clickhouse-cron  kubernetes:batch/v1:CronJob    update
~   vpc-native-agents-digest-cron                              kubernetes:batch/v1:CronJob    update
~   vpc-native-update-source-public-threshold-cron             kubernetes:batch/v1:CronJob    update
~   vpc-native-post-analytics-clickhouse-cron                  kubernetes:batch/v1:CronJob    update
~   vpc-native-expire-super-agent-trial-cron                   kubernetes:batch/v1:CronJob    update
~   vpc-native-bg-deployment                                   kubernetes:apps/v1:Deployment  update
~   vpc-native-clean-stale-user-transactions-cron              kubernetes:batch/v1:CronJob    update
~   vpc-native-user-profile-analytics-clickhouse-cron          kubernetes:batch/v1:CronJob    update
~   vpc-native-clean-zombie-images-cron                        kubernetes:batch/v1:CronJob    update
... and 5 other changes

Use ONE_HOUR_IN_SECONDS to express the 2-hour sitemap cache TTL and align with repository guidance against hardcoded time duration values.

Made-with: Cursor
@nimrodkra nimrodkra enabled auto-merge (squash) March 9, 2026 14:07
@nimrodkra nimrodkra merged commit c98b4bf into main Mar 9, 2026
9 checks passed
@nimrodkra nimrodkra deleted the seo/pr5-cache-reduction branch March 9, 2026 14:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants