Super Simple Cache Warmer

Warm page and image caches by crawling a sitemap.

Visiting HTML alone is not enough for Magento: resized catalog images are created when the image URL is requested (pub/get.php). This tool:

1. Collects page URLs (and optional image:loc entries) from the sitemap
2. GETs each page to warm FPC / CDN page cache
3. Parses HTML for img / srcset / data-src / CSS url() media URLs
4. GETs each unique image (concurrently) so Magento derivatives and CDN edges are warmed before customers hit PLPs

Pages are warmed in Magento-friendly order: CMS → Category → Product (classified from sitemap image:loc + priority).

Category pages are requested with product_list_limit=64. After page 1, the warmer reads the toolbar total / pager and also visits ?p=2…N&product_list_limit=64.

Requirements

• Python 3.9+
• pip

cd /Users/yoeriwassink/Development/super-simple-cache-warmer
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Homebrew Python is externally managed (PEP 668), so always use a venv — do not pip install into the system Python.

Usage

Interactive (prompts for sitemap + cache header):

source .venv/bin/activate
python cache_warmer.py

Non-interactive example for fakobijoux.nl:

source .venv/bin/activate
python cache_warmer.py \
  --sitemap "https://www.fakobijoux.nl/media/sitemap/sitemap.xml" \
  --cache-header "CF-Cache-Status" \
  --hit HIT \
  --miss MISS \
  --images \
  --workers 8

Options

Flag	Meaning
--sitemap	Sitemap URL
--cache-header	Header used to report HIT/MISS (e.g. CF-Cache-Status, X-Cache)
--hit / --miss	Values that mean cache hit / miss
--images / --no-images	Warm images (default: ask, recommended yes)
--workers N	Concurrent image fetches (default: 8)
--all-assets	Include any same-host image-like URL, not only /media/ paths
--allow-external	Also fetch images on other hosts

Settings are remembered in sitemap_cache.txt.

Dynamic rate limiting

Delay starts at 0.2s and:

• doubles on HTTP 429 (capped at 10s)
• slowly decreases after successful 200 responses (floor 0.05s)

Image workers share a lighter per-request pause so concurrency stays polite.

Magento notes

What	Warmed by
Full-page / CDN HTML	Page GET
Magento catalog/product/cache/...	Image GET (this is why --images matters)
Hyvä converted/ / resized/	Often during HTML render; image GET still warms CDN

For a full pre-build of Magento resize files you can also run bin/magento catalog:images:resize on the server; this warmer is the customer-path approach (HTML + real image URLs).