Hiring board eagerly fetches all 57 pages (~22 MB) on load

[KonstantinMB]

Problem

Loading /hiring triggers 350+ network calls to /api/hiring/jobs/paginated, requesting pages 1 through 57 in sequence. Each page is ~400 KB of JSON. Total payload per page load: roughly 22 MB.

This is happening on the homepage too (where the hiring section probably needs only the latest few jobs).

Reproduction (verified via gstack QA)

$ B=~/.claude/skills/gstack/browse/dist/browse
$ $B goto https://exploreyc.com/hiring
$ $B wait --networkidle
$ $B network | grep -c "/api/hiring"
350

$ $B network | grep -oE "page=[0-9]+" | sort -u | wc -l
57   # unique pages requested

$ $B network | grep "/api/hiring/jobs/paginated" | head -5
GET .../api/hiring/jobs/paginated?page=1&per_page=20&sort_by=recent → 200 (168ms, 408520B)
GET .../api/hiring/jobs/paginated?page=2&per_page=20&sort_by=recent → 200 (201ms, 392657B)
GET .../api/hiring/jobs/paginated?page=3&per_page=20&sort_by=recent → 200 (173ms, 401109B)
...

Impact

• Backend cost: 57× the database / serialization work for every visit. With Render's per-CPU pricing this adds up fast.
• User-perceived speed: cumulative ~5-10 seconds of background fetches even after first paint.
• Bandwidth: 22 MB on every visit, mostly thrown away (most users never scroll past page 2-3).
• Open-source attack vector: anyone with a for loop hitting /hiring can drain a lot of server resources cheaply, even with the /api/scrape rate limiter, because /api/hiring/jobs/paginated isn't rate-limited.

Proposed fix

1. Real pagination: only fetch page 1 on load. Prefetch page 2 on idle. Fetch page N when the user hits "Next" or scrolls near the bottom.
2. Rate-limit /api/hiring/jobs/paginated to e.g. 60/min/IP (use the existing ResearchRateLimiter pattern in backend/main.py:46).
3. (Optional) Add Cache-Control: public, max-age=300 headers on this endpoint — the data changes slowly.

Acceptance criteria

• Loading /hiring triggers ≤ 3 calls to /api/hiring/jobs/paginated on first paint (page 1 + maybe analytics + stats)
• Clicking "Next" / scrolling fetches the next page
• Total bytes transferred for the first paint reduced by ~95%
• /api/hiring/jobs/paginated is rate-limited (_enforce_rate_limit pattern, see backend/main.py:91)
• Existing UX (search, filters) still works

Pointers

• frontend/src/pages/HiringBoardPage.tsx (or wherever /hiring is rendered) — likely loops for page in 1..N: fetchPage(page) somewhere
• backend/main.py — find @app.get("/api/hiring/jobs/paginated") to add rate limiting
• The ResearchRateLimiter class at backend/main.py:46 is the reusable pattern

Scope

Touches 1-2 frontend files + 1 backend line for rate limiting. Estimated time: 2-3 hours.

Want to take this on? Comment below and I'll assign it.