Next.js migration: middleware + SEO

[MaxGhenis]

Part of #811

Goal

Migrate Vercel edge middleware and SEO infrastructure to Next.js patterns.

Tasks

• Port middleware.ts bot detection (crawlers, LLM bots, search engines)
• Replace manual OG HTML generation with Next.js generateMetadata on each route
• Replace pre-rendered blog HTML with Next.js SSG (handled by phase 2)
• Port JSON-LD structured data to route-level metadata
• Verify social sharing previews (Facebook, Twitter, LinkedIn, Slack, Discord)
• Port or simplify state legislative tracker proxy logic
• Verify robots.txt and sitemap.xml (use Next.js built-in or route handlers)
• Remove generate-sitemap.ts build script (replace with Next.js sitemap.ts)
• Test with Google Search Console, Facebook Debugger, Twitter Card Validator

What Next.js replaces

Current	Next.js equivalent
middleware.ts bot detection + OG HTML	generateMetadata per route
generate-prerender.ts	Static generation (SSG)
generate-sitemap.ts	app/sitemap.ts route handler
generate-llms-txt.ts	app/llms.txt/route.ts or app/llms-full.txt/route.ts
Manual JSON-LD in middleware	generateMetadata with other field

Notes

• Much of the current middleware complexity exists because the SPA can't serve real HTML to bots — Next.js SSR/SSG eliminates this problem entirely
• Middleware may still be needed for the tracker proxy and country redirects

[SakshiKekre]

Next.js SSG replaces bot-detection middleware — pages serve real HTML to all visitors. Article pages have generateMetadata (OG tags, Twitter cards) and JSON-LD. robots.txt and sitemap.xml via Next.js route handlers (206 URLs). llms.txt restored in PR #905. Remaining improvement (generateMetadata on home/research listing) tracked in #907.