feat: add rank packages function (CM-1310)

[ulemons]

Summary

Feeds sonatype_popularity_score into rank_packages() as a 4th signal, so the Sonatype popularity score actually drives criticality.

Sonatype could not deliver raw monthly Maven download counts, so instead they provide a popularity score derived from their SCA telemetry — normalized 0–100 per ecosystem, tiered P0–P3. Until now Maven criticality relied only on dependent_count and transitive_dependent_count, so industry-critical Maven packages that are under-represented as dependency targets in our graph were missed. This signal is the substitute for the missing downloads data.

This is step 3 of 3:

• Step 1 (separate PR) — schema: sonatype_* columns on packages.
• Step 2 (separate PR) — one-shot CSV ingestion into those columns.
• This PR — feed sonatype_popularity_score into rank_packages() as a 4th signal.

Changes

• New migration V1783123201__rank_packages_sonatype_signal.sql — CREATE OR REPLACE FUNCTION rank_packages(...) adding one signal. Only 3 lines change versus the previous definition:

  • base CTE — the per-row value COALESCE(sonatype_popularity_score, 0) and the per-ecosystem total SUM(...) OVER (PARTITION BY ecosystem).
  • the VALUES unpivot — one extra tuple ('sonatype_popularity', ...).

  Everything downstream (walked, combined, final, the UPDATE) is signal-agnostic, so no other change is needed. No hard-coded signal count, no per-ecosystem branch, no divisor to update.

Type of change

• Bug fix
• New feature
• Refactor / cleanup
• Performance improvement
• Chore / dependency update
• Documentation

JIRA ticket

CM-1310

---

Note

Medium Risk
Replaces the production ranking function and can shift impact, is_critical, and ranks for Maven (and any ecosystem with Sonatype scores); BOOL_OR criticality can only add critical packages, not remove them.

Overview
rank_packages() now includes sonatype_popularity_score as a fourth cumulative-coverage signal alongside downloads and dependency counts, via migration V1783123201__rank_packages_sonatype_signal.sql.

The base CTE adds per-row COALESCE(sonatype_popularity_score, 0) and an ecosystem-level sum; the walked unpivot gains a ('sonatype_popularity', …) row. Signals with zero ecosystem total are still skipped, so ecosystems without Sonatype data keep the same effective signal set as before.

impact, is_critical, rank_in_ecosystem, spotlight overrides, and last_synced_at stamping are unchanged in structure—only the averaged signal set can grow where Sonatype totals are non-zero (notably Maven as a downloads substitute).

^{Reviewed by Cursor Bugbot for commit f68b946. Bugbot is set up for automated code reviews on this repo. Configure here.}

[Copilot]

Pull request overview

This PR adds sonatype_popularity_score as a 4th cumulative-coverage signal to the production rank_packages() ranking function via a new migration. It is step 3 of 3 in wiring Sonatype's popularity score (a substitute for the missing raw Maven download counts) into package criticality scoring, so industry-critical Maven packages that are under-represented as dependency targets in the graph are no longer missed.

Changes:

• New migration V1783123201__rank_packages_sonatype_signal.sql performing CREATE OR REPLACE FUNCTION rank_packages(...).
• Adds the per-row COALESCE(sonatype_popularity_score, 0) value plus its per-ecosystem SUM(...) OVER (PARTITION BY ecosystem) total in the base CTE.
• Adds a ('sonatype_popularity', ...) tuple to the walked unpivot; the existing zero-total WHERE guard leaves ecosystems without Sonatype data (npm/pypi) unchanged.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[mbani01]

LGTM