feat(monitoring): Alert when a large share of recent signups fail

[regdocs]

Signup health is currently invisible until someone looks: gather_hourly_stats pushes counts to Telegram with no threshold, and press/signup_e2e.py (the synthetic canary) only writes a Signup E2E failed Error Log. Nothing pages when real signups start failing.

This adds alert_on_failing_signups to server_monitoring.py, on the hourly scheduler, sending a Raven message when either of two signals crosses its threshold.

Product trial signups that errored out — Product Trial Requests that reached a terminal status (Error or Site Created) in the last hour. Alerts when errors are more than 30% of settled requests, minimum 5, broken down per product trial. Pending/Expired are excluded so a request that hasn't finished yet doesn't dilute the ratio. The window is on modified — when the status last flipped — not on creation: settling takes as long as provisioning does, and a creation window would permanently drop every request that took over an hour to fail, which is exactly the provisioning outage worth alerting on. The tradeoff is that a later touch (is_site_accessible, is_subscription_created) can carry a settled request into a second window; over-counting a failure is recoverable, never seeing it isn't.

Signups that never became a team — Account Requests created between 2h and 1h ago (a one-hour grace period to read the mail and finish) with no Team for that email. Alerts above 90%, minimum 10. The threshold is deliberately near-total: ordinary abandonment at the verification step is the norm, so anything lower would fire every hour. This is the signal that catches verification mail or OTP being broken, which nothing else notices today.

Both exclude the fc-signup-test canary. Ratios over a volume floor rather than absolute counts, so one failure during a quiet hour doesn't page anyone. Thresholds are module constants, easy to retune once we see how it behaves in practice.

Two things worth a reviewer's opinion:

• Placement. Signups have nothing to do with the public server pool, and the file is now ~570 lines. A signup_monitoring.py next to the Account Request doctype would fit taste.md better; it's here because that's where it was asked for. Happy to move it.
• Channel. RAVEN_SIGNUP_ALERTS_CHANNEL currently points at frappe-cloud-server-alerts since there's no signup channel. One-line change if there should be a separate one.

test_server_monitoring.py covers both signals, the volume floors, the grace period, the canary exclusion, and the slow-failure case. The rate tests assert deltas against a baseline read before the fixtures are created, so they neither delete nor depend on rows the site already has — the monitor counts every signup on the site, and scrubbing that would mean deleting other tests' fixtures and real signups. The tests have not been run — I didn't have a site to run them against, so they need a pass before merge.

🤖 Generated with Claude Code

[greptile-apps]

Confidence Score: 4/5

The PR is not safe to merge until trial outcomes are windowed by an actual terminal-status transition time.

Generic post-settlement writes advance modified, allowing old successes or failures to distort later hourly ratios and suppress or repeat alerts.

Files Needing Attention: press/press/doctype/server/server_monitoring.py

_{Reviews (2): Last reviewed commit: "feat(monitoring): Alert when a large sha..." | Re-trigger Greptile}

[greptile-apps]

Test cleanup deletes shared records

When other tests or fixtures have Account Request or Product Trial Request rows, these unfiltered deletes erase them before every test, making the suite destructive and order-dependent. Restrict cleanup to records created by this test.

Context Used: Guidelines for reviewing Frappe Framework applicat... (source)

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 59.33%. Comparing base (9a39716) to head (95e698e).
⚠️ Report is 3 commits behind head on develop.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #7186      +/-   ##
===========================================
+ Coverage    59.23%   59.33%   +0.10%     
===========================================
  Files         1009     1010       +1     
  Lines        91940    92134     +194     
  Branches      1228     1228              
===========================================
+ Hits         54457    54665     +208     
+ Misses       37460    37446      -14     
  Partials        23       23              

Flag	Coverage Δ
dashboard	87.18% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[greptile-apps]

Mutable timestamp miswindows settlements

When a terminal Product Trial Request receives a later write such as the accessibility update during login, its modified timestamp advances and this query counts the old outcome in a new hourly window, causing old successes to suppress real outages or old failures to trigger repeated alerts.

Knowledge Base Used: SaaS Product Trial Signup Flow

[mergify]

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

• Queue this pull request