Skip to content

Observability: Sentry Cron #780

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
AkhileshNegi merged 6 commits into from
Apr 24, 2026
Merged

Observability: Sentry Cron #780

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
97cf4a5
setting up sentry cron observability
AkhileshNegi Apr 22, 2026
68cc418
using cron variable for consistency
AkhileshNegi Apr 22, 2026
011b404
updated with comment
AkhileshNegi Apr 22, 2026
1715ed0
add comments for readability and later reference
AkhileshNegi Apr 22, 2026
9b3b80e
CI fixes
AkhileshNegi Apr 22, 2026
3e1d652
Merge branch 'main' into enhancement/cron-observability
AkhileshNegi Apr 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 29 additions & 8 deletions backend/app/api/routes/cron.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,21 +1,48 @@
import logging

import sentry_sdk
from sentry_sdk.types import MonitorConfig

from app.api.permissions import Permission, require_permission
from fastapi import APIRouter, Depends

from app.api.deps import SessionDep
from app.core.config import settings
from app.crud.evaluations import process_all_pending_evaluations_sync

logger = logging.getLogger(__name__)

router = APIRouter(tags=["Cron"])

EVALUATION_CRON_MONITOR_CONFIG: MonitorConfig = {
# Expected cadence: a check-in every CRON_INTERVAL_MINUTES minutes.
"schedule": {
"type": "interval",
"value": settings.CRON_INTERVAL_MINUTES,
"unit": "minute",
},
# Timezone for the schedule (only affects crontab-style schedules).
"timezone": "UTC",
# Grace period (minutes) before a late check-in is marked as missed.
"checkin_margin": 2,
# Max runtime (minutes) before an in-progress run is marked as timed out.
"max_runtime": 2 * settings.CRON_INTERVAL_MINUTES,
# Consecutive failures/missed/timeouts required to open a Sentry issue.
"failure_issue_threshold": 2,
# Consecutive successful check-ins required to auto-resolve the issue.
"recovery_threshold": 1,
}


@router.get(
"/cron/evaluations",
include_in_schema=False,
dependencies=[Depends(require_permission(Permission.SUPERUSER))],
)
@sentry_sdk.monitor(
monitor_slug="evaluation-cron-job",
monitor_config=EVALUATION_CRON_MONITOR_CONFIG,
)
def evaluation_cron_job(
session: SessionDep,
) -> dict:
Expand All @@ -34,7 +61,6 @@ def evaluation_cron_job(
logger.info("[evaluation_cron_job] Cron job invoked")

try:
# Process all pending evaluations across all organizations
result = process_all_pending_evaluations_sync(session=session)

logger.info(
Expand All @@ -51,10 +77,5 @@ def evaluation_cron_job(
f"[evaluation_cron_job] Error executing cron job: {e}",
exc_info=True,
)
return {
"status": "error",
"error": str(e),
"total_processed": 0,
"total_failed": 0,
"total_still_processing": 0,
}
sentry_sdk.capture_exception(e)
raise
6 changes: 6 additions & 0 deletions backend/app/core/config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,12 @@ def AWS_S3_BUCKET(self) -> str:
CALLBACK_CONNECT_TIMEOUT: int = 3
CALLBACK_READ_TIMEOUT: int = 10

# Evaluation cron invocation interval (minutes). In staging/production the
# endpoint is triggered by AWS EventBridge on this cadence; locally it can
# be driven by scripts/python/invoke-cron.py. The Sentry cron monitor reads
# this same value so its expected schedule stays aligned with the trigger.
CRON_INTERVAL_MINUTES: int = 5
Comment on lines +159 to +163
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🌐 Web query:

What are the valid value constraints for Sentry Python SDK crons MonitorConfig interval schedules with type interval value unit minute?

💡 Result:

For Sentry Python SDK crons MonitorConfig with schedule type "interval", unit "minute", the "value" is a positive integer representing the number of minutes between executions. No explicit upper or lower bounds are documented in official sources, but practical constraints apply: Sentry limits check-ins to 6 per minute per monitor environment, implying a minimum practical value of 10 minutes (to allow for checkin_margin). Examples use values like 3 or 10 minutes. The value must be a non-negative integer (likely >0), as indicated by type definitions and SDK code references.

Citations:

🏁 Script executed:

# Find usage of CRON_INTERVAL_MINUTES
rg "CRON_INTERVAL_MINUTES" --type py -B 2 -A 2

Repository: ProjectTech4DevAI/kaapi-backend

Length of output: 1774

🏁 Script executed:

# Check the config file for imports and current validation patterns
head -50 backend/app/core/config.py

Repository: ProjectTech4DevAI/kaapi-backend

Length of output: 1586

🏁 Script executed:

# Check if Field is already imported in config.py
rg "from pydantic import" backend/app/core/config.py -A 10

Repository: ProjectTech4DevAI/kaapi-backend

Length of output: 322

Add lower bound validation to CRON_INTERVAL_MINUTES to prevent invalid Sentry monitor config.

CRON_INTERVAL_MINUTES is environment-overridable but currently accepts any integer value. Setting it to 0 or negative values would create invalid Sentry monitor schedule configuration (used as interval "value" and in "max_runtime" calculation). Sentry requires positive interval values.

Apply Field(default=5, ge=1) with the necessary import addition.

Suggested fix 
 from pydantic import (
     EmailStr,
+    Field,
     HttpUrl,
     PostgresDsn,
     computed_field,
     model_validator,
 )
@@
-    CRON_INTERVAL_MINUTES: int = 5
+    CRON_INTERVAL_MINUTES: int = Field(default=5, ge=1)
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
# Evaluation cron invocation interval (minutes). In staging/production the
# endpoint is triggered by AWS EventBridge on this cadence; locally it can
# be driven by scripts/python/invoke-cron.py. The Sentry cron monitor reads
# this same value so its expected schedule stays aligned with the trigger.
CRON_INTERVAL_MINUTES: int = 5
# Evaluation cron invocation interval (minutes). In staging/production the
# endpoint is triggered by AWS EventBridge on this cadence; locally it can
# be driven by scripts/python/invoke-cron.py. The Sentry cron monitor reads
# this same value so its expected schedule stays aligned with the trigger.
CRON_INTERVAL_MINUTES: int = Field(default=5, ge=1)
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@backend/app/core/config.py` around lines 159 - 163, Add a lower-bound
validator to CRON_INTERVAL_MINUTES so zero/negative values are rejected: change
the declaration of CRON_INTERVAL_MINUTES in config.py to use Pydantic Field with
default=5 and ge=1 (i.e., Field(default=5, ge=1)) and add the required import
for Field from pydantic at the top of the file; update any type hints that
reference CRON_INTERVAL_MINUTES if necessary to keep it as int.


@computed_field # type: ignore[prop-decorator]
@property
def COMPUTED_CELERY_WORKER_CONCURRENCY(self) -> int:
Expand Down
Loading