Skip to content

feat: graceful shutdown, circuit breaker, retry backoff & batch tx support#171

Merged
AbelOsaretin merged 2 commits into
Heliobond:mainfrom
covenantalade:feature/issues-54-55-56-57
Jun 30, 2026
Merged

feat: graceful shutdown, circuit breaker, retry backoff & batch tx support#171
AbelOsaretin merged 2 commits into
Heliobond:mainfrom
covenantalade:feature/issues-54-55-56-57

Conversation

@covenantalade

Copy link
Copy Markdown
Contributor

Closes #54

Closes #55

Closes #56

Closes #57

What changed

#57 – Graceful shutdown

  • SIGTERM/SIGINT handlers stop all cron tasks, drain in-flight HTTP
    requests via server.close(), then drain the RPC pool (10 s timeout)
  • All cron schedules registered through a scheduleCron() wrapper so
    tasks are tracked and can be stopped cleanly
  • isShuttingDown guard prevents crons from starting new work mid-shutdown
  • Each shutdown step is logged for observability

#56 – Circuit breaker for Stellar RPC

  • New CircuitBreaker class (src/lib/circuit-breaker.ts) with
    CLOSED → OPEN → HALF_OPEN states
  • Configurable via RPC_BREAKER_FAILURE_THRESHOLD and
    RPC_BREAKER_RECOVERY_TIMEOUT_MS env vars
  • withRpcConnection() routes through the breaker; throws
    RpcDegradedError as fallback when OPEN
  • Replaced the three stub functions (isRpcAvailable,
    isRpcOutageExtended, getRpcStatus) with real implementations
  • Breaker metrics exposed on GET /health; GET /ready now includes
    rpc_circuit check

#55 – Transaction retry with exponential backoff

  • New withRetry() utility (src/lib/retry.ts) with configurable
    maxAttempts, baseDelayMs, maxDelayMs, and jitter
  • isTransientError() prevents retries on permanent errors
    (bad auth, insufficient balance, no account, etc.)
  • stellar.ts _executeSignAndSubmitWithRetry delegates to withRetry
    instead of a hand-rolled loop
  • Ships missing src/lib/tx-queue.ts and src/lib/error-limiter.ts
    that index.ts already imported

#54 – Batch transaction support

  • isSorobanBatchAvailable() detection stub — returns false until
    Soroban ships native batching (monitor stellar/stellar-core)
  • runJob() routes to runBatchNative() or runBatchSequential() fallback
  • runBatchNative() scaffolded and ready for implementation
  • Per-item duration_ms tracking added to BatchResult
  • Post-job benchmark summary (total_ms, avg_ms_per_item,
    throughput_per_second) attached to each completed BatchJob

Testing

  • Existing test suite passes unchanged
  • Manual: SIGTERM during active requests confirms clean drain before exit
  • Manual: force-opening the breaker confirms /ready returns 503

Nanle-code and others added 2 commits June 30, 2026 10:28
…port

Closes Heliobond#54, Heliobond#55, Heliobond#56, Heliobond#57

- Heliobond#57: Graceful shutdown — handle SIGTERM/SIGINT, stop all tracked cron
  jobs, drain in-flight HTTP requests via server.close(), drain RPC pool
  with up to 10 s wait; log each step; guard crons with isShuttingDown flag.

- Heliobond#56: Circuit breaker for Stellar RPC — new CircuitBreaker class with
  configurable failure threshold, CLOSED/OPEN/HALF_OPEN states, recovery
  timeout, and fallback behaviour. Wired into withRpcConnection(); breaker
  metrics exposed on /health; /ready now checks rpc_circuit state. RPC
  health tracking (consecutiveFailures, outageDuration) replaces stubs.

- Heliobond#55: Transaction retry with exponential backoff — new withRetry() util
  with configurable maxAttempts, baseDelayMs, maxDelayMs, and jitter.
  Distinguishes transient vs permanent errors (no retry on bad auth etc.).
  _executeSignAndSubmitWithRetry now delegates to withRetry. Also ships
  tx-queue.ts and error-limiter.ts which index.ts already imports.

- Heliobond#54: Batch transaction support — isSorobanBatchAvailable() detection
  stub (returns false until Soroban ships native batching); runJob() routes
  to runBatchNative() or runBatchSequential() fallback; adds per-item
  duration tracking and post-job benchmarking metrics.
@drips-wave

drips-wave Bot commented Jun 30, 2026

Copy link
Copy Markdown

@covenantalade Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@AbelOsaretin AbelOsaretin merged commit d29d2f7 into Heliobond:main Jun 30, 2026
0 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants