feat: general improvements on blacklist management#63
Merged
Conversation
When SMTP rejection indicates IP blacklist detection, convert HardFail to SoftFail to allow automatic retry with a different IP address. This prevents legitimate messages from failing permanently due to transient IP reputation issues. Changes: - SMTPSender: Return boolean from handle_smtp_error_response to indicate blacklist detection - SMTPSender: Convert Net::SMTPFatalError to SoftFail when blacklist detected - QueuedMessage: Use domain-aware IP selection in reallocate_ip_address to respect IP exclusions - SMTPResponseParser: Add iCloud/Apple policy rejection patterns - Add test coverage for iCloud blacklist patterns
The count_healthy_ips method was summing IPs with blacklists and IPs with domain exclusions separately, causing IPs that have both conditions to be counted twice. This resulted in incorrect healthy IP counts (e.g. showing 0 healthy IPs when 10 blacklisted + 5 excluded = 15, but only 13 total IPs with 2 appearing in both categories). Now uses OR query with distinct count to properly count unique IPs that have either blacklist records OR domain exclusions (or both).
- Add retry tracking fields to ip_blacklist_records (next_retry_at, last_retry_at, retry_count, retry_result) - Implement RetryService to test if IP is still blacklisted by sending test SMTP connection - Create scheduled task that automatically retries blacklisted IPs every 2 days - Add manual retry trigger via web UI with rate limiting (5 requests/hour) - Auto-schedule first retry when SMTP blacklist is detected - Resolve blacklist automatically when retry test succeeds and start warmup - Add retry notifications (success/failed/error) to webhook/email/Slack system - Show retry information and 'Retry Now' button in blacklist record detail page - Add comprehensive tests for retry functionality
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces a new manual retry mechanism for SMTP-detected IP blacklist records, improves notification and tracking around blacklist retries, and enhances the accuracy of IP reputation statistics. It also expands SMTP response parsing to better detect iCloud/Apple-specific blocking patterns. The most important changes are grouped below.
Manual Retry for Blacklist Records
retry_nowcontroller action inIPBlacklistRecordsControllerto allow admins to manually trigger an immediate retry test for SMTP-detected blacklist records, with appropriate validation, logging, and error handling.IPBlacklist::RetryService, which encapsulates the logic for performing a retry test by sending a test email, updating the record, and handling success, failure, or error cases.rate_limit_retrymethod. [1] [2] [3]Tracking and Notification Enhancements
IPBlacklistRecordmodel with fields for retry tracking:last_retry_at,next_retry_at,retry_count,retry_result, andretry_result_details, along with supporting indexes. [1] [2]IPBlacklist::Notifier, ensuring operational visibility for these actions. [1] [2]IP Reputation and Health Calculation
UNIONto avoid double-counting IPs that are both blacklisted and excluded, resulting in more accurate statistics.Automatic Retry Scheduling
SMTP Response Parsing Improvements
SMTPResponseParserto detect iCloud/Apple-specific SMTP rejection patterns, improving the accuracy of blacklist source identification. [1] [2] [3] [4]