Investigate 403 Forbidden errors

[elrayle]

Description

There are a large number of 403 Forbidden errors happening during crawler runs. They appear to be primarily for rubygems coordinates. This appears to be a systemic failure and needs to be investigated.

Rationale

Systemic failures need to be addressed.

Definition of Done

• The cause of the error has been identified.
• If fixable, handling of the error is fixed
• if not-fixable, handling of the error is more graceful (What that means needs to be defined.)

[elrayle]

The 403 errors logged by the crawler all seem to be coming from RubyGems.org.