Add support for GitHub pagination and ratelimiting #49
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Create a new
GitHubClientclass which extends theClientclass by modifyingthe
requestmethod and creating four new methodspaginateparameter to the constructorrequestmethod toand if so wait the designated amount of time before making the request.
paginateparameter is true, look in the payload of the responsepage for a link header and if found
extend the fetched data by calling the
get_additional_pagesmethodno_ratelimit_remainingmethod to check if there are noremaining requests allowed
get_additional_pagesmethod which calls theget_next_link_urlmethod to determine if there are additional pages. Ifso, fetch the next page and recursively continue fetching pages until the
last page is fetched
get_next_link_urlmethod which parses the link header andeither returns the next page if there is one or an empty string if not
ratelimit_seconds_remainingmethod which returns the numberof seconds remaining until the ratelimit is cleared
This new functionality can be used like this
Based on conversations with jpaugh and nnja indicating we're going to redo the unit tests from scratch, I removed my in progress unit tests from this branch and figured I'd submit it now to make it available to folks despite the lack of unit tests.
This fixes #10