Change the scraper to get restrictions after getting all the other data, instead of during. This way we can let it run longer but know we have the important data anyway, and it can be rate limited as much as it wants. Other suggestions open.
Change the scraper to get restrictions after getting all the other data, instead of during.
This way we can let it run longer but know we have the important data anyway, and it can be rate limited as much as it wants.
Other suggestions open.