Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Spider for Cuiaba, MT #226

Merged
merged 11 commits into from
Sep 16, 2020
Merged

Add Spider for Cuiaba, MT #226

merged 11 commits into from
Sep 16, 2020

Conversation

andersonberg
Copy link
Contributor

@andersonberg andersonberg commented Aug 27, 2020

This PR adds the spider for Cuiaba-MT as requested at #181
The DOWNLOAD_DELAY setting is important because I was getting a lot of 429 responses (Too many requests). The delay did the job and the spider isn't getting 429 responses anymore

@andersonberg andersonberg changed the title MT Cuiaba Add Spider for Cuiaba, MT Aug 27, 2020
Copy link
Collaborator

@jvanz jvanz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay in the review... =/

Copy link
Collaborator

@jvanz jvanz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you faced issue with timeout while running this spider?

processing/data_collection/gazette/settings.py Outdated Show resolved Hide resolved
@jvanz jvanz merged commit 880c710 into okfn-brasil:main Sep 16, 2020
@jvanz
Copy link
Collaborator

jvanz commented Sep 16, 2020

image

phrfpeixoto pushed a commit to phrfpeixoto/querido-diario that referenced this pull request Oct 2, 2020
@andersonberg andersonberg deleted the mt_cuiaba branch October 13, 2020 17:26
anapaulagomes pushed a commit to anapaulagomes/querido-diario that referenced this pull request Oct 14, 2020
@sergiomario sergiomario added the hacktoberfest-accepted Pull Requests aprovados na Hacktoberfest label Oct 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hacktoberfest-accepted Pull Requests aprovados na Hacktoberfest
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants