New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

抓取网页出现HTTP ERROR处理问题 #52

Open

tottilin opened this issue Oct 16, 2015 · 0 comments

Labels

tottilin commented Oct 16, 2015

1.http error 404 没有丢弃url
2.其他错误在爬虫执行完后，继续尝试，但不能无限次尝试（有的时候会出现爬虫任务根本停不下来）

qinxuye added the question label

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment