GitHub - ParagBadala/WebCrawler: It is WebCrawler which will help in making the sitemap for the website which is basically all the links and pages of the websites. This webcrawler will collect all the links of the website to a queue_file and move the corresponding link to the crawled

ParagBadala / WebCrawler Public

Notifications You must be signed in to change notification settings
Fork 0
Star 0

It is WebCrawler which will help in making the sitemap for the website which is basically all the links and pages of the websites. This webcrawler will collect all the links of the website to a queue_file and move the corresponding link to the crawled_file if that link is crawled.

0 stars 0 forks Branches Tags Activity

Star

Notifications

Branches Tags

Name		Name	Last commit message	Last commit date
Latest commit History 8 Commits
.idea		.idea
Insta_Bot		Insta_Bot
Web_crawler		Web_crawler
domain.py		domain.py
general.py		general.py
link_finder.py		link_finder.py
main.py		main.py
spider.py		spider.py