Simple web crawler script using recursion and puppeteer.
Instructions:
- run
yarnornpm install - Configure .env file (see .env_example)
- URL is required
- DEPTH is not required but suggested
- PROXY is not required
- run
yarn startornpm start
-
Notifications
You must be signed in to change notification settings - Fork 0
A web crawler with Node.js and Puppeteer
License
SeanDoh/NodeCrawler
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
About
A web crawler with Node.js and Puppeteer
Resources
License
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published