TypeScript Openai Puppeteer Web Scraper Template

This is a template repository for building a web scraper using Openai, TypeScript and Puppeteer. With this template, you can quickly set up a project structure to scrape data, save it to a MySQL database and send emails with the data. This template is designed to be used with the Openai platform, but can be used with any other platform that supports TypeScript.

Getting Started

To use this template, you can simply click the "Use this template" button on GitHub, or fork the repository to your own account.

Once you have the repository on your local machine, you can install the dependencies using Yarn or npm:

yarn install
# or
npm install

To compile the scraper, you can use the start command:

npm run compile
# or
npm run dev-compile

To run the scraper, you can use the start command:

yarn start
# or
npm start

This will run the scraper.ts file, which contains the basic code to launch a headless browser and navigate to a web page. You can modify this file to add your own scraping logic.

TypeScript Support

This template includes TypeScript support out of the box. You can write your scraping code in TypeScript and run it using the ts-node package.

Puppeteer

This template includes the Puppeteer package, which provides a high-level API for controlling headless Chrome or Chromium. You can use Puppeteer to navigate web pages, interact with DOM elements, and extract data.

Contributing

If you find any issues with this template, or have suggestions for improvement, please feel free to open an issue or a pull request. We welcome contributions from the community!