Django docs parser

Requirements:

• Python 3.4+

Installation and usage

python3 -m venv env
source env/bin/activate
pip install -r requirements.txt
python parser.py

The above commands will make a virtual environment in a folder called env and install all the requirements listed in requirements.txt into that virtual environment. Once that is done running python parser.py will go to the Django documentation for versions 4.0, 3.2, 3.1, 3.0, 2.2, 2.1, 2.0, 1.11, 1.10 and 1.8, grab their zipped HTML versions and parse them respectively. It will output a file called data.json which you can later use to your avail.

The output JSON file looks like this:

[
  {
    "version": 2.2,
    "id": "django.contrib.postgres.fields.ArrayField",
    "title": "ArrayField",
    "permalink": "https://docs.djangoproject.com/en/2.2/ref/contrib/postgres/fields/#django.contrib.postgres.fields.ArrayField",
    "categories": [
        "class ",
        "contrib",
        "postgres",
        "fields"
    ],
    "content": "A field for storing lists of data. Most field types can be used, you simply pass another field instance as the base_field. You may also specify a size. ArrayField can be nested to store multi-dimensional arrays."
  }
]