Find inspiration in practice

magnet

Practice building a server-end & data-end through python.

Get Started

• Step 1. you need a Python3 environment.
• Step 2. you need a pip3 environment.
• Step 3. you need to create virtualenvs.
• Step 4. pip install following libs.

(magnet) [root@izuf69ibpdra8ui5a6edjtz magnet]# pipenv graph
cfscrape==2.0.7
  - requests [required: >=2.0.0, installed: 2.22.0]
    - certifi [required: >=2017.4.17, installed: 2019.6.16]
    - chardet [required: >=3.0.2,<3.1.0, installed: 3.0.4]
    - idna [required: >=2.5,<2.9, installed: 2.8]
    - urllib3 [required: >=1.21.1,<1.26,!=1.25.1,!=1.25.0, installed: 1.25.3]
Flask==1.1.1
  - click [required: >=5.1, installed: 7.0]
  - itsdangerous [required: >=0.24, installed: 1.1.0]
  - Jinja2 [required: >=2.10.1, installed: 2.10.1]
    - MarkupSafe [required: >=0.23, installed: 1.1.1]
  - Werkzeug [required: >=0.15, installed: 0.15.4]
gunicorn==19.9.0
lxml==4.3.4
selenium==3.141.0
  - urllib3 [required: Any, installed: 1.25.3]

• Step 5. run the following command to launch server.

$ . run.sh

Content

• Douban: web crawler from douban.com.
• Thunder: web crawler from 22tu.cc.
• Beiwo: web crawler from beiwo888.com.
• Piaohua: web crawler from piaohua.com.
• Baidu: web crawler from baidu.com.