Skip to content

kuonumber/crawler

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

80 Commits
.vscode
.vscode
 
 
pycrawler
pycrawler
 
 
.gitignore
.gitignore
 
 
D31_header.ipynb
D31_header.ipynb
 
 
Day018_HW.ipynb
Day018_HW.ipynb
 
 
Day021_HW.ipynb
Day021_HW.ipynb
 
 
Day022_HW.ipynb
Day022_HW.ipynb
 
 
Day025_MultiPage_Sample.ipynb
Day025_MultiPage_Sample.ipynb
 
 
Day026_Scrapy_init.ipynb
Day026_Scrapy_init.ipynb
 
 
Day027_Scrapy_XPath_item_Pipeline.ipynb
Day027_Scrapy_XPath_item_Pipeline.ipynb
 
 
Day028_Scrapy_api.ipynb
Day028_Scrapy_api.ipynb
 
 
Day029_Scrapy_muti_urls.ipynb
Day029_Scrapy_muti_urls.ipynb
 
 
Day030_anti_crawler_HW.ipynb
Day030_anti_crawler_HW.ipynb
 
 
Day032_HW_CAPTCHA_.ipynb
Day032_HW_CAPTCHA_.ipynb
 
 
Day033_cookie_session.ipynb
Day033_cookie_session.ipynb
 
 
Day034_proxy_HW.ipynb
Day034_proxy_HW.ipynb
 
 
Day035_muti_threads.ipynb
Day035_muti_threads.ipynb
 
 
Day036_HW_asyncio.ipynb
Day036_HW_asyncio.ipynb
 
 
Day037_scheduler.ipynb
Day037_scheduler.ipynb
 
 
Day10_homework.ipynb
Day10_homework.ipynb
 
 
Day12_homework_Ettoday_.ipynb
Day12_homework_Ettoday_.ipynb
 
 
Day13_homework_Ptt_.ipynb
Day13_homework_Ptt_.ipynb
 
 
Day14_homework_Yahoo_movie_.ipynb
Day14_homework_Yahoo_movie_.ipynb
 
 
Day15_homework_TW_bank.ipynb
Day15_homework_TW_bank.ipynb
 
 
Day16_homework_Wiki.ipynb
Day16_homework_Wiki.ipynb
 
 
Day17_homework.ipynb
Day17_homework.ipynb
 
 
Day19_homework.ipynb
Day19_homework.ipynb
 
 
Day20_homework.ipynb
Day20_homework.ipynb
 
 
Day8_homework.ipynb
Day8_homework.ipynb
 
 
Day9_homework_download_img.ipynb
Day9_homework_download_img.ipynb
 
 
Day_011_Regular_Expression_HW.ipynb
Day_011_Regular_Expression_HW.ipynb
 
 
Day_023_ETtoday_News_HW.ipynb
Day_023_ETtoday_News_HW.ipynb
 
 
Day_024_104HR_HW.ipynb
Day_024_104HR_HW.ipynb
 
 
README.md
README.md
 
 

Repository files navigation

basic knowledge

html

URL wki 講解
W3C 超連結屬性
關於 robots.txt

selector

quotes_toscrape- 練習網頁
Python Scrapy Tutorial-9-Extracting data w/CSS Selectors
Python Scrapy Tutorial-10-Extracting data w/XPATH
輕鬆學習 Python：在學習網站爬蟲之前

re

regexone
pythex
常用 Regular Expression 範例
regex101-解釋過程

tools for crawler

chrome dev tool
udemy Chrome 網頁除錯功能大解密
google/robotstxt
Python 爬蟲不求人之 Splash HTTP API 篇
selectorgadget - google extension

mutil-crawler

python-threading-multithreaded-programming
Python — 多線程
Python Semaphore does not seem to work in Google Colab
Raymond Hettinger, Keynote on Concurrency, PyBay 2017
Thinking about Concurrency, Raymond Hettinger, Python core developer
What Is Async, How Does It Work, and When Should I Use It? (PyCon APAC 2014)

proxy

4 款爬虫抓包神器
Free Proxy List
spys
最新中国ip地址代理服务器

Address to lat_lng

台灣電子地圖服務網
全國門牌地址定位服務
水利地理資訊平台
國土規劃地理資訊圖台

斷詞工具

CkipTagger
jieba-zh_TW

user-agent

fake-a-browser-visit
user_agent_string

About

No description, website, or topics provided.

Resources

Readme
Activity

Stars

1 star

Watchers

2 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages