Log all requests/responses (debug level)

[elacuesta]

Closes #33, please take a look @pawelmhm

Not adding a specific setting at the moment, since the logger could be made less verbose with

import logging
logging.getLogger("scrapy-playwright").setLevel(logging.INFO)

Sample output:

2021-10-18 12:13:23 [scrapy-playwright] INFO: Launching browser
2021-10-18 12:13:23 [scrapy-playwright] INFO: Browser chromium launched
2021-10-18 12:13:28 [scrapy-playwright] DEBUG: Browser context started: 'default'
2021-10-18 12:13:28 [scrapy-playwright] DEBUG: [Context=default] Request: <GET http://quotes.toscrape.com/scroll> (resource type: document, referrer: None)
2021-10-18 12:13:29 [scrapy-playwright] DEBUG: [Context=default] Response: <200 http://quotes.toscrape.com/scroll> (referrer: None)
2021-10-18 12:13:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET http://quotes.toscrape.com/static/bootstrap.min.css> (resource type: stylesheet, referrer: http://quotes.toscrape.com/scroll)
2021-10-18 12:13:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET http://quotes.toscrape.com/static/main.css> (resource type: stylesheet, referrer: http://quotes.toscrape.com/scroll)
2021-10-18 12:13:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET http://quotes.toscrape.com/static/jquery.js> (resource type: script, referrer: http://quotes.toscrape.com/scroll)
2021-10-18 12:13:29 [scrapy-playwright] DEBUG: [Context=default] Response: <200 http://quotes.toscrape.com/static/bootstrap.min.css> (referrer: None)
2021-10-18 12:13:29 [scrapy-playwright] DEBUG: [Context=default] Response: <200 http://quotes.toscrape.com/static/main.css> (referrer: None)
2021-10-18 12:13:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://fonts.googleapis.com/css?family=Raleway:400,700> (resource type: stylesheet, referrer: http://quotes.toscrape.com/)
2021-10-18 12:13:29 [scrapy-playwright] DEBUG: [Context=default] Response: <200 http://quotes.toscrape.com/static/jquery.js> (referrer: None)
2021-10-18 12:13:29 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://fonts.googleapis.com/css?family=Raleway:400,700> (referrer: None)
2021-10-18 12:13:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET https://fonts.gstatic.com/s/raleway/v22/1Ptxg8zYS_SKggPN4iEgvnHyvveLxVs9pbCIPrQ.ttf> (resource type: font, referrer: https://fonts.googleapis.com/)
2021-10-18 12:13:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET http://quotes.toscrape.com/api/quotes?page=1> (resource type: xhr, referrer: http://quotes.toscrape.com/scroll)
2021-10-18 12:13:29 [scrapy-playwright] DEBUG: [Context=default] Response: <200 https://fonts.gstatic.com/s/raleway/v22/1Ptxg8zYS_SKggPN4iEgvnHyvveLxVs9pbCIPrQ.ttf> (referrer: None)
2021-10-18 12:13:29 [scrapy-playwright] DEBUG: [Context=default] Response: <200 http://quotes.toscrape.com/api/quotes?page=1> (referrer: None)
2021-10-18 12:13:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET http://quotes.toscrape.com/api/quotes?page=2> (resource type: xhr, referrer: http://quotes.toscrape.com/scroll)
2021-10-18 12:13:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET http://quotes.toscrape.com/api/quotes?page=3> (resource type: xhr, referrer: http://quotes.toscrape.com/scroll)
2021-10-18 12:13:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET http://quotes.toscrape.com/api/quotes?page=4> (resource type: xhr, referrer: http://quotes.toscrape.com/scroll)
2021-10-18 12:13:29 [scrapy-playwright] DEBUG: [Context=default] Request: <GET http://quotes.toscrape.com/api/quotes?page=5> (resource type: xhr, referrer: http://quotes.toscrape.com/scroll)
2021-10-18 12:13:30 [scrapy-playwright] DEBUG: [Context=default] Request: <GET http://quotes.toscrape.com/api/quotes?page=6> (resource type: xhr, referrer: http://quotes.toscrape.com/scroll)
2021-10-18 12:13:30 [scrapy-playwright] DEBUG: [Context=default] Request: <GET http://quotes.toscrape.com/api/quotes?page=7> (resource type: xhr, referrer: http://quotes.toscrape.com/scroll)
2021-10-18 12:13:30 [scrapy-playwright] DEBUG: [Context=default] Request: <GET http://quotes.toscrape.com/api/quotes?page=8> (resource type: xhr, referrer: http://quotes.toscrape.com/scroll)
2021-10-18 12:13:30 [scrapy-playwright] DEBUG: [Context=default] Request: <GET http://quotes.toscrape.com/api/quotes?page=9> (resource type: xhr, referrer: http://quotes.toscrape.com/scroll)
2021-10-18 12:13:30 [scrapy-playwright] DEBUG: [Context=default] Request: <GET http://quotes.toscrape.com/api/quotes?page=10> (resource type: xhr, referrer: http://quotes.toscrape.com/scroll)
2021-10-18 12:13:30 [scrapy-playwright] DEBUG: [Context=default] Request: <GET http://quotes.toscrape.com/api/quotes?page=11> (resource type: xhr, referrer: http://quotes.toscrape.com/scroll)
2021-10-18 12:13:30 [scrapy-playwright] DEBUG: [Context=default] Request: <GET http://quotes.toscrape.com/api/quotes?page=12> (resource type: xhr, referrer: http://quotes.toscrape.com/scroll)
2021-10-18 12:13:30 [scrapy-playwright] DEBUG: [Context=default] Response: <200 http://quotes.toscrape.com/api/quotes?page=2> (referrer: None)
2021-10-18 12:13:30 [scrapy-playwright] DEBUG: [Context=default] Response: <200 http://quotes.toscrape.com/api/quotes?page=3> (referrer: None)
2021-10-18 12:13:30 [scrapy-playwright] DEBUG: [Context=default] Response: <200 http://quotes.toscrape.com/api/quotes?page=4> (referrer: None)
2021-10-18 12:13:30 [scrapy-playwright] DEBUG: [Context=default] Response: <200 http://quotes.toscrape.com/api/quotes?page=5> (referrer: None)
2021-10-18 12:13:30 [scrapy-playwright] DEBUG: [Context=default] Response: <200 http://quotes.toscrape.com/api/quotes?page=6> (referrer: None)
2021-10-18 12:13:30 [scrapy-playwright] DEBUG: [Context=default] Response: <200 http://quotes.toscrape.com/api/quotes?page=7> (referrer: None)
2021-10-18 12:13:30 [scrapy-playwright] DEBUG: [Context=default] Response: <200 http://quotes.toscrape.com/api/quotes?page=8> (referrer: None)
2021-10-18 12:13:30 [scrapy-playwright] DEBUG: [Context=default] Response: <200 http://quotes.toscrape.com/api/quotes?page=9> (referrer: None)
2021-10-18 12:13:30 [scrapy-playwright] DEBUG: [Context=default] Response: <200 http://quotes.toscrape.com/api/quotes?page=10> (referrer: None)
2021-10-18 12:13:30 [scrapy-playwright] DEBUG: [Context=default] Response: <200 http://quotes.toscrape.com/api/quotes?page=11> (referrer: None)
2021-10-18 12:13:30 [scrapy-playwright] DEBUG: [Context=default] Response: <200 http://quotes.toscrape.com/api/quotes?page=12> (referrer: None)
2021-10-18 12:13:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET http://quotes.toscrape.com/scroll> (referer: None) ['playwright']
2021-10-18 12:13:31 [scrapy.core.scraper] DEBUG: Scraped from <200 http://quotes.toscrape.com/scroll>
{'url': 'http://quotes.toscrape.com/scroll', 'count': 100}
2021-10-18 12:13:31 [scrapy.core.engine] INFO: Closing spider (finished)

[codecov]

Codecov Report

Merging #37 (44d39ce) into master (9981cd8) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##            master       #37   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            3         3           
  Lines          169       179   +10     
=========================================
+ Hits           169       179   +10     

Impacted Files	Coverage Δ
scrapy_playwright/handler.py	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9981cd8...44d39ce. Read the comment docs.