Skip to content

Commit

Permalink
Pass request to page init callback
Browse files Browse the repository at this point in the history
  • Loading branch information
elacuesta committed Oct 3, 2022
1 parent f9daf64 commit 980fe66
Show file tree
Hide file tree
Showing 5 changed files with 8 additions and 7 deletions.
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -234,13 +234,13 @@ TWISTED_REACTOR = "twisted.internet.asyncioreactor.AsyncioSelectorReactor"
* `playwright_page_init_callback` (type `Optional[Union[Callable, str]]`, default `None`)

A coroutine function (`async def`) to be invoked immediately after creating
a page for the request. It receives the page as its only positional
argument. Useful for initialization code. Invoked only for newly created
a page for the request. It receives the page and the request as positional
arguments. Useful for initialization code. Invoked only for newly created
pages, ignored if the page for the request already exists (e.g. by passing
`playwright_page`).

```python
async def init_page(page):
async def init_page(page, request):
await page.add_init_script(path="./custom_script.js")

class AwesomeSpider(scrapy.Spider):
Expand Down
1 change: 1 addition & 0 deletions changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@

* Remove deprecated code (`PageCoroutine` class, `playwright_page_coroutines` request meta key,
`use_playwright_headers` function).
* `playwright_page_init_callback` meta key (page initialization callback)


### [v0.0.21](https://github.com/scrapy-plugins/scrapy-playwright/releases/tag/v0.0.21) (2022-08-08)
Expand Down
2 changes: 1 addition & 1 deletion examples/init_page.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
import scrapy


async def init_page(page):
async def init_page(page, request):
await page.set_extra_http_headers({"Asdf": "Qwerty"})


Expand Down
2 changes: 1 addition & 1 deletion scrapy_playwright/handler.py
Original file line number Diff line number Diff line change
Expand Up @@ -191,7 +191,7 @@ async def _create_page(self, request: Request) -> Page:
if page_init_callback:
try:
page_init_callback = load_object(page_init_callback)
await page_init_callback(page)
await page_init_callback(page, request)
except Exception as ex:
logger.warning(
"[Context=%s] Page init callback exception for %s (%s)",
Expand Down
4 changes: 2 additions & 2 deletions tests/test_playwright_requests.py
Original file line number Diff line number Diff line change
Expand Up @@ -400,7 +400,7 @@ def should_abort_request_sync(request):

@pytest.mark.asyncio
async def test_page_initialization_ok(self, caplog):
async def init_page(page):
async def init_page(page, request):
await page.set_extra_http_headers({"Extra-Header": "Qwerty"})

settings_dict = {
Expand All @@ -421,7 +421,7 @@ async def init_page(page):

@pytest.mark.asyncio
async def test_page_initialization_fail(self, caplog):
async def init_page(page, unused_arg):
async def init_page(page, request, unused_arg):
await page.set_extra_http_headers({"Extra-Header": "Qwerty"})

settings_dict = {
Expand Down

0 comments on commit 980fe66

Please sign in to comment.