Browser storage for authentication handling

[thernstig]

Hi!

https://github.com/microsoft/playwright/releases/tag/v1.7.0 added the browser storage API. How would this be used with this package? Could it be described somewhere in the docs, or is it not possible?

[elacuesta]

Hi @thernstig, sorry for taking so long. Is the storage.py example script introduced in 8c3ea71 useful to you?
Here's are the relevant bits:

PageCoroutine("evaluate_handle", "window.localStorage.setItem('foo', 'bar');")

$ python examples/storage.py
(...)
2021-03-26 21:14:12 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://example.org> (referer: None) ['playwright']
2021-03-26 21:14:13 [scrapy.core.scraper] DEBUG: Scraped from <200 https://example.org/>
{'url': 'https://example.org/', 'storage_state': {'cookies': [], 'origins': [{'origin': 'https://example.org', 'localStorage': [{'name': 'foo', 'value': 'bar'}]}]}}
(...)

[thernstig]

I am not sure, it seems the Playwright storage API is a bit more powerful in saving all state easily. (cookies, local storage, session storage, IndexedDB etc.)

[elacuesta]

It's only possible to pass storage state to Browser.new_page and Browser.new_context (meaning it's possible to set via the PLAYWRIGHT_CONTEXT_ARGS setting). However, the BrowserContext.new_page method does not take any arguments, so there is currently no way to set storage state when creating new pages.

[thernstig]

I think setting storage state per context is the right approach.

[elacuesta]

Besides defining values on startup as currently possible, #13 will allow to set storage data on dynamically created contexts.