Support importing behaviors from the new Chrome dev tools Recorder panel JSON export format

[pirate]

Chrome recently added in v101 a new framework-agnostic JSON user script export format for their Recording pane.

https://developer.chrome.com/docs/devtools/recorder/reference/#export-flows

It used to only support generating puppeteer JS scripts, but now with the JSON format it can be imported into a wide variety of tools without needing to translate puppeteer JS scripts into playwright/other drivers.

Here's a sample of the JSON format they generate:

{
    "title": "Test Recording 4/11/2023 at 5:15:18 PM",
    "steps": [
        {
            "type": "setViewport",
            "width": 1869,
            "height": 264,
            "deviceScaleFactor": 1,
            "isMobile": false,
            "hasTouch": false,
            "isLandscape": false
        },
        {
            "type": "navigate",
            "assertedEvents": [
                {
                    "type": "navigation",
                    "url": "https://jec.fyi/demo/recorder",
                    "title": "Puppeteer recorder - test page"
                }
            ],
            "url": "https://jec.fyi/demo/recorder"
        },
        {
            "type": "click",
            "target": "main",
            "selectors": [
                [
                    "[data-testid='email']"
                ],
                [
                    "xpath///*[@data-testid=\"email\"]"
                ],
                [
                    "pierce/[data-testid='email']"
                ]
            ],
            "offsetX": 58.1776123046875,
            "offsetY": 8.579559326171875
        },
        {
            "type": "change",
            "target": "main",
            "selectors": [
                [
                    "[data-testid='email']"
                ],
                [
                    "xpath///*[@data-testid=\"email\"]"
                ],
                [
                    "pierce/[data-testid='email']"
                ]
            ],
            "value": "[email protected]"
        }
    ]
}

---

(honestly I recommend opening an issue upstream (done: microsoft/playwright#22345) to add this JSON importing feature to playwright, many people will likely want this)

Replaying docs for playwright where it would be documented if added: https://playwright.dev/docs/next/network#record-and-replay-requests
Chrome docs where it would be documented if playwright added support for replaying this JSON: https://developer.chrome.com/docs/devtools/recorder/reference/#replay-with-external-libraries

[ikreymer]

@pirate thanks for sharing this, potentially exciting that there is a standalone JSON format that's not tied to puppeteer. I wonder if there is a spec for it. There's potentially a few different paths:

• Wait for it to be support in playwright (now that we've switched to playwright)
• Implement support in Browsertrix-Crawler directly
• Explore option of custom extension? I see there's the "Get Extensions" option, which we could implement our own?
• Support in archiveweb.page
• Possible integrate with Browsertrix Behaviors

Very curious about the customization possibility that exists for "Get Extensions".
cc: @Shrinks99 you may be interested in this from design point
cc: @lambdahands something to explore re: behaviors integration?

[pirate]

I just opened an issue in the main playwright repo: microsoft/playwright#22345

I'd give them a few weeks to respond before doing anything to add it to browsertrix-crawler, I'd be very surprised if it takes more than a few months for someone to contribute this to playwright given how useful it would be.

Explore option of custom extension? I see there's the "Get Extensions" option, which we could implement our own?

Yeah here's the docs on extensions and extending Recorder functionality:

• https://developer.chrome.com/docs/devtools/recorder/extensions/#export-extensions
• https://developer.chrome.com/blog/extend-recorder/

Here's an example Chrome extension that implements a custom conversion that's available built into the Dev Tools panel export menu: https://github.com/kobenguyent/codeceptjs-chrome-recorder/blob/main/src/main.ts

(Landing native Browsertrix Behavior export from Chrome dev tools would be awesome with an extension like this)

[ikreymer]

Should also evaluate the general applicability of this, beyond a single page. In some ways, this is similar to what Memento Tracer was trying to do. The overall behavior system is still probably the more general solution, but this might be a useful subset that can be supported within that.