Playwright Cross Platform Considerations

[CodeBunTes]

After accidentally including the Sustainabilty package I'd been trying out on dev in a live deployment and finding myself taking down the site completely, I dug into cross platform support for this a little more given I develop and test on Windows but host the production site in Linux, as this didn't seem like something which should have been possible so easily. I thought I'd open a discussion around the findings to see if it can help provide some ways we can make this experience a bit better.

The problems
The biggest hurdles all seem to stem from how Playwright naturally works cross platform, which differ from how a lot of regular nuget packages function:-

• By default after installing the package, a publish will only work correctly on a single platform/architecture, usually likely to be the same one you're building/developing on. As part of the publish process in dotnet/visual studio, Playwright includes an executable for node which sits underneath the folder .playwright/node/architecture/. This executable is then used to install browsers and run tests and if missing will return a vague 'unable to find assets' error.
• To have it publish the node executables for other platforms you either need to make different publishes for each architecture, or manually edit the .csproj file to include the section and instruct it to include other/all architectures:-

     <PropertyGroup>
		<PlaywrightPlatform>all</PlaywrightPlatform>
    </PropertyGroup>

• Even with the other architectures included, you may still then run into an error running that included version of node. Windows is quite permissive, but on Linux the node executable is required to have the +x bit applied, which may require manually doing a chmod +x .playwright/node/linux-x64/node style command. If this is missing, as may often be the case if building or moving it from one platform to another, it'll still fail with another very vague 'Permission Denied' error.

Solutions?
All of these are inherent limits in how Playwright work (there are debates open about this, but they haven't managed to get much change). But is likely something a regular developer just installing a package in both the nuget and Umbraco world would not expect to have to think about and would assume everything would 'just work (TM)'. While hard to resolve directly, there are a few suggestions I have which may help.

• Change the SustainabilityComposer code to log without throwing an error so that at least if Playwright fails to load, it results in only the Sustainability Package tests not working in back office, rather than halting the entire site, which feels safer behaviour. From testing changing it to something similar to the following should work:-

            var logger = builder.BuilderLoggerFactory.CreateLogger<SustainabilityComposer>();
            try
            {
                var exitCode = Microsoft.Playwright.Program.Main(new[] { "install" });
                if (exitCode != 0)
                {
                    logger.LogError("Umbraco.Community.Sustainability failed to install Playwright browser due to exit code {exitCode}", exitCode);
                }
            }
            catch (Exception ex)
            {
                logger.LogError("Umbraco.Community.Sustainability failed to install Playwright browser due to exception {ex}", ex.ToString());
            }

• The try/catch function catches errors thrown by the executables being missing completely (fixing the 'unable to find assets' error), whereas the exitcode check will catch errors caused by the +x bit not being correctly set. Both of these cases will just then log the failure to Umbraco's regular logs (not sure if this is recommended to be done from a composer given ilogger is not dependency-injectable here, but it is possible to do). And then continue rather than throwing an exception which currently kills the whole Umbraco boot process
• Update the readme with a section on cross platform recommendations around ensuring the correct executables are always included, and the +x bit causing issues. Hopefully if tests can't be run but we've logged some errors, the dev may then go looking in the readme and see such notes then.

References

• Long running discussion around Playwright being quite unintuitive to setup cross platform - [Feature] More intuitive setup and installation of Playwright  microsoft/playwright-dotnet#2286
• Docs on the config option - https://playwright.dev/dotnet/docs/library#bundle-drivers-for-different-platforms

[tcmorris]

Hey @CodeBunTes thanks for raising the discussion and providing some notes on this. I must admit I've also been testing this on Windows so hasn't really been on my radar, will have a read through the options.

Agree we should definitely more gracefully handle the situation where Playwright fails to load, apologies for causing an issue with your live deployment! Our intention here is to have the package added and then do it's thing in the background for when you need to run some tests on website pages.

cc/ @rickbutterfield