A few questions on the implementation

[deviantintegral]

Hello, great to see this!

I've also been working on this on and off over the past month. So far it's all custom code in a single site. Here's a few notes and challenges I ran into, and I'm curious if you've hit them yet. I don't think you have because I don't see workarounds in the code.

1. For writing reports, how are permissions working? While I posted How to get the $uid / $gid variables in custom docker compose services? ddev/ddev#4733 about ZAP, we had exactly the same issue with Playwright.
2. Have you run into this bug yet with Firefox? This plus the above led me to stacking Playwright into the web container, instead of a separate service: [BUG] Running Playwright on Firefox inside a Docker container always hangs forever microsoft/playwright#16491
3. Some tools like https://github.com/chanzuckerberg/axe-storybook-testing also have a Playwright dependency. Any thoughts on how to run them inside the playwright container without having to double-download browsers?

At a high level, what we've ended up with is:

1. A separate yarn project for playwright in the project repo in test/playwright.
2. An additional dockerfile for the web service that is enabled via a ddev playwright command that copies it in. As well, a pre-start host hook that updates it if it already exists. Unfortunately, symlinks and hardlinks aren't working.
3. Adding a VNC server and xvfb for headed operations.
4. Lots of use of of build caches to speed up development and local rebuilds:

# Debian images by default disable apt caching, so turn it on until we finish
# the build.
RUN mv /etc/apt/apt.conf.d/docker-clean /etc/apt/docker-clean-disabled

# Install Playwright browsers and system dependencies.
#
# We want to install whatever browsers are used by the currently locked version
# of playwright in test/playwright. What this does is:
#
# 1. Creates a copy of the playwright directory in the image. We need this
#    because Playwright keeps a hidden .links/<hash> file with the path of
#    what browsers are installed by what apps. Without it, Playwright will re-
#    download browsers even though they are in the directory.
# 2. $username doesn't work in --mount... lines as it's a shell variable set by
#    ddev. Instead, we mount the .cache cache into /dot-cache, and symlink it
#    in.
# 3. Then we remove the symlink, and copy the cache directory into ~/.cache so
#    it is persisted in the image.
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
  --mount=type=cache,target=/var/lib/apt,sharing=locked \
  --mount=type=cache,target=/dot-cache,sharing=locked \
  --mount=type=bind,source=./test/playwright,target=/playwright \
  sudo -u $username mkdir -p /var/www/html/test \
  && cp -a /playwright /var/www/html/test \
  && chown -R $username:$username /var/www/html/test/playwright \
  && chown -R $username:$username /dot-cache \
  && sudo -u $username ln -s /dot-cache /home/$username/.cache \
  && cd /var/www/html/test/playwright \
  && sudo -u $username yarn playwright install-deps \
  && sudo -u $username yarn playwright install \
  && rm -rf /var/www/html/test/playwright \
  && rm /home/$username/.cache \
  && sudo -u $username mkdir -p /home/$username/.cache \
  && sudo -u $username cp -a /dot-cache/* /home/$username/.cache/

# Install a window manager and VNC server.
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
  --mount=type=cache,target=/var/lib/apt,sharing=locked \
  sudo apt-get install -y fluxbox \
    xauth \
    x11vnc

# We're done with apt so disable caching again for the final image.
RUN mv /etc/apt/docker-clean-disabled /etc/apt/apt.conf.d/docker-clean

# Install novnc to access the GUI over a browser.
ENV NOVNC_VERSION="1.4.0" \
    WEBSOCKIFY_VERSION="0.11.0"
RUN  wget -nv -O noVNC.zip \
       "https://github.com/novnc/noVNC/archive/refs/tags/v${NOVNC_VERSION}.zip" \
  && unzip -x noVNC.zip \
  && mkdir -p /opt/bin \
  && mv noVNC-${NOVNC_VERSION} /opt/bin/noVNC \
  && cp /opt/bin/noVNC/vnc.html /opt/bin/noVNC/index.html \
  && rm noVNC.zip \
  && wget -nv -O websockify.zip \
      "https://github.com/novnc/websockify/archive/refs/tags/v${WEBSOCKIFY_VERSION}.zip" \
  && unzip -x websockify.zip \
  && rm websockify.zip \
  && rm -rf websockify-${WEBSOCKIFY_VERSION}/tests \
  && mv websockify-${WEBSOCKIFY_VERSION} /opt/bin/noVNC/utils/websockify

# Copy start scripts for the x server, x11vnc, and novnc.
COPY .ddev/web-build/start-*.sh /usr/local/bin
RUN chmod +rx /usr/local/bin/start-*.sh

# novnc requires a password, so we set it to secret matching Selenium's images.
RUN sudo -u $username mkdir -p /home/$username/.vnc
RUN sudo -u $username x11vnc -storepasswd secret /home/$username/.vnc/passwd

[julienloizelet]

Hi @deviantintegral,
Thanks for your message.

I'm not sure if I can help you much, but I'll try to tell you how I use ddev with playwright. Maybe it will give you some clues.

First, to answer your questions, here is what I can say:

1. As you can see in the test.bats file of this repo, I use a command like ddev exec -s playwright yarn --cwd /var/www/html/yarn test --json --outputFile=./.test-results.json "__tests__/1-simple-test.js" to run test. As a result, the .test-results.json has the following permissions: -rw-r--r-- 1 root root. Since there are read rights, this has never been a problem for me. But I am not sure if you were talking about those kinds of reports.

2. Until now, I only test with Chromium. I just tried Firefox by adding playwright-firefox in the package.json file and modifying jest-playwright.config.js too by adding browsers: ["firefox"],devices: ["Desktop Firefox"]. I also had to run some ddev exec -s playwright apt-get install libxtst6 to make it work. Tests ran successfully, but this maybe too simple.

3. Sorry, but I have no idea for this :)

Then, if you are curious, and I have time, here is how I use playwright :

I only need javascript tools like playwright, jest, etc. to make end-to-end tests for some PHP development (mainly Magento 2 modules and Wordpress plugin but others PHP project too). And I wanted to be able to run test from my host (without headless) and from GitHub actions (headless)

I ended up with the following set up :

• All yarn and js files are in a tests/end-to-end folder in the source files of the tested plugin/module (maybe you can look at some WordPress plugin example)
• In the docker-compose.playwright.yaml file, you can see that the playwright service depends on the web service and that there is a volume : ../:/var/www/html
• To initialize tests (mainly yarn install), I run a script like this one
• And to run tests, I run a script like this one. This is a quite complicated script file and I don't even remember why I ended up with this...but it works for me :) . The interesting part is probably  ddev exec -s playwright xvfb-run --auto-servernum -- yarn --cwd ${YARN_PATH} cross-env for a "GitHub action" case.

If you want to see all the command for a GitHub action, you can take a look here.

Hope it could help.

Thanks

[deviantintegral]

I was curious why you needed playwright-firefox. I think that's because you're using jest for tests instead of playwright/test. Makes sense, and matches the note at https://www.npmjs.com/package/playwright-firefox. I'm really surprised you didn't run into Firefox hangs - but perhaps it's now been fixed upstream.

I'm going to post a repo with the setup we have, and then come back and do a comparison.

[deviantintegral]

Here's the repo: https://github.com/deviantintegral/drupal-playwright

[julienloizelet]

Hi @deviantintegral, thanks for the repo. Very complete !

For information, I just tested a full test suite with Firefox and some WordPress plugin I maintain.

Despite errors on some tests (which could be corrected, I guess, by adding some of the usual workarounds), some tests passed at 100% like this one : https://github.com/julienloizelet/cs-wordpress-bouncer/actions/runs/4508204588/jobs/7936683789

As you said, maybe the bug is fixed upstream.

[deviantintegral]

To follow up, I've been working on converting the example repo to a ddev addon, which I've done here: https://github.com/deviantintegral/ddev-playwright. We've been happy with the "all in one container" approach for a few months now, so I think we'll be sticking with it. Thanks for your feedback!