Adds build steps in makefile for building Chrome and Firefox betas (#…

…1658)

* Added build steps to build Chrome Beta and Firefox Beta nodes and standalone from the makefile.

* Adding steps to deploy the beta browser versions with basic TAG_VERSION

* Create update-dev-beta-browser-images.yml

* fix typo

* removed a generate task from Makefile. Scraping selenium version from the Base/Dockerfile.

* Fixed dependency for standalone beta images. Added in edge, including accidentally deleted standalone_edge...

* Make sure we're building the beta versions when testing them

* Added edge to makefile, also added support for dev versions of the browsers. Also, retagging node-base and hub as dev/beta to make the testing process easier.

* google chrome dev channel is 'google-chrome-unstable'

* Modified NodeChrome Dockerfile to try back one more chromedriver version if the one for dev is not available. Also, added matrix strategy with fail-fast false to build, test, and deploy whatever platforms pass testing.

* Move matrix environment variables from step to job

* try again

* debug why it is failing

* Fix typo

* fix typo

* fix typo

* tag base with channel name as well

* Fix typo

* If dev chromedriver is not available, try with the one previous - fixing the logic on this

* Successful build, but I think the non-dev images are being built prior to testing, so adding SKIP_BUILD argument from seleniarm repo. Adding deploy steps.

* tag node-base and hub and pull from docker hub for faster building and testing. Output browser and driver versions in the logs

* Eliminate rebuilding the images by removing the Makefile node_base dependency.

* Removed unneeded steps and replace NAME with 'selenium'

* Fix note capitalization at the end. Added schedule cron to trigger every 2 days.

* only geckodriver version needs to be displayed

* change namespace to selenium and remove workflow dispatch

.github/workflows/update-dev-beta-browser-images.yml

@@ -0,0 +1,85 @@
+name: Update Dev/Beta Browser Images
+
+on:
+  schedule:
+    # Trigger build every 2 days
+    - cron: '0 2 */2 * *'
+
+jobs:
+
+  deploy:
+
+    runs-on: ubuntu-latest
+    strategy: 
+      fail-fast: false
+      matrix:
+        browser: [chrome,firefox,edge]
+        channel: [dev,beta]
+    env:
+      NAME: selenium
+      BROWSER: ${{ matrix.browser }}
+      CHANNEL: ${{ matrix.channel }}
+
+    steps:
+    - uses: actions/checkout@v3
+    - name: Setup environment variables
+      run: |
+        export SELENIUM_VERSION=$(grep selenium-server Base/Dockerfile | sed 's/.*-\([^-]*\)\.jar \\/\1/' | head -n 1)
+        echo "SELENIUM_VERSION="$SELENIUM_VERSION >> $GITHUB_ENV
+        export BUILD_DATE=$(date '+%Y%m%d')
+        echo "BUILD_DATE="$BUILD_DATE >> $GITHUB_ENV
+        export BROWSER_CAPS=`node -p "process.argv[1][0].toUpperCase() + process.argv[1].toString().substring(1)" $BROWSER`
+        echo "BROWSER_CAPS="$BROWSER_CAPS >> $GITHUB_ENV
+        export CHANNEL_CAPS=`node -p "process.argv[1][0].toUpperCase() + process.argv[1].toString().substring(1)" $CHANNEL`
+        echo "CHANNEL_CAPS="$CHANNEL_CAPS >> $GITHUB_ENV
+        echo "BROWSER is $BROWSER"
+        echo "CHANNEL is $CHANNEL"
+        echo "SELENIUM_VERSION is $SELENIUM_VERSION"
+        echo "BUILD_DATE is $BUILD_DATE"
+
+    - name: Pull hub and node-base and tag them for faster building and testing
+      run: |
+        docker pull $NAME/hub:latest
+        docker pull $NAME/node-base:latest
+        docker tag $NAME/hub:latest $NAME/hub:$CHANNEL
+        docker tag $NAME/node-base:latest $NAME/node-base:$CHANNEL
+        docker tag $NAME/hub:latest $NAME/hub:"$SELENIUM_VERSION"-"$BUILD_DATE"
+        docker tag $NAME/node-base:latest $NAME/node-base:"$SELENIUM_VERSION"-"$BUILD_DATE"
+        
+    - name: Build the Dev/Beta Docker container images
+      run: |
+        echo VERSION=$SELENIUM_VERSION make $BROWSER_$CHANNEL standalone_"$BROWSER"_"$CHANNEL"
+        VERSION=$SELENIUM_VERSION make "$BROWSER"_"$CHANNEL" standalone_"$BROWSER"_"$CHANNEL"        
+
+    - name: Test the Dev/Beta Docker container images
+      run: |
+        export SKIP_BUILD=true
+        export NAMESPACE=$NAME
+        VERSION=$CHANNEL ./tests/bootstrap.sh Node$BROWSER_CAPS
+        VERSION=$CHANNEL ./tests/bootstrap.sh Standalone$BROWSER_CAPS
+
+    - name: Login Docker Hub
+      run: docker login -u="$DOCKER_USERNAME" -p="$DOCKER_PASSWORD"
+      env:
+        DOCKER_USERNAME: ${{secrets.DOCKER_USERNAME}}
+        DOCKER_PASSWORD: ${{secrets.DOCKER_PASSWORD}}  
+
+    - name: Deploy Dev/Beta Docker container images  
+      run: |
+        docker push $NAME/node-$BROWSER:$CHANNEL
+        docker push $NAME/standalone-$BROWSER:$CHANNEL
+        if [ "$BROWSER" = "firefox" ]; then
+          export DRIVER_VERSION_COMMAND="/usr/bin/geckodriver --version | head -n 1"
+          export BROWSER_VERSION_COMMAND="firefox -version"
+        elif [ "$BROWSER" = "chrome" ]; then
+          export DRIVER_VERSION_COMMAND="/usr/bin/chromedriver -version"
+          export BROWSER_VERSION_COMMAND="google-chrome -version"
+        elif [ "$BROWSER" = "edge" ]; then
+          export DRIVER_VERSION_COMMAND="/usr/bin/msedgedriver -version"
+          export BROWSER_VERSION_COMMAND="microsoft-edge -version"
+        fi
+        echo "Push to Docker Hub completed"
+        echo "$BROWSER_CAPS $CHANNEL_CAPS browser version:"
+        docker run --rm $NAME/standalone-$BROWSER:$CHANNEL bash -c "$BROWSER_VERSION_COMMAND"
+        echo "$BROWSER_CAPS $CHANNEL_CAPS WebDriver version:"
+        docker run --rm $NAME/standalone-$BROWSER:$CHANNEL bash -c "$DRIVER_VERSION_COMMAND"

Makefile

@@ -61,12 +61,30 @@ node_base: base
 chrome: node_base
 	cd ./NodeChrome && docker build $(BUILD_ARGS) $(FROM_IMAGE_ARGS) -t $(NAME)/node-chrome:$(TAG_VERSION) .
 
+chrome_dev:
+	cd ./NodeChrome && docker build $(BUILD_ARGS) $(FROM_IMAGE_ARGS) --build-arg CHROME_VERSION=google-chrome-unstable -t $(NAME)/node-chrome:dev .
+
+chrome_beta:
+	cd ./NodeChrome && docker build $(BUILD_ARGS) $(FROM_IMAGE_ARGS) --build-arg CHROME_VERSION=google-chrome-beta -t $(NAME)/node-chrome:beta .
+
 edge: node_base
 	cd ./NodeEdge && docker build $(BUILD_ARGS) $(FROM_IMAGE_ARGS) -t $(NAME)/node-edge:$(TAG_VERSION) .
 
+edge_dev:
+	cd ./NodeEdge && docker build $(BUILD_ARGS) $(FROM_IMAGE_ARGS) --build-arg EDGE_VERSION=microsoft-edge-dev -t $(NAME)/node-edge:dev .
+
+edge_beta:
+	cd ./NodeEdge && docker build $(BUILD_ARGS) $(FROM_IMAGE_ARGS) --build-arg EDGE_VERSION=microsoft-edge-beta -t $(NAME)/node-edge:beta .
+
 firefox: node_base
 	cd ./NodeFirefox && docker build $(BUILD_ARGS) $(FROM_IMAGE_ARGS) -t $(NAME)/node-firefox:$(TAG_VERSION) .
 
+firefox_dev:
+	cd ./NodeFirefox && docker build $(BUILD_ARGS) $(FROM_IMAGE_ARGS) --build-arg FIREFOX_VERSION=devedition-latest -t $(NAME)/node-firefox:dev .
+
+firefox_beta:
+	cd ./NodeFirefox && docker build $(BUILD_ARGS) $(FROM_IMAGE_ARGS) --build-arg FIREFOX_VERSION=beta-latest -t $(NAME)/node-firefox:beta .
+
 docker: base
 	cd ./NodeDocker && docker build $(BUILD_ARGS) $(FROM_IMAGE_ARGS) -t $(NAME)/node-docker:$(TAG_VERSION) .
 
@@ -76,12 +94,30 @@ standalone_docker: docker
 standalone_firefox: firefox
 	cd ./Standalone && docker build $(BUILD_ARGS) $(FROM_IMAGE_ARGS) --build-arg BASE=node-firefox -t $(NAME)/standalone-firefox:$(TAG_VERSION) .
 
+standalone_firefox_dev: firefox_dev
+	cd ./Standalone && docker build $(BUILD_ARGS) --build-arg NAMESPACE=$(NAME) --build-arg VERSION=dev --build-arg BASE=node-firefox -t $(NAME)/standalone-firefox:dev .
+
+standalone_firefox_beta: firefox_beta
+	cd ./Standalone && docker build $(BUILD_ARGS) --build-arg NAMESPACE=$(NAME) --build-arg VERSION=beta --build-arg BASE=node-firefox -t $(NAME)/standalone-firefox:beta .
+
 standalone_chrome: chrome
 	cd ./Standalone && docker build $(BUILD_ARGS) $(FROM_IMAGE_ARGS) --build-arg BASE=node-chrome -t $(NAME)/standalone-chrome:$(TAG_VERSION) .
 
+standalone_chrome_dev: chrome_dev
+	cd ./Standalone && docker build $(BUILD_ARGS) --build-arg NAMESPACE=$(NAME) --build-arg VERSION=dev --build-arg BASE=node-chrome -t $(NAME)/standalone-chrome:dev .
+
+standalone_chrome_beta: chrome_beta
+	cd ./Standalone && docker build $(BUILD_ARGS) --build-arg NAMESPACE=$(NAME) --build-arg VERSION=beta --build-arg BASE=node-chrome -t $(NAME)/standalone-chrome:beta .
+
 standalone_edge: edge
 	cd ./Standalone && docker build $(BUILD_ARGS) $(FROM_IMAGE_ARGS) --build-arg BASE=node-edge -t $(NAME)/standalone-edge:$(TAG_VERSION) .
 
+standalone_edge_dev: edge_dev
+	cd ./Standalone && docker build $(BUILD_ARGS) --build-arg NAMESPACE=$(NAME) --build-arg VERSION=dev --build-arg BASE=node-edge -t $(NAME)/standalone-edge:dev .
+
+standalone_edge_beta: edge_beta
+	cd ./Standalone && docker build $(BUILD_ARGS) --build-arg NAMESPACE=$(NAME) --build-arg VERSION=beta --build-arg BASE=node-edge -t $(NAME)/standalone-edge:beta .
+
 video:
 	cd ./Video && docker build $(BUILD_ARGS) -t $(NAME)/video:$(FFMPEG_TAG_VERSION)-$(BUILD_DATE) .
 

NodeChrome/Dockerfile

@@ -42,7 +42,12 @@ USER 1200
 ARG CHROME_DRIVER_VERSION
 RUN if [ -z "$CHROME_DRIVER_VERSION" ]; \
   then CHROME_MAJOR_VERSION=$(google-chrome --version | sed -E "s/.* ([0-9]+)(\.[0-9]+){3}.*/\1/") \
-    && CHROME_DRIVER_VERSION=$(wget --no-verbose -O - "https://chromedriver.storage.googleapis.com/LATEST_RELEASE_${CHROME_MAJOR_VERSION}"); \
+    && NO_SUCH_KEY=$(curl -ls https://chromedriver.storage.googleapis.com/LATEST_RELEASE_${CHROME_MAJOR_VERSION} | head -n 1 | grep -oe NoSuchKey) ; \
+    if [ -n "$NO_SUCH_KEY" ]; then \
+      echo "No Chromedriver for version $CHROME_MAJOR_VERSION. Use previous major version instead" \
+      && CHROME_MAJOR_VERSION=$(expr $CHROME_MAJOR_VERSION - 1); \
+    fi ; \
+    CHROME_DRIVER_VERSION=$(wget --no-verbose -O - "https://chromedriver.storage.googleapis.com/LATEST_RELEASE_${CHROME_MAJOR_VERSION}"); \
   fi \
   && echo "Using chromedriver version: "$CHROME_DRIVER_VERSION \
   && wget --no-verbose -O /tmp/chromedriver_linux64.zip https://chromedriver.storage.googleapis.com/$CHROME_DRIVER_VERSION/chromedriver_linux64.zip \

NodeFirefox/Dockerfile

@@ -10,7 +10,7 @@ USER root
 # Firefox
 #=========
 ARG FIREFOX_VERSION=latest
-RUN FIREFOX_DOWNLOAD_URL=$(if [ $FIREFOX_VERSION = "latest" ] || [ $FIREFOX_VERSION = "nightly-latest" ] || [ $FIREFOX_VERSION = "devedition-latest" ] || [ $FIREFOX_VERSION = "esr-latest" ]; then echo "https://download.mozilla.org/?product=firefox-$FIREFOX_VERSION-ssl&os=linux64&lang=en-US"; else echo "https://download-installer.cdn.mozilla.net/pub/firefox/releases/$FIREFOX_VERSION/linux-x86_64/en-US/firefox-$FIREFOX_VERSION.tar.bz2"; fi) \
+RUN FIREFOX_DOWNLOAD_URL=$(if [ $FIREFOX_VERSION = "latest" ] || [ $FIREFOX_VERSION = "beta-latest" ] || [ $FIREFOX_VERSION = "nightly-latest" ] || [ $FIREFOX_VERSION = "devedition-latest" ] || [ $FIREFOX_VERSION = "esr-latest" ]; then echo "https://download.mozilla.org/?product=firefox-$FIREFOX_VERSION-ssl&os=linux64&lang=en-US"; else echo "https://download-installer.cdn.mozilla.net/pub/firefox/releases/$FIREFOX_VERSION/linux-x86_64/en-US/firefox-$FIREFOX_VERSION.tar.bz2"; fi) \
   && apt-get update -qqy \
   && apt-get -qqy --no-install-recommends install firefox libavcodec-extra \
   && rm -rf /var/lib/apt/lists/* /var/cache/apt/* \

tests/test.py

@@ -22,6 +22,7 @@
 http_proxy = os.environ.get('http_proxy', '')
 https_proxy = os.environ.get('https_proxy', '')
 no_proxy = os.environ.get('no_proxy', '')
+SKIP_BUILD = os.environ.get('SKIP_BUILD', False)
 
 IMAGE_NAME_MAP = {
     # Hub
@@ -108,15 +109,19 @@ def launch_container(container, **kwargs):
     :param container:
     :return: the container ID
     """
-    # Build the container if it doesn't exist
-    logger.info("Building %s container..." % container)
-    set_from_image_base_for_standalone(container)
-    build_path = get_build_path(container)
-    client.images.build(path='../%s' % build_path,
-                        tag="%s/%s:%s" % (NAMESPACE, IMAGE_NAME_MAP[container], VERSION),
-                        rm=True,
-                        buildargs=FROM_IMAGE_ARGS)
-    logger.info("Done building %s" % container)
+    skip_building_images = SKIP_BUILD == 'true'
+    if skip_building_images:
+        logger.info("SKIP_BUILD is true...not rebuilding images...")
+    else:
+        # Build the container if it doesn't exist
+        logger.info("Building %s container..." % container)
+        set_from_image_base_for_standalone(container)
+        build_path = get_build_path(container)
+        client.images.build(path='../%s' % build_path,
+                            tag="%s/%s:%s" % (NAMESPACE, IMAGE_NAME_MAP[container], VERSION),
+                            rm=True,
+                            buildargs=FROM_IMAGE_ARGS)
+        logger.info("Done building %s" % container)
 
     # Run the container
     logger.info("Running %s container..." % container)