selenium/standalone-chrome-debug can't download files

[piscisferro]

Meta -

Image(s): selenium/standalone-chrome-debug
Docker-Selenium Image Version(s): From 3.4.0-einsteinium to 3.5.2
OS: Ubuntu 16.04

Description

I will explain the setup: I have 2 docker containers, one created by me based in anapsix/alpine-java and this chrome-standalone-debug container connected to a docker network with more containers. With docker-compose I create a volume for both chrome and java container to allow share files between them and my local.

One of the test should download a few files, store it in the specified volume folder and then check if that file exist and is downloaded.

It worked fine until a few days ago, it stopped to work (probably because an update, I was using latest as a tag) and the file didn't download anymore. I tested the code in all versions from 3.4.0-einstenium to 3.5.2 and it didn't work in any of them.

Today I tested it with 3.4.0-chromium version and it worked fine. As an additional info, I ran the code with chromedrive in my local and it worked fine too.

Expected Behavior -

When downloading a file the file downloads successfully.

Actual Behavior -

When downloading a file, it doesn't download anything.

[wlaoh]

Hello, I've been trying to accomplish the same thing using almost the same exact Docker setup. My question is did you have to do anything special in the first place to download the file? Basically what I'm running into is that the OS specific download prompt comes up when the download is initiated, and Selenium is, of course, unable to control this OS prompt.

I'm really curious how it was working for you in the past.

[diemol]

Hi @piscisferro,

There was a bug introduced in 3.4.0-francium related to env vars and default values. This was fixed in 3.5.3-boron. Could you please try again and let us know if it works again?

If not, please help us by providing enough information (docker commands, logs, test script, etc...) for us to reproduce the problem and find a way to fix it.

[piscisferro]

Hi

For the download I define the folder location and the option to download the file directly to that folder using the ChromeOptions:

preferences.put("profile.default_content_settings.popups", 0);
preferences.put("download.prompt_for_download", "false");
preferences.put("download.directory_upgrade", "true");
preferences.put("download.default_directory", /temp/downloads);
preferences.put("plugins.plugins_disabled", new String[] { "Chrome PDF Viewer" });

ChromeOptions options = new ChromeOptions();
options.setExperimentalOption("prefs", preferences);

When clicking on the download button, the browser should download the file directly to the downloads folder in /temp. For this to be "seen" by the anapsix/alpine-java container (which is the container that is executing the script) the two must be connected, I connected them creating a volume in /temp/downloads in the docker-compose file, and as a side effect I have connected it too to my local (so I can see the folder to check manually if required)

The test-script isn't complex, just click on the download button wait a few seconds and then do a linux bash command through java to check that the file downloaded correctly.

downloadButton.click();
String ls = launchBashSentence("bash", "-c", "ls -lrt " + /temp/downloads + "/*.pdf " | tail -1);

The bash sentence don't return nothing because nothing is downloaded.

If I enter to the container (selenium/standalone-chrome) using for example:
docker exec -i -t standalone-chrome-debug bash

In the /temp folder, every time that the download button is clicked it creates a temp file (probably the temporal file that the browser creates when download) but it never "finish" nor get converted to the proper pdf file.

About the env bug, I'm only using env variables to set the URL

i tried with 3.5.3-boron with no luck, I tested it to in 3.6.0, don't work.

--- Edited: Sorry It was late and I was in the mobile phone when I wrote this yesterday, edited to clarify it and provide more information

[piscisferro]

Edited the last post to clarify it and provide more information

[diemol]

Sorry for replying so late.

I was trying this, and I got confused. Before trying something else, can you please check via VNC the container and see what is happening when you click on the download button? You should be able to understand there why the download does not happen.

It also seems that the behaviour in Chrome changed, see:
https://stackoverflow.com/questions/45717426/how-to-disable-chrome-pdf-viewer-in-selenium-and-it-should-auto-download-in-the
https://stackoverflow.com/questions/31672897/how-to-download-a-pdf-file-in-chrome-using-selenium-webdriver

Can you please also share your docker-compose file, to see how the folder is mounted? Maybe you can share a link (with a button to download a PDF) where I can reproduce the issue.

[renanwilliam]

I have the same problem and it's related with file permissions. When docker-compose created a volume it's assign with the root:root permission and the seluser can't write on file

[diemol]

@piscisferro, @wlaoh, @renanwilliam
I am a bit confused here, is the issue that you cannot download files because you get a download prompt or is it that the file is downloaded but you cannot see it in the folder?

I would really appreciate if someone can provide the docker run commands or the docker-compose file used to start the containers. Also a complete test script with a reproducible case, otherwise it is kind of hard to help to find a solution.

[renanwilliam]

Hi @diemol,

To reproduce the error you need to be on a Linux machine, ok?

1. Clone this repository: https://github.com/renanwilliam/selenium-docker-compose-error
2. Go to terminal and execute docker-compose up
3. Open the VNC Viewer for your container, open the shell and type ls -lh
4. You will see that the folder is created with root:root (755) permission:

In a windows machine it's not a problem but in Linux systems the seluser can't save any data in this folder (only with sudo command)

5. If you try open any site and try download a file you will be 'stoped' in this save dialog prompt because the seluser can't save in the default folder:

[renanwilliam]

I think it's more a docker-compose issue but is not only responsible, If selenium docker image runs with root user this behavior will not occurs.

I already have tried:

• Include seluser in root group (not works because permission 755)
• Change the folder owner on Dockerfile (no effect)
• Change the permissions on Dockerfile to 775 or 777 (no effect)

[diemol]

Hi @piscisferro, @renanwilliam

Sorry for the delay.

I tried to reproduce the issue, and for that, I started a standalone Chrome debug container with the following command:

docker run -d -p 5900:5900 -p 4444:4444 \ 
    -v /dev/shm:/dev/shm \
    -v /tmp/downloads:/tmp/downloads \
    selenium/standalone-chrome-debug:3.7.1-beryllium

and then I executed this test

I was able to download the file and verify that it was downloaded in the mounted volume.

If you are still having the issue, could you also please provide (like I just did), a test script that helps us reproduce the issue? A docker command is useful as well (or docker-compose).

[renanwilliam]

Hi @diemol ,

Do you have checked my comment above? There's a test case with docker-compose comments

[ralfthewise]

@renanwilliam docker and docker-compose usually just pass through the filesystem permissions. With your test repository if you do this instead it should work:

rm -rf ./data
mkdir data
chown 777 data
docker-compose up

[diemol]

@renanwilliam
Well, yes, there is a docker-compose file and a set of manual steps in your comment, but I'd prefer some script I can execute and debug. Additionally, the issue is about Chrome and I see Firefox in the screenshot. Moreover, you see the prompt to download the file because I think you need to set some browser options before starting it so the file is downloaded straight away.

I ran the script above in OSX and I did not have any permission issues.

Again, if there is an issue with one of your tests, please help us by providing a script that we can run to reproduce the issue and fix it, because with the current information, we cannot really reproduce it.

[piscisferro]

First I want to remember that this work fine with 3.4.0-chromium version. Last version I tried was a week ago (3.7.0) and still not working.

Simplifying, these are the test scripts I'm using.

docker-compose (I'm using an image created by me based on anapsix/alpine-java to run the tests):

version: "2"
services:
  standalone-chrome-debug:
    container_name: "standalone-chrome-debug"
    image: "selenium/standalone-chrome-debug:latest"
    volumes:
          - ./downloads:/tmp/downloads

  selenium-test-alpine-java:
    container_name: "selenium-test-alpine-java"
    image: "selenium-test-alpine-java:latest"
    depends_on:
          - standalone-chrome-debug
    volumes:
          - ./downloads:/tmp/downloads
    entrypoint: /bin/bash ./test.sh

Selenium Test:

import ...

public class test() {
    public static void main(String[] args) { 
        WebDriver driver = new RemoteWebDriver(remoteAddress, DesiredCapabilities.chrome());

        driver.get(webURL);
        driver.findElement(By.id("DownloadButton")).click();
 
        // Wait a few seconds for the file to download I tried with absurd numbers too.
        try {
            Thread.sleep(5000);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }

        // Launch a linux command in order to get the list and see if the file is downloaded.
        String[] lscmd = {"bash", "-c", "ls -lrt  /tmp/downloads/"};
        String ls = launchBashSentence(lscmd);
        System.out.print(ls);

    }

   private getDesiredcapabilities() {
            Map<String, Object> preferences = new HashMap<String, Object>();
            preferences.put("profile.default_content_settings.popups", 0);
	    preferences.put("download.prompt_for_download", "false");
	    preferences.put("download.directory_upgrade", "true");
	    preferences.put("download.default_directory", "/tmp/downloads");
            
            ChromeOptions options = new ChromeOptions();
	    options.setExperimentalOption("prefs", preferences);

	    DesiredCapabilities capabilitiesChrome = DesiredCapabilities.chrome();
	    capabilitiesChrome.setCapability(ChromeOptions.CAPABILITY, options);
        }

    private launchBashSentence(lscmd) {
        // This method just execute the linux command and get the list of the directory
    }
}

As an additional info: I have the same issue using the selenium/standalone-firefox-debug container so it's not only for the chrome one

I hope this help.

[diemol]

Hi @piscisferro,

I updated the script I am using to reproduce the issue based on your input.

I created this docker-compose file similar to yours, and the selenium-test-alpine-java comes from this Dockerfile. The Java test is already checking that the file is present in the file here.

I executed the test with this command:

    docker-compose -f docker-compose.yaml up --abort-on-container-exit --build

The bottomline is that the test is passing, the image is being downloaded.

Could you please have a look at the scripts I modified and perhaps compare them with yours? Please note that I am using the latest release of the image 3.7.1-beryllium.

[piscisferro]

Ok, after testing further and checking @renanwilliam comment:

I have the same problem and it's related with file permissions. When docker-compose created a volume it's assign with the root:root permission and the seluser can't write on file

Just that:

• When running "docker-compose up" if you don't have the "download" folder created (host machine) it creates the folder with root permissions and with no write permissions preventing chromedriver from downloads files.

• But, If you create the download folder (in the host machine) and give full permissions to it (chmod 777 for example) the tests pass and it download the files successfully (as @ralfthewise said)

Now the thing is, why this restricted permission affect from 3.4.0-chromium onwards but is not affected in 3.4.0-chromium? What is the correct functionality? the 3.4.0-chromium behavior or the 3.7.1-beryllium behavior?

[diemol]

Ah ok, so in the end this was more related to folder permissions, which is more clear when running the images in Ubuntu (this does not affect usually OSX nor Windows).

The images were changed to use the seluser for all the processes, this was done to follow best practices and also because the services don't need root to run. This makes the images more usable in different environments, like hosting services that restrict the use of privileged users.

Sorry if this caused any inconvenience, but just changing the folder permissions fixes it, as @ralfthewise mentions.

[diemol]

@piscisferro, I'll go ahead and close this issue since we found the root cause and a solution to it, (as mentioned in the comment above).

If there are still any related problems, please feel free to reopen.