Downloading a file in new window in headless ChromeDriver

[Dejital]

Meta -

OS:
Windows 10
Selenium Version:
Selenium.WebDriver v3.11.0
Selenium.WebDriver.ChromeDriver v2.37.0
Browser:
Headless ChromeDriver

Expected Behavior -

Attempting to download a file using a headless ChromeDriver. The website being "driven" opens a new window on button click, which then triggers a download of a CSV file.

Actual Behavior -

var submitBtn = _driver.GetById("downloadSubmitBtn");
submitBtn.Click();

This does open a new Window. However, that window is not in focus in the ChromeDriver. If I switch to the Window, I get an error. There is no file downloaded to the default download directory. The following user preferences are set:

var downloadDirectory = "C:\\Temp";
var chromeOptions = new ChromeOptions();
chromeOptions.AddArguments(new List<string>() { "headless" });
chromeOptions.AddUserProfilePreference("download.default_directory", downloadDirectory);
chromeOptions.AddUserProfilePreference("download.prompt_for_download", false);
chromeOptions.AddUserProfilePreference("disable-popup-blocking", "true");
chromeOptions.AddUserProfilePreference("block_third_party_cookies", true);

return new ChromeDriver(chromeOptions);

Running the same code in not headless mode does open a new window, which does successfully down the file to the specified default directory (using same user profile preferences as above).

Steps to reproduce -

Page is unfortunately behind authentication and cannot be linked to directly.

[cgoldberg]

https://bugs.chromium.org/p/chromium/issues/detail?id=696481

[Dejital]

@cgoldberg Sounds like there is no solution yet, but I'm not alone. Is there a way to temporarily "un-headless" the headless browser to just download the file?

[cgoldberg]

@Dejital you can send an HTTP request to fetch the file directly (all the cookies you need are in your driver instance if auth is required)

[Dejital]

@cgoldberg Tried and failed that, since fetching cookies from the driver throws an exception. -- ArgumentException on the Cookie object's Name. Presumably the website I'm authenticating against saves a cookie with no name. Same story for other WebDrivers as well. Somehow they work just fine on their own (authentication), but I can't get the cookies out to make the HTTP request.

[p0deje]

Duplicate of #5159

[ppratikcr7]

It's easy. Call this enabler function after your switch window for your driver:

 def enable_download_in_headless_chrome(driver, download_dir):
    # add missing support for chrome "send_command"  to selenium webdriver
    driver.command_executor._commands["send_command"] = ("POST",'/session/$sessionId/chromium/send_command')
    params = {'cmd': 'Page.setDownloadBehavior', 'params': {'behavior': 'allow', 'downloadPath': download_dir}}
    command_result = driver.execute("send_command", params)

expected_download = 'ur/download/path'
opt = Options()
opt.add_experimental_option("prefs", { \
    'download.default_directory': expected_download,
     'download.prompt_for_download': False,
     'download.directory_upgrade': True,
  })
opt.add_argument('--headless')
opt.add_argument('--window-size=1920,1080');

login_page = "https://www.google.com"
driver = webdriver.Chrome(options=opt)
driver.implicitly_wait(5)
driver.get(login_page)
driver.maximize_window()

#On below click you will be in new tab
scoresheet_tab = driver.find_element_by_xpath("//*[@class='sideNav-item is--scoresheet']").click()

instances = driver.window_handles
driver.switch_to.window(instances[1]) # this is the new browser
#this below function below does all the trick
enable_download_in_headless_chrome(driver, expected_download)

[avmgan]

@ppratikcr7 - Is it possible to call using C#? where I can call this enabler function after the switch window for the driver?

[avmgan]

@ppratikcr7 - Is it possible to call using C#? where I can call this enabler function after the switch window for the driver?

[avmgan]

ppratikcr7 - Is it possible to call using C#? where I can call this enabler function after the switch window for the driver?

[jimevans]

@avmgan There is already support for executing Chrome Debug Protocol commands in the .NET bindings for ChromeDriver, so not all of the plumbing that is required for Python is required here. The C# equivalent would look something like this:

public void EnableDownloadInHeadlessChrome(ChromeDriver driver, string downloadDir)
{
    // There is no need to add support for the Chrome "send_command"; the ChromeDriver
    // class already has it built in.
    var params = new Dictionary<string, object>();
    params["behavior"] = "allow";
    params["downloadPath"] = downloadDir;
    driver.ExecuteChromeCommand("Page.setDownloadBehavior", params);
}

One small issue I'll take with @ppratikcr7's solution is that it assumes the handle with index 1 will be the new window (cf. driver.switch_to.window(instances[1])). This is a common misconception about getting window handles using driver.WindowHandles (driver.window_handles in Python). The handles returned by that property are absolutely not guaranteed to be in any order. Assuming "the handle in the collection with index 1 must be the most recently open window" is a ticking time bomb.

[cgoldberg]

so not all of the plumbing that is required for Python is required here.

FWIW, it's not required in Python either :)

params = {'behavior': 'allow', 'downloadPath': dir}
driver.execute_cdp_cmd('Page.setDownloadBehavior', params)

[rishi1992]

Is it possible in Java?

[phenry9999]

Thank you for the Page.setDownloadBehavior, except I have a new problem. The default browser behavior is when you download file1.xml it download fine, then you download file1.xml again and normally Windows\browser will rename the file to file1 (1).xml (notice the brackets and number count). This number will increment as the same file is downloaded. This is the default behavior. The above code breaks this behavior. How do we get it back?

[amishad02]

@ppratikcr7 how can I do same for java please help.

[KrishnaYadav2102]

Thank you for the Page.setDownloadBehavior, except I have a new problem. The default browser behavior is when you download file1.xml it download fine, then you download file1.xml again and normally Windows\browser will rename the file to file1 (1).xml (notice the brackets and number count). This number will increment as the same file is downloaded. This is the default behavior. The above code breaks this behavior. How do we get it back?

I am also facing the same issue. Is there any solution to download multiple files with same name?