cannot connect to chrome at 127.0.0.1 (Docker compose)

[benjavides]

I was having this issue when trying to run the included docker-compose

E:\Documents\1_PROJECTS\Mudanza_Alemania\busqueda_depto\flathunter>docker-compose up   
[+] Running 1/1
 ✔ Container flathunter-app-1  Recreated                                                                                                                                                                                                                                                  11.0s 
Attaching to app-1
app-1  | [2024/06/08 22:01:40|config.py               |INFO    ]: Using config path /usr/src/app/config.yaml
app-1  | [2024/06/08 22:01:40|chrome_wrapper.py       |INFO    ]: Initializing Chrome WebDriver for crawler...
app-1  | [2024/06/08 22:01:41|patcher.py              |INFO    ]: patching driver executable /root/.local/share/undetected_chromedriver/undetected_chromedriver
app-1  | Traceback (most recent call last):
app-1  |   File "/usr/src/app/flathunt.py", line 99, in <module>
app-1  |     main()
app-1  |   File "/usr/src/app/flathunt.py", line 95, in main
app-1  |     launch_flat_hunt(config, heartbeat)
app-1  |   File "/usr/src/app/flathunt.py", line 35, in launch_flat_hunt
app-1  |     hunter.hunt_flats()
app-1  |   File "/usr/src/app/flathunter/hunter.py", line 56, in hunt_flats
app-1  |     for expose in processor_chain.process(self.crawl_for_exposes(max_pages)):
app-1  |                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-1  |   File "/usr/src/app/flathunter/hunter.py", line 35, in crawl_for_exposes
app-1  |     return chain(*[try_crawl(searcher, url, max_pages)
app-1  |                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-1  |   File "/usr/src/app/flathunter/hunter.py", line 35, in <listcomp>
app-1  |     return chain(*[try_crawl(searcher, url, max_pages)
app-1  |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-1  |   File "/usr/src/app/flathunter/hunter.py", line 27, in try_crawl
app-1  |     return searcher.crawl(url, max_pages)
app-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-1  |   File "/usr/src/app/flathunter/abstract_crawler.py", line 151, in crawl
app-1  |     return self.get_results(url, max_pages)
app-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-1  |   File "/usr/src/app/flathunter/crawler/immobilienscout.py", line 92, in get_results
app-1  |     soup = self.get_page(search_url, self.get_driver(), page_no)
app-1  |                                      ^^^^^^^^^^^^^^^^^
app-1  |   File "/usr/src/app/flathunter/crawler/immobilienscout.py", line 67, in get_driver
app-1  |     self.driver = get_chrome_driver(driver_arguments)
app-1  |                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-1  |   File "/usr/src/app/flathunter/chrome_wrapper.py", line 69, in get_chrome_driver
app-1  |     driver = uc.Chrome(version_main=chrome_version, options=chrome_options) # pylint: disable=no-member
app-1  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-1  |   File "/usr/local/lib/python3.11/site-packages/undetected_chromedriver/__init__.py", line 466, in __init__
app-1  |     super(Chrome, self).__init__(
app-1  |   File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/chrome/webdriver.py", line 45, in __init__
app-1  |     super().__init__(
app-1  |   File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/chromium/webdriver.py", line 61, in __init__
app-1  |     super().__init__(command_executor=executor, options=options)
app-1  |   File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 208, in __init__
app-1  |     self.start_session(capabilities)
app-1  |   File "/usr/local/lib/python3.11/site-packages/undetected_chromedriver/__init__.py", line 724, in start_session
app-1  |     super(selenium.webdriver.chrome.webdriver.WebDriver, self).start_session(
app-1  |   File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 292, in start_session
app-1  |     response = self.execute(Command.NEW_SESSION, caps)["value"]
app-1  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
app-1  |   File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 347, in execute
app-1  |     self.error_handler.check_response(response)
app-1  |   File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response
app-1  |     raise exception_class(message, screen, stacktrace)
app-1  | selenium.common.exceptions.WebDriverException: Message: unknown error: cannot connect to chrome at 127.0.0.1:40595
app-1  | from chrome not reachable
app-1  | Stacktrace:
app-1  | #0 0x55703b22ce3a <unknown>
app-1  | #1 0x55703af162a1 <unknown>
app-1  | #2 0x55703af0096f <unknown>
app-1  | #3 0x55703af50825 <unknown>
app-1  | #4 0x55703af470b4 <unknown>
app-1  | #5 0x55703af91b19 <unknown>
app-1  | #6 0x55703af85253 <unknown>
app-1  | #7 0x55703af551c7 <unknown>
app-1  | #8 0x55703af55b3e <unknown>
app-1  | #9 0x55703b1f327b <unknown>
app-1  | #10 0x55703b1f7327 <unknown>
app-1  | #11 0x55703b1dfdae <unknown>
app-1  | #12 0x55703b1f7df2 <unknown>
app-1  | #13 0x55703b1c474f <unknown>
app-1  | #14 0x55703b21c128 <unknown>
app-1  | #15 0x55703b21c2fb <unknown>
app-1  | #16 0x55703b22bf6c <unknown>
app-1  | #17 0x7f43bd5ae134 <unknown>
app-1  |
app-1  | [2024/06/08 22:02:41|__init__.py             |INFO    ]: ensuring close
app-1 exited with code 0

Adding a couple of things to the Dockerfile solved the issue:
After "# Install Chromium"

RUN apt-get -y update && apt-get install -y chromium \
    && apt-get install -y wget gnupg \
    && wget -q -O - https://dl.google.com/linux/linux_signing_key.pub | apt-key add - \
    && sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list' \
    && apt-get -y update && apt-get install -y google-chrome-stable

At the end

RUN python chrome_driver_install.py

I didn't made a pull request because I'm not sure if all of this is needed. This was chatGPT suggestion which worked after trying a couple of things.

[codders]

I retested the docker container today with the latest code and using Capmonster as the captcha solver. It seemed to work fine and was also able to crawl with Immoscout.

Note that docker-compose is deprecated and has been replaced by a docker plugin: https://docs.docker.com/compose/install/linux/ . I've updated the docs in #654