Breeze won't work with Docker Desktop on Mac M1?

[Bisk1]

I tried to follow instructions on "Contributor's Quick Start":
https://github.com/apache/airflow/blob/main/CONTRIBUTORS_QUICK_START.rst

But when I run the breeze command I get:

(venv) daniel.dylag@MBPM1-OK-163 airflow % breeze --python 3.8 --backend postgres
Good version of Docker: 20.10.20.
Good version of docker-compose: 2.12.0
Docker is not using the default context, used context is: desktop-linux
Please make sure Docker is using the default context.
You can try switching contexts by running: "docker context use default"

Actually, I installed Docker as "Docker Desktop for Mac with Apple Chip" from here: https://www.docker.com/products/docker-desktop/

So when I list my docker contexts I get this:

daniel.dylag@MBPM1-OK-163 projects % docker context list
NAME                TYPE                DESCRIPTION                               DOCKER ENDPOINT                                      KUBERNETES ENDPOINT                ORCHESTRATOR
default             moby                Current DOCKER_HOST based configuration   unix:///var/run/docker.sock                          https://35.184.155.148 (default)   swarm
desktop-linux *     moby                                                          unix:///Users/daniel.dylag/.docker/run/docker.sock

desktop-linux is the context I need to use with Docker Desktop. If I really follow the instructions and switch the context then I get an error that Docker is not running.

(venv) daniel.dylag@MBPM1-OK-163 airflow % docker context use default
default
(venv) daniel.dylag@MBPM1-OK-163 airflow % breeze --python 3.8 --backend postgres
Docker is not running.
Please make sure Docker is installed and running.

And then I restart Docker Desktop - but it switches context back to desktop-linux.
Is there something wrong with my docker or Airflow installation? Do you think it's specific to Mac/Apple chip?

I tried to work around this problem so I manually changed the line:

airflow/dev/breeze/src/airflow_breeze/utils/docker_command_utils.py

Line 336 in 8f5ed4b

expected_docker_context = "default"

to allow desktop-linux.

Then the command starts, but I get new errors during installation:

(...)
Verification complete! Success!

Written /Users/daniel.dylag/projects/airflow/generated/provider_dependencies.json


Building CI Image for Python 3.8

ERROR: use `docker --context=default buildx` to switch to default context
[+] Running 1/0
 ⠿ Container docker-compose-postgres-1  Running                                                                                                                                                                                    0.0s
Error response from daemon: no such image: ghcr.io/apache/airflow/main/ci/python3.8:latest: No such image: ghcr.io/apache/airflow/main/ci/python3.8:latest
Error 1 returned

Do you think it's because of my manual change in the context name that bypassed verification?

[o-nikolas]

CCing @alexkruc who made the change to assert a default context. I forget the context (no pun intended) around why that assertion was added. A Slack thread is linked from the PR but it is unfortunately expired and cannot be read.

[alexkruc]

Hey @o-nikolas - I'll share the context as I remember, I forgot that Slack threads in free tier Slack are deleted :(
So in the organization I work for, we moved from Docker desktop to Colima Docker runtime, which creates a new context.
While working with Colima & breeze, I could not build new breeze images due to buildx errors (perhaps the same one that the author now facing when manually changing the context check). Once I switched to using the default docker context and used Docker desktop, everything worked fine for me.

In the Slack thread, we discussed what is the proper approach to fix this, and it was decided that having a different Docker runtime and using other runtimes than Docker desktop is not according to the guidelines, and we should enforce this.

AFAIK Docker desktop with M1 chip should also use the default context, or else a lot more people would complain on the same issue.. Unfortunately, I still use a Mac with an Intel chip, so I can't confirm, but I do know that a lot of people is using M1 versions and do use the default context.

Perhaps try to re-install Docker desktop? According to this issue (New desktop-linux / desktop-windows contexts cause problems when switching OS mode), default should still be available and working as the ,well, default choice..

[potiuk]

This is what I see on my M1. This is not an M1 feature.

NAME            DESCRIPTION                               DOCKER ENDPOINT                               KUBERNETES ENDPOINT   ORCHESTRATOR
default *       Current DOCKER_HOST based configuration   unix:///var/run/docker.sock                                         swarm
desktop-linux                                             unix:///Users/jarek/.docker/run/docker.sock

Just upgraded to latest version of docker desktop - no problems either.

I think the problem is some buggy version of Docker Desktop that change the defauls to desktop-linux that broke your configuration - you need to change default context to "default" from "desktop-linux" and restart docker.

Those (very similar) issues are on Windows, but I think it might be the same for Mac:

https://developercommunity.visualstudio.com/t/current-context-desktop-linux-is-not-found-on-the/1463380
https://stackoverflow.com/questions/68208567/error-in-publishing-net-core-web-app-on-docker-hub/68237128#68237128
docker/for-win#11721

Basically find default config.json and change currentContext there (or remove it entirely).

https://docs.docker.com/engine/reference/commandline/cli/#configuration-files

[Bisk1]

I dug deeper and the problem is related to the latest release of Docker Desktop for Mac (probably unrelated to the chip type).
https://docs.docker.com/desktop/release-notes/#docker-desktop-4130

By default Docker will not create the /var/run/docker.sock symlink on the host and use the docker-desktop CLI context instead.

This means that you probably couldn't reproduce it because you had older version installed first, which symlinked those (and the symlink was kept after update), but I installed Docker Desktop on a fresh environment.
Also some explanation from maintainers of Docker Desktop for Mac:
docker/for-mac#6529 (comment)

We're trying to make Docker Desktop more userspace-friendly (e.g., fewer root privileges). As part of that, we're migrating the sockets to user-owned directories rather than root-owned directories (like /var/run).

This means that the change is likely to stay (not a bug) and will affect all future Airflow contributors who install Docker Desktop on a fresh environment rather than upgrading an older version.

Just changing the context to default won't work (as I explained in the first message it leads to an error) because there is no default socket, I had to manually symlink my docker socket, and then the default context started to work:
sudo ln -sf "$HOME/.docker/run/docker.sock" /var/run/docker.sock

If it is really necessary to use default context, I think it would be useful for future Airflow contributors to have this symlink requirement documented.

EDIT:
Actually, this change might get rolled back:
docker/for-mac#6529 (comment)

Our current plan is to roll this change back for now and figure out a more orderly upgrade path with docs for client library authors.

[potiuk]

😱 😱 😱 😱 😱 😱 😱 😱 😱 😱

Our current plan is to roll this change back for now and figure out a more orderly upgrade path with docs for client library authors.

Well. I hope they better do :D

[potiuk]

We can find solution to that but I think let's see if they rollback quickly first. And I can add comment in docs.

[potiuk]

PR here: #27354 . Thanks @Bisk1 for all the investigation :) !! 🦾