[BUG] extends with depends_on does not appear to start dependencies with version 2.32.1

[tony-schellenberg]

Description

With docker compose version 2.32.1, using extends with depends_on does not launch dependencies anymore. Downgrading to 2.31.0 works as expected.

If this is user error or not expected to work, please let me know. I'm happy to provide any more information or perform testing as required.

Steps To Reproduce

1. Install version 2.32.1
2. Create two docker files with the content listed below this list
3. Execute docker compose up nginx-deps-work
   • Notice that two containers are running, one for other-nginx as the dependency
4. Clean up with docker compose down -v
   • Verify that no container instances are running
5. Execute docker compose up nginx-extend-deps-fail
   • Notice that only one container is started and the other-nginx dependency is missing
6. Downgrade to version 2.31.0 and repeat the above steps
   • In both of the tests above, the other-nginx dependency is started

docker-compose.yml

services:
  nginx-deps-work:
    image: nginx
    depends_on:
      - other-nginx

  nginx-extend-deps-fail:
    extends:
      file: docker-compose-extends.yml
      service: nginx
    depends_on:
      - other-nginx

  other-nginx:
    image: nginx

docker-compose-extends.yml

services:
  nginx:
    image: nginx

Compose Version

Docker Compose version v2.32.1

Docker Environment

Client: Docker Engine - Community
 Version:    27.4.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.19.3
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.32.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 2
  Running: 0
  Paused: 0
  Stopped: 2
 Images: 15
 Server Version: 27.4.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 88bf19b2105c8b17560993bee28a01ddc2f97182
 runc version: v1.2.2-0-g7cb3632
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.8.0-51-generic
 Operating System: Ubuntu 24.04.1 2024.09.17 LTS (Cubic 2024-09-17 07:02)
 OSType: linux
 Architecture: x86_64
 CPUs: 20
 Total Memory: 62.47GiB
 Name: <REDACTED>
 ID: c886a621-05e7-4a6d-a2a6-df6057ef4d9b
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Default Address Pools:
   Base: 198.18.0.0/22, Size: 26

Anything else?

No response

[ndeloof]

This is the expected behavior, as documented on https://github.com/compose-spec/compose-spec/blob/main/05-services.md#restrictions

earlier release didn't checked this restriction

[ndeloof]

For your information:

This restriction is a legacy rule we inherit from 4c582e4 (see validate_extended_service_dict). The reason this restriction exists, is that extends only target a service, but can't guarantee referred resources exists in the extending compose file. This could bring inconsistencies. Anyway, I wonder we should relax this rule and just let user manage this by himself.

[TroyanOlga]

What about the case when extended service depends on condition, e.g. service_healthy? As I see, with the version 2.32.1 it doesn't work anymore even when depends_on is specified again in the extending file, e.g.:

web-service:
    extends:
      service: web
      file: ../test/docker-compose.base.yml
    depends_on:
      mongodb:
        condition: service_started
      rabbitmq:
        condition: service_healthy
      redis:
        condition: service_started

In this case, web-service doesn't wait for rabbitmq to become healthy (which it did in previous versions). Is that also expected?

[ndeloof]

proposed compose-spec/compose-spec#554

[tony-schellenberg]

Thank you very much for the additional information. I'm not sure if it helps but I can provide a bit more detail on my use case: I have a large docker-compose file that is used to spin up an integration environment for testing in. When developing, I want to use the same structure but I need to make some modifications to the main container that is hosting the software under development for it to be usable in that manner. Extending the service allows me to easily keep the large configuration with just some smaller adjustments to it.

For my use case, the previous behaviour where the depends on clause was extended is beneficial for me. As I re-read the docs after the first comment and thought about it, I was guessing that it could have issues related to transitive dependencies that were defined somewhere else.

I think my desired behaviour would be to have extend pull in all of the stanzas and then give me an error if something couldn't be found. If I wanted to exclude an element in my extended service, I would use !reset or something similar to drop the content from the base service declaration along these lines: https://docs.docker.com/reference/compose-file/merge/#reset-value.

[ndeloof]

compose file is way more flexible today that it used to be, and indeed relying on !reset can help.
IMHO the main challenge is to provide adequate diagnostic if something is missing, so user can understand the issue as compose reports "service db not found" while being required by an extended service, hidden inside 2 nested imports :P

[ldeninski]

@ndeloof I fail to understand how the restrictions are relevant with this compose file:

services:
  baseService:
    image: busybox:stable-musl
    entrypoint: ["/bin/sh"]

  s1:
    image: busybox:stable-musl
    entrypoint: ["/bin/sh"]

  s2:
    extends: baseService
    entrypoint: ["/bin/sh"]
    depends_on:
      - s1
      - asdasdasd
      - sandals

This should be invalid, but I can start the s2 service with docker compose run without any apparent issues.. that is until something in s2 tries to resolve s1. I am pretty sure that this is a bug.

[ndeloof]

sure, this specific issue is same as #12402 already fixed by compose-spec/compose-go#721

[outsinre]

I have the same issue but with include.

If a service of the current compose depends_on another service from include. The depends_on disappear from docker compose convert and not respected when docker compose up.

[ndeloof]

Closing as a duplicate for #12402