Create non-existing bind volumes

[maweki]

I am not sure whether docker or docker-compose does this and conversely I don't know whether the issue lies with podman or podman-compose:

For postgres containers we often have a setup like this:

services:

  db:
    image: postgres
    restart: always
    environment:
      POSTGRES_PASSWORD: example
    volumes:
    - "./pg-data:/var/lib/postgresql/data"

in docker the folder pg-data is created automatically, if it doesn't exist. Via podman-compose I get

Error: error checking path "/tmp/foo/./pg-data": stat /tmp/foo/./pg-data: no such file or directory
125

[manics]

It's an issue with podman, see #175

[akirabaruah]

It seems like this feature is intentionally left out of libpod / podman - see these comments by @rhatdan:

• How to mount new volume(s) into my home folder with podman run(rootless)? podman#4891 (comment)
• create volume path if not exists podman#6234 (comment)

In that case, should we close this issue as working as intended?
It looks like @lcarva's PR provides a workaround regardless of podman's omission of the feature.

FWIW, I also ran into this issue and worked around it by creating the required directories using mkdir on the host before running podman-compose.

[maweki]

This is intentionally breaking drop-in-replaceability of podman/podman-compose with docker/docker-compose. Maybe we can hide it behind a command line switch.

I often have database containers with the data directory of the database mounted into the filesystem. Resetting after I changed something in the build of the container is:

podman-compose down && sudo rm -rf ./data && mkdir data && podman-compose build && podman-compose up instead of just docker-compose down && sudo rm -rf ./data && docker-compose build && docker-compose up and then I often forget to create the data directory before restarting the container.

In the ideal world, podman-compose has a compatibility mode with docker-compose where it tries to behave as a drop-in replacement and I can alias docker-compose to podman-compose -C or something.

We are thinking about enabling podman in the future for our students in the pc laboratory and I stumble over this quite often. I am 100% sure that most students would get this wrong even after explaining it a dozen times.

[lcarva]

I'm happy to add some sort of compatibility flag if that's preferable. Any objections?

[lcarva]

Updated the PR to include the compatibility mode.

[muayyad-alsadi]

I'm happy to add some sort of compatibility flag if that's preferable. Any objections?

the default behavior should be just like docker-compose.
maybe not deleting volumes resulted from legacy code that predates the volume support in podman (maybe we did it when podman had bind volumes only)

but if you think there is a value of having an extra functionality it should not be default.

Not deleting the volumes is something useful I would love a flag to keep it.
But we need to double check to see if docker-compose have a flag to keep the volumes.

[lcarva]

I'm happy to add some sort of compatibility flag if that's preferable. Any objections?

the default behavior should be just like docker-compose.
maybe not deleting volumes resulted from legacy code that predates the volume support in podman (maybe we did it when podman had bind volumes only)

but if you think there is a value of having an extra functionality it should not be default.

Not deleting the volumes is something useful I would love a flag to keep it.
But we need to double check to see if docker-compose have a flag to keep the volumes.

@muayyad-alsadi, this issue and the linked PR #194 are about creating bind volumes, not deleting named volumes (that's my other PR #214 😄 ). docker-compose will create directories for bind volumes if they don't exist. More specifically, docker creates these. As previously mentioned, there was a decision to not implement this behavior in podman which breaks compatibility between podman-compose and docker-compose.

[stellarpower]

If this is still open, had an idea for a different format - would a suffix be possible, as we do for :ro, :Z, etc.?

I agree with podman in this case, and have had some annoying messups when docker has auto-created a folder.

But one case where it is useful is in a wrapper script - I want to wrap a call to podman run ... my-image myCommand ... in my script, so that the container is transparent to the users. In this case, they're passing output filenames into the script, and of course, these don't exist. Currently I can get the basename of the file, bind this directory as the volume, and still create the output file to work around, but would be nice and convenient if I could do the folowing:

podman run --rm -v $inputFile:/input -v $outputFile:/output:c my-ffmpeg-image ffmpeg -i /input -ac ... -v ... /output

if :c means create if it doesn't exist. This way it matches what we usually expect - an input file must exist already, and an output file won't need to, if we don't want to overwrite.

Thoughts?

[lcarva]

@stellarpower, that wouldn't help with keeping compatibility between docker-compose and podman-compose. I like your idea though. I think a better place to implement such feature would be in podman itself. Maybe file an feature request there?

[muayyad-alsadi]

@lcarva @stellarpower please test