Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Permissions on mounted /data volume not correct #7

Closed
marclennox opened this issue Aug 14, 2014 · 20 comments · Fixed by #9
Closed

Permissions on mounted /data volume not correct #7

marclennox opened this issue Aug 14, 2014 · 20 comments · Fixed by #9

Comments

@marclennox
Copy link

I'm running this instance with the /data directory mounted to the host machine with

-v /data/redis:/data

For some reason the permissions on the volume are not getting set properly despite the chown call in the Dockerfile, and therefore Redis pukes when trying to persist the DB.

Any ideas?

@tianon
Copy link
Contributor

tianon commented Aug 14, 2014

Does it work if you manually chown -R 999:999 /data/redis on your host?

(If you're wondering where the "999" came from, see docker run -it --rm redis id, which provides uid=999(redis) gid=999(redis) groups=999(redis).)

When you bind-mount, Docker doesn't touch the permissions of the directory that's being mounted in, which I'm guessing is the problem here.

@marclennox
Copy link
Author

I will try.  I suspect the answer is yes.  However note that the MySQL and Postgres dockers work fine on this respect, presumably because they do the chown in an entry point script?  I'm thinking the redis docker just needs an entry point to do the permission change.—
Sent from Mailbox

On Thu, Aug 14, 2014 at 4:38 PM, Tianon Gravi [email protected]
wrote:

Does it work if you manually chown -R 999:999 /data/redis on your host?
(If you're wondering where the "999" came from, see docker run -it --rm redis id, which provides uid=999(redis) gid=999(redis) groups=999(redis).)

When you bind-mount, Docker doesn't touch the permissions of the directory that's being mounted in, which I'm guessing is the problem here.

Reply to this email directly or view it on GitHub:
#7 (comment)

@marclennox
Copy link
Author

Confirmed that it worked by manually changing permissions to 999:999 on the
host

On 14 August 2014 19:38, Marc Lennox [email protected] wrote:

I will try. I suspect the answer is yes. However note that the MySQL and
Postgres dockers work fine on this respect, presumably because they do the
chown in an entry point script? I'm thinking the redis docker just needs
an entry point to do the permission change.

Sent from Mailbox https://www.dropbox.com/mailbox

On Thu, Aug 14, 2014 at 4:38 PM, Tianon Gravi [email protected]
wrote:

Does it work if you manually chown -R 999:999 /data/redis on your host?

(If you're wondering where the "999" came from, see docker run -it --rm
redis id, which provides uid=999(redis) gid=999(redis) groups=999(redis)
.)

When you bind-mount, Docker doesn't touch the permissions of the
directory that's being mounted in, which I'm guessing is the problem here.


Reply to this email directly or view it on GitHub
#7 (comment).

@ntharim
Copy link

ntharim commented Aug 29, 2014

This issue is kinda annoying. Possible to make it work like MySQL and Postgres?

@yosifkit
Copy link
Contributor

Since we have already set a precedent with the other database images I think this would be ok. @tianon, do you agree?

I won't be able to get to it until after next week, but pull requests are welcome (just follow the examples @ntran13 cited, mysql and postgres).

@tianon
Copy link
Contributor

tianon commented Aug 29, 2014

Yeah, I think this is OK for now since we've already got the precedent set.

@yosifkit
Copy link
Contributor

Sorry for the delay, but a PR to add the auto chowning the data dir is now up (with all the newer versions of redis to boot).

@tianon tianon closed this as completed in #9 Sep 25, 2014
@ElvisTheKing
Copy link

same issue with mounted data dir when using with docker compose without specifying entry point:

for example with

redis:
  image: redis
  volumes:
    - ./data:/data/
  ports: ["6379:6379"]

actual command will be /entrypoint.sh redis which obviously wont work

@yosifkit
Copy link
Contributor

yosifkit commented Sep 2, 2015

The default CMD is redis-server, not redis, so it works just fine (/entrypoint.sh redis-server). Here is the latest image id:

$ docker images redis
REPOSITORY        TAG            IMAGE ID          CREATED           VIRTUAL SIZE
redis             latest         9216d5a4eec8      9 days ago        109.3 MB

@alexgorbatchev
Copy link

I'm having trouble getting this to work with docker machine & compose. Running docker in virtualbox env.

$ rm -fr tmp
$ docker-compose up
Starting test_redis_1
Attaching to test_redis_1
redis_1 | 1:M 17 Feb 20:06:08.714 # Can't open the append-only file: Permission denied
test_redis_1 exited with code 1

docker-compose.yml

version: "2"
services:
  redis:
    image: redis:3
    command: /entrypoint.sh redis-server --appendonly yes
    volumes:
      - ./tmp/redis:/data
    ports:
      - "6379:6379"

The ./tmp/redis folder is created but container can't write to it.

$ docker version
Client:
 Version:      1.10.1
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   9e83765
 Built:        Thu Feb 11 19:14:21 2016
 OS/Arch:      darwin/amd64

Server:
 Version:      1.10.1
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   9e83765
 Built:        Thu Feb 11 20:39:58 2016
 OS/Arch:      linux/amd64

$ docker-compose version
docker-compose version 1.6.0, build d99cad6
docker-py version: 1.7.0
CPython version: 2.7.9
OpenSSL version: OpenSSL 1.0.1j 15 Oct 2014

$ docker-machine version
docker-machine version 0.6.0, build e27fb87

@marclennox
Copy link
Author

I've found when host mounting data volumes, you often have to pre-create them and set the permissions to 999:999. In some cases host mounting is very hard, such as with Postgres. I've stopped doing host-mounting entirely and have embraced volume containers, which make these problems go away.

@yosifkit
Copy link
Contributor

If you still want to mount data to a shared folder on your Mac, then because the entrypoint script currently only does a chown and user drop-down you should be able to do something similar to #47 (comment).

@ralyodio
Copy link

when i pre-create the directory ./mongo-data with ownership of ubuntu:ubuntu and then I run docker-compose up --build it changes the owner to 999 and then docker gives a permission denied error.

@ahma0942
Copy link

Got the exact same problem as @chovy. Any fix to this?

@yosifkit
Copy link
Contributor

@chovy, sorry I missed this. To use a local bind-mounted directory, you need to pre-create the directory (as you do) and then add user to the MongoDB service in the compose file with the ids of the owner of the directory (i.e. your ubuntu user and group; e.g. user: "1000:1000").

If you do not pre-create the directory locally, then docker will create it as root owned when the container starts (and then the mongo container will be unlikely to have access and fail to start).

(The same applies to a bind-mounted redis data directory as well)

https://docs.docker.com/compose/compose-file/#domainname-hostname-ipc-mac_address-privileged-read_only-shm_size-stdin_open-tty-user-working_dir

@Yingsheng-eroad
Copy link

Yingsheng-eroad commented Jun 3, 2020

Have the same issue as @chovy, but mine is postgres. I answered the question in StackOverflow.

@neuronflow
Copy link

same issue with redis and docker-compose..I run this setup on 3 machines and only one is affected on a regular basis, very weird.

@Yingsheng-eroad
Copy link

Yingsheng-eroad commented Jul 23, 2020

same issue with redis and docker-compose..I run this setup on 3 machines and only one is affected on a regular basis, very weird.

This is my solution. Check if it helps.
https://stackoverflow.com/questions/61941469/my-local-volume-for-mongodb-gets-permission-deneied-when-running-docker-compose/62185391#62185391

@ghost
Copy link

ghost commented Jan 20, 2021

I had the same problem.

create a .dockerignore file at the root of the building containing your data folder

image

zspencer added a commit to zinc-collective/convene that referenced this issue Apr 30, 2023
* Add `bundle install` and `bin/setup` to Dockerfile

The `bin/setup` will fail due to not having the database containers up,
but I don't think that is necessary to get whatever other setup is done.

* Prevent caching of apt-get update stuff

Reduce container size

* Fix the `bundle install` step in Dockerfile

Needed some other files in the container!

* Initial chnages to publish and use own devcont img

Can't go the full mile yet as it would break devcontainer usage until
finished, but laying the ground work.
See new github action for possible steps.
I am currently unsure about where we would be pushing to, but we'll
discuss.

* Add missing sudo for removing apt cache

* Update devcontainer workflow for GH Packages

Again, following [this tutorial](https://docs.github.com/en/actions/publishing-packages/publishing-docker-images) (and many other links).
Tagged with commit SHA and `latest`.
Added my branch for temporary testing; just hoping actions registers
in my fork.

* Remove conditional on latest tag

Not sure why, but not working in workflow.

* Actually run workflow on changes to workflow file

* Explicitly state image name

Really need that "-devcontainer" in there to not make ambigous!
This is not a containerized deployment of `convene` itself!

* Fix missing repo name in image name

I don't really know what's going on, but this is something I figured I
could try.

* Remove repeated owner/repo name in image name

* Explicitly forward ports 3000 & 1080

* Use our devcontainer

This will only work in my fork for now!

* Don't rebuild container for any .devcontainer chg

Only for Dockerfile changes, as it should be.

* Add checks and waiting for docker-compose step

Removing 30s compulsory wait.
Also, this may stop the data loss when restarting?

* Add todos for timing and waiting in script

First pass at timing bin/setup call

* Fix logic & comparison operators in startup

* Persist db data across Codespace restarts

Based on ["Persisting data over a rebuild"](https://docs.github.com/en/codespaces/developing-in-codespaces/rebuilding-the-container-in-a-codespace#persisting-data-over-a-rebuild)

* Print commands in startup.bash

Not sure if does what I want, but could be useful?
Lengthen the sleep time of the loop to reduce prints.

* Only run this script if in Codespace

Not technically correct, but just a rudimentary check for now.

* Fix redis data symlink

* Fix some formatting of echos

* Update db file locations persisted

* Fix symlink location; use sudo

* Sudo rm the old db files; check symlinking

* Rework symlink setup significantly

Add chown for the new dirs, as they were owned by root.
This caused issues with the redis container?

Function-ify the symlink setup process. Nicer now.

Keep track of if the symlinks did exist or not.
Should probably move this outside of the function or to separate
function...

* Move data file detection before symlinking step

Since the symlinks definitely won't persist, but our data files may...?

* Don't run `bin/setup` if datafiles existed

* Some whitespace changes

* Recursively chown the /workspaces locations

* Fix typo in setting up db workspaces

* Fix permission on redis container data dir

Based on [this GitHub Issue](redis/docker-library-redis#7), the problem is still permissions.
Fixing it by specifically `chown`-ing the redis container's data
directory seems to work.
The postgres directory doesn't seem to need it, but I have kept a
comment there for posterity.

* Make sure the `_data` subdirectories exist

* Append output of `time` to out file

Or, at least, finally actually do this...

* Always run `bin/setup` on CodeSpace startup

Since `bin/run` won't work otherwise?
This at least helped me get it to work.
I think setting up the gem installation directory to be within the
`/workspaces` directory (like done for database container stuff) should
help?

* Update owner of redis container's vol dir

No idea why 999 did work and now won't, but this 1001 does now work.
My fear is that it is due to some race condition during initialization
of the CodeSpace container or the `docker compose up` call,
but I don't have proof of either yet.
I still need to read the docs...

* Feeback that symlink already existed

* Move the whole docker/volumes dir to workspaces

Instead of just postres and redis data volumes.
This may work the way I want, but maybe not?
Still having some permissions issues with the `dump.rdb` file.

* Detect redis container failure better

Or the postgres one.
Just trying to be more intelligent about this.

* Copy any existing data from original location

Only necessary on first startup (I think).
Mainly for the meta files.

* Ownership changes again

Seems to finally be working!

* Update ruby base container image (3.1->3.2)

* Attempt to scope image to repo

Instead of just organization...?

* Add label to devcontainer for package permissions

See [this link](https://docs.github.com/en/packages/learn-github-packages/connecting-a-repository-to-a-package)

* Add brief description label

Because I can....

* Revert back to `convene-devcontainer` naming

Preferable to me due to how GitHub packages renders the extra
`/` as `%2F` in url and doesn't actually add anything (as far as I can
tell).

* Add labels to devcontainer for pkg perm and desc

See [this link](https://docs.github.com/en/packages/learn-github-packages/connecting-a-repository-to-a-package)

* Update devcontainer.json to use zinc's image

---------

Co-authored-by: Zee <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

10 participants