Gitea PostgreSQL fails to create /bitnami/postgresql directory - permission denied

[rknightion]

Following the k3d quickstart
MacOS
Docker CE 20.10.7
k3d v4.4.7
Single node using rancher.io/local-path storage driver (default)

Gitea fails to deploy. Specifically, the gitea postgres container fails to deploy with the following in the logs:

│ postgresql 12:29:51.56 INFO  ==> ** Starting PostgreSQL setup **                                                                                                     │
│ postgresql 12:29:51.57 INFO  ==> Validating settings in POSTGRESQL_* env vars..                                                                                      │
│ postgresql 12:29:51.58 INFO  ==> Loading custom pre-init scripts...                                                                                                  │
│ postgresql 12:29:51.58 INFO  ==> Initializing PostgreSQL database...                                                                                                 │
│ mkdir: cannot create directory ‘/bitnami/postgresql’: Permission denied                                                                                              │
│ postgresql 12:29:51.59 INFO  ==> Stopping PostgreSQL...

Not sure if this relates to #615, but as different platform logging here.

[andreas-kupries]

Attempting to reproduce failed for me.
Data points from description and my attempts, in tabular form:

OS	Docker	k3d	Epinio	Result
MacOS	20.10.7	4.4.7	?	💥
Linux	20.10.6-ce	4.4.4	v0.0.19-33-g2de03c8	✔️
Linux	20.10.6-ce	4.4.7	v0.0.19-33-g2de03c8	✔️
Linux	20.10.6-ce	4.4.7	v0.0.19	✔️
Linux	20.10.6-ce	4.4.7	v0.0.18	✔️

Differences left to the original report: OS, Docker version, maybe Epinio version.

What version of Epinio did you use ?
Do you have a Linux box available to test with and compare ?

[agracey]

This is an issue with HostPath and NFS volumes. I just opened #684

[kkaempf]

I see this now in Windows with Rancher Desktop (RD) v0.4.0. It didn't happen with RD v0.3.0

[kkaempf]

This is an issue with HostPath and NFS volumes. I just opened #684

Not sure about this.

A local build with this patch

--- a/deployments/gitea.go
+++ b/deployments/gitea.go
@@ -202,7 +202,7 @@ postgresql:
        defer os.Remove(configPath)
 
        if out, err := helpers.RunProc(currentdir, k.Debug,
-               "helm", action, "gitea", "--values", configPath, "--namespace", GiteaDeploymentID, giteaChartURL); err != nil {
+               "helm", action, "gitea", "--values", configPath, "--set", "volumePermissions.enabled=true", "--namespace", GiteaDeploymentID, giteaChartURL); err != nil {
                return errors.New("Failed installing Gitea: " + out)
        }

does not fix this problem.

[jandubois]

@kkaempf Since you are deploying postgresql as a subchart, you have to prefix any value changes with the subchart name:

... --set postgresql.volumePermissions.enabled=true ...

[jandubois]

FWIW, postgresql deployed fine for me with Rancher Desktop 0.4.0 on macOS even without the permissions change, but it should be safer to always include that setting.

[ldevulder]

FYI, I tried on a k3s cluster installed on a Leap15.3 and I had the same error. Here the logs:

k3s-test-vm:~ # k logs -n gitea gitea-postgresql-0 -c gitea-postgresql
postgresql 10:22:04.11 
postgresql 10:22:04.11 Welcome to the Bitnami postgresql container
postgresql 10:22:04.11 Subscribe to project updates by watching https://github.com/bitnami/bitnami-docker-postgresql
postgresql 10:22:04.12 Submit issues and feature requests at https://github.com/bitnami/bitnami-docker-postgresql/issues
postgresql 10:22:04.12 
postgresql 10:22:04.13 INFO  ==> ** Starting PostgreSQL setup **
postgresql 10:22:04.15 INFO  ==> Validating settings in POSTGRESQL_* env vars..
postgresql 10:22:04.16 INFO  ==> Loading custom pre-init scripts...
postgresql 10:22:04.16 INFO  ==> Initializing PostgreSQL database...
mkdir: cannot create directory ‘/bitnami/postgresql’: Permission denied
postgresql 10:22:04.17 INFO  ==> Stopping PostgreSQL...

[ldevulder]

I also tested with k3d installed on Leap15.3 and I have the exact same issue/error message.

[andreas-kupries]

Now that is interesting. I am on Leap 15.3 as well, with k3d and do not have the issue.

Below what I know about my system so that you can compare to your config and actions.

My configuration:

• Docker version 20.10.6-ce, build 8728dd246c3a
• k3d version v4.4.4, k3s version v1.20.6-k3s1 (default)
• kubectl : version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.0", GitCommit:"af46c47ce925f4c4ad5cc8d1fca46c7b77d13b38", GitTreeState:"clean", BuildDate:"2020-12-08T17:59:43Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
• helm version.BuildInfo{Version:"v3.3.1", GitCommit:"249e5215cde0c3fa72e27eb7a30e8d55c9696144", GitTreeState:"clean", GoVersion:"go1.14.7"}
• Epinio Version: v0.0.22, Go Version: go1.14.15

Commands to stand up the k3d/k3s cluster

• k3d cluster create epinio-acceptance --k3s-server-arg --disable --k3s-server-arg traefik
• k3d kubeconfig get epinio-acceptance > tmp/acceptance-kubeconfig

Then installed epinio using

• ep install |& epi.log

ep is link to the locally built epinio, with the version above.
As I tested the v0.0.22 release it was not necessary for me to do the whole dance which would be required for a dev version, like head of main.

Note that the above commands are all done on my Leap host. This host is not a VM, and I am not using a VM to segregate docker and k3d either. with everythng up (cluster and epinio) docker reports:

work@tagetarl:~/SUSE/dev/epinio> d ps
CONTAINER ID   IMAGE                                  COMMAND                  CREATED         STATUS         PORTS                                                              NAMES
05040693de54   rancher/k3d-proxy:v4.4.4               "/bin/sh -c nginx-pr…"   7 minutes ago   Up 7 minutes   80/tcp, 0.0.0.0:40995->6443/tcp                                    k3d-epinio-acceptance-serverlb
fcde048d7fdd   rancher/k3s:v1.20.6-k3s1               "/bin/k3s server --d…"   7 minutes ago   Up 7 minutes                                                                      k3d-epinio-acceptance-server-0
3c6520dec15a   rpardini/docker-registry-proxy:0.6.0   "/entrypoint.sh"         2 days ago      Up 2 days      80/tcp, 8081-8082/tcp, 0.0.0.0:3128->3128/tcp, :::3128->3128/tcp   registry-proxy

Hope that helps

[ldevulder]

@andreas-kupries I'm on a VM, but with all installed on the VM, so should be the same as on bare metal.
I will use the same command as you and try to check the differences.

[andreas-kupries]

I am not as sure that being in a VM is the same a bare metal in this.

When I was working on enhancing the Windows portability of Epinio @kkaempf was able to run RD (pre 0.4) on Windows 10 baremetal. Me trying to use the same RD on a Windows 10 VM hosted on a Leap 15.3 Linux box (or Ubuntu 20.04) RD was unable to start kube, errorring out with an WSL error (i.e. the Windows Subsystem for Linux).

[ldevulder]

@andreas-kupries So I retry ion my VM with your commands but I have an issue, maybe because of the options on the k3d cluster create command. Anyway, I will retry to check why.

I also installed a fresh Leap15.3 in Server mode (like in the VM) on an old laptop that I have and I followed the installation steps from the k3d quickstart and I have the same issue on it. So it seems not related to a VM.

Next step is to find the differences :)

EDIT: the output of epinio installer is a little bit different this time:

✔️  Registry deployed
...
⚠️  Deployment gitea failed with error: failed waiting Gitea postgresql deployment to come up: Failed waiting for gitea-postgresql-0: timed out waiting for the condition
Pod Events: 
Successfully assigned gitea/gitea-postgresql-0 to k3d-epinio-server-0
Container image "cr.l5d.io/linkerd/proxy-init:v1.3.11" already present on machine
Created container linkerd-init
Started container linkerd-init
Pulling image "docker.io/bitnami/postgresql:11.9.0-debian-10-r34"
Successfully pulled image "docker.io/bitnami/postgresql:11.9.0-debian-10-r34" in 51.826012867s
Container image "cr.l5d.io/linkerd/proxy:stable-2.10.2" already present on machine
Created container linkerd-proxy
Started container linkerd-proxy
Started container gitea-postgresql
Container image "docker.io/bitnami/postgresql:11.9.0-debian-10-r34" already present on machine
Created container gitea-postgresql
Back-off restarting failed container

But the detail is the same:

hp-k3d-test:~ # kubectl logs gitea-postgresql-0 -n gitea -c gitea-postgresql
postgresql 12:31:47.95 
postgresql 12:31:47.96 Welcome to the Bitnami postgresql container
postgresql 12:31:47.96 Subscribe to project updates by watching https://github.com/bitnami/bitnami-docker-postgresql
postgresql 12:31:47.97 Submit issues and feature requests at https://github.com/bitnami/bitnami-docker-postgresql/issues
postgresql 12:31:47.97 
postgresql 12:31:48.00 INFO  ==> ** Starting PostgreSQL setup **
postgresql 12:31:48.04 INFO  ==> Validating settings in POSTGRESQL_* env vars..
postgresql 12:31:48.06 INFO  ==> Loading custom pre-init scripts...
postgresql 12:31:48.06 INFO  ==> Initializing PostgreSQL database...
mkdir: cannot create directory ‘/bitnami/postgresql’: Permission denied
postgresql 12:31:48.09 INFO  ==> Stopping PostgreSQL...

[kkaempf]

Confirmed fixed, postgresql runs nicely now. But lead to #732 😠 🤦🏻‍♂️