PostgreSQL backup fails

[imajus]

To Reproduce

1. Deploy PostgreSQL database as a service in Dokploy
2. Set up S3 destination via Dokploy settings
3. Set up PostgreSQL backup schedule
4. Trigger PostgreSQL backup manually

Current vs. Expected behavior

I'm getting the following error in the Dokploy container logs:

Error: Command exited with code 127. Stderr: bash: line 1: rclone: command not found
, command: docker exec c43f***1df sh -c "pg_dump -Fc --no-acl --no-owner -h localhost -U postgres --no-password '***' | gzip" | rclone rcat --s3-access-key-id=*** --s3-secret-access-key=*** --s3-region=*** --s3-endpoint=https://*** --s3-no-check-bucket --s3-force-path-style ":s3:backups//***.sql.gz"
    at Channel.<anonymous> (/app/.next/server/chunks/8786.js:226:1832)
    at Channel.emit (node:events:529:35)
    at Channel.doClose (/app/node_modules/.pnpm/[email protected]/node_modules/ssh2/lib/utils.js:101:21)
    at Object.onceWrapper (node:events:631:28)
    at Channel.emit (node:events:529:35)
    at endReadableNT (node:internal/streams/readable:1400:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)

Provide environment information

OS: Ubuntu 24.04 x86_64
Dokploy version: 0.12.0
PostgreSQL Docker image: postres:17

Which area(s) are affected? (Select all that apply)

Databases

Additional context

No response

[Siumauricio]

I need more details, what s3 provider are you using?, deploying in remote server or dokploy server?

[imajus]

I'm hosting Dokploy on my own VPS. PostgreSQL instance is running on the same VPS as the Dokploy panel.

And it seems that the issue is not related to the S3 destination set up, based on the error message:

rclone: command not found

[Siumauricio]

what VPS provider are you using? is really strange I tested with postgres 17 and is working fine, I don't consider rclone is not a recognized command because we install here https://github.com/Dokploy/dokploy/blob/canary/Dockerfile#L48

The only way it doesn't recognize the command is simple, you are trying to deploy an application on a remote server, but you didn't setup the remote server and therefore it doesn't have rclone installed, that's the only way I can think of

[imajus]

Indeed, I have the rclone installed in the Dokploy container:

# docker exec 8f25f47d7a31 rclone --version
rclone v1.68.2
- os/version: debian 12.8 (64 bit)
- os/kernel: 6.8.0-49-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.23.3
- go/linking: static
- go/tags: none

Interestingly, trying to execute similar command from inside the Dokploy container it works as expected:

# docker exec e13a5714 sh -c "uname -a" | rclone --version
rclone v1.68.2
- os/version: debian 12.8 (64 bit)
- os/kernel: 6.8.0-49-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.23.3
- go/linking: static
- go/tags: none

Also, if I just copy-paste-execute the command from the error message it also works and I get the backup uploaded to the S3 bucket.
It seems that the problem lies in the way the command is triggered from the Dokploy backend.

[Siumauricio]

I have 4 servers and they have no problems, which vps provider are you using?

[imajus]

I've made some further investigation and found this source code in the Dokploy backend:

if (postgres.serverId) {
	const { Id: containerId } = await getRemoteServiceContainer(
		postgres.serverId,
		appName,
	);
	const pgDumpCommand = `docker exec ${containerId} sh -c "pg_dump -Fc --no-acl --no-owner -h localhost -U ${databaseUser} --no-password '${database}' | gzip"`;
	await execAsyncRemote(
		postgres.serverId,
		`${pgDumpCommand} | ${rcloneCommand}`,
	);
} else {
	const { Id: containerId } = await getServiceContainer(appName);
	const pgDumpCommand = `docker exec ${containerId} sh -c "pg_dump -Fc --no-acl --no-owner -h localhost -U ${databaseUser} --no-password '${database}' | gzip"`;
	await execAsync(`${pgDumpCommand} | ${rcloneCommand}`);
}

So apparently, in case of the remote PostgreSQL deployment, the command is executed with execAsyncRemote as opposed to a local deployment for which execAsync is used. And there's a conceptual difference between these two:

• execAsync executes a command inside the Dokploy container
• execAsyncRemote executes a command inside the remote server host system

Even though I have everything installed on a single VPS, I've added that server in the Dokploy settings. And when I created the PostgreSQL database instance, I selected this server, so the serverId column for it in the Dokploy database is non-empty. That causes the backup command to be executed with execAsyncRemote, which fails because I don't have the rclone installed on the host system.

These are all my personal judgements, which may be wrong due to my limited experience with Dokploy. But I don't have any other way to debug the issue locally.

FYI I'm using local VPS provider but I have a full root access to the instance.