after migration, message when try to open stack: could not get the contents of the file 'docker-compose.yml'

[Maximus48p]

Migrate from QNAP Portainer app installation to self installed docker installation.
The Qnap app was not ideally installed (and updated) and therefore i decided to install Portainer by myself (by using docker-compose, which work flawlessly accept the stacks.

I used the backup and restore functionality for migration.

I see all stacks in the list.
But when i try to open an stack, i'm getting the message could not get the contents of the file 'docker-compose.yml'
I'm not able to see the content (docker-compose.yml) of the strack and also not able to edit it. Containers are visible though which are related to the stack.
When i look at the data folder, all compose folders are perfectly migrated, so the content is there.

What i'm i missing? (it's not folder permissions)

Any help very much appreciated, thank you.

[Maximus48p]

it seems that some folder locations are hard copied in the export file "export-1686814745.json"

"com.docker.compose.project.config_files"
and
"com.docker.compose.project.working_dir"

Doesn't feel right. The new data and therefore also compose folder are indeed in the new situation in a new folder.

I have tried to change it to the new folder structure, and restored again with the new data but without luck. same issue.
It's probably also hardcoded in the DB, which i can't open without a password.

Any tricks to change the path to the /data/compose folder?

[Maximus48p]

I found a workaround... Connect the old folder structure to a new folder with the compose files.

The original folder structure from the my post above from the export file was '/share/CACHEDEV1_DATA/.qpkg/Portainer/data/compose'.

First i copied the data/compose folder to a separate folder outside data/, in my case ' /share/docker-volumes/Portainer/oldcompose'

then i created a new volume in my docker-compose file for portainer, like:

    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /share/docker-volumes/Portainer/containerstation:/data
      - /share/docker-volumes/Portainer/oldcompose:/share/CACHEDEV1_DATA/.qpkg/Portainer/data/compose

Done!
1st; The old url now redirects to the 'oldcompose' folder. So no more messages that it cannot be found anymore.
2nd; When you update your stack, it wil be saved in the /data/compose folder. Which is perfect. When you have updated, you can now remove the compose file from the 'oldcompose' folder.

Hope it helps anyone, but it can also be a bug.
Why is the foldername of the data/compose folder hard coded in de DB?

[ThatisMyGitHub]

Hi. I just found your post and I am writing to you because I have experienced the same problem.
My problem, however, is that I don't have an export file because I used the .gzip backup option before upgrading from CE to EE. So now I can't apply your solution because I can't edit the .js with the new folder structure as you so clearly explain.

If you read my message and have the time could you please be so kind to help me to fix this? Everything is running fine in my new Portainer EE instance, but obviously I can't take advantage of all the stacks I'm currently running when I want to modify/redeploy anything.

Is there a way to put the docker-compose.yml for every stack (I have them saved in the backup) in the right place so Portainer could access it when loading the stack in the GUI?

Thanks in advance for reading me.

[ThatisMyGitHub]

After reading your post for the third time I realised that the changes *the mapping between the old and the new directories had to be done in the portainer installation command, not in the already installed dependent containers. Everything is perfect now. No error reading the docker compose files and the stacks can be edited again. Thanks again.

[jamescarppe]

This is one of the drawbacks of using bind mounts, and is why we recommend using a named volume (portainer_data) for your Portainer database. When migrating, you would create a portainer_data volume on the new system, populate it with the contents from the old volume, then mount it as required.

it seems that some folder locations are hard copied in the export file "export-1686814745.json"

Where is this export file coming from? We don't generate one when doing a Portainer backup.

[focher]

I had the same error but the issue was that Portainer was looking for the docker-compose.yml file in the v1 subfolder, which did not exist. I believe the versioning was a new addition. I simply created a folder called v1 and moved docker-compose.yml to it.

[anluoma]

Was also having probles with this one. In our case we moved Portainer from Windows to Linux and there was 2 issues:

1. Backup taken from portainer running windows does not work, it creates backup file structure wrongly. Created a bug for it: Portainer creates backup package structure wrongly when backup is taken in windows server. Could be also charset issue. #11080
2. it is indeed so that compose file path is stored into the database in bucket stacks and in document tree value ProjectPath. In our case for windows it was 'C:\\data\\compose' and when portainer was moved to linux it didn't work as it was searching from '/data/compose'. I couldn't figure any other way than modifying the backup database (portainer.db) and update the ProjectPath value there. Code for that is below if somebody needs it.

That export file mentioned in other comments is just a export and it seems not to be used when restoring db. @jamescarppe If I understood correctly that export is done here:

func (c *DbConnection) ExportJSON(databasePath string, metadata bool) ([]byte, error) {
	log.Debug().Str("databasePath", databasePath).Msg("exportJson")

Go code to modify the ProjectPath in stacks bucket (or any string that has specified chars). It uses portainer.db file directly, which you can get by creating a backup from UI. It expects portainer.db to be in C:\tmp folder.

package main

import (
	"bytes"
	"fmt"
	"log"
	"os"

	bolt "go.etcd.io/bbolt"
)

func main() {
	db, err := bolt.Open("c:/tmp/portainer.db", 0600, nil)
	if err != nil {
		log.Fatal(err)
	}
	defer db.Close()

	err = db.Update(func(tx *bolt.Tx) error {
		bucket := tx.Bucket([]byte("stacks"))
		if bucket == nil {
			return fmt.Errorf("bucket stack not found")
		}

		c := bucket.Cursor()

		for k, v := c.First(); k != nil; k, v = c.Next() {
			fmt.Printf("key=%s, value=%s\n", k, v)
			if bytes.Contains(v, []byte(`C:\\data`)) {
				newValue := bytes.Replace(v, []byte(`C:\\data`), []byte(`/data`), -1)
				err := bucket.Put(k, newValue)
				if err != nil {
					return err
				}
			}
		}

		return nil
	})

	if err != nil {
		fmt.Printf("Could not update the database: %v\n", err)
		os.Exit(1)
	}
}