mergerfs.balance include-path option

[magma1447]

I am sorry if this is a silly question and I am just missing the obvious. But I have tried to use the balance tool as I want it to work twice now (with months in between), without success. Maybe I am just misunderstanding.

I have 4 mounted raids in /mnt/md[0-3].
The mergerfs union of them is mounted in /data.
In /data/foobar/ there are 30 TB of content. This is distributed as 6.9, 6.5, 3.7, 13 TB on each md.
Free disk space on /data is 16 TB, on each md there are 1.6, 1.6, 1.6 and 12 TB. The unbalanced values are because I have recently deleted 10 TB of data, which happened to exist on md3.

So now I want to even it out, but I don't want to move any files available on the source devices, I prefer to just move data within /data/foobar/ , and there should be enough data to even this out.

I have tried different variants with --include-path without success though.
The following three commands outputs Could not find file to transfer: exiting... and exits.

1. /usr/local/bin/mergerfs.balance --include-path /data/foobar/ -p 10 /data/
2. /usr/local/bin/mergerfs.balance --include-path /foobar/ -p 10 /data/
3. /usr/local/bin/mergerfs.balance --include-path foobar/ -p 10 /data/

I have tried to look through all closed issues and some random googling to find an example where --include-path is used, but didn't find any. Am I totally misunderstanding the --include-path option?

[zelmak]

I have the same issue.

/srv/parity1 (snapraid)
/srv/disk1
/srv/disk* /srv/storage (mergerfs)

/srv/storage/docker (docker, duh)
/srv/storage/file-shares (samba)

Added a disk at
/srv/disk2

want to balance some of my larger file shares. tried:
mergerfs.balance -p 50 --include-path file-shares/stuff /srv/storage
mergerfs.balance -p 50 --include-path file-shares/stuff/* /srv/storage
mergerfs.balance -p 50 --include-path "file-shares/stuff/*" /srv/storage
mergerfs.balance -p 50 --include-path /srv/storage/file-shares/stuff /srv/storage
mergerfs.balance -p 50 --include-path "/srv/storage/file-shares/stuff/*" /srv/storage

All give me:
Could not find file to transfer: exiting...
Branches within 50.0% range:

However, I swap it up a bit, and change it to use --exclude-path to exclude docker:
mergerfs.balance -p 50 --exclude-path "/srv/storage/docker" /srv/storage

And it kicked off doing some work. Now, that's not precisely what I want to do, because I was hoping to be able to balance by directory/path.

Am I expecting too much from mergerfs.balance? Am I misunderstanding as to what its goal is? Or is there a better way to do this?

Thanks!

[trapexit]

They are fnmatch pattern matching as the usage docs mention. If you want to match anything with a directory "foobar" then you have to use */foobar/*

Files are just the filename. Paths are the full path. You have to apply the fnmatch. Both just default to *.

[trapexit]

A spread of 50% is very large. Are your drives filled with a spread greater than 50%?

[trapexit]

If you want to balance data in a sub directory within the mount... just do that. Why are you using filters? Just make the start directory /data/foobar. The argument isn't listed as "mergerfs mount" it's listed as "starting directory".

[zelmak]

Your last bit got it working for me. I made the assumption that it was based on mountpoint, not directory.

With the -p 50, I was trying to split the content roughly in half, putting half on the the new disk and leaving the other half where it is.

[trapexit]

That's not what the percentage value means. It's the spread. The difference between the percentages between the drives. What that value should be to stop. 50% means stop when the difference between the lowest filled drive and largest is at least 50%. Like... if you have 1 drive 99% filled and the other 49% filled... stop. If you want them equally filled you set it to 1%. 0% isn't practically achievable. That's why it defaults to 2%. To accommodate for the slop of large files.

[magma1447]

The argument isn't listed as "mergerfs mount" it's listed as "starting directory"

This solved my issue. I said it probably was something silly.

Now if I could only make it move full directories recursively and not only move part of the content for each directory. But I guess that's a bit too much to wish for.

I would say the original issue can be considered close. If anything, maybe some more documentation with examples for it.

[trapexit]

Moving whole directories is an entirely different behavior and far more complicated. You're talking about bin packing. You'd have to scan all drives and attempt to calculate what subdirs are most efficient to move. That's involved. If you want to move data between drives just call rsync on the paths in question like you would for any other. If you're using the balance tool you already don't care where data lives.

[trapexit]

Examples of what exactly? It's usage for your situation is literally mergerfs.balance /some/path/ The fnmatch include and excludes are standards. I can link to the standard but providing examples of how to use something so standard seems odd.