Prune does not reduce repository size

[ppawel]

Have you checked borgbackup docs, FAQ, and open Github issues?

Yes.

Is this a BUG / ISSUE report or a QUESTION?

QUESTION

System information. For client/server mode post info for both machines.

Your borg version (borg -V).

borg 1.1.9

Operating system (distribution) and version.

Kubuntu 19.04

Linux t470s 5.0.0-13-generic #14-Ubuntu SMP Mon Apr 15 14:59:14 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

How much data is handled by borg?

Small repository (size ~ 200MB)

Describe the problem you're observing.

In my repository I am backing up my Firefox profile, like this:

cd ~/.mozilla
borg create --list --filter AME --chunker-params 8,18,16,4095 -s ~/backup/borg::mozilla-"$TSTAMP" . \
  --pattern='-**/datareporting' \
  --exclude-caches

After that I run borg prune to only keep one snapshot, like this:

borg prune -P mozilla- -y 1 -s --save-space --force --list ~/backup/borg

The problem is that the repository size on disk is not being reduced at all even though prune is removing snapshots correctly. For example, I added the exclude pattern for the datareporting directory yesterday, ran the backup and the repository grew by ~2MB - I would expect the old snapshot which contained the datareporting to be removed completely and replaced with the new, smaller snapshot...

Is this intended behavior?

Note: when I run borg recreate, the repository indeed gets smaller but I don't think it should be necessary to recreate the repository during every backup / periodically, or am I wrong?

[ThomasWaldmann]

200MB is relatively tiny, the default segment size of borg is 500MB, so that all would even fit into 1 segment file.

For compaction, there is some threshold, if the hole sizes compared to the segment file size is below some ratio, it won't compact it because the effort would be relatively high compared to the space saving.

Also, check if your repo is append-only, in that case, it of course would never shrink.

[ppawel]

Thank you for the quick answer. I didn't know about the segment size... I guess it's a compile-time parameter, not modifiable at runtime?

Regarding compaction... perhaps would be nice to be able to configure the ratio per repository, if it's not a lot of development effort... just an idea.

[ThomasWaldmann]

segment size is in repodir/config maybe also settable via borg config (or just carefully use a text editor).

you can change it at any time, but it will only affect future segments.

[ppawel]

Wow, very cool, thank you!