Attached images in entry lost

[takedawilliam]

I always capture the QR code for 2FA and save it in the entry as backup. However, when I check the attachment, its size becomes 0B.

Expected Behavior

Here's from my previous backup of the database

Current Behavior

Difference in image size

Normal state

Current state

Possible Solution, but broken

I use my backup database as the base database, then merge my current database into it. However, some attachment are still lost. The database file size becomes smaller than before.

Before merging

After merging

Steps to Reproduce

Context

Debug Info

KeePassXC - Version 2.4.1
Revision: 7bafe65

Qt 5.12.2
Debugging mode is disabled.

Operating system: macOS Mojave (10.14)
CPU architecture: x86_64
Kernel: darwin 18.0.0

Enabled extensions:

• Auto-Type
• Browser Integration
• SSH Agent
• KeeShare (only unsigned sharing)
• YubiKey
• TouchID

Cryptographic libraries:
libgcrypt 1.8.4

[droidmonkey]

Are you using any other program to access your database?

[takedawilliam]

On iPhone, I use Keepass Touch. On Windows, I use KeePass 2.41

[droidmonkey]

Losing attachments is a big deal, we take any type of data loss seriously. I would not read too much into the total size of your database, KeePass databases employ compression before being encrypted so it could very well be the compression engine doing its job.

I am hesitant to point my finger elsewhere for this issue, but you are using three different password manager applications. I use KeePassXC and KeePass2Android and have never lost an attachment.

Please do the following to confirm the extent of the damage (on the broken db):

1. Open your database in KPXC
2. In the search bar type: attach:*. This will show all entries with attachments.
3. Open each entry and confirm if the attachment was zeroed out.
4. Report back your findings.

I am trying to see if there are any patterns. Do all of your attachments have the same file name?

[takedawilliam]

I have 59 items which have attachment and their attachments were all zeroed out. This affects all files attached, not limited to images as you can see in the below image

For 2FA images, they all have the same filename format: 2fa-.png

[droidmonkey]

Please try to replicate this issue on a test database that is NOT opened by other programs (ie, only use KeePassXC). Your problem leads me to believe one of the other programs is not writing the data for the file on saving.

Are you using KDBX4?

[SilverFire]

Hello.
I've faced the same problem on my real database. I also use KeepassXS on my Mac, Keepass 2.41 on Windows and KeePass Touch on my iPhone with the same set of plugins. I use Dropbox to synchronize files between devices.

Are you using KDBX4?

Yes, I do.

First, I've tried to restore the database by synchronizing the latest corrupted version and outdated backup. Nither broken>backup nor backup>broken synchronization works: empty files remain empty.

Then I tried to reproduce the problem on a scratch database but failed to get the same result. Maybe, you have any ideas on how to reproduce the error?

[droidmonkey]

I am wondering if the attachments are being blown away during a merge operation that occurs. Try leaving the database unlocked on your macOS, modify the database on Windows or on iPhone, then watch when the database reloads and merges into the one on macOS.

Try looking into an entry's history to see if a previous copy of the entry is pointing to the intact attachment.

[SilverFire]

Ok, I've reproduced the problem on a clean database using some of the private attachments, I had in my database. I've tried to mess up the database by editing it in KeePassXS/KeePass/KeePass touch, and I succeeded somehow, but not sure that I can do it once more. Nevertheless now I have KDBX that has non-empty attachment in KeePass 2.4.1

but it's empty in KeePassXS

issue3048_broken.kdbx.zip

Use password 3048

[takedawilliam]

I suspect that this problem is due to KeePass Touch. Sometimes it shows error dialog that there's problem when syncing my DB file due to my network is connected/disconnected. Perhaps during the sync process, it messes up my DB file.

I've been using KyPass (iPhone) for 3 weeks now and this problem hasn't occured at all (my network is still the same, up/down a lot since I move around a lot).

@SilverFire you should give KyPass a try

[droidmonkey]

On KeePass Touch's app page it lists the last fix as addressing:

Critical bugfix: For some of you your files disappeared. This patch should fix this again and show all files, imported whatever way used.

Which is very ambiguous, but seems like they have issues...

[SilverFire]

@SilverFire you should give KyPass a try

Thanks, I will!

On KeePass Touch's app page it lists the last fix as addressing:

The fix is a few months old, but yes, I suspect KeePass Touch is guilty.

The KDBX I've provided in my previous post contains uncorrupted files on Windows KeePass 2.4.1 and KeePass Touch, however, KeePassXC shows win_attach.txt as an empty file. @droidmonkey could you check why it happens?

[droidmonkey]

Yes I'll run in through the debugger

[droidmonkey]

OK so here is the problem I observed:

1. There are 4 attachment binaries located in the supplied KDBX
2. There are 5 attachments referenced
3. Attachment 5 (ref id == 4) is win_attach.txt in Entry "Attach from Windows KeePass"
4. Because there is no 5th attachment binary data the ref id points to nothing, thus the 0 byte file

My Opinion:
When KeePass Touch writes the database they have an off-by-one error that only writes n-1 attachment binaries to the database file. Save it multiple times and you get many attachments with 0 bytes.

@lbkvu did you save your database in KeePass Touch many many times?

[droidmonkey]

This should be easy to replicate if you just make 4 saves to the database in KeePass Touch then all the attachments should be 0 bytes (if my hunch is correct)

[SilverFire]

I tried to edit "Attach from Wndows KeePass" record 4 times from KeePass Touch, but it seems that nothing changed: still one corrupted file

issue3048_broken_after_4_edits.kdbx.zip

[SilverFire]

I tried to reproduce everything from scratch:

1. Create a new database on MacOS with plaintext attachment issue3048.kdbx.zip
2. Add a new record with a plaintext attachment from KeePass on Windows issue3048.kdbx.zip
3. Add another plaintext attach to "Windows" record using KeePassXC issue3048.kdbx.zip
4. Rename "Windows" record using KeePassTouch issue3048.kdbx.zip
5. Add plaintext attach to "KeePassXC" record using KeePass on windows. Choose "Binary mode" when adding attach. issue3048.kdbx.zip
6. Rename "KeePassXC" record using KeePass touch
   issue3048.kdbx.zip

Still no luck :( I'm not sure how I managed to mess it up previously.

[SilverFire]

@droidmonkey I am still curious why KeePass on Windows shows win_attach.txt in this database correctly.

[droidmonkey]

OK so I found the issue... some program is adding a duplicate binary data (win_attach.txt) which is against the new kdbx4 standard. When this happens, KeePassXC discards the duplicate binary, however the entry still points to attachment 4 which now does not exist in our internal representation of the attachment data.

This is both a failure of the KDBX4 standard (it should be using UUID or SHA256 hashes to identify attachments, not index values) and the program that is duplicating the binary attachment.

To fix this issue we would have to ignore duplicate attachment binaries, which could result in databases growing in size unnecessarily.

[phoerious]

We should build a map that maps indices to hashes and then use the hashes internally to refer to files. That way we can "correct" erroneous index pointers by mapping them to the same hash, which then points (hopefully) to the correct file. This would automatically fix issues like this when the file is saved eventually.

[SilverFire]

Thank you for the investigation. In there anything else I could do to help you fix the problem?

[droidmonkey]

We still need to figure out what triggers this situation. I am starting to think this is happening in keepass itself!

[SilverFire]

Thank you for the response and prompt fix.
Do you have nightly builds somewhere? I can check the fix on my main KDBX database

[droidmonkey]

We do but they are for develop branch. What OS are you running?

[SilverFire]

Mojave 10.14.5

[droidmonkey]

Here is a build I made, it is unsigned so install it "side-by-side" with your existing KeePassXC install: https://s3.amazonaws.com/droidmonkey-keepassxc/snapshots/KeePassXC-2.4.2-snapshot.dmg

[SilverFire]

Sorry for a late reply.

My original database with corrupted files still remains corrupted, adding new dumb record and saving does not fix old corruptions. However, the sample database I've made for this issue works well: previously missing files are available.

I opened the latest healthy (but outdated) database and merged a corrupted one into the old. The newer changes got imported and the files remain available, so I think the problem is fixed.

[gorrioncillo]

I have a similar problem, I attach a .txt file with original keepass in win10.
When I try to open it with keepassXC2.4.3 in Lubuntu 20.04 I can not because seems to be mess.
Then I figure out that it was compressed in .gzip
So simple change the extension to .gzip and open it.
Hope help others and if someone knows a better way please share