Sharing groups

[ckieschnick]

In many cases passwords should be kept secret and should not be shared with anyone, sometimes it is tiresome to give everybody access to the specific passwords without publishing the rest. Since keeping multiple databases does not do any good, especially when many people may need to access and update them, we created an extension which allows to share groups of entries with multiple peers while keeping others private without the need of multiple databases and regardless of the file location.

Description

The extension allows to export/import or synchronize(=import+export) groups with their contained entries to other parties. The shared data is placed in a share container which may be published by any means (i.e. cloud, network share, ...). Any client can import this container as long he has the password.
The sharing container is protected using a certificate which at least ensures that the source is the expected one, so the client may decline to import data from an untrusted source.
The features does make use of the synchronization (#1992) and merges data in time order (newest data will be on top, older data will be moved to the history).
All data required to work for KeeShare is stored either in KeePassXC-Settings or CustomData, so the extension is fully compatible with existing databases.

Motivation and context

It is often necessary to publish certain secrets to a selected group of people but protecting those secret none the less. Just imagine a company which wants to distribute passwords to some employee groups (i.e. banking data just for the managers, root passwords for admins). Those passwords can be published on internal sites, but keeping them in a password manager makes it easy to protect them. If the password manager takes care, that the passwords are up to date, even when it is not centralized, it would be even better. That's why we introduced the group sharing feature (KeeShare) to KeePassXC. There is no need to keep multiple databases around which may have problems when shared using some technologies.

Furthermore, the extension addresses some more wishes and improvements from the issue list:

• import features resolves Support bidirectional sync with a different database file [$10] #90 Option to Sync (on changes) from another external file (just make the root group an import group)
• export feature resolves Allow user to define automatic DB backup location and name #2062 Allow user to define automatic DB backup location and name (just make the root group an export group and specify the location - to open the backup, one has to import it using the password)
• the merge may be used feature to resolve Bidirectional Merge/Synchronization #637 (Bidirectional Merge/Synchronization)

Problems and Tickets which may gain in importance:

• sharing without history support may lead to data loss
• sharing with autosave on change prevents the user from reviewing and discarding the changes of an import and may lead to unnecessary exports
• Provide a visual indication for entries that contain unsaved changes [$5] #1162 (Provide a visual indication for entries that contain unsaved changes) may make the change tracking of imports easier (depends on the visualization of changes during the import)
• User should be warned before deleting entry with a UID that is being REF'd by another object #852 (User should be warned before deleting entry with a UID that is being REF'd by another object)/Warn user if deleting entries that are referenced. #1744 (Warn user if deleting entries that are referenced.) may improve sharing since references would be used to export an entry to multiple shares

Possible use cases are:

• bidirectional sharing between different individuals or groups without the need of maintaining several databases at the same time (i.e. using a network share or dropbox)
• simple backup of parts of a database or the whole database to a specific location (using only the export feature)
• unidirectional sharing of specific entries (using export on the master and import on the slave client) There may be more use cases for the sync feature which may help with certain restrictions/adaptions (i.e. auto import of a CSV file into a group - may need an adaption of the merge logic to reduce duplicates)

How has this been tested?

There are extensions to the test suite, but a lot of functionality is tested manually. I'm note sure if a UI-test would be justified.

Screenshots (if appropriate):

Please look in the updated QUICKSTART

Types of changes

• ✅ New feature (non-breaking change which adds functionality)

Checklist:

• ✅ I have read the CONTRIBUTING document.
• ✅ My code follows the code style of this project.
• ✅ All new and existing tests passed.
• ✅ I have compiled and verified my code with -DWITH_ASAN=ON.
• ✅ My change requires a change to the documentation and I have updated it accordingly.
• ✅ I have added tests to cover my changes. - As far as the current abstraction allowed.

[phoerious]

Hey, thanks for your follow-up PR. Could you please check why the tests are failing and resolve the merge conflicts? Then we can start reviewing your changes.

[ckieschnick]

CodeFactor fails currently due to an intended duplication in FileDialog. I'm not sure how it was intended to use the methods, therefore I didn't try to put my changes into an existing method to prevent unintended changes at other places.
TeamCity has just one issue (for now) - Release [GCC, AppImage] couldn't fetch packages from a specific ubuntu repository - seems to be a transient failure to me. The other images seem to work now.

[droidmonkey]

That means we need to increment the version number in the docker script so it rebuilds the image. You can ignore the code factor finding as long as what you wrote here is documented in comments.

[droidmonkey]

@ckieschnick sorry for the incredible delay on this PR. We had to clean house a little first. I will rebase this after we merge your first PR. Any additional comments?

[ckieschnick]

I'm currently quite busy, so I couldn't track the progress - therefore I have nothing to comment. In case problems arise, I'll try to support you.

[ratcashdev]

Please consider adding the option of keeping a configurable number of backups (e.g. 3 older backups), not just one.

[droidmonkey]

Some issues I found right away:

• When adding sync to a group, all the other groups were instantly deleted
• It is not possible for two folders in the same database to import from one that exports.
• Inconsistent syncing occurs when two folders in the same database point to the same kdbx.share, they should be independent. One entry went to one folder, the others went to another folder.
• quazip does not have a Qt5 build on Trusty. Investigate storing the signing certificate within the kdbx metadata itself.

[ckieschnick]

For clarification:

• The feature is not intended to sync groups within a database but to sync groups between different databases. It may be possible to have a master slave setup with one group exporting while the slave do only import, but this is more a side effect than an intended feature.
• "Merging" multiple groups into a single kdbx.share (using sync or export) is no use case - each group overwrites the share therefore exporting multiple times from within one database into the same share would lead to lost updates.
• We used QuaZip for simplicity, since the container format is more easy to read and validate (no need to read anything of the container). Another advantage is, that it is easy to use the unpacked container as backup and even checking the signature with third party tools would be easier when the signature is not stored within the container.

Regarding your first point - I didn't have time to check the implementation against the current develop in the last month, but this problem seems weird. I'm currently quite busy, but I may be able to help, if you can provide some more information on this issue.

[droidmonkey]

@ckieschnick no problems, these were more notes for me to investigate further and provide fixes. It is very very likely we are going to drop the QuaZip usage, it currently does not have a port for Qt5 on Trusty and possibly not on macOS either. We could integrate it directly, but I'm sure the Debian folks will have a cow about that.

I am mainly trying to find where the plugin breaks down so we can either add guards to warn or prevent the user from making a mistake.

[dsonck92]

I've been testing this feature today. Very good functionality. However, the first thing which struck me as odd (even though it's mentioned in the docs) is the flat initial syncing.

• While I like the idea of being able to move the items outside of the sync group while maintaining the sync, this should be visually represented in a way. I currently have no "dumb trust" it will remain connected with the sync container.
• Another thing. When I first used it, and it synced everything without hierarchy, I first thought it didn't sync the subitems. I later realized it did in fact sync over, but as I have a huge but organized collection of passwords, it basically makes the first sync very messy. All items are merged into the top group. That means I need to recreate the full structure on the other sync. It might be useful to include a checkmark "Sync subgroup structure". To allow both syncing with and without preserving the paths of the keys. For those that do want to keep the hierarchy synced as well.

[droidmonkey]

@dsonck92 great points, this does require a little bit of work. I think the default behavior should be to maintain structure, it does not make any sense to eliminate structure that is created.

[ckieschnick]

I'll try to address the points in the same order as given:

• Your proposal using KEESHARE and KEESHARE_SECURE is okay for me from the build configuration perspective. Within the code, I'll would like to keep the separation of KEESHARE_SECURE / KEESHARE_INSECURE (it is written in a way which would allow to exclude KeeShare completely, only the secure, only the insecure or include both secure and insecure).
• What do you want to check? For me, the import seems to work as soon as the configuration for the group is accepted. The activated sharing on a group is indicated by the small badge and the "share" tab in the group details.
• I'm not sure which button do you mean.
• Persistent warnings - seems better. I'll look into it.
• Seems like a bug to me, but I'm have no idea how you were able to create multiple warnings for the same Sync-Container since the scope seems to be identical.
• I'm open for suggestions since I'm not happy with terminology too. Secure means that beside the password the source of the container is trusted. Insecure means, that the password matches, but the source cannot be verified so you have to trust the container blind. As long as only trusted people can access the shared filesystem where container resides, it is not insecure - but in an "open" scenario where anybody may read or write the shared container, one may import data blindly which may lead to undesired results (i.e. a changed entry with enough history to remove any valid data, so the data is lost).
• I forgot to check the MSYS-build.

[ckieschnick]

I could reproduce the problem with the multiple dialogs - it's a problem with the FileWatcher which does signal a file change multiple times, but it is expected to be send exactly once. The quick and dirty way would be to delay the permission for some milliseconds to lunch the dialog after all signals were emitted, but I'll try to locate the root cause for now.

[droidmonkey]

Awesome, I tested everything out and it works much better. I think we can do a code review and merge it in afterwards. Any minor ui/ux issues can be cleaned up in a new PR.

Thank you so much for your work on this and the previous PR. Many many people around the world are looking forward to this feature!

[droidmonkey]

Reviewed all files and tested functionality, this is ready for merge.

[droidmonkey]

@ckieschnick great work on this, it is now a part of 2.4.0

[Guite]

Can someone please briefly explain the essential difference of this new addition and the AutoOpen functionality? If I understood the quickstart document correctly there is an additional kdbx file here, too.

[droidmonkey]

AutoOpen keeps the database OUTSIDE of the currently used database. KeeShare brings the database INSIDE to the currently used database. Also the ability to IMPORT, EXPORT, SYNCHRONIZE versus just SYNCHRONIZE with AutoOpen.

[Guite]

Thank you 👍

[AlexanderAmelkin]

So, do I understand it right that you actually can not share the database with other versions of Keepass (e.g. Keepass2Android) and the implementation is essentially useless for mobile users because there is no KeePassXC for Android and no other keepass client has support for this sophisticated mechanism?

[droidmonkey]

You cannot update the share (currently) with KeePass2Android. However, the main database you use will have all shared entries stored within it and are accessible by other keepass clients. KeePass2 has support for the KeeShare implementation via a plugin.

[hi-ko]

The mentioned quickstart guide is no longer available (404) and should be updated to QUICKSTART.md

I also tested the share functionallity in my family and we ended up with a lot of confusion: duplicate entries, keys are not moved to new locations, users do not get when and why entries stop syncrhonization. I think to make "Sharing groups" working the way users expect would require more work than implementing a real sync like available in the C# keepass which is working quite good.
If following the "Sharing groups" aproach the node should contain visible metadata and maybe a visible indicator that it is a shared node and if it is "in sync" - otherwise I wouldn't trust this implementation with my testing experience.

In long term I would prefer a real database approach (postgres, mysql) for managing shared passwords beeing able to define group permissions, audit logs. This could be implemented as a simple and secure REST service in the first step and extended by messaging (e.g. rabbitmq) to "push" changes to the clients while they're connected. I'm willing to provice the server part with my company but we don't have any c skills for the client part.

[droidmonkey]

Please open a new issue (search first please) for any suggestions or bugs you come across. Please don't add to this completed PR.