Remote database support [$276]

[leethax666]

It would be great to have support for WebDav, SFTP, or other protocols to load and save a database. I have a webdav folder on xubuntu, but it doesn't show up on the Open File window.

---

IssueHunt Summary

Backers (Total: $276.00)

• kfdcompiled ($20.00)
• ppmathis ($68.00)
• haizaar ($20.00)
• altendky ($50.00)
• mikescher ($6.00)
• zeezeeeff ($20.00)
• $92.00 have been anonymously funded.

Become a backer now!

Or submit a pull request to get the deposits!

Tips

• Checkout the Issuehunt explorer to discover more funded issues.
• Need some help from other developers? Add your repositories on IssueHunt to raise funds.

[bmelman]

Exctly what i am missing: something like the IOProtocolExt from keepass, this enables simple access to a securely stored keepass database.

[jazz-it]

I fully support the idea... just installed KepassXC for one reason only - to use it as an interface for my shared database on NAS, but unfortunately it's not supported yet.

[droidmonkey]

It is not likely this will be supported in the near future. Every modern OS can mount a WebDAV folder to be made accessible to KeePassXC. Likewise, SMB and NFS folders can be mounted as well. There is really no practical reason to spend significant effort embedding this capability in KeePassXC itself.

@leethax666 I missed this original issue. It is likely you are using the snap version of KeePassXC which is sandboxed and does not let you see some folders. Try mounting your WebDav within your home directory and try again.

[marcbone]

Yes every computer can mount webdav, but this requires that there is a permanent internet connection.
People also want to use their database when there is no internet available.
This is no problem when you use dropbox or nextcloud since they have sync clients which download your files, make them available offline and then taking care of changes.

However I couldnt find any good webdav client for linux. So what I do is the following:

• Mount webdav storage on boot
• use a cron task to save the database every 15 minutes to my home folder

The problem is that when you have no internet, keepassxc opens with an error (database not found or something like that) and I have to manually select my backup database. However, I have to remember to change again to the webdav database when I have Internet otherwise my changes would be overwritten by the cron task.

Thats why it would be convenient if I could just connect to webdav in keepassxc and keepassxc takes care of downloading my database (or use a cached one if there is no internet) and notifying me if there are any conflicts (database changed remotely) when i want to save my database.

I agree that it is a bit out of scope for a password manager to take care of syncing issues with webdav. But it would make using a password manager easier for a lot of people who simply put their database on the cloud storage from their mail provider.

[droidmonkey]

Why not just use a cloud service provider, even syncthing or owncloud? You own all the technical challenges when you use webdav.

[marcbone]

The problem with syncthing is that all my devices are located in my apartment if I am at home. If my apartment burns down, my database is lost. So the proper way to deal with this is rent a vps-server and set up a nextcloud. However it takes some effort and maintenance work to keep everything running and secure (and money 😉 ). I already manage all other things (cardDav, calDav) with my email provider (They use open-Xchange which is really cool, but sadly they dont offer a sync client), so it is just easier and cheaper for me to also let them handle my database.

I guess that is true for many people and thats why it is an often requested feature. So please dont close it with a "wont fix", so someday somebody can come up with a merge request.

[droidmonkey]

Oh it'll remain open, I'm not entirely against this feature. I think it is best paired with plugin/scripting/trigger support though.

[datenwolf]

I'd like to point out another reason for having some form of remote access: Limiting the surface area for brainfart fuckups: Accidently overwriting or deleting a password DB file and the latest backup snapshot didn't yet catch the latest changes.

The other reason might be managing read/write permissions, e.g. for shared access. (Why would you want to share access to a password file you ask: Think organization wide logins to supplier web order systems or similar, where the vendor is incapable of providing sub-accounts for individual persons).

[pandruszkow]

I sympathise with @marcbone, and also found WebDAV very fragile and dependent on connectivity. I experienced some data loss when using mounted WebDAV, which makes me understandably reluctant to entrust it with my password database.

At one point, I mounted my Nextcloud WebDAV share on my Linux box, then moved (admittedly a fairly large, 2GB) file to the share directory. The file disappeared from the source directory and never appeared at the destination, even though I have a fast connection. I no longer trust WebDAV to not lose my data, unless the application supports WebDAV access/mounting natively.

I've evaluated KeepassXC before, and the main reason I didn't switch from Keepass2 was lack of native WebDAV support. I would switch in a heartbeat if this was implemented. IIRC KeepassXC has much better and usable Firefox integration, and it would be worth switching for that reason alone.

[droidmonkey]

Using WebDAV through an application does nothing to resolve reliability issues with the WebDAV protocol itself. If you are concerned with reliability, versioning, and other modern concerns.... don't use WebDAV!

[pandruszkow]

It's easier to know when you failed a password DB upload with native DAV support, and you are able to retry the upload explicitly until you confirm you succeeded for sure. A mounted davfs will simulate a POSIX filesystem, which has different assumptions about the underlying layers of the system, and will tell you everything's fine when it's not. You get no feedback about failures, delays, timeouts, hanged uploads, writes which are only present in the local /tmp cache and which haven't reached the cloud yet (and which may be lost if you happen to shut down the computer right now), and the like. You're basically relying on the davfs layer to recover from the errors and behave well, which is just not guaranteed.

[pandruszkow]

Another benefit of supporting WebDAV is that it's quite ubiquitous, and it's easier to set up a DAV server compared to alternatives. It comes with NextCloud/OwnCloud which is one of the most popular self-hosted "cloud in a box" software packages. NFS and SMB only work reliably on their respective native platforms, and they are both insecure to expose on the public Internet. FTP is quite mature, but less secure and it's actually older than both Unix and TCP/IP. S3 compatible servers like Minio and some others seem like the future, but you can't just put them on a cheap shared PHP host, and they need a special client. Please don't get see what I'm saying as ranting. I've have searched a lot around the Internet to find a good way to expose my files over the Internet with read-write capability, and DAV and FTP are the only two that will work everywhere. It's frustrating, but that seems to be the current state of support.

[droidmonkey]

Please check out syncthing. It is far superior to webdav.

[t00]

It should not be too difficult to implement HTTP(S) GET/PUT with basic authentication. For saving the operations would be:

1. GET
2. Decrypt with the current database key
3. Check for remote changes (diff from loaded)
4. If changed: Confirm -> Merge
5. PUT

When configuring a connection, cloud provider api configurations can be included (endpoint, auth type, request type) so with small modifications i.e. Amazon S3 can be used as well, without any plugins. Rest of the logic will be identical.
Please consider adding this feature as for me as well lack of this feature is stopping me from using KeePassXC.

[ilsergente1993]

+1

[omaidb]

+1

[omaidb]

请支持webdav，好让我的ubuntu不再依赖其他的网盘程序

[reetp]

Keepass2Android has stfp support......

Great cos it works with my ssh keys.

Would be perfect to do likewise on the desktop.

[processor286]

I've just moved from dropbox and g-drive with sync, to direct webdav on a private cloud server (nextcloud). The clients on Android and Windows support DAV directly, and clearly can tell if the WEBDAV save has completed successfully or not.

I did this because I've had data-destroying problems with synchronizing software (mostly insync) , so direct DAV seemed the best solution for this. And is the reason why I don't really want to use a sync solution on Linux, nor do I feel happy trusting davfs really.

Anywyay, wanted to lend some weight to exactly why I think that native webdav support would be a good thing.

[aksdb]

Just leaving another thought here on what would be an advantage of an integrated sync solution like WebDAV: security of the credentials. When KeePassXC manages the connection, I can unlock my local database, it sees the configured WebDAV, connects to it, pulls the database from there, merges, uploads and I can continue to work on the synchronized database. If another tool is required, I would have to permanently store my credentials there for a convenient workflow - so now either the OS or some other layer has Username/Password of my WebDAV share.

So for that reason alone I think the transport layer for the synchronization should reside within KeePassXC as well.

(Btw. I would also not mind having some dedicated sync protocol. That would make some aspects easier, but is all in all probably a lot more effort than "just" using WebDAV which has a ton of different possible backends.)

[altendky]

Just having a file on a locally mounted webdav connection doesn't address synchronization of data inside the database. In KeePass2 I just hit save and it syncs and merges etc up to the server. If I have to manually keep two files and 'merge from file' and so on... that's a lot of overhead vs. ctrl+s.

I understand it's a pile of added complexity so no judgement on it not being implemented presently but I think it does bring important value.

[reetp]

Yup. Local mounting via webdav/nfs/smb for users is just a no no when the database needs to be hosted centrally/cloud so they can easily access via mobile too.
Needs to be a simple one click connect to database across desktop & mobile app.
Sftp/Scp is simple to setup and use and works easily on mobile. It could be that easy on desktop.
Anything more complicated for normal users is just a roadblock.
Damn shame as this prevents us from using it.
My guess is this will go the way of email+PGP. Just too damn difficult, driven round, and distrupted by chat.

[droidmonkey]

The bounty will be satisfied if minimally WebDAV, NFS, and SMB are supported natively within KeePassXC. SSH/SFTP is a bonus, but certainly not a requirement since it requires a significant amount of additional dependencies.

[Lantizia]

In principle... I also agree that, in an ideal world, any mounting/syncing aspects shouldn't be part of the KeePassXC program.

But in reality... it often stinks with ongoing problems and proves to be a major roadblock for new users.

The same argument rages on between users of Keepass2Android v.s. KeePassDX (on Android) and Strongbox v.s. KeePassium (on iOS)... where Keepass2Android & Stongbox both have inbuilt native support for these functions.

Thought to consider...

1. Mounting network locations (without root access) on Linux is still quite unintuitive for newbies. Whilst things like GVfs try and help, it's got very limited configuration options when mounting. usually obscures the real path names, and uses different URI schemes which confuse many applications. Things like FUSE based mounts and davfs2 often require a lot of advanced CLI knowledge, and davfs2 doesn't let you mount anything (as a non-root user) without first adding your user to a special group... and even then has had known bugs (like an odd reliance on needing /etc/fstab to be readable, when it doesn't need it as a non-root user in ad-hoc situations).

2. Also, mounting any network location (on any OS) still means it'll often vanish upon things like hibernation/standby, changing Wi-Fi network, VPN's restating, loss and regaining 4G service, etc... mostly since the OS expects a constant ongoing connection. When it momentarily can't poll it, it'll often be left unmounted or cause crashes, needing to manually reconnect later. But since KeePassXC only needs a momentary connection, this will always prove to be a more positive experience. This isn't an argument for every application to do their own mounts (although some can, like MS Word), but given the KDBX file is both a) being read/written to by potentially multiple KeePass compatible clients by potentially multiple users simultaneously and b) it's the central nexus of all your passwords (and maybe TOTP)... then it's super important that it be always available and accurate... far more than your average file.

3. External tools (e.g. Dropbox, Nextcloud, and again even GVfs) are known to cause issues with the saving, which is the whole reason why there are now two options for an 'alternate saving method', which again isn't an intuitive experience expecting a new user to know what to select.

4. Arguments like "that is the Unix way!" do seem a little disingenuous given how much functionality is wrapped into one application (without even any plugin support) and binary file format. If we're after "the Unix way" then that answer is more like... https://www.passwordstore.org/

5. Too often have I found conflicted copies (i.e. things like Nextcloud have declared that it won't sync a file, so it makes two files) of a KDBX and I've had to merge them manually. The most effective way I can think to do this (with any amount of confidence, although I'd love an easier way)... is to export them both as HTML (as crazy as that is), render that web page in a browser and copy the rendered text to a program like Meld to diff them. As (last I checked) only the HTML exports show what is in additional fields and notes, at least in a way that can be easily diffed. Now I don't know if that is Keepass2Android or KeePassXC (on either Windows, macOS, or Linux... I use it on them all) having a hard time coping with merges, and sometimes machines being offline for long periods... but it's a regular occurrence. I'd hope that by having KeePassXC directly fetch a fresh copy of the file... rather than potentially opening a local copy of a yet-un-synced file (e.g. maybe the Nextcloud client has crashed, or for some reason not running)... that these kinds of conflicts don't occur as often later on. It also gives the user a clear (and "in program") way of knowing if they're using an OFFLINE copy of the database or an ONLINE copy... something Keepass2Android and Strongbox already do.

6. I could go on... but not only am I repeating some things that others have already said, but I do worry that perhaps you might be unmovable on this.

Essentially, although it isn't (and I know that)... I see the KDBX file, to be more like a socket... a named reference to connect to a database/service. It's not just an ordinary file and there are pitfalls when treating it as such.

But most of all, having an easy and intuitive experience for new users I'd argue must be a priority... and this approach of adding custom system calls for external syncing programs - isn't it!

[t-h-e]

I'm happy to continue working on this issue and my PR, but I really would appriciate some feedback/review.

Like,

1. Should the sync be a menu entry similar to "Merge From Database..."? Or should it be in the database settings as suggested here Remote database support [$276] #1775 (comment)
2. Do we only want to support specific commands (scp, curl etc.), like select a command in e.g. in a dropdown, and then set specific parameters for that command as done in Support remote database access using external tools #7222. Or do we want to call any arbitrary system commands (User has to provide program, arguments and input) as suggested in Remote database support [$276] #1775 (comment). Second approach might be more complicated for syncing alone.
3. Is the implementation up to the standards of keepassxc?

@Lantizia

Certainly whatever process is called via #7222 could be doing absolutely anything with the file... with the idea 'syncing' not even coming into it (e.g. it could be used for backing up the database... or sending it off to be exported?). Possibly another way of looking at it might be "Support system calls upon database hooks" ?

At the moment #7222 only calls scp with the possibility to easily support other programs as well. The focus of that PR is on syncing and not arbitrary system calls, but on syncing.

Regarding the argument that other Keepass programs support syncing out of the box. E.g. KeePass also needs a plugin for ssh protocol and that plugin operates in the same way as the PR #7222. The plugin calls an external program for downloading/uploading and in between the merge happens. The only difference is that the plugin for KeePass is only for Windows so it is easy to provide an executable. But the overall concept is the same.

EDIT: Before anyone worries "my favourite protocol is not supported", I will add more protocols/programs before merging the PR. I just would like some initial feedback before going any further.

[t00]

@t-h-e If I may add 2 cents, currently original KeePass allows opening a database from a remote location and then saving it again to a remote location. Merge operation happens "behind the scenes" when there is need to merge between a database loaded in the KeePass and a remote database (regardless if one or another or both changed).

For me personally it would be ideal if it worked that way by allowing initially to Sync instead of expecting the database to be stored locally, otherwise on a new device I would need to somehow manually download it from WebDav/SSH, open and then sync. This also means keeping a script in the database is not that useful for an initial sync - commands still have to be manually entered in terminal or KeePassXC to get a copy of a remote database.

Above might require changing the way how commands work - when syncing, STDIN and STDOUT might need to be utilized instead of local file path so that script can send and receive remote file by streaming to/from it. Or maybe just allow initial load with sync load command? What are your thoughts on it?

One more note - I have not checked how exactly KeePass works with remote WebDAV location but I noticed when saving, it creates a copy of remote database and makes sure nothing is lost when saving by keeping 2 remote copies until sync is finished. Similar process could possibly be implemented with sync commands - obviously these are more generic so while sync process would probably be hardcoded in the app, Load and Save will need to handle remote rename to .temp, sync, save, remote rename to .kdbx. If concurrent device does the sync at the same time, remote rename would fail - things can get complicated there - KeePassXC would need to load new changes - merge - save to temp - compare - rename or something along the lines - already sounds complicated but I think commands are not meant to work with concurrent access. One thing is certain though, preventing data loss is critical if remote database was modified by another device between it was loaded and the moment of saving (between merge operations).

[t-h-e]

@t00: Thanks for clarifying. There was some misunderstanding on my side, because my use case is different. I have a local copy on each device and one copy on a server for synchronisation. Therefore, I'm really interested in getting the sync command feature to be added to KeepassXC.

I see now that the "Remote Database Support" is far more than just the sync command. I can have a look at proper remote database support afterwards (as long as there is any chance of getting it merged).
First I would like to get at least a review of my open PR before investing any more time in working KeepassXC, as I would really like to have that syncing mechanism.

[QORTEC]

Here's some logic that would enhance the usability/functionality of this feature:

• Are we opening a local database, or a cache (of a remote database)?
  Since there is no uptime guarantee of our server, we will probably want to cache data locally.

• Have we edited (added/changed) any entries of the cache?
  If there where changes we'll need to merge, otherwise its safe update/overwrite our cache.

• Which entries where edited?
  If we've only added/changed 2 entries in our cache, we'll probably want to ignore all other discrepancies between our cache and database, and focus on those 2 entries.

• Do we want to push any of the changes form our cache?
  We may or may not want to push our changes to our remote database.

• Do we want to keep our changes in our cache?
  Even if we didn't push a change, we may want to keep it as ether a local change or edit it farther before pushing.

• Do we want to pull/push changes automatically or or manually? (both "server & client" / "database & cache")
  We may want to only pull/push updates explicitly

I run both an NAS and VPN, so there is no reason I couldn't use that as a method of having a "remote database". But as @Lantizia noted merging conflicts in a reasonable way would be annoying at best.

The for the feature to be relatively useful it should support at a minimum, multiple connected clients and tracking of local changes. Wether the remote database is implemented in the client itself or is broken out into its own thing (a CLI server?). Which protocols and tools to use... Those are all implementation details, and in the end it doesn't really matter if it is feature complete, stable, and reliable.

Security is also important, but this feature should only really be used in a LAN or VPN environment... Which is probably a discussion best saved for elsewhere.

[lostever]

i've been waiting this feature for years, hoping fast.

[CumpsD]

Can the code of https://github.com/Kyrodan/KeeAnywhere be used perhaps for this feature? I currently use it with a locked down S3 bucket to "Open from cloud" between devices

[t-h-e]

A short update: The current PR allows opening remote databases as well as syncing and uploading a local database to a remote location: #7222

A remote database is downloaded to a temporary directory. Syncing has to be triggered manually for now. Merging is done locally with the already existing merging logic. Afterwards the database is uploaded again.

[pkoevesdi]

On top of no remote support, there comes this problem with os-wise remote mount: #9395

[KFDCompiled]

I've tested t-h-e's patch from #7222 and it works. I was first to fund this bounty and I think the PR submitted meets all requirements. Any objections to awarding t-h-e the bounty?

Edit: typos

[KaKi87]

When will the feature be released ?
Thanks