Serve web app locally

[SneakIn42]

Problem Statement

Currently the Android app connects to https://app.super-productivity.com to load necessary source code to function. For an ToDo / time tracking app that can and is intended to function locally - except from optional task syncing - that's an unnecessary step just transferring data that could be leaked and adding attack surface that could be abused.

❔ Possible Solution

The best option would probably be to serve the web app locally and therefore embed all needed files inside the Android app itself. That would prevent the former existent connection to be manipulated, spied on or abused otherwise by an attacker.

⤴️ Describe alternatives you've considered

Other alternatives would be:

Give users a option to use a self-hosted instance of the super productivity web app. This wouldn't eliminate the problem completely but users wouldn't have to trust a foreign, uncontrollable and unverifiable web app anymore, because users could set up their own instance or use one they trust. Therefore, the problem would be minimized a bit.

The other option would be to inform the users at the first start of the Android app that a connection to your servers are necessary for super productivity to function properly. This wouldn't avoid the problem, but at least users would know and can actively decide if that's okay with them or not.

➕ Additional context

A pleasant side effect would be that bugs like #3 would probably be avoided :-)

[johannesjo]

Thanks for writing this down! Much appreciated!

[powerpaul17]

The only thing I came across would be to using cordova or something like it. Cordova should also be able to create an electron app for desktop. But I don't know how much work it would be because supposedly there will be some (or a lot) rewriting involved..

[johannesjo]

Thanks for the suggestion, but cordova isn't needed and would not help with the migration problem and make things like the widget a lot harder to do. Basically the app is running in a minimal version of cordova. Only reason I see to use cordova (or preferably capacitor) would be the easy creation of an IOs app.

[powerpaul17]

Ok, sorry, I somehow understood the problem wrong. What is then needed to get the app running locally without having to download it from your server? (I'm working for a company where we build an app which gets released also for Android and iOS through cordova, that's why I wrote it.. 😉 )

[johannesjo]

The most complicated problem would be to transfer data. Currently, it's residing in the browser storage for app.super-productivity.com (the domain of the webapp) but this would change if we were to serve the app via the file protocol (which furthermore might introduce other issues). So we would need some sort of migration process for that. No matter what approach we use for serving the files we would need to solve this problem.

[powerpaul17]

Ok, I was playing around a bit today and came up with two (possible) solutions:

1. Add capacitor to the super-productivity repository and create a android folder (similar to the existing electron folder), then let it sync the built app (in the dist folder). This can also preserve the widgets and other custom android code. It simply loads the app locally. Would also make this repo obsolete then.

2. Rewrite the main activity of this repository to use the capacitor webview logic, etc. and copy the built app from elsewhere into a directory before building the android app.

[powerpaul17]

Additional Info: these were the easiest methods which worked after about half an hour. Trying to get the current implementation to load the web content locally failed after 1.5h of trying..

[johannesjo]

Thanks for digging into this. As mentioned already the setup for integrating the files is not the part I'm worrying about (in the current setup one would only need to make the files available and change the URL which is loaded). The problem is that all user data is saved in the browser instance for the current domain (app.super-productivity.com). We would need to do something like this:
Provide a version that can load both app.super-productivity.com and the files served from the file system. Then the app would need to copy the existing user data over, which is afaics not as trivial as it might sound. E.g. one problem is that one would need to fully load app.super-productivity to access the data there, which takes some time. Currently I have no idea on how to do this without affecting the user experience negatively for people who don't even need a migration.

[powerpaul17]

Oh, ok, now I understand. You are worried about migration of current user data. Yes that's true, it complicates the process. I haven't tried something like that yet, since I'm syncing my data anyway, I did not care about the local storage.. thanks for clarifying though.

[LDuncAndroid]

Currently I have no idea on how to do this without affecting the user experience negatively for people who don't even need a migration.

Could the user be asked if they would like to perform a migration?

I think for an all encompassing solution a REST API would do the trick but got to imagine that's quite a lot of work.

Great project by the way, I've been scouring Github trying to find something to help me manage my ADD and keep me productive. Will be keeping an eye on this and contributing where I can.

[johannesjo]

Could the user be asked if they would like to perform a migration?

Yes, we can do this. But it is not trivial to determine if a user actually needs one and ideally I would not like to show the dialog unnecessarily to new users. No matter how we do it, this might be a bit confusing to non-tech-savvy users.

But maybe it's ok to do this for a single release over the duration of a couple of weeks.

[johannesjo]

I just thought about another possible avenue to take:

1. Add ionic-storage or something similar.
2. Configure the app to always use SQLite as a storage adapter and migrate data if any
3. Make a new release for the android app
4. Wait a couple of weeks, in the hopes that most people will be on the new version after this.
5. Make a new version where files are served from local files

Not gonna lie: This will be a lot of work to set up and will furthermore require an ongoing time investment because would have to make and upload new releases for every new android version. As I am personally ok with how things are now demand would have to be much higher than it currently appears. It doesn't make sense to me to divert so much time to this for just a couple of people because other things will suffer from this.

[Rodeo-McCabe]

Just want to give my +1 on this.

I'd also be totally fine with this:

The other option would be to inform the users at the first start of the Android app that a connection to your servers are necessary for super productivity to function properly. This wouldn't avoid the problem, but at least users would know and can actively decide if that's okay with them or not.

As long as it's clearly explained why and when the app must connect.

[microbearsmb]

You have my +1 as well. I would of course prefer to have all required assets baked into the offline app, but allowing users to run a custom instance of the webserver would work as well.

[Ekleog]

I'm coming here from the CORS issue mentioned at #16.

My impression is doing this from a file URL would also avoid the issues of CORS headers needing to be set, thanks to setAllowUniversalAccessFromFileUrls. So things like needing to add WebAppPassword to Nextcloud would no longer be necessary for the android app at least.

As for the migration issue, maybe one “simple” solution would be to publish the local app as another app, and let people who want to manually migrate their data by exporting/importing if they're not syncing to some remote place anyway? This is basically what happened with riot/riotx from matrix.org; there was a new app created named riotx and the old app was left working for people who still wanted to use it. If people have issues with CORS or the fact that the old app needs internet access to boot up, they can export their data and import it again in the new app, or even just sync with the same remote if they're syncing their app anyway.

Would that “solution” make sense? (it's not technical but social, but sometimes a social solution works as well as a technical one and is much easier to setup)

[lrq3000]

+1, I would really like a locally served version of the Android app.

Wouldn't PR #32 fixing file syncing be helpful for user data migration? Obviously, users would still have to manually sync their data before using the new app, but you can simply provide two versions of the app, one legacy for users who already use the app and want to migrate, and the new app which is served locally, no?

[johannesjo]

@lrq3000 in theory yes, but the users need to be aware of that before they install the new version. Another alternative might be to do the following:

1. Create one new app release that introduces a locally stored flag value, that indicates that there is some data saved to app.super-productivity.com (we already could provide an opt in way to route to the local files instead).
2. Wait a while (a month should hopefully be enough) and create a new release. This release will still route to app.super-productivity.com when the flag value is set, but for all users with a fresh install and no stored data the local files will be served.

This way every new user get's the new variant and everyone who cares can migrate manually by saving the data somewhere and wiping all app data.

Another (possibly even cleaner) approach – not sure if this is possible – might be to provide some sort of proxy to serve the local file content to the app as it would be from app.super-productivity.com. This way we wouldn't need to care about migration at all (hopepfully).

[ishanarora]

@johannesjo how about instead we release the new "local" variant under a new application ID, say com.superproductivity.superproductivity.local . This would allow both variants to co-exist on a phone each with it's own storage.

Then we could also notify the web variant users about the existence of the local variant, if desired (much like we did with deprecation of Google Drive support).

Obviously the local variant would see releases lockstep with the desktop versions. For this reason I think it may be easier to maintain by absorbing this project into the desktop app repo.

[johannesjo]

@ishanarora thanks for the suggestion. I am not a big fan tbh. Release management of all the different app variants is complicated and time consuming enough as is :)

[jiongxuan]

My requirements are a bit different, as I primarily need to support self-hosted services and address WebDAV CORS issues during synchronization. As I mentioned earlier, the current approach requires familiarity with Kotlin syntax and involves multiple code changes, which complicates the process.

I plan to write some code soon to ensure quick modifications for self-hosted access. This will allow us to switch to a self-hosted domain and resolve the issue efficiently. Later, I’ll look into how we can handle URL changes and data migration at runtime, but this will require more time to design.

Regarding a fully offline solution, I agree that the changes would be significant, and it’s something that needs careful consideration. We’ll need to take a closer look at how best to approach it.

[jiongxuan]

@johannesjo @Ekleog @powerpaul17 @ishanarora @LDuncAndroid @lrq3000 I’ve resolved the issue. Please refer to this PR for details:

• Major Android Update: Introducing Offline Support and Resolving CORS Issues #57
• Major Android Update Discussion: Introducing Offline Support and Resolving CORS Issues super-productivity#3527

Considering that many are interested, here’s a brief overview of the changes:

I have taken steps to address the longstanding issues related to offline support and CORS in the Android application. These updates are designed to enhance the user experience by enabling offline functionality, resolving security-related CORS issues, and ensuring a smooth transition for existing users. I welcome any questions or feedback from the community to help refine and improve the solution further.