[MOBILE] Mobile Support - Global Tracking Issue

[guillemcordoba]

After a full iteration of development + deployment + testing cycle with the rostanga mobile app, I've explored a lot of the territory for running holochain on mobile. Right now I'm actually having to maintain a fork of holochain that stores all data in firebase because of the tx5 PendingJoin bug that is a known issue in 0.3.0. Also this makes me less nervous about holochain processes running into these kinds of issues while running for full days.

Anyway, in my view this is a comprehensive list of the issues that holochain core + its dependencies need to fix before we can say that "holochain supports mobile":

UX issues

• Holochain takes a long time to boot up every time you open the app (10 seconds on my android device):
  • 5 seconds of that is running the lair keystore server: Write a test showing InProcKeystore can restart successfully lair#120
  • Investigate what exactly is it that makes it take so long
    • Maybe related? Relax our notion of "joining the network" #3221
  • Run conductor as foreground service on Android -- so it boots up and runs once for all apps. WIP Foreground Service Plugin darksoil-studio/p2p-shipyard#38
• It takes a long time to install apps (in the order of 40 seconds)
  • Make sure that the install process is as streamlined as possible
    • refactor: move wasmer module cache to conductor #3242
    • Use wasm interpreter instead of compiler Feat/wasmer interpreter integration #4206
• Update coordinators developer journey is incomplete #3301

Networking

• Enable cells fails with cells paused because of PendingJoin known issue
• Failed to start SCTP: DTLS not established tx5#79
• Can't write to /data/local/tmp on Android API >= 30 tx5#69

Push Notifications

To achieve push notifications in mobile OSes, we basically need to go through Firebase Cloud Messaging (FCM) and Apple Notification Service (APN). Here I'm assuming tauri as the mobile runtime that holochain is running in.

• I've forked the notification plugin for tauri
  • I've forked tao to be able to set up the notification plugin. That needs to be merged into the main repository.
  • I've forked swift-rs to fix a compilation issue. That needs to be merged into the main repository.
• I've created a PoC of the semi-centralized bridge service to be able to actually send the notifications to offline agents: https://github.com/darksoil-studio/notifications

Android specific issues

• I'm seeing this issue intermitently show up after a minute or so of using the app and calling the conductor. More investigation is needed.

iOS specific issues

Precompiled wasms

This is the area that I have explored the least. So I'm not sure this is ready to go as it is in the code. I do now that @Connoropolous played a big role in the implementation of prepackaged wasms in iOS, so I'll let him comment if he sees fit to do so. But at least we need a:

• Precompiling dylibs via hc bundle currently does not work [BUG] hc dna pack --dylib-ios does not set dylib key in dna manifest #4254
• Good guide on how to precompile and package your wasms in dylibs / iOS frameworks and package it in your iOS app.
• This method could be used for any other platforms as well, including Android. We can modify the implementation to take list of targets -- not just ios

Interpreted wasms

AFAIK, iOS has as a rule that you can't execute or JIT compile anything that wasn't prepackaged as part of your iOS app at compile time.

Note: be careful, as apple actually lets you do things when developing with xcode that then you can't do when publishing an app to the appstore. In my development, holochain worked fine by JIT compiling the wasms, but that stopped working only when I build the "production" app build and downloaded it to the appstore.

wasmer supports a bunch of JIT compile engines, one of them is JavascriptCore which uses the underlying OS based JIT, but in which apple have dropped support for webassembly and will not be bringing it back. I've done some research and wasmer had a vote around implementing more backends, one of which is wasmi, which is a wasm interpreter. They could also implement something like a bridge to WKWebView which does seem to have native WebAssembly support in iOS, but I haven't seem any mention of that in any of their repositories or docs.

With all of this, we are left with no path to implement a runtime like we, neighborhoods or the launcher in iOS. These are runtimes that don't prepackage all installable apps in them, but rather they download them from the internet and install them dynamically in the conductor. I certainly want to enable these kinds of use cases, but it seems like without a big effort they are not possible with the current state of things.

The solutions that I can see are:

• We could re-add wasmi as a possible wasm engine behind a feature flag
• We could help wasmer implement an interpreter backend
• We could help wasmer implement a bridge to WKWebView

All of these seem like a big lift, but this seems crucially important. @thedavidmeister am I missing something in this territory?

I'd be really interested in talking about more options in this territory.

---

I'll try to keep this issue up to date with all the fixes and developments so that we have an overall view of the state of affairs.

[Connoropolous]

hiyas...
this line explains a bit about how the precompiled thing worked, but I have no idea anymore about how wasmer has moved.

holochain/CHANGELOG.md

Line 2116 in 194bc25

- **Experimental**: `hc dna pack` command now takes `--dylib-ios` option, which produces iOS optimized Zomes. These can be utilized by passing `dylib`: `PathBuf` for Zome configurations in `dna.yaml` files and other data structures based on ZomeManifest where Zomes are constructed. [\#2218](https://github.com/holochain/holochain/pull/2218)

There was the additional step that during the Dna read phase you needed to translate the dylib "paths" and read in the bytes. That's in a private repo.

[guillemcordoba]

@Connoropolous thanks! did you need to package your dylib deserialized wasms into iOS frameworks for it to work?

[Connoropolous]

I needed to have those dylib files sitting in the Xcode project folder and make sure they were included on the bundle of things that got codesigned during the app build process. I also needed to develop by testing and running on a hardware device, no simulator.

[guillemcordoba]

That's really helpful @Connoropolous , thanks!

[abe-njama]

@guillemcordoba what's this Tx5 PendingJoin known issue? Could you be referring to the performance issue on Android 33?

[guillemcordoba]

Oh sorry, it's not really tx5 as I got yesterday, it's the issue that @maackle is referring to here, that makes 0.3.x not viable right now, because when you enable the app is app is not enabled and stays in paused mode with PendingJoin as its reason.

[maackle]

FYI #3221 just merged, removing the network join error

[thedavidmeister]

@guillemcordoba is adding wasmi as an engine really that big of a lift?

i'd expect wasmer to basically work as-is because we'd still be using its interface, with wasmi only implemented "under the hood" of wasmer, right?

[guillemcordoba]

@thedavidmeister I don't think it should be that much of a lift no, and for holochain I think that would be the ideal, since then from holochain's point of view it's still wasmer.

Problem is I haven't seen any movement whatsoever in any wasmer repos/discord/discussions/pullrequests/articles that indicate to me that adding wasmi as an engine is in their backlog. So we'd have to request for that I imagine, and put pressure on it so that it is implemented... And meanwhile we have a bunch of time constraints on our side of needing to build products that work now on iOS (at least this is the situation that I'm in right now).

[thedavidmeister]

@guillemcordoba hmm, i have no idea what is involved in making an engine or who to talk to about it, do you?

[guillemcordoba]

Nope... No idea, the wasmer repo and discord is the first thing I would try.

[jost-s]

In my experience the wasmer discord server is not attended much. Github issues and discussions have worked for me to get answers from wasmer devs.

[abe-njama]

@guillemcordoba a couple of the PRs that you tagged in the UX issues section have been merged to develop.
Relax our notion of "joining the network" - Merged and awaiting a release to 0.3.0-beta-dev.35
There's however a related timeout reduction that's pending merging and release as well that will help improve the app experience

refactor: move wasmer module cache to conductor - Merged and awaiting a release to 0.3.0-beta-dev.35

The Networking issues will mostly be addressed by the new endpoint logic in tx5. This work is still in progress pending further integration to Holochain.

[abe-njama]

@zippy @guillemcordoba @thedavidmeister

I've filed an issue in the wasmer repo requesting support or a support model be proposed by the wasmer team so we can confidently work on any one of the Interpreted wasm solutions being suggested by @guillemcordoba in this main issue. This is a follow-up to an email that I'd sent to the wasmer maintainers but I hadn't received a response to.
Please feel free to pile on to the issue so we can get some traction.

[abe-njama]

Hey @guillemcordoba. Are you able to update the test application to 'holochain 0.3.0-beta-dev.37' and run the tests again? The fixes to some of the UX and Networking issues that had been raised have been shipped and we'd like to understand what impact these have had and to take stock of any new issues and to keep focus on outstanding ones.

• Relax our notion of "joining the network"
• 5s network join timeout
• Move wasmer module cache to conductor
• Refactor of endpoint and connection management state logic and its integration to Holochain; Tx5 integration of endpoint/connection state management refactor

[github-actions]

This item has been open for 30 days with no activity.

[mattyg]

Hey @guillemcordoba -- I'm trying to clean this up to make sure everything has a standalone issue in the #4340 Epic

I've forked the notification plugin for tauri
I've forked tao to be able to set up the notification plugin. That needs to be merged into the main repository.

Any progress on getting these forks merged upstream?

I'm seeing wasmerio/wasmer#2861 intermitently show up after a minute or so of using the app and calling the conductor. More investigation is needed.

Do you think this is resolved? I don't think I've seen it.

[mattyg]

I've review this issue and I believe all holochain issues have been moved into standalone tickets for tracking under #4340.

I also created #4421 to track the possibility of re-implementing support for precompiled wasms down the road if ever desired.

@guillemcordoba if you noticed anything I missed please make a standalone issue. I'm going to close this one.