Android Support

[cart]

It should be possible to run Bevy Apps on Android

[PrototypeNM1]

I'm working on this issue now, can I get it assigned to me to help head off accidental parallel implementations?

[cart]

absolutely! thanks for picking it up. this will be huge ❤️

[cart]

haha just noticed @karroffel already beat me to it

[endragor]

@PrototypeNM1 could you please share your progress and what's left to be done? I have some time on weekends that I could dedicate to adding Android support to Bevy. Last weekend I added Android support to cpal, which was one of the major blockers. There are still some issues on android-ndk-rs side. I could look into that, but it would be nice if you broke this issue into subitems and clarify what is done and what isn't.

[PrototypeNM1]

To my knowledge there are 4 main blockers for Android support.

1. Lack of Android support for cpal (wip) (thanks for tacking this!)
   Once fixed, may reveal issues in it's transitive dependents Rodio or winit.
2. cargo apk is broken for how Bevy structures Cargo.toml (PR)
   Alt. removing glob imports from Bevy's Cargo.toml fixes this.
3. Mobile needs input support, touch and IMU
4. bevy-glsl-to-spirv requires a rewrite or replacement.

I'm working on 4. glsl-to-spirv as written depends on a bundled glslangValidator executable. This can't work on Android without hackery. Luckily glslangValidator has a recently added a C API.

@endragor since you started investigating cpal/oboe issues, I recommend figuring out why cargo apk had issues with including multiple shared libraries, doing so is helpful both for the Rust Android ecosystem and Bevy.

[MichaelHills]

@PrototypeNM1 FYI over at #87 I put up #324 where currently uses shaderc-rs instead of glsl-to-spirv for iOS. Feel free to give it a try.

I too have now run into the missing touch support... I might hack something in just so I can play with it, not exactly sure what the correct API is.

[enfipy]

Hey all! I was able to run Bevy 3d example on my Android phone.

Proof and repo:

Current issues are next:

1. Shaders still not working in runtime (so I did it in compile-time). As they precompiled - we can't use macros so that shadows are weird on the screenshot.
2. Bgra was not worked with Vulkan on my phone so I replaced them with Rgba. Probably we can just replace them with Rgba everywhere in Bevy but this should be tested on other platforms.
3. I couldn't make assets work at all on Android. It's pretty weird but with the newest version of android-ndk-rs - there are no assets when I trying to get them from asset_manager.
4. Audio, 3d models, textures, etc were not tested due to problems with assets.

This is the current status. Any feedback and advice are highly appreciated.

[cart]

Fantastic! This is a big step 😄

1. Shaders still not working in runtime (so I did it in compile-time). As they precompiled - we can't use macros so that shadows are weird on the screenshot.

Yeah as we discussed elsewhere I think we'll want to solve this by adding pre-compiled shader support with the various permutations available. We'll want a system like that anyway to improve startup times in release builds. Alternatively we can try adding dynamic compilation support by (1) waiting for Naga or (2) somehow making shaderc compile on android

2. Bgra was not worked with Vulkan on my phone so I replaced them with Rgba. Probably we can just replace them with Rgba everywhere in Bevy but this should be tested on other platforms.

Yeah thats a bit annoying, but it seems like we can work around it by selecting formats according to the platform. Alternatively, it might be worth checking with the wgpu folks to see if its something they can fix on their end.

3. I couldn't make assets work at all on Android. It's pretty weird but with the newest version of android-ndk-rs - there are no assets when I trying to get them from asset_manager.

Hmm maybe we need to add them to some sort of asset manifest? I haven't worked with android for awhile, but i vaguely remember something like that.

[PrototypeNM1]

@enfipy Awesome! Referencing your changes for TextureFormat I got runtime shader generation working on Android.

[enfipy]

After about 10 hours of unsuccessful tries to make @PrototypeNM1 example with bevy-glsl-to-spirv work on my Mac (and Windows) - I added shaderc-rs support for Android (but it's still shitty) and was able to make shaders work in runtime.

Status update:

• Shaders are working on runtime but the build still needs to be enhanced (or I very hope Naga will land soon). (Screenshot_1)
• I added a temporary solution for assets loading on Android. I plan to add a better one to Bevy as @cart will finish asset refactoring.
• PNG images rendered successfully (Screenshot_2).
• Text and UI rendered successfully (Screenshot_1).

Things need to be resolved:

• Audio. This PR looks good and hopefully will enable high-performance audio on Android.
• TextureFormat. It looks like some issues still persist with Bgra/Rgba as I couldn't build a Monkey example and faced a similar issue that was before this (but I don't think it's very serious).
• Shaderc. As I specified above - it's not finished and tested on other platforms yet and hard to build.
• Request Permissions. From Android version 23 and higher, Android requires applications to request permissions at runtime. I want to add jni call to this in a couple of days.
• Touch Support. I hope this will be available soon with this Touch support implementation #696.

This is all I remembered.

Screenshots:

Screenshot_1:

Screenshot_2:

[cart]

Very nice progress! I just merged the asset system changes. I'm making a few more changes to AssetIo to re-add wasm compatibility. It should also make integrating the android AssetIo backend slightly easier (as I'm boxing AssetIo)

[trezm]

I'd like to just say, I'm not working on this issue, but am wildly excited to see its progress. Huge thanks to everyone working on it and know that the community is ecstatic to see this moving forward!!

[PrototypeNM1]

@enfipy @cart Given Enfipy has more time to work on and organize this issue, I think we should transfer this issue to them.

[cart]

I think this is a big enough issue / problem space that I don't think anyone needs to own it.

[PrototypeNM1]

Added support for runtime spirv generation using a rewrite of glsl-to-spirv.

We're still waiting for a fix in cargo-apk's dependencies to land. The alternative of removing glob imports for bevy's workspaces is still a viable alternative if we wanted this working asap.

[mmacedoeu]

Would like to bring attention to proper implementation of winit events Event::Suspended and Event::Resumed at https://github.com/bevyengine/bevy/blob/master/crates/bevy_winit/src/lib.rs#L170 so it can proper release and reload resources

[francesca64]

Would there be any interest in using cargo-mobile to help with building, generating Android Studio projects, running on device, etc?

[cart]

Medium-to-long term I might be interested in adopting higher level abstractions. Short term I'd rather keep it simple, encourage people to become familiar with the "native" mobile tooling, and maintain control over "official" templates. We will likely create an official bevy_template repo in the near future.

A minor note on the current cargo-mobile bevy template: bevy has its own answer to #[mobile_entry_point] called #[bevy_main]. I'd prefer it if you used that instead (to support future non-mobile scenarios). Although thats only available on the master branch right now.

[alice-i-cecile]

Removing from the 0.8 milestone; even if we merge #4913 I don't think this should be closed for 0.8 until we have significantly more testing.

[Leinnan]

Is there any update on that? What is the status of Android in 0.9?

[rib]

Although I haven't looked at Bevy recently we did finally get the android-activity backend merged for Winit which should help with Android support but it will also require some integration work in Bevy when it updates to Winit 0.28.

One of the other notable thing upstreamed in Winit was that all backends now consistently deliver a Resume event which helps with writing portable code that runs on Android (where handling Resume is very important) and other window systems.

I'm not sure atm how tricky it's going to be to rebase #4913 (or maybe something equivalent was done in the mean time) but something equivalent to that will be one of the main blockers still I expect.

[mockersf]

Is there any update on that? What is the status of Android in 0.9?

Works on some device but without audio, doesn't work on others at all.

android-activity backend merged for Winit

Do you know if there has been work to replace it also in things like cpal?

[rib]

cpal has switched over to using ndk-context now so that it's no longer dependent on any glue crate so should hopefully just work now 🤞

E.g. for android-activity I created an example to test cpal here: https://github.com/rib/android-activity/tree/main/examples/agdk-cpal which works with cpal = "0.14"

[rib]

here's the cpal PR that switched to ndk-context: RustAudio/cpal#641 for reference

[enfipy]

Also, our team was working on the toolkit for plugins and build cli for Android (iOS in the future): https://github.com/dodorare/crossbow.

We succeeded in running different Google Play plugins on Android but Bevy was also blocked by stuff like audio, etc. @rib it would be awesome to take a look at how we can integrate with AGDK, we tried to design crossbow similar to Xamarin and Godot - so I think there will be no big problems with it.

Here's an example with Macroquad (but will work with Bevy too):

Honestly, I think if we all here put all this stuff together, we can make pretty fine Android support in Bevy v0.10, IMO.

[rib]

cool, I'd heard of crosssbow recently but hadn't had a chance to take a look so thanks for the pointer @enfipy

Skimming that example I see it has a main entry point like

#[macroquad::main("Macroquad UI")]
async fn main() -> anyhow::Result<()> {
}

which makes me think that you're possibly defining your own glue layer between native and jvm code? Or does this maybe currently build on the ndk-glue macro perhaps?

Winit (which Bevy usually builds on) is currently (for 0.28) implies needing to use android-activity as a glue crate (previously ndk-glue) for handling integration with the Activity class (either NativeActivity or GameActivity from AGDK) and both of those Activity crates also generally define the architecture for spawning a native thread that will run your application's main() function.

I'd need to poke into crossbow more to understand how it defines its main entry point macro to see if we can figure out how to join the dots with android-activity.

You should have some flexibility from the proc macro I guess but one notable thing about the entry point ABI for android-activity is that it is passed an app: AndroidApp argument that represents the app and represents a kind of connection between a native thread and JVM Activity (to avoid global static state) and I wonder how that would fit with crossbow's current main function definition.

[rib]

okey, I realize now that the example main() entry point is specific to the macroquad library - not something imposed by crossbow

[rib]

btw another build tool that's possibly in a similar space to crosstool is xbuild. I guess you've maybe seen it already @enfipy but figured I'd mentioned it while I didn't see it in the list to existing tools mentioned in the readme for crosstool.

[slyedoc]

Got bevy_audio working if I tell cpal to enable oboe "shared-stdcxx" feature, meaning I am having an issue with "c++_static" linking. Still working on why, and not sure if its just me or not.

[rib]

In case it might give some insight/clue you could maybe poke at this minimal cpal example: https://github.com/rib/android-activity/tree/main/examples/agdk-cpal

That example doesn't currently do anything special to choose the stdc++_shared vs static implementation and seemed to work.

There's currently no standard convention amongst Rust Android crates for being able to configure whether to use the shared or static implementation which is a bit awkward, since it wouldn't really be desirable to have a mix and match amongst crates.

It sounds kinda surprising that it works when enabling shared stdcxx. I would have thought you'd also have to make sure to copy the libc++_shared.so out of your NDK/toolchain into your package for that to run - but maybe cargo apk automagically does that.

[rib]

oh sorry, I lost track of which bug I was responding too and see I already linked that example quite recently in this thread so guess you've probably already seen it.

[slyedoc]

but maybe cargo apk automagically does that.

It does, and your examples have been my starting point for everything in this PR. Used na-winit-wgpu quite a bit, and made sure I could get agdk-cpal working before anything else this morning.

I do want to add AGDK bevy example (you put in so much work getting that working), plan to after I am happy with native activity with apk.

[MarijnS95]

It sounds kinda surprising that it works when enabling shared stdcxx. I would have thought you'd also have to make sure to copy the libc++_shared.so out of your NDK/toolchain into your package for that to run - but maybe cargo apk automagically does that.

cargo-apk takes care of all your library linking problems 😉, and we landed the same feature in xbuild just a few days ago 🙂

[rib]

@MarijnS95 does cargo apk maybe default to making crates link against libc++_shared.so as opposed to static linking?

As a guess, that seem like it could explain a discrepancy between the agdk-cpal example working and needing to enable shared-stdcxx with Bevy (while building with cargo apk)

[MarijnS95]

@rib cargo apk looks at what libraries the final .so requests linking against; if you configure the library to not link against dynamic libc++ (via i.e. the cc crate when compiling native code), it won't embed libc++_shared.so in the final APK.

[rib]

@rib cargo apk looks at what libraries the final .so requests linking against; if you configure the library to not link against dynamic libc++ (via i.e. the cc crate when compiling native code), it won't embed libc++_shared.so in the final APK.

right, makes sense

[alice-i-cecile]

Android is reasonably well supported now, following #9937. #10158 will fix an important bug with audio playing while suspended, but in general, it should be functional.

It needs more and better documentation still, and better multi-touch + haptic support, but the foundations are all there. If you run into further problems, please open a new issue!