winit inside Android GLSurfaceView

[TroyNeubauer]

Description

Hi,
I have a requirement to embed an OpenGL app into Android's GLSurfaceView, which lives inside an existing jetpack compose app (Unable to use NativeActivity orGameActivity). I would like to somehow have winit wrap this GLSurfaceView into a winit::window::Window so that I can re-use platform-agnostic Rust rendering code (egui).

It looks like this use case isnt supported so I was curious if this is something you would be interested in a PR for, of if this is best left outside of the library.

Similar to the web, GLSurfaceView "owns" the lifecycle (handles OpenGL context creation, redraw interval, redraw thread, etc.).

1. What do you think would be involved in accomplishing this? Off the top of my head:
   a. New feature flag
   b. New API for connecting a winit window to an existing GLSurfaceView
   c. Subscribe to android events form the GLSurfaceView, feed these to winit
   d. ...
2. Is this enough of a tear up that I should handle separately, without winit?

Relevant platforms

Android

[madsmtm]

CC @MarijnS95

Also, how does Glutin support Android?

[kchibisov]

with EGL, you just use it + raw-window-handle and that's about it. They are talking about android specific view here they get from somewhere else IIRC, which is not glutin related at all.

[MarijnS95]

Unfortunately wrapping an arbitrary Surface in winit is not possible without completely rewriting the Android backend. winit hosts an entire NativeActivity or GameActivity, lifecycle and all.

In addition, GLSurfaceView seems very tailored to Java and developer simplicity, by currenting a context for you and providing already-loaded GL function pointers. There is currently no Rust abstraction available around this that I know of, so you'll have to write all that. But the interface to GLSurfaceView seems customizable enough that you might also be able to (re)load function pointers in Rust via the industry standard gl_generator crate as long as the onDrawFrame() callback is called through into Rust via JNI / native methods.
That still requires you to process input and pass it on to the Rust side in "some way".

If you were to perform (EGL) context management yourself in Rust (i.e. via the already mentioned glutin crate) you're much better off using the already-existing NativeWindow/Surface bindings, and wrapping those in an EGLSurface to render to, as I demonstrate in https://github.com/MarijnS95/AndroidNativeSurface ¹.

---

What we are planning on the winit side is to provide multiple Windows, each wrapping a single NativeActivity/GameActivity with a shared event loop for the entire Application (like on other platforms).

Separate from that, subsurfaces are going to be a great fit for individual Surfaces, of which Android can compose many (from the same app) on screen. These are hierarchical, so winit could be made to wrap a single Surface inside which more Surfaces are created, managed and updated.

---

And finally, Android added a new AInputReceiver for ASurfaceControl recently in Android 15. This is the kind of API that we could use to natively receive inputs for individual Surfaces on screen without having to own a global input receiver and translating inputs (and/or dealing with precedence).

---

With refactors ongoing in winit to support out-of-tree backends via dyn (IIRC), I see a potential for designing multiple (layered!) Android backends to support various usecases using the many methods mentioned here without creating a cfg/combinatorial hell inside the backend. Such backends could even be made official, with various helpers such as the Java interface compiled right in.

Footnotes

1. Disregard this examples' usage of SurfaceTexture which is less useful for you as it takes frames from an arbitrary "buffer producer", "uploads" them into GL textures, and typically uses those textures via TextureView. ↩

[TroyNeubauer]

Thanks for digging in here! This all makes sense.

Turns out the number of things you need here is pretty minimal and I got this working last night: https://github.com/MerchGuardian/egui-android-test/blob/master/egui-view/native_gl_surface/src/lib.rs#L163-L238. I wish I knew about your https://github.com/MarijnS95/AndroidNativeSurface repo sooner 😅

Given how simple this turned out, I think that rewriting the backend to make this usecase fit doesnt make sense.

👍 for AInputReceiver. I will try this out

[MarijnS95]

Glad to hear that you got it working by relying on the already-currented (with surface!) context from GLSurfaceView 👍

AInputReceiver is still ways out from marketed phones, and we'll probably need some more significant refactors to android-activity and winit to make it useful but overall the possibilities of different kinds of (embedded) winit backends for Android are almost endless.