Remove WebKit from AIR and replace it with Ultralight

[al-sabr]

Description

Since the HTML rendering is now pretty much outdated the only viable solution right now with AIR is to use https://github.com/tuarua/WebViewANE which is pretty massive and mixed up of C#, C++ and Adobe AIR. This is an embedded Chromium which is injected inside an AIR windows with no possibility to have other DisplayObject over it because it is not rendered in the composition pipeline of the Stage.

Either implement a way which allows external rendering contexts being injected in DisplayObject so that we can render external apps code inside of AIR so that when we scale the AIR app the external context can adjust itself. I am not only talking about Browser it can also be an OpenGL app like imgui which can be also used like a native AIR button seamlessly.

Or replace the massive WebKit engine with https://ultralig.ht/ which is brutal in terms of optimization.

• Will this change the current API? How?

In case of replacing the internal HTML renderer, the API will stay the same.

In case of injection mechanism then every DisplayObject should have a new OpenGL context rendering ID that can be passed to the external ANE so that the initialization code can know where to do his job.

• Who will benefit from this feature?

Everybody who would like to use the latest features of HTML5 and inject SAP inside of AIR. Everyone is happy when the size of the final app is reduced in term of size and memory usage.

Everybody could contribute external components not only written in AS3 but it can also be in Golang, C++, Rust, etc...

[ajwfrost]

Thanks for the suggestions. We've found that it's impractical to try to replicate the tight integration that Adobe had initially created between the AS3 API and the JavaScript APIs and HTML DOM that is a feature of the "HTMLLoader" mechanism (customised webkit from 10 years ago) but also that no one (?) actually uses such a tight integration. There are other, more standardised ways to communicate between the languages if that's necessary. So the idea of using a more modern browser engine is then a little simpler....

Currently we've got the possibility of using the native "StageWebView" i.e. the version that uses the component provided by the operating system, which means it would be a more modern mechanism (on Windows there's a choice of IE-based or Edge/WebView2-based). But the restrictions here are that this can only be associated with a single window and can't be moved, or merged in with the rest of the displaylist content. Which I notice has just been raised as a request...

Your concept here is an interesting one, to have an 'injection' mechanism so that the third party browser engine can render directly into the AIR display list. We can look at that perhaps, since we have implemented a mechanism in the past that allows this (third-party textures loaded into the AIR display list or rendered via Stage3D) although it relied on the EGLImage and some OpenGL ES extensions.. Windows builds of course don't use OpenGL but we may be able to look at some mechanism here... it may of course require specific integration points being available from the third party software. So eg. if this Ultralight component can render to some bitmap in shared memory, we can use that as a display object.

Will get one of the team to have a bit of a play...

thanks

[al-sabr]

This could be much more promising than expected because we could literally inject Qt5 & Qt6 applications directly in the Flash rendering engine and make it look like an internal component.

It is important to mention that it should have some kind of mechanism handling State changing. I mean MXML States when creating components inside of Flex. Something like when a state is changed then the API notify the ANE so that it pauses its rendering loop maybe ?

[al-sabr]

I found out that there's some plans for ARM32 & ARM64 for Ultralight so if you guys can test the Windows, Mac & Linux version with a proof of concept... It might be usable by the community and no need to internalize the HTML rendering. I think it is a good idea to remove the Webkit from the AIR binaries. It makes it really big.

ultralight-ux/Ultralight#88
ultralight-ux/Ultralight#370

List of ongoing missing features : ultralight-ux/Ultralight#178