WebView2 control loses pointer capture when when moving mouse outside of the control

[ghord]

Describe the bug

I'm hosting a monaco editor in WebView2 control, and I have focus issue that I think is a bug.

I've added following javascript at the startup to illustrate the issue:

window.addEventListener("gotpointercapture", event => {
    console.log("gotpointercapture");
});

window.addEventListener("lostpointercapture", event => {
    console.log("lostpointercapture");
});

window.addEventListener("mousemove", event => {
    console.log("mousemove");
});

As you can see in the gif, lostpointercapture event fires straight away when you move the mouse outside of WebView2.

This makes small input controls hosted using WebView practically unusable, as both scrolling and text selection break once you move at least 1 pixel outside of webview.

Steps to reproduce the bug

1. Host any html content in WebView2
2. Hook up gotpointercapture/lostpointercapture event
3. Capture pointer in javascript on mouse down
4. Move pointer outside of the host WebView2, while holding down mouse button.
5. Pointer capture gets released. Moreover, mousemove events do not seem to be sent to the content.

Expected behavior

Focus does not get lost and hosted window gets onmousemove messages correctly until user releases mouse button.

Screenshots

No response

NuGet package version

WinUI 3 - Windows App SDK 1.3.2: 1.3.230602002

Windows version

Windows 11 (22H2): Build 22621

Additional context

This issue does not occur in WPF implementation of WebView2 control, so this is WinUI3 specific.

I've seen some code in backported WinUI2 code here that claims to work around this specific issue:

https://github.com/microsoft/microsoft-ui-xaml/blob/3b77866fbdb2f2eda3fc24a6c92568439b700540/dev/WebView2/WebView2.cpp#L186-L198C21

    if (deviceType == winrt::PointerDeviceType::Mouse)
    {
        // WebView takes mouse capture to avoid missing pointer released events that occur outside of the element that
        // end pointer pressed state inside the webview. Example, scrollbar is being used and mouse is moved out
        // of webview bounds before being released, the webview will miss the released event and upon reentry into
        // the webview, the mouse will still cause the scrollbar to move as if selected.
        m_hasMouseCapture = this->CapturePointer(args.Pointer());

I don't think this works in WinUI3.

I would be very grateful to at least get a workaround for this issue.

[ghord]

The same issue (I think) can be reproduced with WinUI3 Controls gallery easily:

As you can see, once you drag the mouse outside of the WebView2, selection is no longer being updated.

[ghord]

Workaround is possible for this issue (although hacky):

1. Subscribe to WebView2 PointerMove in c# code, and send message to browser process if mouse is held down (using PostWebMessageAsJson), with current coordinates relative to WebView2.

private void PART_WebView_PointerMoved(object sender, PointerRoutedEventArgs e)
{
    if (PART_WebView == null)
        return;

    var point = e.GetCurrentPoint(PART_WebView);
    if (point.Properties.IsLeftButtonPressed)
    {
        var position = point.Position;

        var message = new EditorControlMessage(EditorControlMessageKind.MouseMove, null, null, (int)position.X, (int)position.Y);
        var messageJson = JsonSerializer.Serialize(message, options_);
        PART_WebView.CoreWebView2.PostWebMessageAsJson(messageJson);
    }
}

2. In javascript, on receiving the message, synthesize pointermove event and dispatch it to last element that had focus (gotpointercapture event.target).

var lastFocus;

window.addEventListener("gotpointercapture", event => {
    lastFocus = event.target;
});

window.chrome.webview.addEventListener('message', event => {

if (event.data.messageKind === 'mouseMove') {
    if (lastFocus !== undefined) {

        var evt = new PointerEvent("pointermove", {
            clientX: event.data.x,
            clientY: event.data.y,
            buttons: 1,
            bubbles: true,
            cancelable: true
        });

        lastFocus.dispatchEvent(evt);
    }
}

Note that this hack is monaco editor specific (different components could require different js events/parameters to work), which is enough for me.
Ideally hosted content in WebView2 should not lose mouse capture so easily.

[Fydar]

Just bumping this issue and sharing the GIFs I produced demonstrating this issue.

_{Figure 1: Expected behaviour demonstrated using WPF.}

_{Figure 2: Actual behaviour demonstrated in a MAUI Blazor Hybrid application.}

[ghord]

Just to anyone curious, 1.5.4 changes:

Fixed an issue where WebView2 sent extra blur and focus events on every click. For more info, see GitHub issue #9288. This also fixes problems with dropdowns not responding, as seen in GitHub issue #9566.

Do not fix this issue. Mouse capture is still lost when you drag outside the WebView2 control no matter what.

[Thomr77]

Hi everyone,

Is it possible that this could get attention as it really makes scrolling a Maui Blazor website very hard for the user - especially with larger vertical content?

We had a look at implementing the above workaround, but the WebView2 PointerMoved event is not called - the same goes for PointerGestureRecognizer PointerMoved.

A solution would be very greatly appreciated for Maui Blazor!

[Fydar]

Would be a wonderful Christmas present to have this fixed 🥺🎄

I love .NET Blazor Hybrid, apart from the lack of cursor graphic support and the pointer capture issues 🥲

[Thomr77]

Hi @codendone - that is truly awesome - thank you!! 😁