Manually pass mouse wheel messages to TermControls (#5131)

## Summary of the Pull Request

As we've learned in #979, not all touchpads are created equal. Some of them have bad drivers that makes scrolling inactive windows not work. For whatever reason, these devices think the Terminal is all one giant inactive window, so we don't get the mouse wheel events through the XAML stack. We do however get the event as a `WM_MOUSEWHEEL` on those devices (a message we don't get on devices with normally functioning trackpads).

This PR attempts to take that `WM_MOUSEWHEEL` and manually dispatch it to the `TermControl`, so we can at least scroll the terminal content.

Unfortunately, this solution is not very general purpose. This only works to scroll controls that manually implement our own `IMouseWheelListener` interface. As we add more controls, we'll need to continue manually implementing this interface, until the underlying XAML Islands bug is fixed. **I don't love this**. I'd rather have a better solution, but it seems that we can't synthesize a more general-purpose `PointerWheeled` event that could get routed through the XAML tree as normal. 

## References

* #2606 and microsoft/microsoft-ui-xaml#2101 - these bugs are also tracking a similar "inactive windows" / "scaled mouse events" issue in XAML

## PR Checklist
* [x] Closes #979
* [x] I work here
* [ ] Tests added/passed
* [n/a] Requires documentation to be updated

## Detailed Description of the Pull Request / Additional comments

I've also added a `til::point` conversion _to_ `winrt::Windows::Foundation::Point`, and some scaling operators for `point`

## Validation Steps Performed

* It works on my HP Spectre 2017 with a synaptics trackpad
  - I also made sure to test that `tmux` works in panes on this laptop
* It works on my slaptop, and DOESN'T follow this hack codepath on this machine.

.github/actions/spell-check/whitelist/whitelist.txt

@@ -1134,6 +1134,7 @@ IIo
 IList
 ime
 Imm
+IMouse
 Impl
 implementingtextandtextrange
 inbox
@@ -2423,6 +2424,7 @@ TOPDOWNDIB
 TOPLEFT
 TOPRIGHT
 tosign
+touchpad
 tounicodeex
 towlower
 towupper
@@ -2434,6 +2436,7 @@ tracelogging
 traceloggingprovider
 trackbar
 TRACKCOMPOSITION
+trackpad
 trackpads
 transcoder
 transitioning

src/cascadia/TerminalApp/lib/pch.h

@@ -9,6 +9,8 @@
 
 #define WIN32_LEAN_AND_MEAN
 
+// Manually include til after we include Windows.Foundation to give it winrt superpowers
+#define BLOCK_TIL
 #include <LibraryIncludes.h>
 // This is inexplicable, but for whatever reason, cppwinrt conflicts with the
 //      SDK definition of this function, so the only fix is to undef it.
@@ -63,3 +65,6 @@ TRACELOGGING_DECLARE_PROVIDER(g_hTerminalAppProvider);
 #include <winrt/Microsoft.Terminal.TerminalConnection.h>
 
 #include <CLI11/CLI11.hpp>
+
+// Manually include til after we include Windows.Foundation to give it winrt superpowers
+#include "til.h"

src/cascadia/TerminalConnection/pch.h

@@ -12,18 +12,21 @@
 
 #define BLOCK_GSL
 
+// Manually include til after we include Windows.Foundation to give it winrt superpowers
+#define BLOCK_TIL
 #include <LibraryIncludes.h>
 
 // Must be included before any WinRT headers.
 #include <unknwn.h>
-
+#include <winrt/Windows.Foundation.h>
 #include <wil/cppwinrt.h>
 
-#include "winrt/Windows.Foundation.h"
 #include "winrt/Windows.Security.Credentials.h"
 #include "winrt/Windows.Foundation.Collections.h"
 #include <Windows.h>
 
 #include <TraceLoggingProvider.h>
 TRACELOGGING_DECLARE_PROVIDER(g_hTerminalConnectionProvider);
 #include <telemetry/ProjectTelemetry.h>
+
+#include "til.h"

src/cascadia/TerminalControl/IMouseWheelListener.idl

@@ -0,0 +1,16 @@
+// Copyright (c) Microsoft Corporation.
+// Licensed under the MIT license.
+
+namespace Microsoft.Terminal.TerminalControl
+{
+
+    // This interface is a hack for GH#979. Controls should implement this
+    // interface to be able to be notified of mousewheel events even on devices
+    // who's trackpads won't scroll inactive windows.
+
+    [uuid("65b8b8c5-988f-43ff-aba9-e89368da1598")]
+    interface IMouseWheelListener
+    {
+        Boolean OnMouseWheel(Windows.Foundation.Point coord, Int32 delta);
+    }
+}

src/cascadia/TerminalControl/TermControl.cpp

@@ -1172,6 +1172,8 @@ namespace winrt::Microsoft::Terminal::TerminalControl::implementation
     // - Event handler for the PointerWheelChanged event. This is raised in
     //   response to mouse wheel changes. Depending upon what modifier keys are
     //   pressed, different actions will take place.
+    // - Primarily just takes the data from the PointerRoutedEventArgs and uses
+    //   it to call _DoMouseWheel, see _DoMouseWheel for more details.
     // Arguments:
     // - args: the event args containing information about t`he mouse wheel event.
     void TermControl::_MouseWheelHandler(Windows::Foundation::IInspectable const& /*sender*/,
@@ -1183,22 +1185,49 @@ namespace winrt::Microsoft::Terminal::TerminalControl::implementation
         }
 
         const auto point = args.GetCurrentPoint(*this);
-
-        if (_CanSendVTMouseInput())
+        auto result = _DoMouseWheel(point.Position(),
+                                    ControlKeyStates{ args.KeyModifiers() },
+                                    point.Properties().MouseWheelDelta(),
+                                    point.Properties().IsLeftButtonPressed());
+        if (result)
         {
-            _TrySendMouseEvent(point);
             args.Handled(true);
-            return;
         }
+    }
 
-        const auto delta = point.Properties().MouseWheelDelta();
+    // Method Description:
+    // - Actually handle a scrolling event, whether from a mouse wheel or a
+
+    //   touchpad scroll. Depending upon what modifier keys are pressed,
+    //   different actions will take place.
+    //   * Attempts to first dispatch the mouse scroll as a VT event
+    //   * If Ctrl+Shift are pressed, then attempts to change our opacity
+    //   * If just Ctrl is pressed, we'll attempt to "zoom" by changing our font size
+    //   * Otherwise, just scrolls the content of the viewport
+    // Arguments:
+    // - point: the location of the mouse during this event
+    // - modifiers: The modifiers pressed during this event, in the form of a VirtualKeyModifiers
+    // - delta: the mouse wheel delta that triggered this event.
+    bool TermControl::_DoMouseWheel(const Windows::Foundation::Point point,
+                                    const ControlKeyStates modifiers,
+                                    const int32_t delta,
+                                    const bool isLeftButtonPressed)
+    {
+        if (_CanSendVTMouseInput())
+        {
+            // Most mouse event handlers call
+            //      _TrySendMouseEvent(point);
+            // here with a PointerPoint. However, as of #979, we don't have a
+            // PointerPoint to work with. So, we're just going to do a
+            // mousewheel event manually
+            return _terminal->SendMouseEvent(_GetTerminalPosition(point),
+                                             WM_MOUSEWHEEL,
+                                             _GetPressedModifierKeys(),
+                                             ::base::saturated_cast<short>(delta));
+        }
 
-        // Get the state of the Ctrl & Shift keys
-        // static_cast to a uint32_t because we can't use the WI_IsFlagSet macro
-        // directly with a VirtualKeyModifiers
-        const auto modifiers = static_cast<uint32_t>(args.KeyModifiers());
-        const auto ctrlPressed = WI_IsFlagSet(modifiers, static_cast<uint32_t>(VirtualKeyModifiers::Control));
-        const auto shiftPressed = WI_IsFlagSet(modifiers, static_cast<uint32_t>(VirtualKeyModifiers::Shift));
+        const auto ctrlPressed = modifiers.IsCtrlPressed();
+        const auto shiftPressed = modifiers.IsShiftPressed();
 
         if (ctrlPressed && shiftPressed)
         {
@@ -1210,8 +1239,25 @@ namespace winrt::Microsoft::Terminal::TerminalControl::implementation
         }
         else
         {
-            _MouseScrollHandler(delta, point);
+            _MouseScrollHandler(delta, point, isLeftButtonPressed);
         }
+        return false;
+    }
+
+    // Method Description:
+    // - This is part of the solution to GH#979
+    // - Manually handle a scrolling event. This is used to help support
+    //   scrolling on devices where the touchpad doesn't correctly handle
+    //   scrolling inactive windows.
+    // Arguments:
+    // - location: the location of the mouse during this event. This location is
+    //   relative to the origin of the control
+    // - delta: the mouse wheel delta that triggered this event.
+    bool TermControl::OnMouseWheel(const Windows::Foundation::Point location,
+                                   const int32_t delta)
+    {
+        const auto modifiers = _GetPressedModifierKeys();
+        return _DoMouseWheel(location, modifiers, delta, false);
     }
 
     // Method Description:
@@ -1284,7 +1330,11 @@ namespace winrt::Microsoft::Terminal::TerminalControl::implementation
     // - Scroll the visible viewport in response to a mouse wheel event.
     // Arguments:
     // - mouseDelta: the mouse wheel delta that triggered this event.
-    void TermControl::_MouseScrollHandler(const double mouseDelta, Windows::UI::Input::PointerPoint const& pointerPoint)
+    // - point: the location of the mouse during this event
+    // - isLeftButtonPressed: true iff the left mouse button was pressed during this event.
+    void TermControl::_MouseScrollHandler(const double mouseDelta,
+                                          const Windows::Foundation::Point point,
+                                          const bool isLeftButtonPressed)
     {
         const auto currentOffset = ScrollBar().Value();
 
@@ -1301,11 +1351,11 @@ namespace winrt::Microsoft::Terminal::TerminalControl::implementation
         //      for us.
         ScrollBar().Value(newValue);
 
-        if (_terminal->IsSelectionActive() && pointerPoint.Properties().IsLeftButtonPressed())
+        if (_terminal->IsSelectionActive() && isLeftButtonPressed)
         {
             // If user is mouse selecting and scrolls, they then point at new character.
             //      Make sure selection reflects that immediately.
-            _SetEndSelectionPointAtCursor(pointerPoint.Position());
+            _SetEndSelectionPointAtCursor(point);
         }
     }
 

src/cascadia/TerminalControl/TermControl.h

@@ -84,6 +84,8 @@ namespace winrt::Microsoft::Terminal::TerminalControl::implementation
 
         bool OnF7Pressed();
 
+        bool OnMouseWheel(const Windows::Foundation::Point location, const int32_t delta);
+
         ~TermControl();
 
         Windows::UI::Xaml::Automation::Peers::AutomationPeer OnCreateAutomationPeer();
@@ -198,9 +200,10 @@ namespace winrt::Microsoft::Terminal::TerminalControl::implementation
         winrt::fire_and_forget _TerminalScrollPositionChanged(const int viewTop, const int viewHeight, const int bufferSize);
         winrt::fire_and_forget _TerminalCursorPositionChanged();
 
-        void _MouseScrollHandler(const double delta, Windows::UI::Input::PointerPoint const& pointerPoint);
+        void _MouseScrollHandler(const double mouseDelta, const Windows::Foundation::Point point, const bool isLeftButtonPressed);
         void _MouseZoomHandler(const double delta);
         void _MouseTransparencyHandler(const double delta);
+        bool _DoMouseWheel(const Windows::Foundation::Point point, const ::Microsoft::Terminal::Core::ControlKeyStates modifiers, const int32_t delta, const bool isLeftButtonPressed);
 
         bool _CapturePointer(Windows::Foundation::IInspectable const& sender, Windows::UI::Xaml::Input::PointerRoutedEventArgs const& e);
         bool _ReleasePointerCapture(Windows::Foundation::IInspectable const& sender, Windows::UI::Xaml::Input::PointerRoutedEventArgs const& e);

src/cascadia/TerminalControl/TermControl.idl

@@ -1,6 +1,8 @@
 // Copyright (c) Microsoft Corporation.
 // Licensed under the MIT license.
 
+import "IMouseWheelListener.idl";
+
 namespace Microsoft.Terminal.TerminalControl
 {
     delegate void TitleChangedEventArgs(String newTitle);
@@ -30,7 +32,7 @@ namespace Microsoft.Terminal.TerminalControl
         void HandleClipboardData(String data);
     }
 
-    [default_interface] runtimeclass TermControl : Windows.UI.Xaml.Controls.UserControl, IF7Listener
+    [default_interface] runtimeclass TermControl : Windows.UI.Xaml.Controls.UserControl, IF7Listener, IMouseWheelListener
     {
         TermControl();
         TermControl(Microsoft.Terminal.Settings.IControlSettings settings, Microsoft.Terminal.TerminalConnection.ITerminalConnection connection);

src/cascadia/TerminalControl/TerminalControl.vcxproj

@@ -76,6 +76,7 @@
       <DependentUpon>TermControl.xaml</DependentUpon>
     </Midl>
     <Midl Include="TermControlAutomationPeer.idl" />
+    <Midl Include="IMouseWheelListener.idl" />
     <Midl Include="TSFInputControl.idl">
       <DependentUpon>TSFInputControl.xaml</DependentUpon>
     </Midl>

src/cascadia/TerminalControl/pch.h

@@ -9,6 +9,8 @@
 
 #define WIN32_LEAN_AND_MEAN
 
+// Manually include til after we include Windows.Foundation to give it winrt superpowers
+#define BLOCK_TIL
 #include <LibraryIncludes.h>
 // This is inexplicable, but for whatever reason, cppwinrt conflicts with the
 //      SDK definition of this function, so the only fix is to undef it.
@@ -23,6 +25,7 @@
 #include <unknwn.h>
 #include <winrt/Windows.Foundation.h>
 #include <winrt/Windows.Foundation.Collections.h>
+#include <winrt/Windows.system.h>
 #include <winrt/Windows.Graphics.Display.h>
 #include <winrt/windows.ui.core.h>
 #include <winrt/Windows.ui.input.h>
@@ -46,3 +49,5 @@
 #include <TraceLoggingProvider.h>
 TRACELOGGING_DECLARE_PROVIDER(g_hTerminalControlProvider);
 #include <telemetry/ProjectTelemetry.h>
+
+#include "til.h"

src/cascadia/TerminalCore/ControlKeyStates.hpp

@@ -47,6 +47,27 @@ class Microsoft::Terminal::Core::ControlKeyStates
         return *this;
     }
 
+#ifdef WINRT_Windows_System_H
+    ControlKeyStates(const winrt::Windows::System::VirtualKeyModifiers& modifiers) noexcept :
+        _value{ 0 }
+    {
+        // static_cast to a uint32_t because we can't use the WI_IsFlagSet
+        // macro directly with a VirtualKeyModifiers
+        const auto m = static_cast<uint32_t>(modifiers);
+        _value |= WI_IsFlagSet(m, static_cast<uint32_t>(winrt::Windows::System::VirtualKeyModifiers::Shift)) ?
+                      SHIFT_PRESSED :
+                      0;
+
+        // Since we can't differentiate between the left & right versions of Ctrl & Alt in a VirtualKeyModifiers
+        _value |= WI_IsFlagSet(m, static_cast<uint32_t>(winrt::Windows::System::VirtualKeyModifiers::Menu)) ?
+                      LEFT_ALT_PRESSED :
+                      0;
+        _value |= WI_IsFlagSet(m, static_cast<uint32_t>(winrt::Windows::System::VirtualKeyModifiers::Control)) ?
+                      LEFT_CTRL_PRESSED :
+                      0;
+    }
+#endif
+
     constexpr DWORD Value() const noexcept
     {
         return _value;

src/cascadia/WindowsTerminal/AppHost.cpp

@@ -51,7 +51,7 @@ AppHost::AppHost() noexcept :
                                                 _logic,
                                                 std::placeholders::_1,
                                                 std::placeholders::_2));
-
+    _window->MouseScrolled({ this, &AppHost::_WindowMouseWheeled });
     _window->MakeWindow();
 }
 
@@ -338,3 +338,48 @@ void AppHost::_ToggleFullscreen(const winrt::Windows::Foundation::IInspectable&,
 {
     _window->ToggleFullscreen();
 }
+
+// Method Description:
+// - Called when the IslandWindow has received a WM_MOUSEWHEEL message. This can
+//   happen on some laptops, where their trackpads won't scroll inactive windows
+//   _ever_.
+// - We're going to take that message and manually plumb it through to our
+//   TermControl's, or anything else that implements IMouseWheelListener.
+// - See GH#979 for more details.
+// Arguments:
+// - coord: The Window-relative, logical coordinates location of the mouse during this event.
+// - delta: the wheel delta that triggered this event.
+// Return Value:
+// - <none>
+void AppHost::_WindowMouseWheeled(const til::point coord, const int32_t delta)
+{
+    if (_logic)
+    {
+        // Find all the elements that are underneath the mouse
+        auto elems = winrt::Windows::UI::Xaml::Media::VisualTreeHelper::FindElementsInHostCoordinates(coord, _logic.GetRoot());
+        for (const auto& e : elems)
+        {
+            // If that element has implemented IMouseWheelListener, call OnMouseWheel on that element.
+            if (auto control{ e.try_as<winrt::Microsoft::Terminal::TerminalControl::IMouseWheelListener>() })
+            {
+                try
+                {
+                    // Translate the event to the coordinate space of the control
+                    // we're attempting to dispatch it to
+                    const auto transform = e.TransformToVisual(nullptr);
+                    const til::point controlOrigin{ til::math::flooring, transform.TransformPoint(til::point{ 0, 0 }) };
+
+                    const til::point offsetPoint = coord - controlOrigin;
+
+                    if (control.OnMouseWheel(offsetPoint, delta))
+                    {
+                        // If the element handled the mouse wheel event, don't
+                        // continue to iterate over the remaining controls.
+                        break;
+                    }
+                }
+                CATCH_LOG();
+            }
+        }
+    }
+}

src/cascadia/WindowsTerminal/AppHost.h

@@ -35,4 +35,5 @@ class AppHost
                       const winrt::Windows::UI::Xaml::ElementTheme& arg);
     void _ToggleFullscreen(const winrt::Windows::Foundation::IInspectable& sender,
                            const winrt::TerminalApp::ToggleFullscreenEventArgs& arg);
+    void _WindowMouseWheeled(const til::point coord, const int32_t delta);
 };