Show a window with tabs as 1 window instead of 1 window per tab

[lwouis]

From discussion in #256 (comment)

Windows are grouped together as tabs. This is built-in macOS mechanisms where multiple NSWindows are grouped together. They appear as individual windows in AltTab since the accessibility API lists them that way.

There may be a way, potentially using private APIs, to notice tabs, and group them as 1 window in AltTab.

[lwouis]

After discussing with another user, i think this, if technically doable, should be a preference: "show tabs as separate windows". Some people will want the split and some will want to see 1 window per tab group.

[lwouis]

I have been investigating this further and it's a snake pit. Indeed the OS treats windows in a strange way when they implement standard NSWindow tabs (i.e. using AppKit).

I checked all AX notifications, and none of them detect windows merging into tabs. Same for splitting into individual windows. Basically merging into tabs means that windows become children of other windows, instead of the application. Splitting means they become children of the app again.

Current state in AltTab

Because we are not aware of these changes, windows or tabs created after AltTab is running will appear as individual windows. Merging them into tabs won't affect that, which is a bug from the user perspective.

Conversely, tabbed windows made before AltTab was open will be seen as only 1 window by AltTab as they are already created, and asking the app for its children won't reveal the windows children of the main window.

Moving forwards

Since we can't observe changes of children/parent, it means we have to detect this last minute: when the user summons the UI. This sucks because it means more computation and more latency.

I can imagine 2 ways to deal with this. On summon, we:

1. Ask every app their windows again, and when it doesn't match our current understanding (because some windows have been tabbed/split), we sync it up.

2. Ask every window we observe for their children. Then look for a child with role AXTabGroup which indicates that the window has children windows. We could then sync it up. I like that this approach only checks for the existing windows, which is a way smaller list than the apps, so less work. However, I don't know how robust it is to check for this AXTabGroup.

3. Ask every window we observe for their parent. If it's not an application, then remove it from our list.

Other considerations:

• Option 2 and 3 don't handle one use-case: AltTab starts but they are already tabbed windows. They will never be known to AltTab as the app reports 1 window, and we never know that it has children windows itself.

• Checking for AXTabGroup like in option 2 is the only way that would allow to implement a preference to display tabs as individual windows. Given that cmd-tab + down arrow or Mission Control don't let users navigate tabs, and given that nobody has asked for that extra feature so far, I think this will go to the back-burner. I think it would be a great feature for people who have lots of tabs, but since no-one asked for it yet, I guess not many people have this workflow.

• With any option, it may be an opportunity to also fix issues where windows go invalid without us getting noticed, and then we display incorrect windows. This seems to still be an issue, even though it's hard to pin-point what's at play: Mail.app has an extra window showing in AltTab #292 Phantom windows created by app Optimus Player #200. I also noticed similar issues working on Stops working when Octave-app runs #182 "Steam" windows do not show up #236, however I fixed a deeper cause that prevented these symptoms from showing up, for those 2 tickets. We could verify at the last second if the windows are still valid.

• HyperSwitch handles this correctly. I suspect they do option 1.

[lwouis]

Damn I found more complexity :/

Focusing different tabs doesn't trigger the kAXFocusedWindowChangedNotification notification that we rely on to determine chronology of user interaction. This means that even if we implement one of the workarounds above, we are still having issues: when a tab is split into a window, where do we put it in the chronology? We don't know when the user interacted with that tab last. Same thing when the user switches tabs within a window.

[lwouis]

Hehe checked how our friends at yabai are handling this, and I got a good laugh out of it. I can feel their pain after having looked into this mess for a few days.

It makes me really wonder how professional assistive apps like screen readers for visually impaired people deal with this lack of API from macOS...

[koekeishiya]

Focusing different tabs doesn't trigger the kAXFocusedWindowChangedNotification notification

It does trigger kAXMainWindowChangedNotification, if you wish to continue down this rabbithole. Unfortunately, I didn't actually document all of my findings..

[lwouis]

It does trigger kAXMainWindowChangedNotification, if you wish to continue down this rabbithole. Unfortunately, I didn't actually document all of my findings..

I actually found this soon after posting! Thanks for sharing as always though @koekeishiya! 👍

[lwouis]

Good news! I got this working! I even made it a preferences: "Show standard tabs as windows":

• If unchecked, the user will see each tab group as 1 window
• If checked, the user will see each tab as 1 window

Unchecked is the default to be in line with Mission Control.

The only bug/limitation is if AltTab is started after the user has already created a tab group and they check the new preference, then we will not show the other tabs in the tab group until the user focuses them manually. There is simply no way to get these windows before the user focuses them as the app is not listing them.

Everything else works: minimized, hidden, other spaces, using Window > Merge All Windows, using Window > Move Tab to New Window, drag-and-dropping tabs into/out-of windows, etc. Even doing any of these while AltTab UI is displayed.

[ryenus]

Wow, I was thinking why vscode tabs were displayed as separate windows the other day, and now you just worked on it, thanks!

Nevertheless, I just tried v3.19.1 and below are my observations:

With "Show standard tabs as windows" unchecked

Only windows from current space are displayed with it unchecked, even though the option show windows from all spaces is active.

With "Show standard tabs as windows" checked

The option show windows from all spaces is now respected, however, for some apps, at least vscode, the tabs actually do not show up as separate windows, but as one window. BTW, I use native tabs in vscode (via "window.nativeTabs": true), with all the tabs merged into one window.
When checked later, I'd say the result varies.

[lwouis]

Hey @ryenus! Thanks so much for finding this bug! I can reproduce it locally. I'm sad this new feature introduced a regression. I'm sad I introduced a regression. There is just too much to test these days after a change.

I'll see how this can be mitigated as soon as possible. I hope it's not a big blocker, as Spaces are often a hard issue to handle due to there being no public API to handle them.

[nathang21]

Hey Louis, it happens, we appreciate all the work you do! Plus reducing the auto-update frequency as you did recently helps reduce the impact of hiccups like this.

…

On Wed, May 6, 2020 at 7:49 PM lwouis ***@***.***> wrote: Hey @ryenus <https://github.com/ryenus>! Thanks so much for finding this bug! I can reproduce it locally. I'm sad this new feature introduced a regression. I'm sad I introduced a regression. There is just too much to test <https://github.com/lwouis/alt-tab-macos/blob/master/docs/QA.md> these days after a change. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#258 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB4CUJQMLSEQFA5M7UFTVRDRQIOS5ANCNFSM4MMVADBA> .

[lwouis]

@nathang21 thanks for the encouraging words! I pushed a workaround. While working on it I discovered another bug in macOS:

// macOS bug: if you tab a window, then move the tab group to another space, other tabs from the tab group will stay on the current space
// you can use the Dock to focus one of the other tabs and it will teleport that tab in the current space, proving that it's a macOS bug
// note: for some reason, it behaves differently if you minimize the tab group after moving it to another space

There is nothing I can do here as it is OS behavior, reproducible without AltTab.

Another point is that the only workaround I could think to detect tabs was to subscribe to kAXFocusedUIElementChangedNotification. As the name implies, it happens pretty much all the time when you interact with the app's UI.

[ryenus]

Thank you @lwouis for all the efforts and more specifically the diligence you've put into this :-)

With "Show standard tabs as windows" unchecked in v3.19.2, now I see some strange layout issues, assuming below is the list of windows and an x stands of a window:

xx xx
xxxxx

Another thing I noticed is that when pressing native cmd+tab quickly, to just switch back to the most recent app, the apps are switched meanwhile the apps list isn't shown at all within a noticeable delay. Is it possible to simulate that in alt-tab? So the windows list is only displayed after pressing alt-tab and holding alt key for a while, e.g., 300ms or something which might be configurable. Or maybe this is already possible just the default delay is too short?

[lwouis]

@ryenus Thank you for the kind words 👍

With "Show standard tabs as windows" unchecked in v3.19.2, now I see some strange layout issues, assuming below is the list of windows and an x stands of a window:

Could you please attach a screenshot? A way to do it is to launch the Screenshot app and put a 5s timer. That way you can activate AltTab, wait, and snap.

Another alternative is to install Loom. A user sent me a video recording with that app a few weeks back, and it's such a delight to see a problem with a nice video and audio. I'm using it myself now and it's just one click and you get a link to share, no need to upload anywhere. Really convenient.

Another thing I noticed is that when pressing native cmd+tab quickly, to just switch back to the most recent app, the apps are switched meanwhile the apps list isn't shown at all within a noticeable delay. Is it possible to simulate that in alt-tab? So the windows list is only displayed after pressing alt-tab and holding alt key for a while, e.g., 300ms or something which might be configurable. Or maybe this is already possible just the default delay is too short?

All of this is already implemented in AltTab. There are 2 sides to it:

• Artificial delay. This is for users who are not pro-gamers and want a fixed large delay with no UI displayed when using AltTab. For example 300ms where you're guaranteed no UI will show, then it does. This is a preference called "Apparition delay":

• Intrinsic delay. If you have a high-dpi screen, and lots of windows, AltTab will take more milliseconds to display. Let's say it takes 200ms. You can release the shortcut before that for sure. If you do, I made sure that we drop UI calculations, and switch to the window immediately. However, if you have few windows, AltTab will be so fast to compute its UI that it's going to be faster than your fingers so there is no way to avoid that. I assume that this is what's happening for you. AltTab must show the UI within 16ms, and beat your display 60hz refresh rate and show on the very next frame. Thus we can't cancel it. If you want to avoid this "flashing" of a few frames of AltTab, see the artificial delay preference above 👍

[mfn]

While working on it I discovered another bug

You should write a blog 😄