Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update MinGW-w64 to dcd7fefc703fb4b12187235386900d34cc13fdc5 #22156

Merged
merged 6 commits into from
Dec 14, 2024

Conversation

alexrp
Copy link
Member

@alexrp alexrp commented Dec 5, 2024

The most notable change here is the addition of winpthreads support, which is now included in the produced mingw32.lib. Accordingly, -lpthread and -lwinpthread are now special-cased as libc names for MinGW.

Closes #10989.

@alexrp
Copy link
Member Author

alexrp commented Dec 5, 2024

@andrewrk would like your OK on adding winpthreads support here.

@alexrp alexrp requested a review from andrewrk December 5, 2024 17:22
@linusg
Copy link
Contributor

linusg commented Dec 5, 2024

cc @kassane, sounds like this will make https://github.com/kassane/winpthreads-zigbuild redundant.

@andrewrk
Copy link
Member

The sheer amount of code here is giving me pause. Also, I think nobody should really be using winpthreads. While it's a cool hack, it's far better to do conditional compilation per OS and implement threads using Windows APIs instead.

On the other hand, if this is a dependency of someone's project, and they can't compile with Zig's mingw-w64 then that is a problem.

Can you please share the change in installation size that this brings?

@alexrp
Copy link
Member Author

alexrp commented Dec 12, 2024

The sheer amount of code here is giving me pause. Also, I think nobody should really be using winpthreads. While it's a cool hack, it's far better to do conditional compilation per OS and implement threads using Windows APIs instead.

On the other hand, if this is a dependency of someone's project, and they can't compile with Zig's mingw-w64 then that is a problem.

It seems like there are C/C++ projects out there that do rely on winpthreads for Windows support (see #10989, which resulted in https://github.com/kassane/winpthreads-zigbuild being created).

I do agree that there's no good reason to be using winpthreads in Zig code.

Can you please share the change in installation size that this brings?

directory size (bytes) notes
lib/libc/mingw 7,620,059 master
lib/libc/mingw 7,708,937 mingw upgrade
lib/libc/mingw 7,854,626 mingw upgrade + winpthreads
lib/libc/include/any-windows-any 73,057,906 master
lib/libc/include/any-windows-any 79,555,645 mingw upgrade
lib/libc/include/any-windows-any 79,604,089 mingw upgrade + winpthreads

Doesn't seem too bad? Most of the size increase is just from the MinGW upgrade itself, and more specifically, from IDL-generated headers. The worst offenders are mshtml.h, windows.applicationmodel.datatransfer.h, windows.data.json.h, and windows.ui.notifications.h.

#18477 would certainly help here. But an IDL compiler is a non-trivial piece of software and my to-do list is way too long as it is... so I guess I'll just leave it up to you whether we should take this MinGW upgrade as-is.

@andrewrk
Copy link
Member

Alright, go ahead and merge it.

I think last time I deleted the worst N offenders out of spite and nobody noticed. To be honest if it were me working on this upgrade, I would do it again.

@alexrp
Copy link
Member Author

alexrp commented Dec 12, 2024

FWIW, it seems like we could save a decent chunk by just nuking the WinRT fluff in particular:

cat $(grep -RIl "Autogenerated by WIDL") | wc -c
47669477cat $(grep -RIl "Autogenerated by WIDL" | grep -v "windows") | wc -c
35545010

I don't know how relevant this stuff is for MinGW. Does anyone actually use it? Perhaps it's fine to get rid of it until somebody complains.

@alexrp
Copy link
Member Author

alexrp commented Dec 12, 2024

I don't know how relevant this stuff is for MinGW. Does anyone actually use it? Perhaps it's fine to get rid of it until somebody complains.

Hmm, unfortunately, it seems like there is code in the wild that assumes the presence of WinRT headers with MinGW: https://github.com/search?q=%28language%3Ac+OR+language%3Ac%2B%2B%29+NOT+%28path%3Apch.cpp+OR+path%3Apch.h+OR+path%3Astdafx.h+OR+path%3Aany-windows-any+OR+path%3Amingw-w64-headers%29+%2F%23include+%3Cwindows%28%5C.%5Cw*%29%2B%5C.h%3E%2F+mingw&type=code

@alexrp
Copy link
Member Author

alexrp commented Dec 14, 2024

Ok, I nuked the WinRT headers anyway.

 delete mode 100644 lib/libc/include/any-windows-any/eventtoken.h
 delete mode 100644 lib/libc/include/any-windows-any/ivectorchangedeventargs.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.applicationmodel.activation.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.applicationmodel.background.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.applicationmodel.core.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.applicationmodel.datatransfer.dragdrop.core.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.applicationmodel.datatransfer.dragdrop.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.applicationmodel.datatransfer.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.applicationmodel.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.data.json.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.data.xml.dom.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.devices.bluetooth.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.devices.enumeration.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.devices.geolocation.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.devices.haptics.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.devices.input.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.devices.power.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.devices.radios.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.devices.usb.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.foundation.collections.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.foundation.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.foundation.metadata.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.foundation.numerics.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.gaming.input.custom.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.gaming.input.forcefeedback.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.gaming.input.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.gaming.ui.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.globalization.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.graphics.capture.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.graphics.capture.interop.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.graphics.directx.direct3d11.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.graphics.directx.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.graphics.effects.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.graphics.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.graphics.holographic.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.graphics.imaging.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.management.deployment.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.media.capture.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.media.closedcaptioning.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.media.devices.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.media.effects.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.media.faceanalysis.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.media.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.media.render.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.media.speechrecognition.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.media.speechsynthesis.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.networking.connectivity.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.networking.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.perception.spatial.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.perception.spatial.surfaces.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.security.authentication.onlineid.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.security.authorization.appcapabilityaccess.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.security.credentials.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.security.credentials.ui.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.security.cryptography.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.security.enterprisedata.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.security.exchangeactivesyncprovisioning.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.security.isolation.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.storage.fileproperties.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.storage.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.storage.search.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.storage.streams.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.system.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.system.power.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.system.profile.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.system.profile.systemmanufacturers.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.system.threading.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.system.userprofile.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.ui.composition.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.ui.composition.interop.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.ui.core.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.ui.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.ui.input.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.ui.notifications.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.ui.viewmanagement.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.ui.xaml.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.ui.xaml.hosting.desktopwindowxamlsource.h
 delete mode 100644 lib/libc/include/any-windows-any/windows.ui.xaml.interop.h
 delete mode 100644 lib/libc/include/any-windows-any/windowscontracts.h

That gets us down from 79,604,089 bytes to 67,468,048 bytes, vs the current 73,057,906 bytes.

I doubt anyone is actually building software that needs these with zig cc/zig c++. Worst case, if there really is someone doing that, we can add these bloated headers when an issue is opened.

Does this sound reasonable @andrewrk?

@andrewrk
Copy link
Member

Yes, looks quite reasonable. Thank you!

@alexrp alexrp enabled auto-merge December 14, 2024 09:02
@alexrp alexrp merged commit 70de2f3 into ziglang:master Dec 14, 2024
10 checks passed
@alexrp alexrp deleted the mingw-update branch December 15, 2024 00:36
alexrp added a commit to alexrp/zig that referenced this pull request Dec 15, 2024
It should not use the CRT ones, and it also needs a few flags of its own that I
forgot to add in ziglang#22156.

Follow-up fix for ziglang#10989.
alexrp added a commit that referenced this pull request Dec 16, 2024
It should not use the CRT ones, and it also needs a few flags of its own that I
forgot to add in #22156.

Follow-up fix for #10989.
@jayschwa
Copy link
Contributor

Ok, I nuked the WinRT headers anyway.

I doubt anyone is actually building software that needs these with zig cc/zig c++. Worst case, if there really is someone doing that, we can add these bloated headers when an issue is opened.

SDL has a joystick implementation that uses windows.gaming.input.h, but I'm guessing it's not commonly used compared to its other joystick implementations for Windows.

@alexrp
Copy link
Member Author

alexrp commented Dec 16, 2024

Is that a primary implementation or one you have to opt into? Is it enabled by default at configure time?

@marler8997
Copy link
Contributor

If Zig did ever want to get in the business of shipping WinRT headers, there's also an option of shipping the underlying "winmd" binary files that Microsoft distributes for these APIs and generate the headers ourselves. I'm currently working on a parser for these in Zig.

@alexrp
Copy link
Member Author

alexrp commented Dec 16, 2024

How large is the full set of WinMD files?

@marler8997
Copy link
Contributor

marler8997 commented Dec 16, 2024

I'm not sure how to find the full set of winmd files, but, here is a subset that happens to currently be installed on my machine courtesy of whatever set of Visual Studio packages I happen to have:

> Get-ChildItem -Path "C:\Program Files (x86)\Windows Kits\10\References\10.0.22621.0" -Recurse -Filter "*.winmd" | Sort-Object Length | ForEach-Object { "{0,10:N2} KB {1}" -f ($_.Length / 1KB), $_.Name }
      3.00 KB Windows.Media.Capture.CameraCaptureUIContract.winmd
      3.50 KB Windows.Gaming.Input.GamingInputPreviewContract.winmd
      3.50 KB Windows.Gaming.UI.GamingUIProviderContract.winmd
      3.50 KB Windows.Management.Workplace.WorkplaceSettingsContract.winmd
      3.50 KB Windows.ApplicationModel.Background.BackgroundAlarmApplicationContract.winmd
      3.50 KB Windows.Media.Protection.ProtectionRenewalContract.winmd
      3.50 KB Windows.UI.ViewManagement.ViewManagementViewScalingContract.winmd
      3.50 KB Windows.System.Profile.ProfileSharedModeContract.winmd
      3.50 KB Windows.Perception.Automation.Core.PerceptionAutomationCoreContract.winmd
      4.00 KB Windows.System.Profile.ProfileHardwareTokenContract.winmd
      4.00 KB Windows.Globalization.GlobalizationJapanesePhoneticAnalyzerContract.winmd
      4.00 KB Windows.System.UserProfile.UserProfileLockScreenContract.winmd
      4.00 KB Windows.Devices.Printers.PrintersContract.winmd
      4.00 KB Windows.Devices.Portable.PortableDeviceContract.winmd
      4.00 KB Windows.Management.Deployment.Preview.DeploymentPreviewContract.winmd
      4.00 KB Windows.ApplicationModel.Activation.WebUISearchActivatedEventsContract.winmd
      4.00 KB Windows.UI.Shell.SecurityAppManagerContract.winmd
      4.00 KB Windows.ApplicationModel.Activation.ActivationCameraSettingsContract.winmd
      4.00 KB Windows.ApplicationModel.Preview.InkWorkspace.PreviewInkWorkspaceContract.winmd
      4.50 KB Windows.Gaming.UI.GameChatOverlayContract.winmd
      4.50 KB Windows.Media.Playlists.PlaylistsContract.winmd
      4.50 KB Windows.Networking.NetworkOperators.LegacyNetworkOperatorsContract.winmd
      4.50 KB Windows.Media.Capture.AppCaptureMetadataContract.winmd
      4.50 KB Windows.ApplicationModel.StartupTaskContract.winmd
      4.50 KB Windows.Networking.Connectivity.WwanContract.winmd
      4.50 KB Windows.ApplicationModel.Calls.LockScreenCallContract.winmd
      5.00 KB Windows.ApplicationModel.FullTrustAppContract.winmd
      5.00 KB Windows.ApplicationModel.Wallet.WalletContract.winmd
      5.50 KB Windows.ApplicationModel.CommunicationBlocking.CommunicationBlockingContract.winmd
      5.50 KB Windows.UI.ApplicationSettings.ApplicationsSettingsContract.winmd
      6.00 KB Windows.Media.AppBroadcasting.AppBroadcastingContract.winmd
      6.00 KB Windows.Devices.Printers.Extensions.ExtensionsContract.winmd
      6.00 KB Windows.System.UserProfile.UserProfileContract.winmd
      6.00 KB Windows.ApplicationModel.Activation.ActivatedEventsContract.winmd
      6.00 KB Windows.System.Profile.SystemManufacturers.SystemManufacturersContract.winmd
      6.00 KB Windows.System.Profile.ProfileRetailInfoContract.winmd
      6.00 KB Windows.Devices.SmartCards.SmartCardBackgroundTriggerContract.winmd
      6.50 KB Windows.ApplicationModel.Resources.Management.ResourceIndexerContract.winmd
      6.50 KB Windows.Networking.Sockets.ControlChannelTriggerContract.winmd
      6.50 KB Windows.Devices.Custom.CustomDeviceContract.winmd
      7.00 KB Windows.Media.MediaControlContract.winmd
      7.00 KB Windows.UI.Core.CoreWindowDialogsContract.winmd
      7.00 KB Windows.Media.Devices.CallControlContract.winmd
      7.50 KB Windows.Media.Capture.GameBarContract.winmd
      7.50 KB Windows.ApplicationModel.Preview.Notes.PreviewNotesContract.winmd
      7.50 KB Windows.ApplicationModel.Search.Core.SearchCoreContract.winmd
      8.00 KB Windows.UI.Core.AnimationMetrics.AnimationMetricsContract.winmd
      8.00 KB Windows.Media.AppRecording.AppRecordingContract.winmd
      9.00 KB Windows.ApplicationModel.Activation.ContactActivatedEventsContract.winmd
      9.50 KB Windows.Web.Http.Diagnostics.HttpDiagnosticsContract.winmd
     10.00 KB Windows.UI.Core.Preview.Communications.PreviewCommunicationsContract.winmd
     10.00 KB Windows.Management.Deployment.SharedPackageContainerContract.winmd
     10.00 KB Windows.ApplicationModel.Calls.Background.CallsBackgroundContract.winmd
     10.50 KB Windows.UI.WebUI.Core.WebUICommandBarContract.winmd
     10.50 KB Windows.Security.ExchangeActiveSyncProvisioning.EasContract.winmd
     10.50 KB Windows.ApplicationModel.Calls.CallsVoipContract.winmd
     10.50 KB Windows.Gaming.Preview.GamesEnumerationContract.winmd
     10.50 KB Windows.Gaming.XboxLive.StorageApiContract.winmd
     12.00 KB Windows.UI.UIAutomation.UIAutomationContract.winmd
     13.00 KB Windows.ApplicationModel.Search.SearchContract.winmd
     13.00 KB Windows.UI.Xaml.Hosting.HostingContract.winmd
     13.50 KB Windows.Devices.Scanners.ScannerDeviceContract.winmd
     14.50 KB Windows.System.SystemManagementContract.winmd
     14.50 KB Windows.Services.TargetedContent.TargetedContentContract.winmd
     16.00 KB Windows.Services.Maps.LocalSearchContract.winmd
     16.50 KB Windows.Devices.Sms.LegacySmsApiContract.winmd
     17.50 KB Windows.ApplicationModel.SocialInfo.SocialInfoContract.winmd
     17.50 KB Windows.AI.MachineLearning.Preview.MachineLearningPreviewContract.winmd
     18.00 KB Windows.Networking.XboxLive.XboxLiveSecureSocketsContract.winmd
     19.27 KB Windows.Networking.NetworkOperators.NetworkOperatorsFdnContract.WinMD
     19.50 KB Windows.Management.Update.WindowsUpdateContract.winmd
     19.77 KB Windows.Embedded.DeviceLockdown.DeviceLockdownContract.winmd
     20.00 KB Windows.Media.Capture.AppCaptureContract.winmd
     20.00 KB Windows.Services.Maps.GuidanceContract.winmd
     20.50 KB Windows.Storage.Provider.CloudFilesContract.winmd
     20.77 KB Windows.Phone.StartScreen.DualSimTileContract.WinMD
     23.50 KB Windows.Security.EnterpriseData.EnterpriseDataContract.winmd
     24.00 KB Windows.Security.Isolation.Isolatedwindowsenvironmentcontract.winmd
     24.00 KB Windows.Foundation.FoundationContract.winmd
     29.00 KB Windows.Graphics.Printing3D.Printing3DContract.winmd
     29.00 KB Windows.ApplicationModel.Calls.CallsPhoneContract.winmd
     30.00 KB Windows.Devices.DevicesLowLevelContract.winmd
     31.00 KB Windows.Devices.SmartCards.SmartCardEmulatorContract.winmd
     33.00 KB Windows.Services.Store.StoreContract.winmd
     34.00 KB Windows.AI.MachineLearning.MachineLearningContract.winmd
     37.50 KB Windows.Media.Capture.AppBroadcastContract.winmd
     79.77 KB Windows.Phone.PhoneContract.winmd
    100.00 KB Windows.UI.Xaml.Core.Direct.XamlDirectContract.winmd
  5,910.00 KB Windows.Foundation.UniversalApiContract.winmd

The majority of the size in this case is the last winmd file being 5.9 MB, the total is around 6.9 MB. These are uncompressed binary files, If I compress the big 6 MB winmd with simple zip it comes down to about 1.6MB, probably better gzipped. There's also some unnecessary sections in these files we could easily strip out, although, I'm not sure how much is actually unnecessary. My parser says there's 60 KB of just string data in the 100 KB binary (Windows.UI.Xaml.Core.Direct.XamlDirectContract.winmd) so a guess is maybe 90% is the "metadata" we care about?

@castholm
Copy link
Contributor

Is that a primary implementation or one you have to opt into? Is it enabled by default at configure time?

SDL (at least SDL3) defaults to including the Windows.Gaming.Input implementation in the build if _WIN32_MAXVER >= 0x0A00 (= Windows 10 SDK) (MSBuild) or if the windows.gaming.input.h header can be found at configure time (CMake). SDL usually includes multiple joystick backends and tries to pick the best available one at runtime. As I understand it, the Windows.Gaming.Input backend is one of the preferred ones for Xbox controllers because it rounds out a few quirks that other backends can't solve on their own, but it's not necessary for controller input to be functional.

It would be nice to keep the WinRT headers but for SDL specifically it's not a big deal if it's dropped (a properly built dynamically linked SDL lets power users override almost everything with environment variables anyway). Perhaps they can be reintroduced later if the winmd header generation thing happens.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

zig c++: 'pthread.h' file not found for target x86_64-windows-gnu
6 participants