Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

InternetOpenUrl() failed - 0x80072f05 : unknown error #2956

Closed
tigerinus opened this issue Feb 12, 2023 · 25 comments
Closed

InternetOpenUrl() failed - 0x80072f05 : unknown error #2956

tigerinus opened this issue Feb 12, 2023 · 25 comments
Labels
Issue-Bug It either shouldn't be doing this or needs an investigation.
Milestone

Comments

@tigerinus
Copy link

Brief description of your issue

image

Looks like https://cdn.winget.microsoft.com/cache/source.msix does not come with a valid SSL certificate:

image

Steps to reproduce

winget upgrade --all --verbose

Expected behavior

it should just work

Actual behavior

see screenshot.

Environment

Windows Package Manager v1.4.10173
Copyright (c) Microsoft Corporation. All rights reserved.

Windows: Windows.Desktop v10.0.19045.2486
System Architecture: X64
Package: Microsoft.DesktopAppInstaller v1.19.10173.0

Logs: %LOCALAPPDATA%\Packages\Microsoft.DesktopAppInstaller_8wekyb3d8bbwe\LocalState\DiagOutputDir

User Settings: %LOCALAPPDATA%\Packages\Microsoft.DesktopAppInstaller_8wekyb3d8bbwe\LocalState\settings.json

Links
---------------------------------------------------------------------------
Privacy Statement   https://aka.ms/winget-privacy
License Agreement   https://aka.ms/winget-license
Third Party Notices https://aka.ms/winget-3rdPartyNotice
Homepage            https://aka.ms/winget
Windows Store Terms https://www.microsoft.com/en-us/storedocs/terms-of-sale
@microsoft-github-policy-service microsoft-github-policy-service bot added the Needs-Triage Issue need to be triaged label Feb 12, 2023
@hawkowl
Copy link

hawkowl commented Feb 12, 2023

Seconded here:

2023-02-12 12:49:51.440 [CORE] WinGet, version [1.4.10173], activity [{302FD314-8C54-40CD-801A-0E06E020267B}]
2023-02-12 12:49:51.440 [CORE] OS: Windows.Desktop v10.0.22621.1105
2023-02-12 12:49:51.440 [CORE] Command line Args: "C:\Users\Amber\AppData\Local\Microsoft\WindowsApps\winget.exe" update --verbose-logs
2023-02-12 12:49:51.440 [CORE] Package: Microsoft.DesktopAppInstaller v1.19.10173.0
2023-02-12 12:49:51.440 [CORE] IsCOMCall:0; Caller: winget-cli
2023-02-12 12:49:51.443 [CLI ] WinGet invoked with arguments: 'update' '--verbose-logs'
2023-02-12 12:49:51.443 [CLI ] Found subcommand: update
2023-02-12 12:49:51.443 [CLI ] Leaf command to execute: root:upgrade
2023-02-12 12:49:51.445 [CORE] Setting action: Get, Type: Secure, Name: admin_settings
2023-02-12 12:49:51.445 [CORE] Admin settings was not found
2023-02-12 12:49:51.446 [CORE] Setting action: Get, Type: Secure, Name: admin_settings
2023-02-12 12:49:51.446 [CORE] Admin settings was not found
2023-02-12 12:49:51.446 [CORE] Setting action: Get, Type: Secure, Name: admin_settings
2023-02-12 12:49:51.446 [CORE] Admin settings was not found
2023-02-12 12:49:51.446 [CLI ] Executing command: upgrade
2023-02-12 12:49:51.446 [REPO] Additional sources GP is not enabled.
2023-02-12 12:49:51.446 [CORE] Setting action: Get, Type: Secure, Name: user_sources
2023-02-12 12:49:51.446 [CORE] Setting action: Get, Type: Secure, Name: admin_settings
2023-02-12 12:49:51.446 [CORE] Admin settings was not found
2023-02-12 12:49:51.446 [CORE] Adding chain to pinning configuration [Microsoft Store Source]:
DigiCert Global Root G2 : PublicKey
  Microsoft Azure TLS Issuing CA 01 : Subject | Issuer
    sfdataservice.microsoft.com : Subject | Issuer
2023-02-12 12:49:51.454 [CORE] Setting action: Get, Type: Standard, Name: sources_metadata
2023-02-12 12:49:51.454 [YAML] Detected UTF-8
2023-02-12 12:49:51.454 [REPO] GetCurrentSourceRefs: Source named 'microsoft.builtin.desktop.frameworks' from origin Default is hidden and is dropped.
2023-02-12 12:49:51.454 [REPO] Default source requested, multiple sources available, adding all to source references.
2023-02-12 12:49:51.454 [REPO] Adding to source references msstore
2023-02-12 12:49:51.454 [REPO] Adding to source references winget
2023-02-12 12:49:51.454 [REPO] Source past auto update time [5 mins]; it has been at least 27936109 mins
2023-02-12 12:49:51.539 [FAIL] WindowsPackageManager.dll!00007FF8F23FCA12: ReturnHr(1) tid(56d0) 80072F05     Msg:[winrt::hresult_error: The date in the certificate is invalid or has expired] 

2023-02-12 12:49:51.539 [FAIL] WindowsPackageManager.dll!00007FF8F2284431: LogHr(2) tid(56d0) 80072F05 
2023-02-12 12:49:51.539 [FAIL] D:\a\_work\1\s\external\pkg\src\AppInstallerRepositoryCore\RepositorySource.cpp(53)\WindowsPackageManager.dll!00007FF8F2407F41: (caller: 00007FF8F22D0DBE) LogHr(3) tid(56d0) 80072F05     Msg:[winrt::hresult_error: The date in the certificate is invalid or has expired] 

2023-02-12 12:49:51.539 [REPO] Source add/update failed, waiting a bit and retrying: winget
2023-02-12 12:49:53.599 [FAIL] WindowsPackageManager.dll!00007FF8F23FCA12: ReturnHr(2) tid(56d0) 80072F05     Msg:[winrt::hresult_error: The date in the certificate is invalid or has expired] 

2023-02-12 12:49:53.599 [FAIL] WindowsPackageManager.dll!00007FF8F2284431: LogHr(5) tid(56d0) 80072F05 
2023-02-12 12:49:53.599 [FAIL] D:\a\_work\1\s\external\pkg\src\AppInstallerRepositoryCore\RepositorySource.cpp(540)\WindowsPackageManager.dll!00007FF8F2407523: (caller: 00007FF8F21C8E1D) LogHr(6) tid(56d0) 80072F05     Msg:[winrt::hresult_error: The date in the certificate is invalid or has expired] 

2023-02-12 12:49:53.600 [REPO] Failed to update source: winget
2023-02-12 12:49:53.600 [REPO] Multiple sources available, creating aggregated source.

The served certificate is here, fingerprint F0:AA:13:A5:24:3B:AC:A6:00:3E:75:FA:59:5E:2F:20:36:54:BC:B6:09:BD:B2:71:CC:ED:98:60:93:FB:D7:95:

cdn-winget-microsoft-com.pem.txt

@beirlaenl
Copy link

beirlaenl commented Feb 12, 2023

Looks like the certificate for https://cdn.winget.microsoft.com/cache expired about 1.5 hours ago.

image

@qilme
Copy link

qilme commented Feb 12, 2023

You can add a source like https://winget.azureedge.net/cache using the command below.
sudo winget source add -n winget https://winget.azureedge.net/cache.
https://learn.microsoft.com/en-us/windows/package-manager/winget/source

@peanut996
Copy link

You can add a source like https://winget.azureedge.net/cache using the command below. sudo winget source add -n winget https://winget.azureedge.net/cache. https://learn.microsoft.com/en-us/windows/package-manager/winget/source

it works. thx!

@ax-zwk
Copy link

ax-zwk commented Feb 12, 2023

'sudo' is not recognized as an internal or external command,
operable program or batch file.

@peanut996
Copy link

peanut996 commented Feb 12, 2023

'sudo' is not recognized as an internal or external command, operable program or batch file.

powershell do not need sudo if you are admin account.
just first remove then re-add

winget source remove -n winget

winget source add -n winget https://winget.azureedge.net/cache

@trparky
Copy link

trparky commented Feb 12, 2023

Is it safe to do this? Will we have to go back to the old URL, https://cdn.winget.microsoft.com/cache?

@qilme
Copy link

qilme commented Feb 12, 2023

maybe. But its a aliases of cdn.winget.microsoft.com. you can nslookup it.

Aliases: cdn.winget.microsoft.com
winget.azureedge.net
winget.ec.azureedge.net
scdn283e9.wpc.14d639.nucdn.net

@adamhl8
Copy link

adamhl8 commented Feb 12, 2023

Is it safe to do this? Will we have to go back to the old URL, cdn.winget.microsoft.com/cache?

You can always run winget source reset --force (as admin) to get back to defaults.

@Viajaz
Copy link

Viajaz commented Feb 12, 2023

Does each Microsoft Product Team manage the certificates of their infrastructure themselves? How are there not processes and automated checks to ensure various Product Teams CDNs etc. don't have their certificates expire on live services? Microsoft even operates the Issuing CA for the expired certificate in this instance.

@riverar
Copy link

riverar commented Feb 12, 2023

@denelon Red alert.

@kashif-khan
Copy link

'sudo' is not recognized as an internal or external command, operable program or batch file.

powershell do not need sudo if you are admin account. just first remove then re-add

winget source remove -n winget

winget source add -n winget https://winget.azureedge.net/cache

This worked for me.

@denelon denelon self-assigned this Feb 12, 2023
@denelon denelon added Issue-Bug It either shouldn't be doing this or needs an investigation. and removed Needs-Triage Issue need to be triaged labels Feb 12, 2023
@denelon
Copy link
Contributor

denelon commented Feb 12, 2023

Hey all. We're working on getting the certificate renewed.

@R-Adrian
Copy link

R-Adrian commented Feb 12, 2023

according to the certificate transparency logs it should have already been renewed 3 times, most recently in november 2022... but none of those renewed certificates were actually picked up by the server.
If the certificate files on disk were changed, was the server process restarted to re-read the certificate files? Was it even restarted for scheduled updates?

image

@12932
Copy link

12932 commented Feb 12, 2023

That error message also needs to be fixed. What a terrible terrible error message

@denelon
Copy link
Contributor

denelon commented Feb 12, 2023

The new certificate has been imported. We're waiting for the provisioning to complete.

The expectation is 6 - 8 hours for full propagation.

@denelon
Copy link
Contributor

denelon commented Feb 12, 2023

Here is a screenshot from an endpoint that has the new certificate.
Note the timestamp:

The URL is https://cdn.winget.microsoft.com/cache

cert

The actual file used by the client is https://cdn.winget.microsoft.com/cache/source.msix

I'm going to keep the issue open until tomorrow so we can make sure the certificate has fully propagated before closing this issue.

@denelon
Copy link
Contributor

denelon commented Feb 12, 2023

fingerprints

@tigerinus
Copy link
Author

seems to be working now.

@denelon, what's the action taken to ensure this does not happen again?

@denelon
Copy link
Contributor

denelon commented Feb 12, 2023

We're going to do a root cause on Monday. The certificate was issued in November, and it was published to the secret store, but it didn't get configured on the endpoints. I'm not sure yet if it was code that we wrote not doing the right thing, or something else.

@denelon
Copy link
Contributor

denelon commented Feb 12, 2023

Since we haven't had any new reports of certificate problems, I'm going to go ahead and close this issue now.

@denelon denelon closed this as completed Feb 12, 2023
@denelon denelon added this to the v1.5-Client milestone Feb 12, 2023
@denelon denelon added this to WinGet Feb 12, 2023
@denelon denelon moved this to Released in WinGet Feb 12, 2023
@denelon denelon unpinned this issue Feb 13, 2023
@denelon
Copy link
Contributor

denelon commented Feb 14, 2023

All,
We're still investigating the root cause here. We're reaching out to other teams to understand if anything changed with our upstream dependencies or not. We did have certificate rotation configured, and @R-Adrian shared the certificates that had been generated via auto-renewal. The issue seems to be with the process to update the certificates with our CDN provider.

@R-Adrian
Copy link

R-Adrian commented Feb 14, 2023

i think the problem for the server refusing to pick the new certificates is that the "renewals" were issued from a different certification authority.

The first certificate (the one that expired on February 12th) was issued from Microsoft Azure TLS Issuing CA 05 but the renewals were from Microsoft Azure TLS Issuing CA 01.
Getting a certificate from a different authority is usually considered a new issuance, not a simple renewal, or the server might have been configured with a specific CA only. (pinned issuing CA certificate?).

edit:
BTW, both Issuing CA 01 and Issuing CA 05 are scheduled to expire on June 27th 2024.
if they keep issuing 1-year-long certificates, that means they must stop issuing certificates at least 365 days before their CA certificates will expire themselves, so, probably June 26th 2023 is the cutoff date.

I suspect they will be superseded a bit before that, at the latest in April or May 2023 they will probably be replaced and will stop issuing any new certificates a bit earlier than the cutoff.

@denelon
Copy link
Contributor

denelon commented Feb 15, 2023

Root Cause

While our certificate was being properly auto renewed, the new certificate was not being loaded into the CDN endpoints. After discussing with our Azure infrastructure team, we were informed that autorotation is not supported with our specific configuration. Additionally, it does not appear that any monitoring was in place to check the certificate that was actually in use on the endpoint for validity. 

Our planned path forward

In consultation with the Azure Front Door team – we have determined that the Azure Front Door CDN does support certificate autorotation. We will beging working on changes to our Azure Front Door CDN so that autorotation is supported. We will also add monitoring to verify that the certificates have been configured on the CDN endpoints.

@HeyItsJono
Copy link

HeyItsJono commented Mar 22, 2023

For what it's worth, I'm still having issues as documented in #1656. My post from there:

Can confirm I'm also getting Failed in attempting to update the source: winget, in Australia.

Things I've tried that don't work:

  • winget source reset --force then winget upgrade; issue continues to occur
  • winget source update --name winget; just continuously loads for hours until I manually break the command with Ctrl+C
  • winget source update; updates msstore source without issue then hangs again on winget source
  • Use a USA NordVPN node; no change to any of the above outcomes

For me winget source list shows winget points to https://cdn.winget.microsoft.com/cache.
Visiting this page shows a valid and in-date SSL certificate, but the page itself throws the following error:

<Error>
<Code>ResourceNotFound</Code>
<Message>The specified resource does not exist. RequestId:c8d43b9b-101e-004d-748d-5c6f85000000 Time:2023-03-22T07:08:08.5530177Z</Message>
</Error>

I can curl https://cdn.winget.microsoft.com/cache/source.msix --output ./source.msix and end up with source.msix, SHA-256 1b94f4cad9e735f2b650f579ed7e9d83ff84e4a8217951d77fd3853f89968dba, size 6.24mb.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Issue-Bug It either shouldn't be doing this or needs an investigation.
Projects
None yet
Development

No branches or pull requests