Publishing is down again (failed to update source) #2718

infinitepower18 · 2022-11-22T22:32:47Z

Brief description of your issue

Same problem that previously occured #2666 has started again. And downloading the msix from https://cdn.winget.microsoft.com/cache/source.msix returns a 0kb file.

Steps to reproduce

Run winget source update

Expected behavior

winget should have updated successfully

Actual behavior

Winget will attempt to update the source when running any command but will return the message Failed in attempting to update the source: winget. If you run winget source update it will respond with Cancelled

Environment

Windows Package Manager v1.3.2691
Windows: Windows.Desktop v10.0.22621.819
System Architecture: X64
Package: Microsoft.DesktopAppInstaller v1.18.2691.0

The text was updated successfully, but these errors were encountered:

mdanish-kh · 2022-11-22T22:35:14Z

FYI: microsoft/winget-pkgs#86160 (comment)

Vzz1c · 2022-11-23T09:58:19Z

+1

Psi085 · 2022-11-23T14:42:22Z

+1

simkoG · 2022-11-23T16:00:36Z

+1

M43R · 2022-11-24T12:42:14Z

winget source list
-----------------------------------------------------
msstore https://storeedgefd.dsx.mp.microsoft.com/v9.0
winget  https://cdn.winget.microsoft.com/cache

ResourceNotFoundThe specified resource does not exist.

vtvy · 2022-11-24T17:53:12Z

I have the same problem "Failed in attempting to update the source: winget". After following this solution, Winget will work perfectly.

mgthantzin · 2022-11-25T06:51:46Z

Another workaround is to use a VPN. Use a location far from your current continent.

simkoG · 2022-11-25T07:55:57Z

My bad, that was my UK VPN previously, sorry. It is working if I connect to a VPN server far away from me.

VincentJoshuaET · 2022-11-26T07:46:21Z

When will this be fixed?

denelon · 2022-11-26T19:20:20Z

We believe we've found the root cause. We've triggered another purge to reset the cache endpoints and we're working with our CDN provider to ensure all regions are updated. We've got a production change schedule for Monday to hopefully address this permanently.

denelon · 2022-11-28T19:25:44Z

We've gotten an update on the CDN endpoints. There were two endpoints that needed to be manually purged. We've been told these have been completed.

MarcoPeraza · 2022-11-28T22:49:33Z

Still not working for me.

PS C:\Users\User\code\BoxSetup> winget source reset --force
Resetting all sources...Done
PS C:\Users\User\code\BoxSetup> winget source update
Updating all sources...
Updating source: msstore...
Done
Updating source: winget...
Cancelled

Downloading https://cdn.winget.microsoft.com/cache/source.msix through a web browser produces a 0KB file.

MarcoPeraza · 2022-11-29T17:09:55Z

Working now

denelon · 2022-11-29T17:16:12Z

All, we've had our CDN provider flush a couple of the pesky nodes that weren't getting updated, and we're purging after each publish run. This should have been resolved around 5:00 PM Pacific yesterday.

simkoG · 2022-11-29T17:47:45Z

@denelon It is fixed for me (Hungary, EU).

efie45 · 2022-12-19T16:34:58Z

Still not working for me. Getting 0 byte file from the CDN in US.

cdn.winget.microsoft.com resolves to 152.195.19.97 for me.

R-Adrian · 2022-12-28T19:45:26Z

i posted the issue #2807 referenced above and it was pointed out to me that there is already a pinned issue ... so i experimented a bit more with the info i found in here.

(GitHub places the pinned issues OUTSIDE the list of current issues - i did not even notice it because it is placed in such a way that is confused with an advertising banner... and i developed a habit to ignore anything placed in such locations... thus the pinned issues area is usually ignored)

i tried downloading the https://cdn.winget.microsoft.com/cache/source.msix file with:

internet explorer: result = error INET_E_DOWNLOAD_FAILURE
MS Edge Chromium, result = downloads ok
Google Chrome, result = downloads ok
Mozilla Firefox, result = downloads ok
winget upgrade --verbose-logs command line (winget v1.4.3531), result = Failed in attempting to update the source: winget An error occurred in the secure channel support

2022-12-28 21:26:41.640 [REPO] Source past auto update time [5 mins]; it has been at least 27870926 mins
2022-12-28 21:26:42.095 [FAIL] WindowsPackageManager.dll!00007FF89ECBBF22: ReturnHr(1) tid(3c90) 80072F7D     Msg:[winrt::hresult_error: An error occurred in the secure channel support] 

2022-12-28 21:26:42.096 [FAIL] WindowsPackageManager.dll!00007FF89EB43CF1: LogHr(2) tid(3c90) 80072F7D 
2022-12-28 21:26:42.096 [FAIL] D:\a\_work\1\s\external\pkg\src\AppInstallerRepositoryCore\RepositorySource.cpp(53)\WindowsPackageManager.dll!00007FF89ECC7451: (caller: 00007FF89EB906FE) LogHr(3) tid(3c90) 80072F7D     Msg:[winrt::hresult_error: An error occurred in the secure channel support] 

2022-12-28 21:26:42.096 [REPO] Source add/update failed, waiting a bit and retrying: winget
2022-12-28 21:29:45.049 [FAIL] WindowsPackageManager.dll!00007FF89ECBBF22: ReturnHr(2) tid(3c90) 80072F7D     Msg:[winrt::hresult_error: An error occurred in the secure channel support] 

2022-12-28 21:29:45.049 [FAIL] WindowsPackageManager.dll!00007FF89EB43CF1: LogHr(5) tid(3c90) 80072F7D 
2022-12-28 21:29:45.050 [FAIL] D:\a\_work\1\s\external\pkg\src\AppInstallerRepositoryCore\RepositorySource.cpp(540)\WindowsPackageManager.dll!00007FF89ECC6A33: (caller: 00007FF89EA88B3D) LogHr(6) tid(3c90) 80072F7D     Msg:[winrt::hresult_error: An error occurred in the secure channel support] 

2022-12-28 21:29:45.050 [REPO] Failed to update source: winget

ping cdn.winget.microsoft.com

Pinging sni1gl.wpc.nucdn.net [152.199.21.175] with 32 bytes of data:
Reply from 152.199.21.175: bytes=32 time=6ms TTL=58
Reply from 152.199.21.175: bytes=32 time=6ms TTL=58
Reply from 152.199.21.175: bytes=32 time=6ms TTL=58
Reply from 152.199.21.175: bytes=32 time=5ms TTL=58

Ping statistics for 152.199.21.175:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 5ms, Maximum = 6ms, Average = 5ms

denelon · 2022-12-28T20:16:07Z

@R-Adrian thanks! That's some helpful information. I wonder if something with the user agent might be at play. I'll share this with the engineering team.

R-Adrian · 2022-12-28T21:23:02Z

i think i nailed it... the error message is almost correct.... it is a SChannel problem but is actually a problem with Windows (10? 22H2?) ... the operating system is not actually TLS 1.3 compliant and has to use TLS 1.2
The download via the modern browsers works because the browser handles TLS 1.3 internally... but internet explorer and winget rely on the OS to establish the TLS connection... which is not working.

I managed to reproduce the issue on another computer like this:

a) defined the registry key

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.3\Client]
"DisabledByDefault"=dword:00000000
"Enabled"=dword:00000001

b) rebooted the system

c) and then managed to cause the schannel winget error by simply enabling the TLS 1.3 checkbox in the Advanced Internet Options ( inetcpl.cpl ) while at the same time having the TLS 1.3 SChannel registry setting above.

d) when the TLS 1.3 checkbox is disabled in inetcpl.cpl, even if the SChannel registry settings for TLS 1.3 client remain enabled the checkbox overrides the registry setting, and winget (and internet explorer) works again... but it uses TLS 1.2 connections

R-Adrian · 2022-12-28T22:01:23Z

P.S. found further support material, TLS 1.3 seems is not intended to be supported in Windows 10

https://learn.microsoft.com/en-us/windows/win32/secauthn/protocols-in-tls-ssl--schannel-ssp-

R-Adrian · 2022-12-29T06:57:33Z

P.S. 2.. after sleeping on it.. i think this TLS 1.3 bug with Microsoft Azure connections might also be the reason for Windows Update and the Microsoft Store app itself randomly failing instantly when you open them during the past few months ... and then they start to mysteriously work again if you leave them alone for a few minutes without doing anything to them.
(i already checked NTP clock sync, that is not the issue)

On some computers at the office i configured them to use a proxy for updates and my squid proxy logs are full of failed tunnel connections to these endpoints even if i have allowed them in the proxy config and they work via the modern browsers:
slscr.update.microsoft.com:443
sls.update.microsoft.com:443
not surprisingly, i have also found the cdn.winget.microsoft.com:443 endpoint in the squid proxy logs there.

BuzzwordChief · 2022-12-30T15:15:03Z

I get the same error on Windows 11 Home 22H2.

C:\Users\DaveM>winget upgrade --verbose-logs
Failed in attempting to update the source: winget

cdn.winget.microsoft.com resolves to 152.199.21.175 for me.
winget version v1.3.2691

R-Adrian · 2022-12-30T15:34:58Z

I get the same error on Windows 11 Home 22H2.

can you please try temporarily disabling TLS 1.3 from the Internet Options -> Advanced control panel and trying again with that? (start->run-> inetcpl.cpl) Make sure TLS 1.2 remains enabled there.

(i do not have any Windows 11 system to test with)

BuzzwordChief · 2022-12-30T15:40:13Z

Does not seem to help:

Do I have to restart or anything?

R-Adrian · 2022-12-30T15:48:54Z

Do I have to restart or anything?

Not sure how Win11 behaves here, i only had to restart when i enabled TLS 1.3 in registry for SChannel, not when using the control panel on Win10 22H2.

Maybe try completely disabling TLS 1.3 via the registry options for SChannel and then rebooting?

R-Adrian · 2023-01-01T09:33:44Z

Happy New Year, and i tried testing another thing...

on Win 10, toggling TLS 1.3 on/off in inetcpl.cpl still causes winget errors when TLS 1.3 is on
according to Qualys SSL Labs https://cdn.winget.microsoft.com appears to be serving different content depending on the connection being on TLS 1.2 or TLS 1.3 and also if SNI is negotiated or not negotiated.
https://www.ssllabs.com/ssltest/analyze.html?d=cdn.winget.microsoft.com

i cannot make SSLLabs analyze the European side of the CDN - SSLLabs does not let me select 152.199.21.175 and it auto-selects to the North America one, 152.195.19.97.. so i added that to my hosts file and tested again with that.

result: Same error happens with winget and 152.195.19.97 present in the hosts file (on Windows 10 22H2)

fluentmoheshwar · 2023-01-12T17:13:26Z

+1 Solutions didn't work for me.

denelon · 2023-01-26T21:35:52Z

We've been working with our CDN provider to squash the nasty zero-byte file being cached and served. Is anyone still getting this problem? If you are, can you share the IP address you're hitting?

R-Adrian · 2023-01-26T21:47:31Z

TLS 1.3 error is still there - the first few tries are with TLS 1.3 enabled then it works after disabling it again.

C:\WINDOWS\system32>ping cdn.winget.microsoft.com

Pinging sni1gl.wpc.nucdn.net [2606:2800:233:1cb7:261b:1f9c:2074:3c] with 32 bytes of data:
Reply from 2606:2800:233:1cb7:261b:1f9c:2074:3c: time=11ms
Reply from 2606:2800:233:1cb7:261b:1f9c:2074:3c: time=11ms
Reply from 2606:2800:233:1cb7:261b:1f9c:2074:3c: time=11ms
Reply from 2606:2800:233:1cb7:261b:1f9c:2074:3c: time=11ms

Ping statistics for 2606:2800:233:1cb7:261b:1f9c:2074:3c:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 11ms, Maximum = 11ms, Average = 11ms

BuzzwordChief · 2023-01-27T08:05:10Z

Yes, same error...

sebaFlame · 2023-01-28T14:30:58Z

+1 EU (Belgium)

PS C:\Users\C_B-A> winget upgrade
Failed in attempting to update the source: winget
PS C:\Users\C_B-A> ping cdn.winget.microsoft.com

Pinging sni1gl.wpc.nucdn.net [152.199.21.175] with 32 bytes of data:
Reply from 152.199.21.175: bytes=32 time=17ms TTL=54
Reply from 152.199.21.175: bytes=32 time=32ms TTL=54
Reply from 152.199.21.175: bytes=32 time=14ms TTL=54
Reply from 152.199.21.175: bytes=32 time=14ms TTL=54

Ping statistics for 152.199.21.175:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 14ms, Maximum = 32ms, Average = 19ms

Tarrowren · 2023-02-02T02:29:49Z

+1

C:\Users\tarro>ping cdn.winget.microsoft.com

Pinging sni1gl.wpc.nucdn.net [152.199.39.108] with 32 bytes of data:
Reply from 152.199.39.108: bytes=32 time=377ms TTL=50
Reply from 152.199.39.108: bytes=32 time=362ms TTL=50
Reply from 152.199.39.108: bytes=32 time=378ms TTL=50
Reply from 152.199.39.108: bytes=32 time=372ms TTL=50

Ping statistics for 152.199.39.108:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 362ms, Maximum = 378ms, Average = 372ms

Hobart · 2023-02-07T17:19:02Z

C:\>winget upgrade
Failed in attempting to update the source: winget
Failed when searching source; results will not be included: winget
No installed package found matching input criteria.

C:\>winget source list
Name    Argument
-----------------------------------------------------
msstore https://storeedgefd.dsx.mp.microsoft.com/v9.0
winget  https://cdn.winget.microsoft.com/cache

C:\>ping cdn.winget.microsoft.com

Pinging sni1gl.wpc.nucdn.net [152.195.19.97] with 32 bytes of data:
Reply from 152.195.19.97: bytes=32 time=5ms TTL=58
Reply from 152.195.19.97: bytes=32 time=5ms TTL=58
Reply from 152.195.19.97: bytes=32 time=5ms TTL=58
Reply from 152.195.19.97: bytes=32 time=5ms TTL=58

Ping statistics for 152.195.19.97:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 5ms, Maximum = 5ms, Average = 5ms

skieast · 2023-02-12T00:55:29Z

Same:
C:\Windows\System32>ping cdn.winget.microsoft.com

Pinging sni1gl.wpc.nucdn.net [2606:2800:11f:1cb7:261b:1f9c:2074:3c] with 32 bytes of data:
Reply from 2606:2800:11f:1cb7:261b:1f9c:2074:3c: time=16ms
Reply from 2606:2800:11f:1cb7:261b:1f9c:2074:3c: time=17ms
Reply from 2606:2800:11f:1cb7:261b:1f9c:2074:3c: time=21ms
Reply from 2606:2800:11f:1cb7:261b:1f9c:2074:3c: time=16ms

Ping statistics for 2606:2800:11f:1cb7:261b:1f9c:2074:3c:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 16ms, Maximum = 21ms, Average = 17ms

An extract from the upgrade log. Seems to be certificate expired or invalid. I tried this both with TLS 1.3 enabled and disabled.

2023-02-11 17:21:45.205 [FAIL] WindowsPackageManager.dll!00007FFDB296D8D1: LogHr(2) tid(4cfc) 80072F05 
2023-02-11 17:21:45.205 [FAIL] D:\a\_work\1\s\external\pkg\src\AppInstallerRepositoryCore\RepositorySource.cpp(53)\WindowsPackageManager.dll!00007FFDB2AF1BA1: (caller: 00007FFDB29BA2CE) LogHr(3) tid(4cfc) 80072F05     Msg:[winrt::hresult_error: The date in the certificate is invalid or has expired] 

2023-02-11 17:21:45.205 [REPO] Source add/update failed, waiting a bit and retrying: winget
2023-02-11 17:21:47.285 [FAIL] WindowsPackageManager.dll!00007FFDB2AE6472: ReturnHr(2) tid(4cfc) 80072F05     Msg:[winrt::hresult_error: The date in the certificate is invalid or has expired] 

2023-02-11 17:21:47.285 [FAIL] WindowsPackageManager.dll!00007FFDB296D8D1: LogHr(5) tid(4cfc) 80072F05 
2023-02-11 17:21:47.285 [FAIL] D:\a\_work\1\s\external\pkg\src\AppInstallerRepositoryCore\RepositorySource.cpp(540)\WindowsPackageManager.dll!00007FFDB2AF1183: (caller: 00007FFDB28A937D) LogHr(6) tid(4cfc) 80072F05     Msg:[winrt::hresult_error: The date in the certificate is invalid or has expired] 

2023-02-11 17:21:47.285 [REPO] Failed to update source: winget

KK-Designs · 2023-02-12T01:05:14Z

I'm still getting the error:

mohitcv · 2023-02-12T04:23:46Z

Looks like it might be related to an expired certificate. Get the following error when navigating to https://cdn.winget.microsoft.com/cache/source.msix

R-Adrian · 2023-02-12T04:36:32Z

the expired certificate is actually a slightly different problem than the primary issue of this thread. Still related to the general CDN infrastructure problems tho.

denelon · 2023-02-12T06:37:26Z

We're working to renew the certificate.

InternetOpenUrl() failed - 0x80072f05 : unknown error #2956

fluentmoheshwar · 2023-02-14T10:14:49Z

Fixed after uninstalling internet download manager.

R-Adrian · 2023-02-14T11:42:14Z

Fixed after uninstalling internet download manager.

i do not have internet download manager installed (or ever had it) and can still reliably trigger the CDN bug on multiple WIndows 10 computers that i use (both at home and at the office) by simply force-enabling TLS 1.3 support (which is "experimental" in Windows 10).

Maybe Internet Download Manager was messing with the system TLS support in SChannel and uninstalling it just reconfigured SChannel back to the defaults?

fluentmoheshwar · 2023-02-14T15:55:05Z

Fixed after uninstalling internet download manager.

i do not have internet download manager installed (or ever had it) and can still reliably trigger the CDN bug on multiple WIndows 10 computers that i use (both at home and at the office) by simply force-enabling TLS 1.3 support (which is "experimental" in Windows 10).

Maybe Internet Download Manager was messing with the system TLS support in SChannel and uninstalling it just reconfigured SChannel back to the defaults?

Probably, Disabling TLS 1.3 didn't help

bepstein111 · 2023-02-20T00:31:21Z

This issue just began a few days ago for me, about two days after reinstalling (but kept my personal files). Nothing suggested so far has helped =[

fluentmoheshwar · 2023-02-21T07:33:46Z

This issue just began a few days ago for me, about two days after reinstalling (but kept my personal files). Nothing suggested so far has helped =[

Try installing Cloudflare WARP it solved the issue for me. 😄

fluentmoheshwar · 2023-02-21T07:36:24Z

Update: It happened again but installing Cloudflare WARP solved the issue, I think the problem isn't with winget CDN, but my ISP (Grameenphone), because I'm having same issue with twitter and GitHub (sometimes).

bepstein111 · 2023-02-21T15:24:45Z

Update:
My problem was solved. It was caused by installing Private Internet Access which automatically turned on it's "VPN Kill switch" not allowing any traffic to flow via windows built-in networking. Browsers worked fine though which is why I was confused. I turned off the kill switch and everything appears to work fine now.

denelon · 2023-02-22T00:17:45Z

Hey all, we've addressed the zero-byte file issue in the CDN.

Publishing has been working consistently since then, so I'm going to go ahead and close this issue.

We're aware there are users who are still having issues updating the source in some scenarios, so we'll deal with those on a more case by case basis.

ghost added the Needs-Triage Issue need to be triaged label Nov 22, 2022

denelon pinned this issue Nov 23, 2022

denelon changed the title ~~winget is down again (failed to update source)~~ Publishing is down again (failed to update source) Nov 23, 2022

denelon added Issue-Bug It either shouldn't be doing this or needs an investigation. and removed Needs-Triage Issue need to be triaged labels Nov 23, 2022

denelon mentioned this issue Nov 23, 2022

Can't upgrade or install packages #2696

Closed

mdanish-kh mentioned this issue Nov 25, 2022

[Package Issue]: VideoLAN.VLC.Nightly microsoft/winget-pkgs#89632

Closed

2 tasks

mdanish-kh mentioned this issue Nov 28, 2022

Installer hash does not match; to override this check use --force (google chrome) microsoft/winget-pkgs#89874

Closed

denelon mentioned this issue Dec 28, 2022

Failed in attempting to update the source: winget #2807

Closed

R-Adrian mentioned this issue Dec 30, 2022

2316632087 (0x8A150017) Error when using winget install #2808

Closed

denelon added this to the v1.5-Client milestone Feb 14, 2023

denelon closed this as completed Feb 22, 2023

denelon unpinned this issue Feb 22, 2023

Publishing is down again (failed to update source) #2718

Publishing is down again (failed to update source) #2718

Comments

infinitepower18 commented Nov 22, 2022

Brief description of your issue

Steps to reproduce

Expected behavior

Actual behavior

Environment

mdanish-kh commented Nov 22, 2022

Vzz1c commented Nov 23, 2022

Psi085 commented Nov 23, 2022

simkoG commented Nov 23, 2022

M43R commented Nov 24, 2022

vtvy commented Nov 24, 2022

mgthantzin commented Nov 25, 2022

simkoG commented Nov 25, 2022 • edited Loading

VincentJoshuaET commented Nov 26, 2022

denelon commented Nov 26, 2022

denelon commented Nov 28, 2022

MarcoPeraza commented Nov 28, 2022 • edited Loading

MarcoPeraza commented Nov 29, 2022

denelon commented Nov 29, 2022

simkoG commented Nov 29, 2022

efie45 commented Dec 19, 2022

R-Adrian commented Dec 28, 2022

denelon commented Dec 28, 2022 • edited Loading

R-Adrian commented Dec 28, 2022

R-Adrian commented Dec 28, 2022

R-Adrian commented Dec 29, 2022 • edited Loading

BuzzwordChief commented Dec 30, 2022

R-Adrian commented Dec 30, 2022 • edited Loading

BuzzwordChief commented Dec 30, 2022

R-Adrian commented Dec 30, 2022

R-Adrian commented Jan 1, 2023 • edited Loading

fluentmoheshwar commented Jan 12, 2023

denelon commented Jan 26, 2023

R-Adrian commented Jan 26, 2023

BuzzwordChief commented Jan 27, 2023

sebaFlame commented Jan 28, 2023 • edited Loading

Tarrowren commented Feb 2, 2023

Hobart commented Feb 7, 2023

skieast commented Feb 12, 2023 • edited Loading

KK-Designs commented Feb 12, 2023

mohitcv commented Feb 12, 2023

R-Adrian commented Feb 12, 2023

denelon commented Feb 12, 2023

fluentmoheshwar commented Feb 14, 2023

R-Adrian commented Feb 14, 2023

fluentmoheshwar commented Feb 14, 2023

bepstein111 commented Feb 20, 2023

fluentmoheshwar commented Feb 21, 2023

fluentmoheshwar commented Feb 21, 2023

bepstein111 commented Feb 21, 2023

denelon commented Feb 22, 2023

simkoG commented Nov 25, 2022 •

edited

Loading

MarcoPeraza commented Nov 28, 2022 •

edited

Loading

denelon commented Dec 28, 2022 •

edited

Loading

R-Adrian commented Dec 29, 2022 •

edited

Loading

R-Adrian commented Dec 30, 2022 •

edited

Loading

R-Adrian commented Jan 1, 2023 •

edited

Loading

sebaFlame commented Jan 28, 2023 •

edited

Loading

skieast commented Feb 12, 2023 •

edited

Loading