Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Installer fails at curl due to self-signed certificate #19

Closed
BarronRauEA opened this issue Dec 2, 2022 · 21 comments
Closed

Installer fails at curl due to self-signed certificate #19

BarronRauEA opened this issue Dec 2, 2022 · 21 comments
Assignees
Milestone

Comments

@BarronRauEA
Copy link

I am installing on a machine that is behind a proxy and uses a company certificate. The failure is:

windows_install.bc: [INFO] Downloading https://github.com/msys2/msys2-installer/releases/download/2022-01-28/msys2-base-x86_64-20220128.sfx.exe
windows_install.bc: [INFO] Running command: C:\Users\BARRON~2\AppData\Local\Temp\7z17ECACAC\sg\staging-curl\windows_x86_64\bin\curl.exe -L -o C:\Users\BARRON~2\AppData\Local\Temp\path_eval_77c67b\curlo01eb9f https://github.com/msys2/msys2-installer/releases/download/2022-01-28/msys2-base-x86_64-20220128.sfx.exe

% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0

curl: (60) SSL certificate problem: self-signed certificate in certificate chain
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not establish a secure connection to it. To learn more about this situation and how to fix it, please visit the web page mentioned above.

FATAL [e6435b12]. A transient failure occurred.

Root cause: The conformant command had exit code 60: C:\Users\BARRON~2\AppData\Local\Temp\7z17ECACAC\sg\staging-curl\windows_x86_64\bin\curl.exe -L -o C:\Users\BARRON~2\AppData\Local\Temp\path_eval_77c67b\curlo01eb9f https://github.com/msys2/msys2-installer/releases/download/2022-01-28/msys2-base-x86_64-20220128.sfx.exe
A transient failure occurred.
dkml-package.bc: [ERROR] FATAL [5f927a8b].
A transient failure occurred.
Root cause: The command had exit code 20:
C:\Users\BARRON~2\AppData\Local\Temp\7z17ECACAC\bin\dkml-install-user-runner.exe
install-user-network-unixutils
--verbosity=info --color=always
--prefix C:\Users\BarronRauEA\AppData\Local\Programs\DiskuvOCaml
--staging-files C:\Users\BARRON~2\AppData\Local\Temp\7z17ECACAC\sg
A transient failure occurred.

It appears that the installer uses a copy of curl from a temp file at C:\Users\BarronRauEA\AppData\Local\Temp\7z6912243C\sg\staging-curl\windows_x86_64\bin. I tried adding my cert to the end of the curl-ca-bundle.crt file and rerunning the installer but had the same problem. I also tried removing the staged cert file and curl.exe and replaced them with a shortcut to my local curl installation, with the same failure.

Is it possible for the installer to use my existing instance of curl instead of the staged one or is there a way for the staged instance to use my certificate?

@jonahbeckford jonahbeckford self-assigned this Dec 2, 2022
@jonahbeckford
Copy link
Contributor

The only good solution is to generate a more offline setup that bundles as much content as possible into the installer.

I'll prioritize this issue. I can't bundle everything, so I may need some back and forth with you in this ticket to get an offline setup that works.

  • Bundle msys2-base-x86_64-20220128.sfx.exe into the installer so it doesn't need to be downloaded
  • Cut a prerelease for the issue creator
  • Check with issue creator to see if the new uninstaller + installer fixes the issue

@BarronRauEA
Copy link
Author

I hate to inconvenience you, but I'd love to muddle through testing to help get it working. Let me know what I can do and when.

@jonahbeckford
Copy link
Contributor

jonahbeckford commented Dec 2, 2022

If you are willing to get your hands dirty, can you:

  1. Run the uninstaller (https://github.com/diskuv/dkml-installer-ocaml/releases/download/v1.0.1/uninstall-diskuv-ocaml-windows_x86_64-1.0.1.exe) to remove any partial files.
  2. Install 7zip if you don't have it yet
  3. Uncompress the DKML setup-*.exe file.
  4. There should be curl.exe and curl-ca-bundle.crt in sg\staging-curl\windows_x86_64\bin\ one or two places in the directory tree ... replace them both.
  5. I'll update this ticket shortly with how to run the installer directly from the uncompressed directory tree Run: setup.exe -v -v to run with verbose logging.
  6. Update the ticket with your progress

@BarronRauEA
Copy link
Author

New error code from curl now:
curl: (35) schannel: next InitializeSecurityContext failed: Unknown error (0x80092012) - The revocation function was unable to check revocation for the certificate.

I tried with both the original curl-ca-bundle.crt with my cert appended and with my cert renamed to curl-ca-bundle.crt. Copying my curl in is necessary to get the new error code.

@jonahbeckford
Copy link
Contributor

Can you try running your system (company) curl directly and reply with the output?

curl -v -L -o msys2-base.exe https://github.com/msys2/msys2-installer/releases/download/2022-01-28/msys2-base-x86_64-20220128.sfx.exe

@BarronRauEA
Copy link
Author

Sorry, looks like that is my problem. I'll check with someone here and let you know if I still have a problem. Do you want me to close this issue now or wait until my install is successful?

C:\Users\BarronRauEA>curl -v -L -o msys2-base.exe https://github.com/msys2/msys2-installer/releases/download/2022-01-28/msys2-base-x86_64-20220128.sfx.exe
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 140.82.112.4:443...

  • Connected to github.com (140.82.112.4) port 443 (#0)
  • schannel: disabled automatic use of client certificate
  • ALPN: offers http/1.1
  • ALPN: server accepted http/1.1

GET /msys2/msys2-installer/releases/download/2022-01-28/msys2-base-x86_64-20220128.sfx.exe HTTP/1.1
Host: github.com
User-Agent: curl/7.83.1
Accept: /

@jonahbeckford
Copy link
Contributor

Let's keep it open. Can you try to add -k to the curl command and see if that works?

(DKML uses sha256 checksum validation after downloading any content, so the insecure mode (-k) of curl can be safe to use if the checksum validation can be guaranteed.)

@BarronRauEA
Copy link
Author

BarronRauEA commented Dec 2, 2022 via email

jonahbeckford added a commit to diskuv/dkml-component-unixutils that referenced this issue Dec 3, 2022
@jonahbeckford
Copy link
Contributor

No, it is not possible to pass the curl flag down. But am making a change to the installer to embed the msys2-base-x86_64-20220128.sfx.exe inside the installer so curl is not needed.

@jonahbeckford jonahbeckford added this to the v1.0.2 milestone Dec 3, 2022
@jonahbeckford jonahbeckford reopened this Dec 3, 2022
@BarronRauEA
Copy link
Author

Good morning Jonah,

Thanks for all your work on this. When I go to https://github.com/diskuv/dkml-installer-ocaml/releases and click on https://github.com/diskuv/dkml-installer-ocaml/releases/download/v1.0.2/setup-diskuv-ocaml-windows_x86_64-1.0.2.exe I get a 404 error. Let me know when you want me to try it again or if there is some other way I should be downloading the installer.

@BarronRauEA
Copy link
Author

BarronRauEA commented Dec 3, 2022

Well, we bypassed the first problem. Looks like the certificate is causing another problem. I had my MSP looking at it end of the day Friday and expect to hear more from him Monday. Output now is (I edited out the repetitive lines):

2022-12-03T18:41:11.974Z [MSYS2] pacman -Syu --noconfirm
:: Synchronizing package databases...
error: failed retrieving file 'mingw32.db' from mirror.msys2.org : SSL certificate problem: self signed certificate in certificate chain
error: failed retrieving file 'mingw64.db' from mirror.msys2.org : SSL certificate problem: self signed certificate in certificate chain
error: failed retrieving file 'ucrt64.db' from mirror.msys2.org : SSL certificate problem: self signed certificate in certificate chain

warning: too many errors from mirror.msys2.org, skipping for the remainder of this transaction

error: failed retrieving file 'clang64.db' from mirror.msys2.org : SSL certificate problem: self signed certificate in certificate chain

error: failed retrieving file 'msys.db' from mirror.msys2.org : SSL certificate problem: self signed certificate in certificate chain

error: failed to synchronize all databases (download library error)
Setup did not complete because an error occurred.
MSYS2 command failed! Exited with 1. Command was: env MSYSTEM=CLANG64 MSYSTEM_PREFIX=/clang64 HOME=/home/BarronRauEA PATH=/clang64/bin:/usr/bin:/bin env PATH=/c/Program Files/Git/cmd:/clang64 /bin:/usr/bin:/bin DKML_TMP_PARENTDIR=/c/Users/BARRON~2/AppData/Local/Temp/PA1A29~1/0 pacman -Syu --noconfirm

at Invoke-MSYS2Command, C:\Users\BarronRauEA\AppData\Local\Temp\7zAF6DF2B4\sg\network-ocamlcompiler\windows_x86_64\dkmldir\vendor\drd\src\windows\UnixInvokers\UnixInvokers.psm1: line 165 at Invoke-MSYS2CommandWithProgress, C:\Users\BarronRauEA\AppData\Local\Temp\7zAF6DF2B4\sg\network-ocamlcompiler\windows_x86_64\setup-userprofile.ps1: line 833 at <ScriptBlock>, C:\Users\BarronRauEA\AppData\Local\Temp\7zAF6DF2B4\sg\network-ocamlcompiler\windows_x86_64\setup-userprofile.ps1: line 1037 at <ScriptBlock>, <No file>: line 1 at <ScriptBlock>, <No file>: line 1

[Advanced] MSYS2 commands can be run with: C:\Users\BARRON~2\AppData\Local\Programs\DISKUV~1\tools\MSYS2\msys2_shell.cmd

Bug Reports can be filed at https://github.com/diskuv/dkml-installer-ocaml/issues
Please copy the error message and attach the log file available at C:\Users\BARRON~2\AppData\Local\Programs\DISKUV~1\setup-userprofile.full.log

FATAL [a0d16230]. A transient failure occurred.

Root cause: The conformant command had exit code 1: C:\Users\BARRON~2\AppData\Local\Temp\7zAF6DF2B4\sg\network-ocamlcompiler\windows_x86_64\setup-userprofile.bat
-AllowRunAsAdmin
-InstallationPrefix
C:\Users\BARRON~2\AppData\Local\Programs\DISKUV~1
-MSYS2Dir
C:\Users\BARRON~2\AppData\Local\Programs\DISKUV~1\tools\MSYS2
-OpamExe C:\Users\BARRON~2\AppData\Local\Programs\DISKUV~1\bin\OPAM-R~1.EXE
-DkmlPath C:\Users\BARRON~2\AppData\Local\Temp\7ZAF6D~1\sg\NETWOR~1\WINDOW~1\dkmldir
-GlobalCompileDir C:\Users\BARRON~2\AppData\Local\Temp\7ZAF6D~1\sg\STAGIN~1\generic
-NoDeploymentSlot -DkmlHostAbi windows_x86_64 -TempParentPath C:\Users\BARRON~2\AppData\Local\Temp\PA1A29~1 -SkipProgress

A transient failure occurred.

dkml-package.bc: [ERROR] FATAL [5f927a8b]. A transient failure occurred.

Root cause: The command had exit code 20: C:\Users\BARRON~2\AppData\Local\Temp\7zAF6DF2B4\bin\dkml-install-user-runner.exe
install-user-network-ocamlcompiler --verbosity=info --color=always --prefix
C:\Users\BarronRauEA\AppData\Local\Programs\DiskuvOCaml --staging-files
C:\Users\BARRON~2\AppData\Local\Temp\7zAF6DF2B4\sg

A transient failure occurred.

Press "y" and ENTER to exit the installer.

@jonahbeckford
Copy link
Contributor

Sigh. You really need access to the MSYS2 package manager. There is a flow documented at https://www.msys2.org/docs/faq/ for accepting company TLS certificates, but there is no way to do that today in the middle of the installation process.

The modifications to support that will have to wait until mid-next week at the earliest, and then I want to make sure everyone in your company who wants to use it has simple steps. Can you do a Zoom chat (or some alternative) on Tuesday?

@BarronRauEA
Copy link
Author

BarronRauEA commented Dec 3, 2022

Perhaps a bit of explanation -

I'm an accountant and program some for work and some for hobby. This will be mostly hobby for now and I'm the only one who will need it installed. The only internet connection I have is through my business. We're part of a conservative Mennonite church that requires we only access the internet through a church-approved filter. The filter uses the self-signed certificate to man-in-the-middle the traffic for filtering purposes. I feel bad having you put so much time into this, but perhaps you want to work the bugs out of the installer for others.

I'd be happy to do a screen share or some other connection if you like. We don't use the video capabilities on zoom so let me know what works best, Zoom, Teams, TeamViewer, etc are all options from my end.

Would it work to add the update-ca-trust command to msys command line before synchronizing the package databases? I could copy the certificate into the installer \sg\offline-unixutils\windows_x86_64\bin\msys2-base.sfx.exe\msys64\etc\pki\ca-trust\source\anchors\ before running it. I only half know what I am doing, so don't wonder if you're missing something if the idea is crazy.

Enjoy your Lord's day and thanks for your patience.

@jonahbeckford
Copy link
Contributor

Oh no problem. Like you mentioned I think the proxy environment you have is fairly normal. I'd like to get it working for you and others.

Can you schedule some time with https://calendly.com/diskuv/30-minute-dkml-installation? We won't be able to solve the problem, but I want to test that copying the certificates actually works. We'll test it outside the installer. Then I'll know that a change to the installer will work.

Have a blessed day as well!

@BarronRauEA
Copy link
Author

On for Monday the 12th. I'm out of the office Wed. through Fri. next week (7th - 12th).

@jonahbeckford
Copy link
Contributor

Meeting Summary:

  • Can replicate the issue in a standalone MSYS2 shell (independent of the DKML installer): pacman -S vim fails with self-signed certificate in certificate chain
  • Copying a .pem or .cer certificate over (in this situation, a .der converted with openssl) per https://www.msys2.org/docs/faq/ does let pacman -S vim succeed
  • Possible options include:
    1. Completely offline msys2 packages
    2. Copy .pem or .cer from a pre-specified location into MSYS2 during install
    3. Work with a pre-existing MSYS2

Internal notes: Oddly enough the third option (pre-existing MSYS2) already is possible ... that is how DKML is tested on GitHub and GitLab which come bundled with MSYS2. But there is not a specific option in setup.exe to pass the MSYS2 option down to the MSYS2 component (dkml-component-unixutils), nor is there a parameters file that can be read by the MSYS2 component. But I think there is an MSYS2 override option in setup-userprofile.ps1 (will check).

@jonahbeckford
Copy link
Contributor

With that test we did, I think you should be able to complete most of the installation right now.

Using the links you had earlier in this thread,

  1. You want to keep the already-installed files that are in
    C:\Users\BARRON~2\AppData\Local\Programs\DISKUV~1. If that directory
    is empty or non-existent, rerun the uninstaller and then the installer
    so that it fails at the "self-signed certificate" part.
  2. Decompress the installer (using 7zip) into a directory.

Then run the following in PowerShell:

# Whatever directory you use to decompress the installer
cd C:\Users\beckf\Downloads\unsigned-diskuv-ocaml-windows_x86_64-i-1.0.1

if (!(test-path "$env:TEMP\dkml-issue-19")) { new-item -itemtype directory "$env:TEMP\dkml-issue-19" }

# Change the MSYS2Dir to your customized installation!
sg\network-ocamlcompiler\windows_x86_64\setup-userprofile.bat `
    -AllowRunAsAdmin `
    -InstallationPrefix `
    C:\Users\BARRON~2\AppData\Local\Programs\DISKUV~1 `
    -MSYS2Dir `
    C:\Users\BARRON~2\AppData\Local\Programs\DISKUV~1\tools\MSYS2 `
    -OpamExe C:\Users\BARRON~2\AppData\Local\Programs\DISKUV~1\bin\OPAM-R~1.EXE `
    -DkmlPath "$PWD\sg\NETWOR~1\WINDOW~1\dkmldir" `
    -GlobalCompileDir "$PWD\sg\STAGIN~1\generic" `
    -NoDeploymentSlot -DkmlHostAbi windows_x86_64 `
    -TempParentPath "$env:TEMP\dkml-issue-19" `
    -SkipProgress

Please tell me if anything else errors.

@BarronRauEA
Copy link
Author

BarronRauEA commented Dec 12, 2022 via email

@jonahbeckford
Copy link
Contributor

jonahbeckford commented Dec 21, 2022

Spoke too soon. Please do not use it yet. I will update this when I have a better pre-release to give you!

I have pre-release "17" that you can use.

Installer: https://github.com/diskuv/dkml-installer-ocaml/releases/download/v1.1.0-prerel17/unsigned-diskuv-ocaml-windows_x86_64-i-1.0.1.exe
Uninstaller: https://github.com/diskuv/dkml-installer-ocaml/releases/download/v1.1.0-prerel17/unsigned-diskuv-ocaml-windows_x86_64-u-1.0.1.exe

Here is the relevant part of the upcoming release notes:

New security:

  • (Advanced; experimental) If you are behind a corporate firewall that uses
    man-in-the-middle (MITM) TLS proxying, you can install your corporate CA chain
    so DKML, in particular MSYS2, does not reject connections. Only persons with
    write access to $env:ProgramData\DiskuvOCaml\conf\unixutils.sexp will be
    able to define the allowed MITM TLS chain; you may need access
    from your corporate Administrator. An example unixutils.sexp is:

    (
        (trust_anchors ("C:\\conf\\my.pem" "D:\\conf\\my.cer"))
    )

    You specify one or more .pem or .cer CA files, making sure to use two
    backslashes to escape your paths. Your Administrator may have already placed
    the CA files on your machine; otherwise use the guide at
    https://www.msys2.org/docs/faq/#how-can-i-make-msys2pacman-trust-my-companys-custom-tls-ca-certificate
    to copy them from your web browsers.

(I don't have an ETA when the full release will be available, although it will be fairly soon)

@jonahbeckford
Copy link
Contributor

You can use https://github.com/diskuv/dkml-installer-ocaml/releases/tag/v1.1.0_r2 . Search for trust_anchors on that release page which will tell you how to use your .pem / .cer CA files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

2 participants