Cannot use ftp protocol in install.sh for downloading non-existent packages in secure environments

[singleterry]

• OpenCoarrays Version: <2.2.0>
• Fortran Compiler: <gcc 7.3.0>
• C compiler used for building lib: <ggc 7.3.0>
• Installation method: <install.sh>
• Output of uname -a: Linux hze.larc.nasa.gov 2.6.32-431.11.2.el6.x86_64 tests dis_transpose: test passed  #1 SMP Tue Mar 25 19:59:55 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
• MPI library being used: <3.2>
• Machine architecture and number of physical cores: not relevant
• Version of CMake: <3.4.0>

When using the install.sh script, the prerequisites downloaded with the ftp protocol will fail in secure environments. The protocol used to download those files should be changed.

[zbeekman]

@singleterry I'm not sure what you mean by "secure" environments. Can you please give some more details, preferably including some output with the error message.

Did curl AND wget fail for you?

[singleterry]

Assume I can just reply back!?! The ftp protocol is forbidden in secure environments. Therefore when curl (and I assume wget) tried to access ftp://url, it cannot find the site because the ftp protocol is blocked. Of course, when I http://url, it finds it fine, I download what it was trying to download and rerun the install script with success. Maybe change to https:// or sftp:// This did not happen with all of the curl/wget calls. Some were to https:// and they worked fine! I hope that helps!?! Robert ******************************************************************************* Robert C. Singleterry Jr., Ph.D. Research Engineer and NASA Administrator’s Fellow (Cohort 6: 2002-2004) Durability, Damage Tolerance, & Reliability Branch Langley Research Center, National Aeronautics and Space Administration Building 1205, Room 285, Mail Stop 188E 2 West Reid Street Hampton, VA 23681-2199 Work Voice: (757) 864-1437 FAX: (757) 864-4385 Personal Cell: (757) 371-4848 ******************************************************************************* From: zbeekman <[email protected]> Sent: Friday, October 5, 2018 10:25 To: sourceryinstitute/OpenCoarrays <[email protected]> Cc: Singleterry, Robert C. (LARC-D309) <[email protected]>; Mention <[email protected]> Subject: Re: [sourceryinstitute/OpenCoarrays] Cannot use ftp protocol in install.sh for downloading non-existent packages in secure environments (#585) @singleterry<https://github.com/singleterry> I'm not sure what you mean by "secure" environments. Can you please give some more details, preferably including some output with the error message. Did curl AND wget fail for you? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#585 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AEL6W1KK50NvF7sDHKUryQmG4LTyKMlEks5uh2u6gaJpZM4W12lm>.

[zbeekman]

Yes, email replies get forwarded. 🙂

Thanks for clarification. We certainly should review all of our download URLs and favor encrypted variants wherever possible via https and, if/where needed, sftp. Sometimes, however, people are on systems lacking CA certs, and curl or wget fail with SSL errors, but I’d rather someone not be able to install OC, than be the vector used to compromise a machine through a MITM software supply chain attack.

I’ll assign this issue to myself to ensure I review the urls and update to use encryption wherever possible.

[singleterry]

Hello, Maybe make that a question to ask the user?? Robert ******************************************************************************* Robert C. Singleterry Jr., Ph.D. Research Engineer and NASA Administrator’s Fellow (Cohort 6: 2002-2004) Durability, Damage Tolerance, & Reliability Branch Langley Research Center, National Aeronautics and Space Administration Building 1205, Room 285, Mail Stop 188E 2 West Reid Street Hampton, VA 23681-2199 Work Voice: (757) 864-1437 FAX: (757) 864-4385 Personal Cell: (757) 371-4848 ******************************************************************************* From: zbeekman <[email protected]> Sent: Tuesday, October 9, 2018 15:02 To: sourceryinstitute/OpenCoarrays <[email protected]> Cc: Singleterry, Robert C. (LARC-D309) <[email protected]>; Mention <[email protected]> Subject: Re: [sourceryinstitute/OpenCoarrays] Cannot use ftp protocol in install.sh for downloading non-existent packages in secure environments (#585) Yes, email replies get forwarded. 🙂 Thanks for clarification. We certainly should review all of our download URLs and favor encrypted variants wherever possible via https and, if/where needed, sftp. Sometimes, however, people are on systems lacking CA certs, and curl or wget fail with SSL errors, but I’d rather someone not be able to install OC, than be the vector used to compromise a machine through a MITM software supply chain attack. I’ll assign this issue to myself to ensure I review the urls and update to use encryption wherever possible. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#585 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AEL6W43LbON9chXpfeO8HawpOky9OuITks5ujPKvgaJpZM4W12lm>.

[rouson]

@singleterry Thanks for submitting this.

@zbeekman The URL for each package the installer knows how to install is listed here. The only ones that use the ftp protocol are the ones on the GNU ftp server ftp.gnu.org. If there's a reliable mirror that uses something other than ftp, the only change that is needed is to change each corresponding URL in the aforementioned file. Most likely, everything will work from there.

In case it helps, if you ever want to see what URL the installer will use for a given package, pass the package name with the-U argument:

$ ./install.sh -U gcc
ftp://ftp.gnu.org:/gnu/gcc/gcc-8.2.0/gcc-8.2.0.tar.gz

[rouson]

I see there's a GNU mirror list here: https://www.gnu.org/prep/ftp.en.html.

GNU Mirror List - GNU Project - Free Software Foundation

[zbeekman]

@rouson I think we want: https://ftpmirror.gnu.org/gcc/gcc-8.2.0/gcc-8.2.0.tar.xz

I think this redirects to a suitable mirror that is closest to the user (and has https/ssl encryption) so it should be fast.

[rouson]

I'll test the switch and push a fix.

[rouson]

This is now fixed in PR #594 and will appear in the next release.