MS Windows native TLS (via Schannel SSP), IDN, and DNS SRV handling. GitHub releases for binaries.

[mlt]

Let's have an option to build msmtp using libraries provided by MS Windows. The produced portable executable is just 190kb with no external dependencies. It would be also nice to automatically upload a binary to GitHub Releases whenever a new version is tagged. I believe it should "just work" if merged.

Ultimately, I would like to have msmtp available via Chocolatey. I "packaged" it a while ago when it was hosted on SF but it needs an update badly.

There is some hiccup with IDN/locale when built with NLS/gettext. I have no idea what is causing it and why _create_locale(LC_ALL, "") works while setlocale(LC_ALL, "") does not. My main use-case is with Emacs so I personally don't need NLS much.

I can clean up/split/rebase commits as necessary.

Upddate: It looks like the implications of ENABLE_NLS affect console output only somehow. I suggest to remove that conditional code completely commit as is with broken output as long as IDN resolution works and wait until someone complains. It is impossible to test it properly on GitHub because if stdout/stderr are redirected it is "almost" not noticeable. Here is what I get on my local machine from NLS-enabled build with enabled UTF-8.

Click me

Command Prompt

C:\dev\c\msmtp\123\src>chcp
Active code page: 65001
C:\dev\c\msmtp\123\src>msmtp --serverinfo --host=mx3.почта.рус --port=1234
msmtp: cannot connect to mx3.почта.руÑ?, port 1234: connection refused
C:\dev\c\msmtp\123\src>msmtp --serverinfo --host=mx3.почта.рус --port=1234 > output.log 2>&1
C:\dev\c\msmtp\123\src>type output.log
msmtp: cannot connect to mx3.почта.ру�?, port 1234: connection refused

MSYS2

mlt@DESKTOP-K26J5U0 UCRT64 /c/dev/c/msmtp/123
$ ./src/msmtp --serverinfo --host=mx3.почта.рус --port=1234
msmtp.exe: cannot connect to mx3.почта.рус, port 1234: connection refused

PowerShell

(base) PS C:\dev\c\msmtp\123\src> chcp
Active code page: 65001
(base) PS C:\dev\c\msmtp\123\src> .\msmtp --debug --serverinfo --host=mx3.почта.рус --port=1234
using account specified on command line
host = mx3.почта.руÑ
…
msmtp.exe: cannot connect to mx3.почта.руÑ?, port 1234: connection refused
(base) PS C:\dev\c\msmtp\123\src> .\msmtp --debug --serverinfo --host=mx3.почта.рус --port=1234 2>&1 | tee .\output.log
.\msmtp : msmtp.exe: cannot connect to mx3.почта.ру�?, port 1234: connection refused
At line:1 char:1
+ .\msmtp --debug --serverinfo --host=mx3.почта.рус --port=1234 2>&1 |  ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (msmtp.exe: cann...nection refused:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError

using account specified on command line
host = mx3.почта.рус
…

[marlam]

The Windows platform is already supported out of the box, and up-to-date binary packages exist: https://packages.msys2.org/packages/mingw-w64-x86_64-msmtp?repo=mingw64

Why add more than 1000 lines of code that do not benefit any other platform, when that platform is already supported by code shared across all platforms?

If the only benefits are avoiding dependencies on free libraries or reducing the binary size, then I don't think it's worth it.

[mlt]

Indeed, the proposed changes do not benefit users of other platforms GitHub Actions aside. I think it is worth adding build tests for Linux and MacOS while at it unless you test it elsewhere (but I haven't seen anything mentioned on your web site or GH repo). However, I would like to make a case for Windows users.

• I have nothing against MSYS2 and I do use it to build msmtp with GitHub Actions. That said, it does require an installation. Whereas proposed changes allow for making msmtp ultraportable (single EXE) and well suited for non-MSYS2 distribution channel like Chocolatey.
• Also, it looks like there is a minor delay: the currently available version is 1.8.23. While I believe, it is not critical but nevertheless it is not up-to-date so to speak.
• It is not only about install [dependencies] and forget but keeping certificates and revocation lists up-to-date. In case of Windows and SSPI it is taken care of by MS automagically.
• Adoption in other software… libcurl implemented Schannel support in 2012 although they had support for 6 SSL libraries already.
• Schannel integrates nicely in current msmtp layout with conditional compilation of mtls-sspi.c
• It does not have to be "default" build but an option. I mean I don't have an implementation for what GNU SASL offers.

I did not mean to drop a code bomb. It is just after Schannel it was only logical to use native implementation for the rest. I am willing to split PRs.

[marlam]

I still see some need for discussions. Regarding your points:

• You can get a single executable file for Windows already, via static linking.
• Regarding build requirements, there are no differences between 1.8.23 and the current version, so you could just bump the package version in MSYS2
• Regarding Schannel:
  • Chocolatey does not seem to have GnuTLS, but it does have OpenSSL. Improving the existing but outdated OpenSSL code in msmtp would benefit all platforms, in contrast to adding support for a Windows-specific API
  • curl may support Schannel and even more TLS libraries, but it's a much larger project with more developers. msmtp on the other hand is struggling to keep its OpenSSL support alive; adding support for yet another API won't help

[mlt]

• You can get a single executable file for Windows already, via static linking.

I guess it is a faux pas to talk about file sizes in 2024🤷That is true static linking is a thing. However, I see no point to compile and link in something having functionality already provided by OS.

Until "recently", there were too many ways to build stuff on Windows that were not ABI compatible. That is if you want Visual Studio and all, you'd have to build dependencies with MS compiler especially if it comes to C++. Definitely, UCRT64 (environment of MSYS2) and vcpkg make it easier to build 3rd party libraries and link now.

Also Windows users aren't particularly inclined to deal with PEM files. Certificates and revocations should be handled by OS updates automagically. That is my main concern as it is very tempting to just set things once and never look back.

• Regarding Schannel:
  • Chocolatey does not seem to have GnuTLS, but it does have OpenSSL.

That is a can of worms dependency hell I would not open. As close as it gets to be a "package manager", Chocolatey is mostly about binary end products and not libraries that could be compiled with different options.

There are also other use cases like https://portableapps.com/ to make it possible to run things without proper "installation". Which brings another issue.. fallback check for msmtp configuration file next to EXE but that is not urgent.

Improving the existing but outdated OpenSSL code in msmtp would benefit all platforms, in contrast to adding support for a Windows-specific API

Why not both? I am in the more the better camp. Although not relevant for other platform, some might find it useful on Windows. In the end, it is just a file sitting in a repo that won't have any security implications for users of other platforms.

• curl may support Schannel and even more TLS libraries, but it's a much larger project with more developers. msmtp on the other hand is struggling to keep its OpenSSL support alive; adding support for yet another API won't help

That was news to me (Apparently I didn't read news). I just had an impression the discouragement came from the tainted reputation from a while ago.

It looks like quite a few API functions could be used but they aren't: time conversion (ASN1_TIME_to_tm), SNI host set up (SSL_set_tlsext_host_name), restricting cipher suites, and verification (using X509_VERIFY_PARAM). Also looks like newer version are automatically initialized and random seeded (I'm not sure if there is an easy way to check if it was explicitly disable during the build) and reseeded. Most of the grievances you brought up seem to be obsolete. If you have a personal issue with OpenSSL, I probably can take a stab at removing old code and using appropriate API while dropping old OpenSSL compatibility.

I have a rather dumb question regarding restricting ciphers. What is the point to go into details with priorities and bits? It seems like every library has the capability to use only strong (whatever that means at a given instant) cryptography, e.g. set_security_level to 5.

[marlam]

• You can get a single executable file for Windows already, via static linking.

I guess it is a faux pas to talk about file sizes in 2024🤷That is true static linking is a thing. However, I see no point to compile and link in something having functionality already provided by OS.

The point would be to avoid using system-specific libraries and give the existing portable code a wider exposure. Fixing a bug or implementing an improvement would then benefit all platforms, and could be done collaboratively by users on all platforms. That's an important point for small projects that lack resources.

Until "recently", there were too many ways to build stuff on Windows that were not ABI compatible. That is if you want Visual Studio and all, you'd have to build dependencies with MS compiler especially if it comes to C++. Definitely, UCRT64 (environment of MSYS2) and vcpkg make it easier to build 3rd party libraries and link now.

With static binaries, there's no need for ABI compatibility.

Also Windows users aren't particularly inclined to deal with PEM files. Certificates and revocations should be handled by OS updates automagically. That is my main concern as it is very tempting to just set things once and never look back.

GnuTLS already uses the Windows system default trust store when msmtp is used with tls_trust_file system, which is the default. So you already get the Windows trust model including updates by default, and there's no need for manual updates or PEM file fiddling.

• Regarding Schannel:

  • Chocolatey does not seem to have GnuTLS, but it does have OpenSSL.

That is a can of worms dependency hell I would not open. As close as it gets to be a "package manager", Chocolatey is mostly about binary end products and not libraries that could be compiled with different options.

Seems like a shortcoming of Chocolatey to me ;)

Improving the existing but outdated OpenSSL code in msmtp would benefit all platforms, in contrast to adding support for a Windows-specific API

Why not both? I am in the more the better camp. Although not relevant for other platform, some might find it useful on Windows. In the end, it is just a file sitting in a repo that won't have any security implications for users of other platforms.

Yes, it has an impact: it reduces the exposure of the portable code, and reduces the number of people that can work on fixing bugs or implementing improvements.

In this sense a fourth TLS implementation amounts to Technical Debt; it is a liability rather than an asset.

• curl may support Schannel and even more TLS libraries, but it's a much larger project with more developers. msmtp on the other hand is struggling to keep its OpenSSL support alive; adding support for yet another API won't help

That was news to me (Apparently I didn't read news). I just had an impression the discouragement came from the tainted reputation from a while ago.

It looks like quite a few API functions could be used but they aren't: time conversion (ASN1_TIME_to_tm), SNI host set up (SSL_set_tlsext_host_name), restricting cipher suites, and verification (using X509_VERIFY_PARAM). Also looks like newer version are automatically initialized and random seeded (I'm not sure if there is an easy way to check if it was explicitly disable during the build) and reseeded. Most of the grievances you brought up seem to be obsolete. If you have a personal issue with OpenSSL, I probably can take a stab at removing old code and using appropriate API while dropping old OpenSSL compatibility.

Yes, there are many ways to improve the existing OpenSSL code, and for someone familiar with OpenSSL it's probably easy to do. I am personally frustrated with OpenSSL (and it shows in the news entry I wrote; sorry for that) and I just don't have the time to deal with a frustrating library when the alternatives are (in my personal view) far better, but of course I would be happy to accept any patches by others.

Unfortunately the OpenSSL code in msmtp will not simply go away, as there are special purpose Linux distributions that depend on it and cannot currently use GnuTLS or libretls. So in effect we already experience technical debt and code as a liability.

I don't want to add to the problem: once Windows users use an Schannel msmtp version, we cannot simply remove the code when nobody steps up to implement necessary improvements, and we get the same problem all over again.

I have a rather dumb question regarding restricting ciphers. What is the point to go into details with priorities and bits? It seems like every library has the capability to use only strong (whatever that means at a given instant) cryptography, e.g. set_security_level to 5.

There's always someone who wants full control, but you can still use simple presets. For example, with GnuTLS, you can use keywords such as "NORMAL" or "SECURE192" instead of fiddling with details. OpenSSL may be different; I did not look.

[mlt]

With static binaries, there's no need for ABI compatibility.

What I meant is the way to build it, not run it. I'm not sure how you acquire Windows users and contributors. It is appealing when one can simply open Visual Studio and hit F5 to start debugging in a cozy and familiar IDE. However that route involves building of all underlying dependencies using MS tools. vcpkg project helps with those, but

(base) PS C:\dev> vcpkg install libgnutls
Computing installation plan...
libgnutls is only supported on '!windows | mingw', which does not match x64-windows. This usually means that there are known build failures, or runtime problems, when building other platforms. To ignore this and attempt to build libgnutls anyway, rerun vcpkg with `--allow-unsupported`.

GnuTLS already uses the Windows system default trust store when msmtp is used with tls_trust_file system, which is the default. So you already get the Windows trust model including updates by default, and there's no need for manual updates or PEM file fiddling.

Apparently it is the case with OpenSSL 3.2+ as well, but for root certificates only. 😞

Seems like a shortcoming of Chocolatey to me ;)

The main benefit is the ease of getting updates on that platform from diverse sources. So, yeah, no place for dependencies.

It just seems "wrong" to pull dependencies from msys2 just to repack.

Yes, it has an impact: it reduces the exposure of the portable code, and reduces the number of people that can work on fixing bugs or implementing improvements.
In this sense a fourth TLS implementation amounts to Technical Debt; it is a liability rather than an asset.

I am hesitating between dropping msmtp from Chocolatey completely or keeping a derivative patched up version with a clear indication of that for as long as it lasts 🤔

What somewhat bothers me about msys2 is the lack of "portability" in a sense of dropping necessary small stuff on a USB flash drive and run from there.

Yes, there are many ways to improve the existing OpenSSL code, and for someone familiar with OpenSSL it's probably easy to do. I am personally frustrated with OpenSSL (and it shows in the news entry I wrote; sorry for that) and I just don't have the time to deal with a frustrating library when the alternatives are (in my personal view) far better, but of course I would be happy to accept any patches by others.

I don't mind to learn that library/suffer but it is a work in progress.

Unfortunately the OpenSSL code in msmtp will not simply go away, as there are special purpose Linux distributions that depend on it and cannot currently use GnuTLS or libretls.

I didn't know that.

I don't want to add to the problem: once Windows users use an Schannel msmtp version, we cannot simply remove the code when nobody steps up to implement necessary improvements, and we get the same problem all over again.

Understandable.

There's always someone who wants full control, but you can still use simple presets. For example, with GnuTLS, you can use keywords such as "NORMAL" or "SECURE192" instead of fiddling with details. OpenSSL may be different; I did not look.

I am leaning toward using SSL_CONF_cmd in place of priorities besides using msmtp section in openssl.cnf.

What is you take on testing? I am about to open PR on IDN while using --configure.

[marlam]

With static binaries, there's no need for ABI compatibility.

What I meant is the way to build it, not run it. I'm not sure how you acquire Windows users and contributors. It is appealing when one can simply open Visual Studio and hit F5 to start debugging in a cozy and familiar IDE. However that route involves building of all underlying dependencies using MS tools. vcpkg project helps with those, but

(base) PS C:\dev> vcpkg install libgnutls
Computing installation plan...
libgnutls is only supported on '!windows | mingw', which does not match x64-windows. This usually means that there are known build failures, or runtime problems, when building other platforms. To ignore this and attempt to build libgnutls anyway, rerun vcpkg with `--allow-unsupported`.

Catering to the shortcomings of proprietary IDEs does not help to improve the quality of any software in my experience. That problem is better addressed with opening a feature or support request at your IDE vendor.

Seems like a shortcoming of Chocolatey to me ;)

The main benefit is the ease of getting updates on that platform from diverse sources. So, yeah, no place for dependencies.

It just seems "wrong" to pull dependencies from msys2 just to repack.

And it probably is. I recommend using MXE to build a cross-compiler and the required libraries, and then simply build a Windows static executable from a Linux system, so things can be scripted and automated. Something similar is probably also possible on Windows, it should even be simpler since there is no cross-compilation involved.

What is you take on testing? I am about to open PR on IDN while using --configure.

Thanks for that; it's merged :)