Outgoing emails corrupt / garbled

[DavidAnderson684]

We're running a test install (Freescout 1.7.30, PHP 8.0.15). Outgoing mails from Freescout (both customer replies, and other system emails, e.g. user invite email) are corrupt, resulting in being wrongly displayed in a variety of email clients.

Here is the raw source of a mail sent by Freescout (a reply to a test customer), as shown by Thunderbird's "Show source" feature:
test.txt

You will notice that some of the headers have unwanted preceding spaces, causing them to be treated as part of the preceding header and effectively ignored.

The same corruption is seen in the headers when the mail is collected instead in Alpine, and in Roundcube webmail, so is clearly occurring at source (i.e. not introduced by the email user agent).

The mailserver (Exim, latest version) handles outgoing email for a over 200 other hosted websites, without any similar issues seen

The top of the mail in Thunderbird looks much like you might expect, given that the headers don't correctly set up the MIME encoding:

[DavidAnderson684]

The same issue occurs on emails sent using the "Sent Test" button in Manage -> Settings -> Mail Settings under "System Emails".

[DavidAnderson684]

I added some logging of the $headers parameter in Swift_Transport_SimpleMailInvoker::mail(). The lines were all separated by a newline (char(10)), only. https://www.php.net/manual/fr/function.mail.php says that they should be separated by CRLF (\r\n).
I added a naive str_replace() (which of course is not a real fix), so that it was CRLF - and the mails are now correctly formatted.

[freescout-helpdesk]

Strange. You seem to be the only one experiencing this issue.

[DavidAnderson684]

If you do a file_put_contents() of $headers in that method, then what is the separator for you?

[freescout-helpdesk]

Swift_Transport_SimpleMailInvoker::mail() is not called in our case. Probably something else is actually performs sending.

[freescout-helpdesk]

What sending method are you using:

• PHP's mail() function
• Sendmail
• SMTP

[DavidAnderson684]

@freescout-helpdesk

In the settings, I had chosen to use PHP's mail() function (which was why I then searched the source for mail() and found it in the above method in order to debug the issue). Swift_Transport_SimpleMailInvoker::mail() is certainly being called for me: I putfile_put_contents() calls in there to log what was being sent to mail() in order to debug the problem.... and the str_replace() hack changed the result.

I've now added code in Swift_Transport_SimpleMailInvoker::mail() to save the stack trace, and here are the results:
trace.txt

[freescout-helpdesk]

We've checked. With "PHP's mail() function" as the sending method headers are separated with \n. But in Gmail the email and source headers look fine:

Message-ID: <[email protected]>
Date: Mon, 14 Feb 2022 01:52:23 +0000
...

Can you send an email to Gmail and check?
What is you operating system and PHP version?

[DavidAnderson684]

As per original message, PHP is 8.0.15, and the corrupt headers are the same in every mail client (Thunderbird, Alpine and Roundcube being mentioned), since they are corrupt in the mail in the inbox. Here it is in GMail, with the headers concatenated as per the email RFC (headers beginning with a space are taken as a continuation of the current header):

Resulting in corrupt display:

Operating system is CentOS 7.9 (fully updated).

[freescout-helpdesk]

It's hard to tell why it happens for you and does not happen for others. It can be system or PHP configuration issue.

[freescout-helpdesk]

Try to downgrade PHP to 7.4 or upgrade to 8.1

[DavidAnderson684]

Interestingly, it works in PHP 7.4 (version 7.4.28). I have not tried PHP 8.1.

The mail configuration in both is identical; the only uncommented lines in the configuration relating to mail are:

sendmail_path = /usr/sbin/sendmail -t -i
mail.add_x_header = On

It looks likely that this is related - a change made in PHP 8 only: https://bugs.php.net/bug.php?id=47983

[freescout-helpdesk]

Works fine in PHP 8.1. It must be PHP 8.0.x issue.

[DavidAnderson684]

Further investigation confirms that in PHP 8.0, if you supply a string to the $additional_headers parameter of https://www.php.net/manual/en/function.mail.php, and if you separate your header with \n (contrary to the parameter specification that "Multiple extra headers should be separated with a CRLF (\r\n)."), then this problem occurs. There are work-arounds (and it works on other PHP versions), but the real fix would be for Freescout to follow the PHP manual and use \r\n instead of \n.

[Erudition]

Having the same issue with PHP 8.2 and 8.3. It messed up the password reset link so I had to manually edit the database to get admin access. Only then could I change it away from PHP sendmail