gh-76511: Fix email.Message.as_string() for non-ASCII message with ASCII charset #116125
Conversation
serhiy-storchaka
commented
Feb 29, 2024
•
serhiy-storchaka
commented
- Issue: Email parser creates a message object that can't be flattened #76511
…ith ASCII charset
serhiy-storchaka
added
needs backport to 3.11
| @@ -0,0 +1,2 @@ | |||
| Fix :meth:`email.Message.as_string` for non-ASCII message with ASCII | |||
| charset. | |||
There was a problem hiding this comment.
This doesn't really describe the change very clearly. I would say something like "Fix exception in message_as_string that results when a message that claims to be in the ascii character set actually has non-ascii characters by replacing the non-ascii characters with ?, like the 'replace' parameter of encode does." Or something like that.
I don't want this comment to be a blocker, though.
There was a problem hiding this comment.
Thank you for your suggestion @bitdancer.
|
Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.11, 3.12. |
…ith ASCII charset (pythonGH-116125) (cherry picked from commit f97f25e) Co-authored-by: Serhiy Storchaka <[email protected]>
|
GH-116364 is a backport of this pull request to the 3.12 branch. |
…ith ASCII charset (pythonGH-116125) (cherry picked from commit f97f25e) Co-authored-by: Serhiy Storchaka <[email protected]>
|
GH-116365 is a backport of this pull request to the 3.11 branch. |
…with ASCII charset (GH-116125) (GH-116364) (cherry picked from commit f97f25e) Co-authored-by: Serhiy Storchaka <[email protected]>
…with ASCII charset (GH-116125) (GH-116365) (cherry picked from commit f97f25e) Co-authored-by: Serhiy Storchaka <[email protected]>
…ith ASCII charset (pythonGH-116125)
…ith ASCII charset (pythonGH-116125)
|
fwiw this seems to have caused a user-visible regression in django (at least) -- django/django#17979 oddly enough I don't think surrogates should have been involved at all -- the messages we're seeing were entirely utf-8-encodable and using a utf-8 charset. only internally after this series of changes between 3.11.6 and 3.11.9 they seem to have surrogates 🤔 my gut is something is wrong with this or one of the other recent unicode-related email patches and should probably be looked at. I can spend some time finding a minimal reproduction if that'd be useful |
Django SafeMIMEText corner case behaves differently in Python 3.11.9+, 3.12.3+, likely due to python/cpython#116125
