-
-
Notifications
You must be signed in to change notification settings - Fork 31k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
gh-91156: Use locale.getencoding()
instead of getpreferredencoding
#91732
Changes from 3 commits
0bf6bfc
8cd88b9
3005858
951875a
38601b0
b9f088b
d8e8c26
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -37,7 +37,7 @@ Linux and the BSD variants of Unix. | |
|
||
import locale | ||
locale.setlocale(locale.LC_ALL, '') | ||
code = locale.getpreferredencoding() | ||
code = locale.getencoding() | ||
|
||
Then use *code* as the encoding for :meth:`str.encode` calls. | ||
|
||
|
@@ -924,7 +924,7 @@ the following methods and attributes: | |
Encoding used to encode method arguments (Unicode strings and characters). | ||
The encoding attribute is inherited from the parent window when a subwindow | ||
is created, for example with :meth:`window.subwin`. By default, the locale | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Would you mind to replace "locale encoding" with "current locale encoding"? Just to remind that it can be changed at runtime. For example, the encoding used by the readline module is the currrent encoding, the encoding is not stored anywere. Its C code uses PyUnicode_EncodeLocale() and PyUnicode_DecodeLocale(): current LC_CTYPE locale encoding. |
||
encoding is used (see :func:`locale.getpreferredencoding`). | ||
encoding is used (see :func:`locale.getencoding`). | ||
|
||
.. versionadded:: 3.3 | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -363,7 +363,7 @@ class TestEnUSCollation(BaseLocalizedTest, TestCollation): | |
locale_type = locale.LC_ALL | ||
|
||
def setUp(self): | ||
enc = codecs.lookup(locale.getpreferredencoding(False) or 'ascii').name | ||
enc = codecs.lookup(locale.getencoding() or 'ascii').name | ||
if enc not in ('utf-8', 'iso8859-1', 'cp1252'): | ||
raise unittest.SkipTest('encoding not suitable') | ||
if enc != 'iso8859-1' and (sys.platform == 'darwin' or is_android or | ||
|
@@ -533,6 +533,12 @@ def test_defaults_UTF8(self): | |
if orig_getlocale is not None: | ||
_locale._getdefaultlocale = orig_getlocale | ||
|
||
def test_getencoding(self): | ||
# Invoke getencoding to make sure it does not cause exceptions. | ||
enc = locale.getencoding() | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I suggest to test the type: add Maybe also ensure that the string is not empty? add |
||
# make sure it is valid | ||
codecs.lookup(enc) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. _PyUnicode_InitEncodings() fails it config.filesystem_encoding or config.stdio_encoding is not known by codecs.lookup(name). So this call should not fail. If tomorrow this test fails, I suggest to remove it and only check that the string is non-empty. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. When UTF-8 mode is enabled, both of stdio encoding and filesystem encoding are UTF-8, not locale encoding. |
||
|
||
def test_getpreferredencoding(self): | ||
# Invoke getpreferredencoding to make sure it does not cause exceptions. | ||
enc = locale.getpreferredencoding() | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -145,11 +145,6 @@ def test_guess_all_types(self): | |
self.assertNotIn('.no-such-ext', all) | ||
|
||
def test_encoding(self): | ||
getpreferredencoding = locale.getpreferredencoding | ||
self.addCleanup(setattr, locale, 'getpreferredencoding', | ||
getpreferredencoding) | ||
locale.getpreferredencoding = lambda: 'ascii' | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This hack doesn't work for most cases. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Please remove This code is correct. I don't think that the locale encoding is still used: MimeType.read() calls |
||
filename = support.findfile("mime.types") | ||
mimes = mimetypes.MimeTypes([filename]) | ||
exts = mimes.guess_all_extensions('application/vnd.geocube+xml', | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remark unrelated to your PR.
This is wrong: Python now always call
setlocale(LC_CTYPE, "")
at startup.Calling
locale.setlocale(locale.LC_ALL, '')
is no longer needed.Moreover, curses likely use mbstowcs() and wcstombs() functions, rather than nl_langinfo() (nl_langinfo(CODESET)?).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK. Let's remove this note.