Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bpo-47000: Add locale.getencoding() #32068

Merged
merged 13 commits into from
Apr 9, 2022
Merged

bpo-47000: Add locale.getencoding() #32068

merged 13 commits into from
Apr 9, 2022

Conversation

methane
Copy link
Member

@methane methane commented Mar 23, 2022 •
edited
Loading

@methane methane force-pushed the locale-get-encoding branch from c99ef77 to 8fc120e Compare March 31, 2022 06:13
@methane methane force-pushed the locale-get-encoding branch from 04fd5b0 to 8b50eb1 Compare April 4, 2022 03:16
@methane methane changed the title bpo-47000: Add locale.get_encoding() bpo-47000: Add locale.getencoding() Apr 4, 2022
@methane methane changed the title bpo-47000: Add locale.getencoding() bpo-47000: Add locale.getencoding() Apr 4, 2022
@methane methane force-pushed the locale-get-encoding branch from ffc9ba1 to a0204c2 Compare April 4, 2022 04:58
@methane methane requested a review from vstinner April 6, 2022 03:27
vstinner
vstinner reviewed Apr 6, 2022
View reviewed changes
Doc/library/locale.rst Outdated Show resolved Hide resolved
vstinner
vstinner reviewed Apr 6, 2022
View reviewed changes
Copy link
Member

@vstinner vstinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In some places, "utf-8" is used, in some other places "UTF-8" is used. I suggest to use lower case "utf-8" everywhere: doc, C code, _Py_STR(), etc. You may have to update getpreferredencoding() doc.

Doc/glossary.rst Show resolved Hide resolved
Doc/glossary.rst Outdated Show resolved Hide resolved
Doc/glossary.rst Show resolved Hide resolved
Doc/glossary.rst Outdated Show resolved Hide resolved
Doc/library/locale.rst Outdated Show resolved Hide resolved
Doc/library/locale.rst Outdated Show resolved Hide resolved
Doc/whatsnew/3.11.rst Outdated Show resolved Hide resolved
Lib/locale.py Outdated Show resolved Hide resolved
Modules/_io/textio.c Outdated Show resolved Hide resolved
@methane methane force-pushed the locale-get-encoding branch from 39e99bc to 7720b10 Compare April 7, 2022 04:18
vstinner
vstinner reviewed Apr 7, 2022
View reviewed changes
Doc/library/locale.rst Outdated
argument are ignored.

The :ref:`Python preinitialization <c-preinit>` configures the LC_CTYPE
locale. See also the :term:`filesystem encoding and error handler`.

.. versionchanged:: 3.7
The function now always returns ``UTF-8`` on Android or if the
The function now always returns ``"UTF-8"`` on Android or if the
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The function now always returns ``"UTF-8"`` on Android or if the
The function now always returns ``"utf-8"`` on Android or if the

Doc/library/locale.rst Outdated
:ref:`Python UTF-8 Mode <utf8-mode>` is enabled.

.. versionchanged:: 3.11
The function now returns ``"utf-8"`` instead of ``"UTF-8"`` on Android
or if the :ref:`Python UTF-8 Mode <utf8-mode>` is enabled.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am sure that the upper case versus lower case is worth to be mentioned. But I'm fine with keeping if you consider that it's worth it.

Doc/library/locale.rst
@@ -327,16 +327,37 @@ The :mod:`locale` module defines the following exception and functions:
is not necessary or desired, *do_setlocale* should be set to ``False``.

On Android or if the :ref:`Python UTF-8 Mode <utf8-mode>` is enabled, always
return ``'UTF-8'``, the :term:`locale encoding` and the *do_setlocale*
return ``'utf-8'``, the :term:`locale encoding` and the *do_setlocale*
argument are ignored.

The :ref:`Python preinitialization <c-preinit>` configures the LC_CTYPE
locale. See also the :term:`filesystem encoding and error handler`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest to copy/paste this paragraph to getencoding() doc.

Doc/whatsnew/3.11.rst Outdated
------

* Add :func:`locale.getencoding` to get the current locale encoding. It is similar to
``locale.getpreferredencoding(False)`` but ignores :ref:`UTF-8 Mode <utf8-mode>`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like repeating "Python" to remind that it's unrelated to the Unix locale, but really something specific to Python which ignores the locale.

Suggested change
``locale.getpreferredencoding(False)`` but ignores :ref:`UTF-8 Mode <utf8-mode>`.
``locale.getpreferredencoding(False)`` but ignores the :ref:`Python UTF-8 Mode <utf8-mode>`.

Misc/NEWS.d/next/Library/2022-04-06-11-54-53.bpo-47000.2nmAR1.rst Outdated
@@ -0,0 +1 @@
Add :func:`locale.getencoding`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can copy/paste the Doc/whatsnew/3.11.rst entry entry.

Doc/glossary.rst Outdated
encoding.
On Android and VxWorks, Python uses ``"utf-8"`` as the locale encoding.

``locale.getencoding()`` can be used to get the locale encoding.

Python uses the :term:`filesystem encoding and error handler` to convert
between Unicode filenames and bytes filenames.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO it would be helpful to mention here the Python UTF-8 Mode which ignores the locale encoding and always uses UTF-8. What do you think?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so. This paragraph doesn't describe where the locale encoding is used.
This paragraph describes what the locale encoding is.
With this pull request, locale encoding is locale encoding even in UTF-8 mode.

On the other hand, following this paragraph looks unnecessary:

      Python uses the :term:`filesystem encoding and error handler` to convert
      between Unicode filenames and bytes filenames.

I will replace it with See also :term:`filesystem encoding and error handler`.

vstinner
vstinner approved these changes Apr 8, 2022
View reviewed changes
Copy link
Member

@vstinner vstinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks.

IMO it's important that the current is complete and explains well this function, since it's a complex topic. If the doc is unclear, people will misuse it.

Thanks for adding "See also the filesystem encoding and error handler" in the glossary, that's good!

I'm not sure why the PR is still a draft.

@methane methane marked this pull request as ready for review April 9, 2022 00:54
@methane methane merged commit 6773203 into python:main Apr 9, 2022
@methane methane deleted the locale-get-encoding branch April 9, 2022 00:54
@methane methane mentioned this pull request Apr 10, 2022
Closed
@jopemachine jopemachine mentioned this pull request Sep 9, 2022
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vstinner vstinner vstinner approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@methane @vstinner @the-knights-who-say-ni @bedevere-bot