Improve the documentation of PEP 495 features without referencing the PEP

[abalkin]

Well, ideally we want the docs to avoid having to reference PEPs to explain things as far as the use of the current implementation goes, as opposed to the historical design decisions that shaped the eventual implementation in CPython.

That said, if you really want, it's the https://github.com/python/cpython repo along with the rest of the source code. You can find a link to the source code of every docs page at the top of said page:

The link for this particular page is https://github.com/python/cpython/tree/main/Lib/datetime.py

Originally posted by @CAM-Gerlach in python/peps#2959 (comment)

[CAM-Gerlach]

Thanks, @abalkin . Just to clarify, I didn't mean to imply that the CPython docs should never reference PEPs, but rather just that they shouldn't rely on PEPs to "explain things", i.e. for the often-neglected Explanation type content critical to reader understanding of the key concepts behind the implementation, and that parts of the PEP content should be ported over to the canonical docs, if likely to be useful to a sufficient cross-section of future users of a feature to teach, describe, instruct and explain the current implementation (as opposed to being more of historical/background interest, or more relevant to the decision-making process).

---

Also, porting over and summarizing some discussion relevant to this issue from python/peps#2959 :

@bxparks reported:

...the behavior of the datetime library (or any timezone library in any language) around the Gap is non-trivial, and it took me a long time (1-2 years) to understand all the nuances fully.

I replied:

If text in the PEP was important to your understanding of the behavior of the datetime module, then perhaps it would be worth porting the relevant missing portions of the PEP's explanation to the Python docs, as an explanation-type section. Currently the docs cover the (API) "reference" topic thoroughly, but are somewhat more sparse in serving the other areas, particularly explanation. As the goal of explanation-type docs content is to serve understanding of the thing documented, this would seem like a good fit. if you're interested in something like that, I could help guide you through it (especially since I'm also on the Python Docs team).

@bxparks replied:

My opinion is that an average software developer would find it very difficult to understand the fold parameter in the datetime library without reading PEP 495, with the correct Gap diagram. During an overlap (e.g. 02:00->01:00 shift in the fall-back), the fold can be hand-wave explained, as currently done in the datetime.rst page. But during a gap (e.g. 02:00->03:00 spring-forward), an accurate explanation of fold in the datetime docs page would need PEP 495.

An extreme example of such a gap is something like the Pacific/Apia time zone in Samoa, which has a 24 hour gap in Dec 2011, going from 2011-12-29 23:59:59 to 2011-12-31 00:00:00, skipping Dec 30 entirely. I think it would be very difficult to explain how the fold parameter handles this case without PEP 495.

and I clarified:

It's certainly not any sort of hard rule; PEPs are referenced a number of places to explain the motivation, rationale and history of a specific feature, and it's better to reference the PEP than to simply leave the reader hanging completely when it comes to explanation-type content serving their understanding.

What I was suggesting, rather, is that while adding references to the PEP (:pep:`495`) is okay as a short term stopgap, what would be really helpful is if you could help migrate the relevant existing portions of the PEP that are important to reader understanding of the current fold behavior (as opposed to, e.g., justifying the changes, outlining backward compatibility, outlining alternatives, or specifying the API implementation) to a new top-level explanation section in the datetime docs on the topic.

[hugovk]

Is there something concrete to do here, or can we close this issue?

[bxparks]

There are several problems with the current datetime.rst file.

High level:

• it is a single page that is far too long, which makes it hard to quickly find information
• there are insufficient examples that illustrate the common usage of most classes and methods
• there are a few examples which are mostly obsolete and interfere with the flow of the document,
  • e.g. the tzinfo.fromutc() shows how to write a custom tzinfo class, which I suspect almost no one cares about, because I think most people would use the zoneinfo or pytz library instead

Low level, related to the fix and comments that I made in python/peps#2959:

• the fold parameter is inadequately explained in datetime.rst
  • the document current has this to say (which I think is not sufficient): "In [0, 1]. Used to disambiguate wall times during a repeated interval. (A repeated interval occurs when clocks are rolled back at the end of daylight saving time or when the UTC offset for the current zone is decreased for political reasons.) The value 0 (1) represents the earlier (later) of the two moments with the same wall time representation."
  • it does not adequately explain that the fold parameter is both an output parameter and an input parameter
  • it does not adequately explain the effect of the fold parameter during a "gap"
• the document does not adequately explain the result of certain operations (e.g. the datetime.datetime() constructor) inside a gap

Even if we added additional content to explain the problems that I listed above, I remarked (somewhere) that I think it is almost impossible for an average Python developer to understand the explanation without reading PEP495. Specifically, the diagrams in PEP495 are critical to understand how the fold parameter affect various functions and how the results are to be interpreted. Without the ability to reference or include the diagrams into datetime.rst, I (personally) don't think that adding additional content in datetime.rst would be effective.

My other concern is that I am not sure if I am the best person to do this edit. Yes, I have some low-level domain expertise on this topic, since I have implemented several DateTime/TimeZone libraries in various languages (C, C++, Python, Go) in the past few years. But I am not a full time Python developer. I am not familiar with the Python development process, or the documentation review process. The datetime.rst file is 2654 lines long (9th longest by lines, 5th longest by word count). I don't know the RST mark up language. It's intimidating. I am afraid that the editing, reviewing, and bike-shedding process will suck up months, maybe 1-2 years, of my time, and I don't know if I have the energy to go through that, especially since Python is not my main development environment.

[picnixz]

Hi, Sphinx maintainer here.

it is a single page that is far too long, which makes it hard to quickly find information

Hard to split it up without breaking Sphinx references. We had a similar discussion #126053 for the data model page so we probably can't do anything for now.

there are insufficient examples that illustrate the common usage of most classes and methods

If there are insufficient examples, we could add a new page with such examples. That could alleviate the length of the document but I'd suggest opening a separate issue.

there are a few examples which are mostly obsolete and interfere with the flow of the document, e.g., the tzinfo.fromutc() shows how to write a custom tzinfo class, which I suspect almost no one cares about, because I think most people would use the zoneinfo or pytz library instead

Even if you suspect it, it's an existing API so we should document it. Not everyone wants to rely on third-party library and not everyone can make the migration to zoneinfo if they learned tzinfo first. So I'm against removing them.

the fold parameter is inadequately explained in datetime.rst

Let's open a separate issue for that.

the document does not adequately explain the result of certain operations (e.g. the datetime.datetime() constructor) inside a gap

Similarly let's open another issue.

Without the ability to reference or include the diagrams into datetime.rst, I (personally) don't think that adding additional content in datetime.rst would be effective.

It's possible to include diagrams using Sphinx but we probably need to create them in advance since not everyone may have graphiz installed.

---

Since most of your concerns can be addressed separately, I suggest closing this specific issue and, if desired, create other related issues but AFAICT, we should probably create other pages for explaining in-depth concepts.