grass.temporal.datetime_math: Return a TypedDict from compute_datetime_delta

[echoix]

This PR started by wanting to annotate compute_datetime_delta, which ended up needing a TypedDict to fully describe the keys that are returned and can't be omitted. A TypedDict can be defined in two ways, with a class syntax or functionnal syntax. In both cases, at runtime it is a plain dict, and the class syntax doesn't really define a class, as no values can be assigned, only annotations. https://mypy.readthedocs.io/en/stable/typed_dict.html#class-based-syntax

Either way, after editing the lines so that the dictionary can be initialized only once with all required keys, I found a hole in the logic that makes it possible for the month key to not always be assigned. Correctly factorized, linters or type checkers (pylint and pylance this time), correctly identify the same issue I found earlier:
By commenting out the variable initializations,

After going through the month logic:

We get warned that month can be not set when used

I added an extra test case for this. We can also see the effects on the tests if we use a value different from 0 for the initialization value of months, like 110, where multiple tests will now fail.

So what should be the correct logic here?

Lastly, some tests already existed, but in a file "unit_tests.py" that doesn't seem to run, and the file header mentions they are deprecated. So, I copied the test inputs over to a new pytest test (as it is a python-only method). Since these tests only checked for one (or sometimes more) values from the dict, I tried to calculate the expected values manually without looking at the doctests of the function. Even at the end, I was still making mistakes on when a key should contain the total of accumulated time vs 0. So maybe take another fresh look there.

[echoix]

I see that I mistakenly committed some changes that were intended for another branch/PR when amending in the file python/grass/temporal/datetime_math.py

[echoix]

I see that I mistakenly committed some changes that were intended for another branch/PR when amending in the file python/grass/temporal/datetime_math.py

Reverted them

[echoix]

I did a commit to remove that highlights the tests that depend on the month value being initialized to a value, by changing the initialized value to 110.

[echoix]

The generated docs link to the TypedDict class:

Note that the order of the docs is changed to be alphabetical. It is not an OrderedDict, order doesn't matter (even if dicts in 3.7 are guaranteed to keep insertion order).

[echoix]

Note: Currently pytest wasn't configured to accept the default naming scheme, so my purposely "broken" commit didn't fail. I probably won't have time today for this kind of edit

[wenzeslaus]

Looks good in general, but what's up with all the underscores?

[echoix]

Looks good in general, but what's up with all the underscores?

A typed dict needs to have all keys filled, and we can create them with a dict literal. This means I couldn't have it empty, add a key, and use the existing values to build a next value for another key. It needed to be set at once. So I used local variables for the internal variables, and didn't want to collide with special names (like min).

[echoix]

Ok, please now take a look at the tests I made sure to make failing. These are the ones that did not set any value for the "month" key before. I made the tests by setting the initial value to 0, and then changed it to another value (110) so we would see were the 110 value leaked. Tests didn't run before (only possibly doctest, but they are not enforced everywhere as they fail)

[echoix]

If I could have a review on the logic side (ie, what should the function return) before the weekend, I would have the opportunity to fix it there and anyone could review the Python part.

[echoix]

@ninsbl do you have an idea on what the values should be, by default, when not handled by the previous logics?

The failing test case was made on purpose to show that there was some "holes" in the situations not all covered.

[echoix]

Looks good in general, but what's up with all the underscores?

A typed dict needs to have all keys filled, and we can create them with a dict literal. This means I couldn't have it empty, add a key, and use the existing values to build a next value for another key. It needed to be set at once. So I used local variables for the internal variables, and didn't want to collide with special names (like min).

I don't know why I didn't think of it before, but there's a way to initialize the typed dictionary with values and use them, thus keeping pretty much the original shape of the code, reducing the diff.

[petrasovaa]

Based on my current understanding of this, it would be ok to initialize month to 0 as a default value. In some cases, the month key was missing, and looking at the code using this function in temporal_granularity.py, missing key or key equal zero acts the same way.
Does this answer your question? The comments here are somewhat confusing.

[petrasovaa]

I noticed some of the docstring examples seem incorrect (don't match the actual results), e.g. here months shouldn't be 12:

         >>> start = datetime(2011, 12, 1, 0, 0, 0)
         >>> end = datetime(2012, 6, 1, 0, 0, 0)
         >>> compute_datetime_delta(start, end)
         {'hour': 0, 'month': 12, 'second': 31622405, 'max_days': 366, 'year': 1, 'day': 0, 'minute': 0}

There may be more, I honestly had hard time understanding what exactly this function computes. Would it be possible to fix them?

[echoix]

I noticed some of the docstring examples seem incorrect (don't match the actual results), e.g. here months shouldn't be 12:

         >>> start = datetime(2011, 12, 1, 0, 0, 0)
         >>> end = datetime(2012, 6, 1, 0, 0, 0)
         >>> compute_datetime_delta(start, end)
         {'hour': 0, 'month': 12, 'second': 31622405, 'max_days': 366, 'year': 1, 'day': 0, 'minute': 0}

There may be more, I honestly had hard time understanding what exactly this function computes. Would it be possible to fix them?

I think its a one off. There was already the same inputs earlier in the doctest + in the pytest tests, but with a different expected value. This one used the expected value of the example above it. So I just removed it.

[petrasovaa]

Forgot about this before, now the month key would be present in all the outputs, so it should probably be added to the docstrings.

[echoix]

Forgot about this before, now the month key would be present in all the outputs, so it should probably be added to the docstrings.

How do make the docstring tests run?

[petrasovaa]

Forgot about this before, now the month key would be present in all the outputs, so it should probably be added to the docstrings.

How do make the docstring tests run?

There is temporal/testsuite/test_temporal_doctests.py, but it's deactivated.

[echoix]

I managed to get the doctest running for that file:

Open a grass shell, then:

python -m grass.temporal.datetime_math 

I didn't find any doctest option that ignores the dict order, so I just replaced the expected result with the new, constant order