Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: no longer raise user warning when plotting tz aware time series #31207

Merged
merged 9 commits into from
Feb 3, 2020

Conversation

MarcoGorelli
Copy link
Member

@MarcoGorelli MarcoGorelli commented Jan 22, 2020

@MarcoGorelli MarcoGorelli changed the title 🐛 no longer raise user warning when plotting tz aware time series [BUG] no longer raise user warning when plotting tz aware time series Jan 22, 2020
@MarcoGorelli MarcoGorelli force-pushed the issue-31205 branch 4 times, most recently from b283e9e to 66adf3b Compare January 23, 2020 15:15
Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

likely this was introduced in master and is not apparent on 0.25.3 ? if so we don't need a whatsnew note (@jorisvandenbossche )

data = data.to_period(freq=freq)
with warnings.catch_warnings():
warnings.filterwarnings("ignore", category=UserWarning)
data = data.to_period(freq=freq)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't do this, instead I think just

data = data.tz_localize(None).to_period(freq=freq)

should suffice

@jreback jreback added this to the 1.0.0 milestone Jan 24, 2020
@jreback jreback added Timezones Timezone data dtype Visualization plotting labels Jan 24, 2020
@jorisvandenbossche jorisvandenbossche changed the title [BUG] no longer raise user warning when plotting tz aware time series BUG: no longer raise user warning when plotting tz aware time series Jan 24, 2020
@jorisvandenbossche
Copy link
Member

@MarcoGorelli tiny nitpicky workflow comment: can you use "BUG:" instead of "[BUG]" for PR titles? (just for consistency, that's how the majority of PRs do it)

@@ -251,7 +251,7 @@ def _maybe_convert_index(ax, data):
freq = frequencies.get_period_alias(freq)

if isinstance(data.index, ABCDatetimeIndex):
data = data.to_period(freq=freq)
data = data.tz_localize(None).to_period(freq=freq)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To preserve the existing behaviour, this might need to be tz_convert instead of tz_localize ? (the first will give the underlying UTC data, the other converts to naive local time)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would maybe check with a small actual example and see what plotting it gives (on 0.25.3 and on this PR).

Because intuitively, converting to local time zone sounds more useful, though. But not fully sure what the current behaviour is

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's the output of

df = pd.DataFrame(np.random.randn(100, 3), index=pd.date_range("2012", freq='H', periods=100, tz='UTC'), columns=['a', 'b', 'c']) 
df.head().plot() 

on 0.25.3:
image

On this branch:
image

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MarcoGorelli can you try with something else as UTC? (as UTC is the one timezone where UTC and local is the same :-))

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jorisvandenbossche sure :)

df = pd.DataFrame(
    np.random.randn(100, 3),
    index=pd.date_range(
        "2012", freq="H", periods=100, tz=pytz.timezone("Africa/Gaborone")
    ),
    columns=["a", "b", "c"],
)
df.head().plot()

0.25.3
image

This branch:
image

@jorisvandenbossche
Copy link
Member

likely this was introduced in master and is not apparent on 0.25.3 ? if so we don't need a whatsnew note (@jorisvandenbossche )

It seems this is already happening in released pandas as well (checked 0.25.3)

@MarcoGorelli
Copy link
Member Author

Can confirm this was already happening in 0.25.3. So, the whatsnew note is still needed?

@TomAugspurger
Copy link
Contributor

Yes, a release note will be needed if it's fixing something present in 0.25.3.

Are we backporting this to 1.0.0?

@MarcoGorelli
Copy link
Member Author

@TomAugspurger sure, whatsnew entry added

@TomAugspurger
Copy link
Contributor

@MarcoGorelli can you merge master to fix the CI failure?

If this is fixing a bug that was present in 0.25, then I think we should target 1.0.1 instead.

@MarcoGorelli
Copy link
Member Author

MarcoGorelli commented Jan 27, 2020

@TomAugspurger

I think we should target 1.0.1 instead.

Sure, I'll wait for the PR which creates that file to be merged

@MarcoGorelli
Copy link
Member Author

@TomAugspurger merged, whatsnew v1.0.1 note added

@jorisvandenbossche
Copy link
Member

@MarcoGorelli workflow related question: can you only push new commits? (instead of squashing/rebasing and force pushing) That makes it easier to see what changed in the GitHub interface

@jorisvandenbossche
Copy link
Member

It's not critical to backport this to 1.0.1, since it was already present in 0.25 and before. But it seems a safe fix (.index.to_period() basically does .index.tz_localize(None).to_period()), so I am fine with backporting.

tz = tz_aware_fixture
index = date_range("1/1/2011", periods=2, freq="H", tz=tz)
ts = Series([188.5, 328.25], index=index)
_check_plot_works(ts.plot)
with tm.assert_produces_warning(None):
_check_plot_works(ts.plot)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add something to check that the points are actually in the correct date-time, e.g. we removed the timezone (checking first/last prob enough)

@MarcoGorelli
Copy link
Member Author

I checked (visually) the output of the test function:

from pandas import date_range, Series
import matplotlib.pyplot as plt
import pytz

index = date_range("1/1/2011", periods=2, freq="H", tz=pytz.timezone('Europe/London'))
ts = Series([188.5, 328.25], index=index)
ts.plot()
plt.show()

and it looks like this:
image

Is this right? Shouldn't both ticks be labeled?

Anyway, this isn't affected by the current change (it's present on master as well) so I believe it would count as a separate issue, but have written the current test assuming it's expected output

@pep8speaks
Copy link

pep8speaks commented Feb 1, 2020

Hello @MarcoGorelli! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2020-02-02 17:27:38 UTC

@MarcoGorelli
Copy link
Member Author

pytest pandas/tests/plotting/test_datetimelike.py works locally but not during CI, looking into it

# Extract H:M component, check first point is in correct timezone.
# NOTE: this test could be updated once GH 31548 is fixed,
# so that the last point is checked as well.
assert labels = []
Copy link
Member Author

@MarcoGorelli MarcoGorelli Feb 2, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just for debugging CI purposes

@MarcoGorelli MarcoGorelli changed the title BUG: no longer raise user warning when plotting tz aware time series (wip) BUG: no longer raise user warning when plotting tz aware time series Feb 2, 2020
@MarcoGorelli
Copy link
Member Author

MarcoGorelli commented Feb 2, 2020

In one of the tests from the CI (linux py36_local), the axis labels are

['01 00:01', ...1 00:51', ...]

I don't understand where that extra minute is coming from (nor can I understand why it only appears during CI), but I think it'll take me longer than the v1.0.1 deadline to figure this out.

@jreback jreback changed the title (wip) BUG: no longer raise user warning when plotting tz aware time series BUG: no longer raise user warning when plotting tz aware time series Feb 2, 2020
@jreback
Copy link
Contributor

jreback commented Feb 2, 2020

cc @TomAugspurger @jorisvandenbossche ok as is?

@MarcoGorelli
Copy link
Member Author

MarcoGorelli commented Feb 2, 2020

@jreback Should I remove the test that checks the labels (which fails during CI) and open a separate issue about that?

@jreback
Copy link
Contributor

jreback commented Feb 2, 2020

@jreback Should I remove the test that checks the labels (which fails during CI) and open a separate issue about that?

yes

@MarcoGorelli
Copy link
Member Author

@jreback Done, and have opened #31580

@jorisvandenbossche jorisvandenbossche merged commit 44782c0 into pandas-dev:master Feb 3, 2020
@jorisvandenbossche
Copy link
Member

Thanks @MarcoGorelli !

meeseeksmachine pushed a commit to meeseeksmachine/pandas that referenced this pull request Feb 3, 2020
jorisvandenbossche pushed a commit that referenced this pull request Feb 3, 2020
@MarcoGorelli MarcoGorelli deleted the issue-31205 branch February 5, 2020 18:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Timezones Timezone data dtype Visualization plotting
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: plotting regular tz-aware timeseries gives UserWarning
5 participants