Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: raise when sort_index with ascending=None #39451

Merged
merged 14 commits into from
Feb 26, 2021

Conversation

ivanovmg
Copy link
Member

Added validation of ascending for sort_index methods.
Proposed a solution to raise ValueError when either of the elements of ascending turned out to be None.

pandas/core/generic.py Show resolved Hide resolved
@jreback jreback added Bug Reshaping Concat, Merge/Join, Stack/Unstack, Explode labels Jan 28, 2021
@jreback jreback added this to the 1.2.2 milestone Jan 28, 2021
@ivanovmg ivanovmg force-pushed the sort_values branch 3 times, most recently from 971b3a9 to 7717f8b Compare January 29, 2021 11:16
pandas/util/_validators.py Show resolved Hide resolved
Value to be validated.
arg_name : str
Name of the argument. To be reflected in the error message.
none_allowed : bool, optional
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add default

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added

Name of the argument. To be reflected in the error message.
none_allowed : bool, optional
Whether to consider None to be a valid boolean.
int_allowed : bool, optional
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added

@@ -11772,3 +11774,12 @@ def _doc_params(cls):
The required number of valid values to perform the operation. If fewer than
``min_count`` non-NA values are present the result will be NA.
"""


def validate_ascending(ascending):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you move to pandas/util/validators.py

shouldn't this raise for a list-like case if something is wrong?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

e.g. this shouldn't return anything

pls add some typing

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It raises for a list-like case if any of the items in ascending list is not treated as a valid boolean.
This construct does just that.

return [validate_bool_kwarg(item, "ascending", **kwargs) for item in ascending]

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can simply be part of validate_bool_kwarg or another function but should be co-located with validate_bool_kwarg

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved the function to pandas/util/_validators.py.
It is not clear for me at the moment how to make this validation anyhow different (as a part of validate_bool_kwarg) due to the fact that multiple types are acceptable for ascending.

@jreback jreback removed this from the 1.2.2 milestone Feb 1, 2021
@@ -11772,3 +11774,12 @@ def _doc_params(cls):
The required number of valid values to perform the operation. If fewer than
``min_count`` non-NA values are present the result will be NA.
"""


def validate_ascending(ascending):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can simply be part of validate_bool_kwarg or another function but should be co-located with validate_bool_kwarg

@jreback
Copy link
Contributor

jreback commented Feb 11, 2021

can you merge master and fixup to comments.

pandas/core/series.py Show resolved Hide resolved
pandas/core/series.py Show resolved Hide resolved
pandas/util/_validators.py Show resolved Hide resolved
pandas/tests/frame/methods/test_sort_index.py Show resolved Hide resolved
Comment on lines 768 to 769
(True, None),
(False, "True"),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you make these lists

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and maybe add an empty list too

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently an empty list does not raise an error, so the test fails.
Should I change the code, so that an empty list raises?

pandas/core/frame.py Show resolved Hide resolved
pandas/core/series.py Show resolved Hide resolved
Copy link
Member

@simonjayhawkins simonjayhawkins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ivanovmg can you resolve conflicts and address comments

@simonjayhawkins simonjayhawkins added this to the 1.2.3 milestone Feb 23, 2021
@jreback
Copy link
Contributor

jreback commented Feb 26, 2021

ok looks fine. can you add a whtasnew note for 1.2.3 in regression section. merge master and ping on green.

@jreback jreback merged commit 78371cc into pandas-dev:master Feb 26, 2021
@jreback
Copy link
Contributor

jreback commented Feb 26, 2021

thanks @ivanovmg

@jreback
Copy link
Contributor

jreback commented Feb 26, 2021

@meeseeksdev backport 1.2.x

@lumberbot-app

This comment has been minimized.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: Regression in sort_index with ascending=None
3 participants