Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

REF: Use np.result_type instead of np.find_common_type #53343

Merged
merged 6 commits into from
May 23, 2023
Merged

REF: Use np.result_type instead of np.find_common_type #53343

merged 6 commits into from
May 23, 2023

Conversation

mroeschke
Copy link
Member

@mroeschke mroeschke added Compat pandas objects compatability with Numpy or Python functions Dependencies Required and optional dependencies labels May 22, 2023
@mroeschke mroeschke added this to the 2.0.2 milestone May 22, 2023
@mroeschke
Copy link
Member Author

cc @jbrockmendel Could I get a review to get the CI back to green?

jbrockmendel
jbrockmendel reviewed May 23, 2023
View reviewed changes
pandas/core/dtypes/cast.py
# was only one dtype)
common_dtype = np.dtype("O")

except TypeError:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you give an example of what gets here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

In [2]: np.result_type(np.dtype(float), np.dtype("M8"))
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[2], line 1
----> 1 np.result_type(np.dtype(float), np.dtype("M8"))

File <__array_function__ internals>:200, in result_type(*args, **kwargs)

TypeError: The DType <class 'numpy.dtype[float64]'> could not be promoted by <class 'numpy.dtype[datetime64]'>. This means that no common DType exists for the given inputs. For example they cannot be stored in a single array unless the dtype is `object`. The full list of DTypes is: (<class 'numpy.dtype[float64]'>, <class 'numpy.dtype[datetime64]'>)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAICT pretty much anything that would resolve to object before would return a TypeError now

jbrockmendel
jbrockmendel reviewed May 23, 2023
View reviewed changes
pandas/core/dtypes/cast.py
try:
common_dtype = np.result_type(*dtypes)
if common_dtype.kind in "mMSU":
# NumPy promotion currently (1.25) misbehaves for for times and strings,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there some numpy version at which we can remove this check?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure cc @seberg any thoughts?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a PR that would fix it, so I hope next release this will happen. That was the PR which get stuck a bit on the discussion of what == should do.

@mroeschke mroeschke merged commit 71cfd3a into pandas-dev:main May 23, 2023
@mroeschke mroeschke deleted the ref/np/result_type branch May 23, 2023 20:35
@lumberbot-app
Copy link

lumberbot-app bot commented May 23, 2023

Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

  1. Checkout backport branch and update it.
git checkout 2.0.x
git pull
  1. Cherry pick the first parent branch of the this PR on top of the older branch:
git cherry-pick -x -m1 71cfd3a2d56ff81404e13b7e5c6a9720d52a3a81
  1. You will likely have some merge/cherry-pick conflict here, fix them and commit:
git commit -am 'Backport PR #53343: REF: Use np.result_type instead of np.find_common_type'
  1. Push to a named branch:
git push YOURFORK 2.0.x:auto-backport-of-pr-53343-on-2.0.x
  1. Create a PR against branch 2.0.x, I would have named this PR:

"Backport PR #53343 on branch 2.0.x (REF: Use np.result_type instead of np.find_common_type)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the Still Needs Manual Backport label once the PR gets merged.

If these instructions are inaccurate, feel free to suggest an improvement.

mroeschke added a commit to mroeschke/pandas that referenced this pull request May 23, 2023
@mroeschke
Backport PR pandas-dev#53343: REF: Use np.result_type instead of np.f…
8bb9d67 
…ind_common_type
mroeschke added a commit that referenced this pull request May 24, 2023
@mroeschke
Backport PR #53343: REF: Use np.result_type instead of np.find_common…
e821e1b 
…_type (#53359)

* Backport PR #53343: REF: Use np.result_type instead of np.find_common_type

* Update pandas/core/internals/concat.py

* Update pandas/core/arrays/sparse/dtype.py

* Update pandas/core/dtypes/concat.py
DeaMariaLeon pushed a commit to DeaMariaLeon/pandas that referenced this pull request May 24, 2023
@mroeschke @DeaMariaLeon
REF: Use np.result_type instead of np.find_common_type (pandas-dev#53343
5660717 
)

* REF: Use np.result_type instead of np.find_common_type

* Fall back to object

* Create a np_find_common_dtype

* Address circular import

* Use generator

* Typing
@jrbourbeau jrbourbeau mentioned this pull request May 24, 2023
Merged
@neutrinoceros neutrinoceros mentioned this pull request Jun 18, 2023
Closed
3 tasks
Daquisu pushed a commit to Daquisu/pandas that referenced this pull request Jul 8, 2023
@mroeschke
REF: Use np.result_type instead of np.find_common_type (pandas-dev#53343
c71f2ab 
)

* REF: Use np.result_type instead of np.find_common_type

* Fall back to object

* Create a np_find_common_dtype

* Address circular import

* Use generator

* Typing
@mroeschke mroeschke mentioned this pull request Sep 21, 2023
Merged
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@seberg seberg seberg left review comments

@jbrockmendel jbrockmendel jbrockmendel approved these changes

Assignees
No one assigned
Labels
Compat pandas objects compatability with Numpy or Python functions Dependencies Required and optional dependencies
Projects
None yet
Milestone
2.0.2
Development

Successfully merging this pull request may close these issues.

BUG: np.find_common_type is deprecated in numpy main branch
3 participants
@mroeschke @seberg @jbrockmendel