BUG: issubdtype is inconsistent on types and dtypes #9505

eric-wieser · 2017-08-01T11:08:49Z

Previously, issubdtype(x, y) and issubdtype(dtype(x), dtype(y)) could give different results

Fixes #9480 and #9506 and #2334 (duped by #5711 and #3218)

This deprecates all of the following strange behaviors, with the intent in the distant future to invert the results:

>>> np.issubdtype(np.float32, 'float64')
True
>>> np.issubdtype(np.float32, 'f8')
True
>>> np.issubdtype(np.int32, basestring)
True
>>> np.issubdtype(np.int32, 'int64')
True
>>> np.issubdtype(np.unicode_, 'void')
True
>>> np.issubdtype(np.int32, int)          # should have been spelt np.integer
True
>>> np.issubdtype(np.float32, float)      # should have been spelt np.floating
True
>>> np.issubdtype(np.complex64, complex)  # should have been spelt np.complexfloating
True
>>> np.issubdtype(np.float32, "float")
True
>>> np.issubdtype(np.float64, "f")
True

eric-wieser · 2017-08-01T11:10:07Z

test_numerictypes seemed like the obvious place to put a test for numerictypes.py.

However, if you actually look at that file, it seems to not be related to numerictypes.py at all

charris · 2017-08-01T16:14:39Z

Yeah, the numpy type hierarchy is a mess. Not conceptually, but as implemented. There are things like

In [3]: dtype(np.integer)
Out[3]: dtype('int64')

that, IMHO, should raise errors. Perhaps we should make things like np.integer abstract classes now that those are available. The main problem with cleaning things up will be the number of backward compatibility problems that will surely arise. The issue should probably be raised on the mailing list because of its potential impact.

eric-wieser · 2017-08-01T17:23:19Z

I've thought about this patch some more, and decided the best approach is just to fix dtype inputs, and FutureWarning in all other cases

eric-wieser · 2017-08-01T17:26:18Z

Perhaps we should make things like np.integer abstract classes

They already are not creatable - np.integer(1) fails. Problem is that dtypes don't care whether their type has a constructor.

eric-wieser · 2017-08-01T17:56:54Z

@charris: See updated PR. Apart from the DEP commit, which I can split off if needed, I don't think this needs a mailing list consultation

eric-wieser · 2017-08-01T18:01:56Z

numpy/core/numerictypes.py

+            # float, and similar less obvious things, such as np.generic from
+            # basestring
+            mro = arg2.mro()
+            arg2 = mro[1] if len(mro) > 1 else mro[0]


Note the only way that len(mro) == 1 is if arg2 is object, which should never happen - but we had it before too, so...

eric-wieser · 2017-08-01T18:20:06Z

numpy/core/numerictypes.py

+            warnings.warn(
+                "Conversion of the second argument of issubdtype from `{raw}` "
+                "to `{abstract} is deprecated. In future, it will be treated as "
+                "`{concrete} == np.dtype({raw}).type`.".format(


Example message:

In [1]: np.issubdtype(complex, bytearray) True

FutureWarning: Conversion of the second argument of issubdtype from bytearray to np.generic is deprecated. In future, it will be treated as np.object_ == np.dtype(bytearray).type.

eric-wieser · 2017-08-01T18:30:30Z

A quick search reveals that matplotlib makes the issubdtype(..., int) mistake in some rarely-used code (combined with the false assumption that np.int != int), and scipy makes a few of these too.

This will also catch out people who wrote np.float but meant np.floating, which scipy is guilty of

charris · 2017-08-05T02:25:25Z

LGTM. Could you add an entry in the release notes under Future Changes.

eric-wieser · 2017-08-05T17:19:56Z

Happy to rebase this myself later.

charris · 2017-08-05T17:25:16Z

The test queue looks stuffed. Yeah, go ahead and fix things up and add the release note.

charris · 2017-08-05T21:24:47Z

Close and reopen to see if the tests restart.

…ation Fixes numpygh-9506, unsigned exponentiation

…subdtype

Fixes numpy#9480

In many cases, this coercion is surprising, or would be if the user knew about it: * [('a', int)] -> np.flexible * str - > str (!) - not even a numpy type * 'float32' -> np.floating (discards size) * int -> np.signed_integer (not np.integer, as is usually meant)

charris · 2017-08-06T16:36:20Z

Thanks Eric.

Numpy is deprecating this usage (see numpy/numpy#9505).

Fixes issue #419 that prompts a FutureWarning due to a numpy change (as can be seen in numpy/numpy#9505)

Fix future warning on use of `np.issubdtype`. Fixes: ``` /home/travis/build/nipy/dipy/venv/lib/python3.4/site-packages/nibabel/streamlines/array_sequence.py:23: FutureWarning: Conversion of the second argument of issubdtype from `bool` to `np.generic` is deprecated. In future, it will be treated as `np.bool_ == np.dtype(bool).type`. ``` observed in projects using Nibabel, e.g. DIPY: https://travis-ci.org/nipy/dipy/jobs/471947136 More information about the issue and the fix at: numpy/numpy#2334 numpy/numpy#9505

This finishes the deprecation started in numpygh-9505 removing behaviour that allowed strings/types representing specific dtypes to behave like their more generic supertypes (e.g. the python float would map to floating instead of float64 which it typically maps to).

This finishes the deprecation started in gh-9505 removing behaviour that allowed strings/types representing specific dtypes to behave like their more generic supertypes (e.g. the python float would map to floating instead of float64 which it typically maps to). Co-Authored-By: Eric Wieser <[email protected]>

eric-wieser added 00 - Bug component: numpy._core labels Aug 1, 2017

eric-wieser force-pushed the fix-issubdtype branch 2 times, most recently from 6b9830f to d602521 Compare August 1, 2017 13:25

eric-wieser force-pushed the fix-issubdtype branch from 42b84a5 to 1517171 Compare August 1, 2017 17:52

eric-wieser commented Aug 1, 2017

View reviewed changes

eric-wieser force-pushed the fix-issubdtype branch from 1517171 to 6b191cd Compare August 1, 2017 18:18

eric-wieser commented Aug 1, 2017

View reviewed changes

eric-wieser force-pushed the fix-issubdtype branch 2 times, most recently from c666800 to 1e06594 Compare August 1, 2017 19:33

charris closed this Aug 5, 2017

charris reopened this Aug 5, 2017

eric-wieser added 7 commits August 5, 2017 21:49

BUG: Be explicit about allowing all integer types in matrix exponenti…

6a119c9

…ation Fixes numpygh-9506, unsigned exponentiation

MAINT: Stop using the undocumented coercion-then-downcast feature of …

fb004f8

…subdtype

MAINT: Simplify test for string-likes

0ab1a2b

BUG: abstract types did not compare correctly with issubdtype

dda2034

BUG: dtypes are ignored and downcast in issubdtype

44a4b71

Fixes numpy#9480

DOC: Make example work in python 3

4cac8cb

eric-wieser force-pushed the fix-issubdtype branch from 7ccc47d to 091b8c3 Compare August 6, 2017 02:57

eric-wieser mentioned this pull request Aug 6, 2017

Stop using np.{builtin}, and fix bugs due to the previous confusion matplotlib/matplotlib#8996

Merged

charris merged commit f307cec into numpy:master Aug 6, 2017

pv mentioned this pull request Aug 7, 2017

TST: _lib: make test_parallel_threads to not fail falsely scipy/scipy#7711

Merged

This was referenced Sep 27, 2017

issubdtype fails with string representations #3218

Closed

issubdtype(<whatever>, bool) malfunctions (Trac #1739) #2334

Closed

eric-wieser added this to the 1.14.0 release milestone Sep 27, 2017

eric-wieser mentioned this pull request Sep 27, 2017

issubdtype(double, bool) #5711

Closed

effigies added a commit to effigies/h5py that referenced this pull request Oct 25, 2017

ENH: Call np.issubdtype with np.floating

d317e57

Numpy is deprecating this usage (see numpy/numpy#9505).

effigies mentioned this pull request Oct 25, 2017

ENH: Call np.issubdtype with np.floating h5py/h5py#942

Closed

shoyer mentioned this pull request Jan 9, 2018

Test Failure: test_datetime_line_plot pydata/xarray#1813

Closed

astrojuanlu mentioned this pull request Jan 24, 2018

FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated h5py/h5py#961

Closed

ibenes mentioned this pull request Jan 26, 2018

Training decoupling beer-asr/beer#7

Merged

benjello added a commit to openfisca/openfisca-survey-manager that referenced this pull request Jan 30, 2018

Fix numpy deprecation of np.float in issubdtype numpy/numpy#9505

2e7701d

ariddell pushed a commit to stan-dev/pystan2 that referenced this pull request Jan 31, 2018

Update misc.py (#420)

3bf068b

Fixes issue #419 that prompts a FutureWarning due to a numpy change (as can be seen in numpy/numpy#9505)

bletham mentioned this pull request Feb 2, 2018

pystan issubdtype warning facebook/prophet#427

Closed

This was referenced Feb 11, 2018

Cannot install via conda when using python 3 bio-phys/MDBenchmark#33

Closed

Pin numpy to versions below 1.14 bio-phys/MDBenchmark#34

Closed

bwohlberg added a commit to bwohlberg/sporco that referenced this pull request Feb 24, 2018

Address change in numpy numpy/numpy#9505

5010498

ccnobbli mentioned this pull request Apr 9, 2018

Recursively_convert_to_json_serializable doesn't check all numpy types great-expectations/great_expectations#265

Closed

EricCousineau-TRI mentioned this pull request Apr 26, 2018

numpy: Require NumPy >= 0.15.0 RobotLocomotion/drake#8608

Closed

jhlegarreta mentioned this pull request Dec 25, 2018

BUG: Fix future warning on use of np.issubdtype. nipy/nibabel#707

Merged

seberg mentioned this pull request Mar 18, 2020

DEP: Make issubdtype consistent for types and dtypes #15773

Merged

ccwang002 mentioned this pull request Apr 9, 2020

update to htslib 1.10 brentp/cyvcf2#147

Merged

ramtingh mentioned this pull request Jul 27, 2021

Fix deprecated numpy issubdtype calls vmtk/vmtk#377

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BUG: issubdtype is inconsistent on types and dtypes #9505

BUG: issubdtype is inconsistent on types and dtypes #9505

eric-wieser commented Aug 1, 2017 •

edited

Loading

eric-wieser commented Aug 1, 2017

charris commented Aug 1, 2017

eric-wieser commented Aug 1, 2017

eric-wieser commented Aug 1, 2017

eric-wieser commented Aug 1, 2017

eric-wieser Aug 1, 2017

eric-wieser Aug 1, 2017 •

edited

Loading

eric-wieser commented Aug 1, 2017

charris commented Aug 5, 2017

eric-wieser commented Aug 5, 2017

charris commented Aug 5, 2017

charris commented Aug 5, 2017

charris commented Aug 6, 2017

BUG: issubdtype is inconsistent on types and dtypes #9505

BUG: issubdtype is inconsistent on types and dtypes #9505

Conversation

eric-wieser commented Aug 1, 2017 • edited Loading

eric-wieser commented Aug 1, 2017

charris commented Aug 1, 2017

eric-wieser commented Aug 1, 2017

eric-wieser commented Aug 1, 2017

eric-wieser commented Aug 1, 2017

eric-wieser Aug 1, 2017

Choose a reason for hiding this comment

eric-wieser Aug 1, 2017 • edited Loading

Choose a reason for hiding this comment

eric-wieser commented Aug 1, 2017

charris commented Aug 5, 2017

eric-wieser commented Aug 5, 2017

charris commented Aug 5, 2017

charris commented Aug 5, 2017

charris commented Aug 6, 2017

eric-wieser commented Aug 1, 2017 •

edited

Loading

eric-wieser Aug 1, 2017 •

edited

Loading