Fix the use of linspace for numpy 1.18 #354

ppwwyyxx · 2019-12-25T19:25:00Z

The original code cannot run under latest numpy 1.18:

╰─$ np.linspace(.5, 0.95, np.round((0.95 - .5) / .05) + 1, endpoint=True)                                          
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
~/.local/lib/python3.8/site-packages/numpy/core/function_base.py in linspace(start, stop, num, endpoint, retstep, dtype, axis)
    116     try:
--> 117         num = operator.index(num)
    118     except TypeError:

TypeError: 'numpy.float64' object cannot be interpreted as an integer

During handling of the above exception, another exception occurred:

TypeError                                 Traceback (most recent call last)
<ipython-input-2-62ed634c156c> in <module>
----> 1 np.linspace(.5, 0.95, np.round((0.95 - .5) / .05) + 1, endpoint=True)

<__array_function__ internals> in linspace(*args, **kwargs)

~/.local/lib/python3.8/site-packages/numpy/core/function_base.py in linspace(start, stop, num, endpoint, retstep, dtype, axis)
    117         num = operator.index(num)
    118     except TypeError:
--> 119         raise TypeError(
    120             "object of type {} cannot be safely interpreted as an integer."
    121                 .format(type(num)))

TypeError: object of type <class 'numpy.float64'> cannot be safely interpreted as an integer.

This fix addresses this issue. The issue is originally reported at facebookresearch/detectron2#580

cc @rbgirshick @agrimgupta92: lvis seems to have the same issue

WarrenWeckesser · 2019-12-27T21:24:37Z

The change looks good. Accepting a floating point value for the num argument of numpy.linspace has been deprecated since NumPy 1.12. That deprecation has been converted to an error in NumPy 1.18, so anyone using cocoapi with NumPy 1.18 will get an error such as the one reported here or at numpy/numpy#15192.

eric-wieser · 2019-12-27T21:31:41Z

PythonAPI/pycocotools/cocoeval.py

@@ -503,8 +503,8 @@ def setDetParams(self):
        self.imgIds = []
        self.catIds = []
        # np.arange causes trouble.  the data point on arange is slightly larger than the true value
-        self.iouThrs = np.linspace(.5, 0.95, np.round((0.95 - .5) / .05) + 1, endpoint=True)
-        self.recThrs = np.linspace(.0, 1.00, np.round((1.00 - .0) / .01) + 1, endpoint=True)
+        self.iouThrs = np.linspace(.5, 0.95, int(np.round((0.95 - .5) / .05)) + 1, endpoint=True)


Tempting to rewrite this as

Suggested change

self.iouThrs = np.linspace(.5, 0.95, int(np.round((0.95 - .5) / .05)) + 1, endpoint=True)

self.iouThrs = np.arange(10, 19+1) / 20

Read as "between 10/20 and 19/20, inclusive"

Or If you prefer hundredths:

Suggested change

self.iouThrs = np.linspace(.5, 0.95, int(np.round((0.95 - .5) / .05)) + 1, endpoint=True)

self.iouThrs = np.arange(50, 95+1, 5) / 100

Did you mean np.arange(10, 19+1) / 20?

Other than that, I agree with your suggestion to use some form of arange instead of the floating point shenanigans in np.round((0.95 - .5) / .05).

Fixed the typo above, also for the hundredths case - good catch.

Best use floats in the division for Python 2.7 compatibility.

CDahmsCellarEye · 2019-12-27T22:15:53Z

Somebody please integrate this ASAP!! Until this change is merged, pycocotools will cause a crash for both TensorFlow and PyTorch object detection during the evaluation phase of training!!

It took me many hours of troubleshooting to work out that backdating to Numpy 1.17.4 was the only currently available resolution.

See:

https://stackoverflow.com/questions/59493606/pytorch-and-tensorflow-object-detection-evaluate-object-of-type-class-nump

pytorch/vision#1700

numpy/numpy#15192

eaidova · 2020-02-03T12:29:12Z

@pdollar do you plan update pycocotools package on pypi (https://pypi.org/project/pycocotools/)?
Because this version still contains numpy1.18 incompatible code.

aCuria · 2020-03-03T03:07:14Z

Has this been integrated? The pytorch tutorials dont work and I am wondering which version of pycocotools to upgrade to

CDahmsCellarEye · 2020-03-03T17:35:54Z

Has this been integrated? The pytorch tutorials dont work and I am wondering which version of pycocotools to upgrade to

For the moment the easiest solution is to make everything current, except numpy, which can be backdated to 1.17.4, see my post here for more info:

https://stackoverflow.com/questions/59493606/pytorch-and-tensorflow-object-detection-evaluate-object-of-type-class-nump

aCuria · 2020-03-04T03:01:14Z

What is preventing cocoapi from being fixed? I will downgrade numpy for now, but Numpy 1.18 was released months ago

CDahmsCellarEye · 2020-03-04T03:15:48Z

What is preventing cocoapi from being fixed? I will downgrade numpy for now, but Numpy 1.18 was released months ago

The pull request to cocoapi was merged in, but it seems the apt-get package has not been updated per this change so you still have to back to Numpy 1.17.4 for the moment.

ppwwyyxx · 2020-03-04T03:21:20Z

It was already fixed two months ago. No one should use cocoapi from apt-get or pip install or conda because cocoapi was never officially released on those places. The correct way to install cocoapi is pip install cython; pip install -U 'git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI'.

UPDATE: pip install pycocotools becomes official since Jun, 2020

CDahmsCellarEye · 2020-03-04T03:57:26Z

It was already fixed two months ago. No one should use cocoapi from apt-get or pip install or conda because cocoapi was never officially released on those places. The correct way to install cocoapi is pip install cython; pip install -U 'git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI'

The reality of the situation is that most people will use pip packages (I meant to say pip rather than apt-get) rather than installing from a git repo.

aCuria · 2020-03-04T06:19:05Z

@CDahmsCellarEye This didnt work for me,
The following line

pip install -U "git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI"

Failed with the error

``cl : Command line error D8021 : invalid numeric argument '/Wno-cpp'`

ppwwyyxx · 2020-03-04T07:05:03Z

@aCuria google can tell that your issue is #51

aCuria · 2020-03-04T10:08:02Z

I did see #51 but that's a workaround for a bug unfixed since 2017. All other libraries I have seen so far work with pip install .... and don't require you to download / modify source before it would run. I don't usually use python as much but its been harder to get pycocotools to work than even C/C++ libraries so far.

Michael-J98 · 2020-04-05T15:16:37Z

Has this already been fixed? I still get the same error.
I install it on colab by lines:
"
git clone https://github.com/cocodataset/cocoapi.git
cd cocoapi/PythonAPI
python setup.py build_ext install
".
Anyone can help me?

glenn-jocher · 2020-05-16T03:48:49Z

I still get this error in May 2020. pycocotools with numpy 18.4:

loading annotations into memory...
Done (t=0.38s)
creating index...
index created!
Loading and preparing results...
DONE (t=5.31s)
creating index...
index created!
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/numpy/core/function_base.py", line 117, in linspace
    num = operator.index(num)
TypeError: 'numpy.float64' object cannot be interpreted as an integer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "test.py", line 257, in <module>
    opt.augment)
  File "test.py", line 214, in test
    cocoEval = COCOeval(cocoGt, cocoDt, 'bbox')
  File "/usr/local/lib/python3.6/dist-packages/pycocotools/cocoeval.py", line 76, in __init__
    self.params = Params(iouType=iouType) # parameters
  File "/usr/local/lib/python3.6/dist-packages/pycocotools/cocoeval.py", line 527, in __init__
    self.setDetParams()
  File "/usr/local/lib/python3.6/dist-packages/pycocotools/cocoeval.py", line 507, in setDetParams
    self.iouThrs = np.linspace(.5, 0.95, np.round((0.95 - .5) / .05) + 1, endpoint=True)
  File "<__array_function__ internals>", line 6, in linspace
  File "/usr/local/lib/python3.6/dist-packages/numpy/core/function_base.py", line 121, in linspace
    .format(type(num)))
TypeError: object of type <class 'numpy.float64'> cannot be safely interpreted as an integer.

AKMourato · 2021-08-23T15:50:24Z

@ppwwyyxx I've already gathered that. The problem is that I've installed the latest version of pycocotools from pip and source and I'm still getting this error on numpy 1.20. Any advice?

ppwwyyxx · 2021-08-23T17:26:18Z

Most likely you're not using what you "installed". See https://ppwwyyxx.com/blog/2019/On-Environment-Packaging-in-Python/ for some possible reasons.

Fix the use of linspace for numpy 1.18

6c3b394

WarrenWeckesser mentioned this pull request Dec 27, 2019

change in numpy 1.18.0 causes error with TensorFlow or PyTorch object detection evaluation numpy/numpy#15192

Closed

eric-wieser reviewed Dec 27, 2019

View reviewed changes

ppwwyyxx mentioned this pull request Jan 3, 2020

Explicitly change type to int for np.linspace #357

Open

pdollar merged commit e140a08 into cocodataset:master Jan 3, 2020

fmassa mentioned this pull request Jan 8, 2020

PyTorch object detection - evaluate function call - object of type <class 'numpy.float64'> cannot be safely interpreted as an integer pytorch/vision#1700

Closed

This was referenced Jan 22, 2020

Fixed numpy.float64 cannot be safely interpreted as an integer issue philferriere/cocoapi#8

Closed

Fix numpy.float64 cannot be safely interpreted as an integer issue philferriere/cocoapi#9

Open

heavengate mentioned this pull request Feb 18, 2020

object of type <class 'numpy.float64'> cannot be safely interpreted as an integer. PaddlePaddle/PaddleDetection#245

Closed

peterbell10 mentioned this pull request Mar 2, 2020

Crash in torchvision tutorial sample code pytorch/pytorch#33535

Closed

fmassa mentioned this pull request Mar 26, 2020

torchvision_finetuning_instance_segmentation notebook error pytorch/vision#2014

Closed

michaelisc mentioned this pull request Apr 2, 2020

Fix linspace for numpy 1.18 lvis-dataset/lvis-api#12

Merged

This was referenced Apr 10, 2020

TorchVision Object Detection finetuning tutorial pytorch/vision#2088

Closed

Fix pycocotools version in tutorial pytorch/vision#2090

Merged

ppwwyyxx deleted the patch-1 branch May 16, 2020 03:50

glenn-jocher mentioned this pull request May 18, 2020

Pycocotools incompatable with numpy 1.18 googlecolab/colabtools#1260

Closed

This was referenced May 28, 2020

TypeError in cocoeval.py #402

Open

cocoeval.py: np.linspace issues with numpy 1.18.0+ and Python3.5+ #356

Open

sraimund mentioned this pull request Jun 2, 2020

object of type <class 'numpy.float64'> cannot be safely interpreted as an integer tensorflow/models#8576

Closed

ppwwyyxx mentioned this pull request Jun 9, 2020

AttributeError: 'COCO' object has no attribute 'get_cat_ids' open-mmlab/mmdetection#2913

Closed

xkzy mentioned this pull request Jul 3, 2020

fixes according to the numpy update 1.18.1 philferriere/cocoapi#10

Open

ppwwyyxx mentioned this pull request Aug 22, 2021

TypeError: 'numpy.float64' object cannot be interpreted as an integer [np == 1.20.1] facebookresearch/detectron2#3392

Closed

rbavery mentioned this pull request Nov 18, 2021

numpy 1.18 change breaks pycocotools waspinator/pycococreator#42

Open

xuzhao9 mentioned this pull request Jan 26, 2022

Remove the numpy version requirement pytorch/benchmark#715

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix the use of linspace for numpy 1.18 #354

Fix the use of linspace for numpy 1.18 #354

ppwwyyxx commented Dec 25, 2019

WarrenWeckesser commented Dec 27, 2019

eric-wieser Dec 27, 2019 •

edited

Loading

WarrenWeckesser Dec 27, 2019

WarrenWeckesser Dec 27, 2019

eric-wieser Dec 27, 2019

charris Dec 28, 2019

CDahmsCellarEye commented Dec 27, 2019

eaidova commented Feb 3, 2020

aCuria commented Mar 3, 2020 •

edited

Loading

CDahmsCellarEye commented Mar 3, 2020

aCuria commented Mar 4, 2020 •

edited

Loading

CDahmsCellarEye commented Mar 4, 2020

ppwwyyxx commented Mar 4, 2020 •

edited

Loading

CDahmsCellarEye commented Mar 4, 2020

aCuria commented Mar 4, 2020 •

edited

Loading

ppwwyyxx commented Mar 4, 2020 •

edited

Loading

aCuria commented Mar 4, 2020

Michael-J98 commented Apr 5, 2020

glenn-jocher commented May 16, 2020

AKMourato commented Aug 23, 2021

ppwwyyxx commented Aug 23, 2021

	self.iouThrs = np.linspace(.5, 0.95, int(np.round((0.95 - .5) / .05)) + 1, endpoint=True)
	self.iouThrs = np.arange(10, 19+1) / 20

	self.iouThrs = np.linspace(.5, 0.95, int(np.round((0.95 - .5) / .05)) + 1, endpoint=True)
	self.iouThrs = np.arange(50, 95+1, 5) / 100

Fix the use of linspace for numpy 1.18 #354

Fix the use of linspace for numpy 1.18 #354

Conversation

ppwwyyxx commented Dec 25, 2019

WarrenWeckesser commented Dec 27, 2019

eric-wieser Dec 27, 2019 • edited Loading

Choose a reason for hiding this comment

WarrenWeckesser Dec 27, 2019

Choose a reason for hiding this comment

WarrenWeckesser Dec 27, 2019

Choose a reason for hiding this comment

eric-wieser Dec 27, 2019

Choose a reason for hiding this comment

charris Dec 28, 2019

Choose a reason for hiding this comment

CDahmsCellarEye commented Dec 27, 2019

eaidova commented Feb 3, 2020

aCuria commented Mar 3, 2020 • edited Loading

CDahmsCellarEye commented Mar 3, 2020

aCuria commented Mar 4, 2020 • edited Loading

CDahmsCellarEye commented Mar 4, 2020

ppwwyyxx commented Mar 4, 2020 • edited Loading

CDahmsCellarEye commented Mar 4, 2020

aCuria commented Mar 4, 2020 • edited Loading

ppwwyyxx commented Mar 4, 2020 • edited Loading

aCuria commented Mar 4, 2020

Michael-J98 commented Apr 5, 2020

glenn-jocher commented May 16, 2020

AKMourato commented Aug 23, 2021

ppwwyyxx commented Aug 23, 2021

eric-wieser Dec 27, 2019 •

edited

Loading

aCuria commented Mar 3, 2020 •

edited

Loading

aCuria commented Mar 4, 2020 •

edited

Loading

ppwwyyxx commented Mar 4, 2020 •

edited

Loading

aCuria commented Mar 4, 2020 •

edited

Loading

ppwwyyxx commented Mar 4, 2020 •

edited

Loading