addcyclic giving error on map + list

[xylar]

The following simple test:

#!/usr/bin/env python

from mpl_toolkits.basemap import addcyclic
import numpy

lon = numpy.arange(0., 360., 0.5)
lat = numpy.arange(-90., 90.5, 0.5)

Lon, Lat = numpy.meshgrid(lon, lat)

field = numpy.sin(numpy.deg2rad(Lat))

fieldCyc, latCyc, lonCyc = addcyclic(field, Lat, Lon)

produces this error:

  File "./test_addcyclic.py", line 13, in <module>
    fieldCyc, lonCyc = addcyclic(field, Lon)
  File "/home/xylar/miniconda3/envs/mpas_analysis_py3/lib/python3.6/site-packages/mpl_toolkits/basemap/__init__.py", line 5111, in addcyclic
    return list(map(_addcyclic,arr[:-1]) + [_addcyclic_lon(arr[-1])])
TypeError: unsupported operand type(s) for +: 'map' and 'list'

Details:

My conda environment is as follows:

# packages in environment at /home/xylar/miniconda3/envs/mpas_analysis_py3:
#
# Name                    Version                   Build  Channel
asn1crypto                0.24.0                py36_1003    conda-forge
atomicwrites              1.2.1                      py_0    conda-forge
attrs                     18.2.0                     py_0    conda-forge
basemap                   1.2.0            py36h50ae964_0    conda-forge
blas                      1.1                    openblas    conda-forge
bokeh                     1.0.1                 py36_1000    conda-forge
bottleneck                1.2.1            py36h7eb728f_1    conda-forge
bzip2                     1.0.6                h470a237_2    conda-forge
ca-certificates           2018.10.15           ha4d7672_0    conda-forge
certifi                   2018.10.15            py36_1000    conda-forge
cffi                      1.11.5           py36h5e8e0c9_1    conda-forge
cftime                    1.0.2.1          py36h7eb728f_0    conda-forge
chardet                   3.0.4                 py36_1003    conda-forge
click                     7.0                        py_0    conda-forge
cloudpickle               0.6.1                      py_0    conda-forge
cmocean                   1.2                        py_0    conda-forge
cryptography              2.3.1            py36hdffb7b8_0    conda-forge
cryptography-vectors      2.3.1                 py36_1000    conda-forge
curl                      7.62.0               h74213dd_0    conda-forge
cycler                    0.10.0                     py_1    conda-forge
cytoolz                   0.9.0.1          py36h470a237_1    conda-forge
dask                      0.20.0                     py_0    conda-forge
dask-core                 0.20.0                     py_0    conda-forge
dbus                      1.13.0               h3a4f0e9_0    conda-forge
distributed               1.24.0                py36_1000    conda-forge
esmf                      7.1.0r               h35eb876_3    conda-forge
expat                     2.2.5                hfc679d8_2    conda-forge
fontconfig                2.13.1               h65d0f4c_0    conda-forge
freetype                  2.9.1                h6debe1e_4    conda-forge
geos                      3.6.2                hfc679d8_3    conda-forge
gettext                   0.19.8.1             h5e8e0c9_1    conda-forge
glib                      2.56.2               h464dc38_1    conda-forge
gsl                       2.2.1           blas_openblashddceaf2_6  [blas_openblas]  conda-forge
gst-plugins-base          1.12.5               hde13a9d_0    conda-forge
gstreamer                 1.12.5               h5856ed1_0    conda-forge
h5netcdf                  0.6.2                      py_0    conda-forge
h5py                      2.8.0            py36h097b052_4    conda-forge
hdf4                      4.2.13               h951d187_2    conda-forge
hdf5                      1.10.3               hc401514_2    conda-forge
heapdict                  1.0.0                 py36_1000    conda-forge
icu                       58.2                 hfc679d8_0    conda-forge
idna                      2.7                   py36_1002    conda-forge
jinja2                    2.10                       py_1    conda-forge
jpeg                      9c                   h470a237_1    conda-forge
kiwisolver                1.0.1            py36h2d50403_2    conda-forge
krb5                      1.16.2               hbb41f41_0    conda-forge
libcurl                   7.62.0               hbdb9355_0    conda-forge
libedit                   3.1.20170329         haf1bffa_1    conda-forge
libffi                    3.2.1                hfc679d8_5    conda-forge
libgcc-ng                 7.2.0                hdf63c60_3    conda-forge
libgfortran               3.0.0                         1    conda-forge
libiconv                  1.15                 h470a237_3    conda-forge
libnetcdf                 4.6.1               h9cd6fdc_11    conda-forge
libpng                    1.6.35               ha92aebf_2    conda-forge
libssh2                   1.8.0                h5b517e9_2    conda-forge
libstdcxx-ng              7.2.0                hdf63c60_3    conda-forge
libtiff                   4.0.9                he6b73bb_2    conda-forge
libuuid                   2.32.1               h470a237_2    conda-forge
libxcb                    1.13                 h470a237_2    conda-forge
libxml2                   2.9.8                h422b904_5    conda-forge
libxslt                   1.1.32               h88dbc4e_2    conda-forge
locket                    0.2.0                      py_2    conda-forge
lxml                      4.2.5            py36hc9114bc_0    conda-forge
markupsafe                1.1.0            py36h470a237_0    conda-forge
matplotlib                2.2.3            py36h8e2386c_0    conda-forge
more-itertools            4.3.0                 py36_1000    conda-forge
mpi                       1.0                       mpich    conda-forge
mpich                     3.2.1                h26a2512_5    conda-forge
msgpack-python            0.5.6            py36h2d50403_3    conda-forge
nco                       4.7.7                he007dc3_1    conda-forge
ncurses                   6.1                  hfc679d8_1    conda-forge
netcdf-fortran            4.4.4               h71ea97b_10    conda-forge
netcdf4                   1.4.2            py36hac939d9_0    conda-forge
numpy                     1.15.4          py36_blas_openblashb06ca3d_0  [blas_openblas]  conda-forge
olefile                   0.46                       py_0    conda-forge
openblas                  0.3.3                ha44fe06_1    conda-forge
openssl                   1.0.2p               h470a237_1    conda-forge
packaging                 18.0                       py_0    conda-forge
pandas                    0.23.4           py36hf8a1672_0    conda-forge
partd                     0.3.9                      py_0    conda-forge
pcre                      8.41                 hfc679d8_3    conda-forge
pillow                    5.3.0            py36hc736899_0    conda-forge
pip                       18.1                  py36_1000    conda-forge
pluggy                    0.8.0                      py_0    conda-forge
progressbar2              3.38.0                     py_1    conda-forge
proj4                     4.9.3                h470a237_8    conda-forge
psutil                    5.4.8            py36h470a237_0    conda-forge
pthread-stubs             0.4                  h470a237_1    conda-forge
py                        1.7.0                      py_0    conda-forge
pycparser                 2.19                       py_0    conda-forge
pyopenssl                 18.0.0                py36_1000    conda-forge
pyparsing                 2.3.0                      py_0    conda-forge
pyproj                    1.9.5.1          py36h508ed2a_6    conda-forge
pyqt                      5.6.0            py36h8210e8a_7    conda-forge
pyshp                     2.0.0                      py_0    conda-forge
pysocks                   1.6.8                 py36_1002    conda-forge
pytest                    3.10.0                py36_1000    conda-forge
pytest-runner             4.2                        py_1    conda-forge
python                    3.6.6                h5001a0f_3    conda-forge
python-dateutil           2.7.5                      py_0    conda-forge
python-utils              2.3.0                      py_1    conda-forge
pytz                      2018.7                     py_0    conda-forge
pyyaml                    3.13             py36h470a237_1    conda-forge
qt                        5.6.2                hf70d934_9    conda-forge
readline                  7.0                  haf1bffa_1    conda-forge
requests                  2.20.0                py36_1000    conda-forge
scipy                     1.1.0           py36_blas_openblashb06ca3d_202  [blas_openblas]  conda-forge
setuptools                40.5.0                   py36_0    conda-forge
sip                       4.18.1           py36hfc679d8_0    conda-forge
six                       1.11.0                py36_1001    conda-forge
sortedcontainers          2.0.5                      py_0    conda-forge
sqlite                    3.25.3               hb1c47c0_0    conda-forge
tblib                     1.3.2                      py_1    conda-forge
tk                        8.6.8                ha92aebf_0    conda-forge
toolz                     0.9.0                      py_1    conda-forge
tornado                   5.1.1            py36h470a237_0    conda-forge
udunits2                  2.2.27.6             h3a4f0e9_1    conda-forge
urllib3                   1.23                  py36_1001    conda-forge
wheel                     0.32.2                   py36_0    conda-forge
xarray                    0.10.9                   py36_0    conda-forge
xorg-libxau               1.0.8                h470a237_6    conda-forge
xorg-libxdmcp             1.1.2                h470a237_7    conda-forge
xz                        5.2.4                h470a237_1    conda-forge
yaml                      0.1.7                h470a237_1    conda-forge
zict                      0.1.3                      py_0    conda-forge
zlib                      1.2.11               h470a237_3    conda-forge

The same test works fine for this environment:

# packages in environment at /home/xylar/miniconda3/envs/mpas_analysis:
#
# Name                    Version                   Build  Channel
alabaster                 0.7.10                   py36_1    conda-forge
asn1crypto                0.24.0                   py36_0    conda-forge
attrs                     18.1.0                     py_0    conda-forge
babel                     2.5.3                    py36_0    conda-forge
basemap                   1.1.0                    py36_4    conda-forge
blas                      1.0                         mkl  
bokeh                     0.12.16                  py36_0    conda-forge
bottleneck                1.2.1                    py36_1    conda-forge
ca-certificates           2018.4.16                     0    conda-forge
certifi                   2018.4.16                py36_0    conda-forge
cffi                      1.11.5                   py36_0    conda-forge
cftime                    1.0.0                    py36_0    conda-forge
chardet                   3.0.4                    py36_0    conda-forge
click                     6.7                        py_1    conda-forge
cloudpickle               0.5.3                      py_0    conda-forge
cmocean                   1.1                        py_0    conda-forge
commonmark                0.5.4                      py_1    conda-forge
cryptography              2.2.1                    py36_0    conda-forge
curl                      7.59.0                        1    conda-forge
cycler                    0.10.0                   py36_0    conda-forge
cytoolz                   0.9.0.1                  py36_0    conda-forge
dask                      0.17.5                     py_0    conda-forge
dask-core                 0.17.5                     py_0    conda-forge
dbus                      1.11.0                        0    conda-forge
distributed               1.21.8                   py36_0    conda-forge
docutils                  0.14                     py36_0    conda-forge
esmf                      7.1.0r                        1    conda-forge
expat                     2.2.5                         0    conda-forge
fontconfig                2.12.6                        0    conda-forge
freetype                  2.8.1                         0    conda-forge
future                    0.16.0                   py36_0    conda-forge
geos                      3.6.2                         1    conda-forge
gettext                   0.19.8.1                      0    conda-forge
glib                      2.55.0                        0    conda-forge
gsl                       2.2.1                h0c605f7_3  
gst-plugins-base          1.8.0                         0    conda-forge
gstreamer                 1.8.0                         1    conda-forge
gsw                       3.2.1                    py36_1    conda-forge
h5netcdf                  0.5.1                      py_0    conda-forge
h5py                      2.8.0            py36h470a237_0    conda-forge
hdf4                      4.2.13                        0    conda-forge
hdf5                      1.10.1                        2    conda-forge
heapdict                  1.0.0                    py36_0    conda-forge
icu                       58.2                          0    conda-forge
idna                      2.6                      py36_1    conda-forge
imagesize                 1.0.0                    py36_0    conda-forge
intel-openmp              2018.0.0                      8  
jinja2                    2.10                     py36_0    conda-forge
jpeg                      9b                            2    conda-forge
kiwisolver                1.0.1                    py36_1    conda-forge
krb5                      1.14.6                        0    conda-forge
libffi                    3.2.1                         3    conda-forge
libgcc                    7.2.0                h69d50b8_2  
libgcc-ng                 7.2.0                hdf63c60_3  
libgfortran               3.0.0                         1  
libgfortran-ng            7.2.0                hdf63c60_3  
libiconv                  1.15                          0    conda-forge
libnetcdf                 4.6.1                         2    conda-forge
libpng                    1.6.34                        0    conda-forge
libssh2                   1.8.0                         2    conda-forge
libstdcxx-ng              7.2.0                hdf63c60_3  
libtiff                   4.0.9                         0    conda-forge
libxcb                    1.13                          0    conda-forge
libxml2                   2.9.8                         0    conda-forge
libxslt                   1.1.32                        0    conda-forge
locket                    0.2.0                    py36_1    conda-forge
lxml                      4.2.1                    py36_0    conda-forge
markupsafe                1.0                      py36_0    conda-forge
matplotlib                2.2.2                    py36_1    conda-forge
mkl                       2018.0.2                      1  
mkl_fft                   1.0.2                    py36_0    conda-forge
mkl_random                1.0.1                    py36_0    conda-forge
more-itertools            4.1.0                      py_0    conda-forge
mpi                       1.0                       mpich    conda-forge
mpich                     3.2.1                         0    conda-forge
msgpack-python            0.5.6            py36h2d50403_2    conda-forge
nco                       4.7.4                         1    conda-forge
ncurses                   5.9                          10    conda-forge
netcdf-fortran            4.4.4                         7    conda-forge
netcdf4                   1.4.0                    py36_0    conda-forge
numpy                     1.14.2           py36hdbf6ddf_1  
numpydoc                  0.8.0                    py36_0    conda-forge
olefile                   0.45.1                   py36_0    conda-forge
openblas                  0.2.20                        7    conda-forge
openssl                   1.0.2o                        0    conda-forge
packaging                 17.1                       py_0    conda-forge
pandas                    0.23.0                   py36_1    conda-forge
partd                     0.3.8                    py36_0    conda-forge
pcre                      8.41                          1    conda-forge
pillow                    5.1.0                    py36_0    conda-forge
pip                       9.0.3                    py36_0    conda-forge
pluggy                    0.6.0                      py_0    conda-forge
progressbar2              3.37.0                   py36_0    conda-forge
psutil                    5.4.5                    py36_0    conda-forge
py                        1.5.3                      py_0    conda-forge
pycparser                 2.18                     py36_0    conda-forge
pygments                  2.2.0                    py36_0    conda-forge
pyopenssl                 18.0.0                   py36_0    conda-forge
pyparsing                 2.2.0                    py36_0    conda-forge
pyproj                    1.9.5.1                  py36_0    conda-forge
pyqt                      5.6.0                    py36_5    conda-forge
pyshp                     1.2.12                     py_0    conda-forge
pysocks                   1.6.8                    py36_1    conda-forge
pytest                    3.5.1                    py36_0    conda-forge
pytest-runner             4.2                        py_0    conda-forge
python                    3.6.5                         1    conda-forge
python-dateutil           2.7.3                      py_0    conda-forge
python-utils              2.3.0                    py36_0    conda-forge
pytz                      2018.4                     py_0    conda-forge
pyyaml                    3.12                     py36_1    conda-forge
qt                        5.6.2                         7    conda-forge
readline                  7.0                           0    conda-forge
recommonmark              0.4.0                      py_1    conda-forge
requests                  2.18.4                   py36_1    conda-forge
scipy                     1.1.0            py36hfc37229_0  
setuptools                39.2.0                   py36_0    conda-forge
sip                       4.18                     py36_1    conda-forge
six                       1.11.0                   py36_1    conda-forge
snowballstemmer           1.2.1                    py36_0    conda-forge
sortedcontainers          1.5.10                   py36_0    conda-forge
sphinx                    1.7.4                    py36_0    conda-forge
sphinx_rtd_theme          0.3.1                    py36_0    conda-forge
sphinxcontrib-websupport  1.0.1                    py36_0    conda-forge
sqlite                    3.20.1                        2    conda-forge
tabulate                  0.8.2                      py_0    conda-forge
tblib                     1.3.2                    py36_0    conda-forge
tk                        8.6.7                         0    conda-forge
toolz                     0.9.0                      py_0    conda-forge
tornado                   5.0.2                    py36_0    conda-forge
typing                    3.6.4                    py36_0    conda-forge
udunits2                  2.2.26                        0    conda-forge
urllib3                   1.22                     py36_0    conda-forge
wheel                     0.31.0                   py36_0    conda-forge
xarray                    0.10.4                   py36_0    conda-forge
xorg-libxau               1.0.8                         3    conda-forge
xorg-libxdmcp             1.1.2                         3    conda-forge
xz                        5.2.3                         0    conda-forge
yaml                      0.1.7                         0    conda-forge
zict                      0.1.3                      py_0    conda-forge
zlib                      1.2.11                        0    conda-forge

So far, I haven't been able to figure out which package change is responsible, but I will investigate further.

[xylar]

@ocefpaf, the line giving the error is one you edited 7 months ago. I don't think that edit is responsible for the problem but maybe you would have an idea of what's causing this error.

[xylar]

Hmm, I tested with simple conda environments where I installed only python=3.6, numpy (latest) and either basemap 1.2.0 and 1.1.0. Things seem to have broken in 1.2.0, so it does seem like the change from 7 months ago may actually be responsible.

[xylar]

I think the fix is that
https://github.com/matplotlib/basemap/blob/master/lib/mpl_toolkits/basemap/__init__.py#L5102
should be:

return list(map(_addcyclic,arr[:-1])) + [_addcyclic_lon(arr[-1])]

That is, the map should be converted to a list and then added to the other list, rather than the map being added to a list and then converted to a list.

I'll make a PR for this as soon as I'm able.

[ocefpaf]

That is, the map should be converted to a list and then added to the other list, rather than the map being added to a list and then converted to a list.

Yep. I probably botched that in the 2to3 removal. Do you want to send a PR fixing it?

[xylar]

Yes, I'll do that soon. Basemap takes awhile to clone...

[davidmnielsen]

This fix leads me to another error:

TypeError: invalid indexer array, does not have integer dtype: array([slice(None, None, None), slice(0, 1, None)], dtype=object)

Could you please give me a hand on this?

[davidmnielsen]

The error is reproducible by following the steps posted in here:

https://github.com/davidmnielsen/polarplots.git

[xylar]

@davidmnielsen, I think you should probably open a separate issue for your error, since this one has been closed. I don't have the expertise to help you with your error myself, I'm afraid.

[severin1992]

Any news on that? I encounter the same issue:

<ipython-input-13-e5896fefedc7> in <module>
     29 
     30 # Make the plot continuous
---> 31 temp_cyclic, lons_cyclic = addcyclic(tTemp, lons)
     32 # Shift the grid so lons go from -180 to 180 instead of 0 to 360.
     33 #air_cyclic, lons_cyclic = shiftgrid(180., air_cyclic, lons_cyclic, start=False)

~/anaconda3/envs/AirPred/lib/python3.6/site-packages/mpl_toolkits/basemap/__init__.py in addcyclic(*arr, **kwargs)
   5109         return _addcyclic_lon(arr[-1])
   5110     else:
-> 5111         return list(map(_addcyclic,arr[:-1]) + [_addcyclic_lon(arr[-1])])
   5112 
   5113 def _choosecorners(width,height,**kwargs):

TypeError: unsupported operand type(s) for +: 'map' and 'list'

After trying:

my_example_nc_file = '/Users/Severin/Desktop/ecmwf_era5_17010100.nc'
#fh becomes the file handle of the open netCDF file
fh = Dataset(my_example_nc_file, mode='r') 

import matplotlib.pyplot as plt
from mpl_toolkits.basemap import Basemap, addcyclic, shiftgrid
import matplotlib.cm as cm
import numpy as np

lons = fh.variables['lon'][:]
lats = fh.variables['lat'][:]
tTemp = fh.variables['T2'][0,:,:]

fh.close()

fig=plt.figure(figsize=(12, 8) )

m=Basemap(projection='mill', lat_ts=10, llcrnrlon=lons.min(), \
  urcrnrlon=lons.max(),llcrnrlat=lats.min(),urcrnrlat=lats.max(), \
  resolution='c')

m.drawcoastlines()
m.drawmapboundary()

# Make the plot continuous
temp_cyclic, lons_cyclic = addcyclic(tTemp, lons)

FYI:
basemap 1.2.0 py36h0acbc05_0
matplotlib 3.0.2 py36h54f8f79_0

conda update basemap doesn't ensure that the according line is changed to

return list(map(_addcyclic,arr[:-1])) + [_addcyclic_lon(arr[-1])]

I don't know whether that's a smart thing to do, but I just changed the line by myself. Now I got this message:

/Users/Severin/anaconda3/envs/AirPred/lib/python3.6/site-packages/numpy/ma/core.py:3174: 
FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; 
use `arr[tuple(seq)]` instead of `arr[seq]`. 
In the future this will be interpreted as an array index, `arr[np.array(seq)]`, 
which will result either in an error or a different result.
  dout = self.data[indx] ```

[xylar]

This issue has been fixed but there hasn't been a new release of basemap since then. @WeatherGod, is there any plan for a release anytime soon?

[WeatherGod]

I am sorry, I hadn't realized that this critical fix was unreleased. I'll tag a new release now.

[samayamadhavan]

Hi, can you tell me what the fix to this issue is?

[matheusdpv]

Any news about how to fix it?

[xylar]

I believe this was fixed in the last release (0.18.0). We have been using https://scitools.org.uk/cartopy/docs/latest/cartopy/util/util.html#cartopy.util.add_cyclic_point successfully since that release. For example:
https://github.com/MPAS-Dev/MPAS-Analysis/blob/develop/mpas_analysis/shared/plot/climatology_map.py#L144