Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] xr.merge converts automatically variables into float64 #3969

Closed
clausmichele opened this issue Apr 14, 2020 · 2 comments
Closed

[BUG] xr.merge converts automatically variables into float64 #3969

clausmichele opened this issue Apr 14, 2020 · 2 comments

Comments

@clausmichele
Copy link
Contributor

clausmichele commented Apr 14, 2020

Merging two datasets using xr.merge converts all the variables to float64 type.

MCVE Code Sample

import numpy as np
import xarray as xr

x = np.arange(0,10)
y = np.arange(0,10)
time = [0,1]
data = np.zeros((10,10), dtype=bool)
dataArray1 = xr.DataArray([data], coords={'time': [time[0]], 'y': y, 'x': x},
                             dims=['time', 'y', 'x'])
dataArray2 = xr.DataArray([data], coords={'time': [time[1]], 'y': y, 'x': x},
                             dims=['time', 'y', 'x'])
dataArray1 = dataArray1.to_dataset(name='data')
dataArray2 = dataArray2.to_dataset(name='data')
xr.merge([dataArray1,dataArray2])

Current Output

<xarray.Dataset>
Dimensions:  (time: 2, x: 10, y: 10)
Coordinates:
  * time     (time) int64 0 1
  * y        (y) int64 0 1 2 3 4 5 6 7 8 9
  * x        (x) int64 0 1 2 3 4 5 6 7 8 9
Data variables:
    data     (time, y, x) float64 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0 0.0

Expected Output

<xarray.Dataset>
Dimensions:  (time: 2, x: 10, y: 10)
Coordinates:
  * time     (time) int64 0 1
  * y        (y) int64 0 1 2 3 4 5 6 7 8 9
  * x        (x) int64 0 1 2 3 4 5 6 7 8 9
Data variables:
    data     (time, y, x) bool False False False False False ... False False False False False

Problem Description

The merge function should not convert data types into float64.
In this case is increasing the memory usage compared to what is expected.

Versions

INSTALLED VERSIONS ------------------ commit: None python: 3.6.8 (default, May 7 2019, 14:58:50) [GCC 8.3.0] python-bits: 64 OS: Linux OS-release: 4.15.0-88-generic machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: C.UTF-8 LOCALE: en_US.UTF-8 libhdf5: None libnetcdf: None

xarray: 0.15.1
pandas: 1.0.3
numpy: 1.18.2
scipy: None
netCDF4: None
pydap: None
h5netcdf: None
h5py: None
Nio: None
zarr: None
cftime: None
nc_time_axis: None
PseudoNetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: None
dask: None
distributed: None
matplotlib: 3.2.0
cartopy: None
seaborn: None
numbagg: None
setuptools: 46.1.3
pip: 9.0.1
conda: None
pytest: None
IPython: 7.13.0
sphinx: 2.4.3

@kmuehlbauer
Copy link
Contributor

@clausmichele According to the docs on combining xr.concat should be used for this example.

xr.concat([dataArray1,dataArray2], dim='time')
<xarray.Dataset>
Dimensions:  (time: 2, x: 10, y: 10)
Coordinates:
  * y        (y) int64 0 1 2 3 4 5 6 7 8 9
  * x        (x) int64 0 1 2 3 4 5 6 7 8 9
  * time     (time) int64 0 1
Data variables:
    data     (time, y, x) bool False False False False ... False False False

@clausmichele
Copy link
Contributor Author

I've just noticed that using xr.concat inverts the coordinates order, I opened a new issue #4072

@dcherian dcherian closed this as completed Dec 7, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants