Skip to content
This repository was archived by the owner on Aug 29, 2023. It is now read-only.

Add time dim in open_xarray_dataset() and use file chunking by default #632

Closed
wants to merge 12 commits into from

Conversation

forman
Copy link
Member

@forman forman commented May 1, 2018

Hi @JanisGailis, @papesci, please review this asap as it closes

forman and others added 3 commits May 1, 2018 14:29
falling back to default chunk sizes from NetCDF (experimental!) to optimize image display
@forman forman self-assigned this May 1, 2018
@forman forman requested review from JanisGailis and papesci May 1, 2018 15:50
@codecov-io
Copy link

codecov-io commented May 1, 2018

Codecov Report

Merging #632 into master will increase coverage by 0.01%.
The diff coverage is 90.66%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #632      +/-   ##
==========================================
+ Coverage   76.42%   76.43%   +0.01%     
==========================================
  Files          81       81              
  Lines       11714    11741      +27     
==========================================
+ Hits         8952     8974      +22     
- Misses       2762     2767       +5
Impacted Files Coverage Δ
cate/core/opimpl.py 95.43% <90%> (-0.93%) ⬇️
cate/core/ds.py 77.48% <91.42%> (+0.77%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e5c3f68...813c969. Read the comment docs.

@JanisGailis
Copy link
Member

It has to be checked if Use Case 6 happy path still works on entry level machines (4GB memory) when using this branch. I'll try to do this tomorrow afternoon.

In addition, it's a good idea to try and download a large subset of SST data and try to do daily -> monthly aggregation on it.

Copy link
Member

@JanisGailis JanisGailis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just tested that UC6 still runs on a VM with 4GB memory. So that's that. I say - merge this and we'll see what happens.

@forman forman requested a review from hans-permana May 2, 2018 11:22
Copy link
Contributor

@hans-permana hans-permana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@forman forman closed this May 8, 2018
@forman forman deleted the 623-nf-add_time_dim_on_open_dataset branch May 8, 2018 14:06
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants