Approximation downloaded data #126
Conversation
renaudjester
changed the base branch from
main
to
copernicusmarine-toolbox-v2
| estimated_size_message += f"{estimate_size:.3f} MB" | ||
|
|
||
| download_estimated_size = 0 | ||
| for variable_name in dataset.data_vars: |
There was a problem hiding this comment.
why do we need to iterate through the variables? do they have different points or different types of description within the same dataset? (I would expect that the see temperature and the wind might be have values over the same points, so we would only need to multiply for the number of variables, no?)
There was a problem hiding this comment.
Because they might have different types (not sure but possible)
But imagine the wind is in float64 and the thetao is in float32
There was a problem hiding this comment.
And what would be the problem there? the amount of information per step (per chunk) would be different, but not the way to check for the amount of points to account, no? I think I'm not yet fully understanding the methodology followed, still checking the code hehe
There was a problem hiding this comment.
It wouldn't be a problem but it would change the result.
Also some variables don't have depth, so they are lighter than others
| for var in service.variables | ||
| if var.short_name == variable_name | ||
| ][0].coordinates | ||
| if coord.coordinate_id == coordinate_name |
There was a problem hiding this comment.
why do we need to have a for for all the dimensions and then do them one by one? is there any case in which we are using 'lon' and then 'longitude' or something like that? or what is this logic for?
There was a problem hiding this comment.
it's just that we are in a loop over the coordinates of the dataset object and what we want for this given coordinate is its metadata, so this loop is to get the metadata of the specific coordinate
There was a problem hiding this comment.
Maybe you are right though, there could be some performance improvements haha
renaudjester
force-pushed
the
approximation-downloaded-data
branch
from
d34b64c to
b3b48d7
Compare
|
Then I assume is good, no? although maybe I would like a short explanation backend wise, I don't think anything is going to break and we can upgrade the methodology with the feedback we get, no? |
We now display an approximation of the amount of data that will be downloaded next to the expected size of the final file.
We now display an approximation of the amount of data that will be downloaded next to the expected size of the final file.
Fix CMT-63