Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use s3 product dataset mapping #151

Merged

Conversation

renaudjester
Copy link
Collaborator

@renaudjester renaudjester commented Sep 27, 2024

Finishing the task on stop using cache

Also started to document the custom errors of the toolbox

@renaudjester renaudjester changed the base branch from main to copernicusmarine-toolbox-v2 September 27, 2024 12:18
Comment on lines +1 to +5
Common and Custom errors
==========================

Custom errors
-------------
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I started doing this so I'll finish it here but the most important of this pull requests is the fact that we are using the mapping from the s3 metadata instead of a local file

@renaudjester renaudjester requested a review from uriii3 September 27, 2024 12:59
- The dataset id is incorrect and not present in the catalog.
- The dataset has been retired.

Please verifiy that the dataset id is can be found in
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some typo I understand, no? without the 'is' makes more sense

"""
Exception raised when other options are passed with create_template.

Please note that create_template should be passed with no other optiond
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo



class CredentialCannotBeNone(Exception):
"""
Exception raised when a credential are not set.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

plural or singular? credentials are not set or credential is not set (no?)

Comment on lines +109 to +114

.. _pypi: https://pypi.org/project/copernicusmarine/
.. _conda-forge: https://anaconda.org/conda-forge/copernicusmarine
.. _dockerhub: https://hub.docker.com/r/copernicusmarine/copernicusmarine
.. _copernicusmarine-register: https://data.marine.copernicus.eu/register
.. _Anaconda: https://www.anaconda.com/products/individual
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is all of this? I understand the correction is because I messed it up, but what does it do?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a way to put all the links at the same place but it's just some synthax from restructuredText

Comment on lines +28 to +30
MARINE_DATA_STORE_ROOT_METADATA_URL = (
"https://s3.waw3-1.cloudferro.com/mdl-metadata"
)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the new link no? to update it, the action is somewhere else I understand, right?

Comment on lines +85 to +86
product_id = connection.get_json_file(dataset_product_mapping_url).get(
dataset_id
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand that this has been check that it works, right? do yo feel it is more or less the time we anticipated?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Super fast

@renaudjester renaudjester merged commit c322edc into copernicusmarine-toolbox-v2 Sep 30, 2024
2 checks passed
@renaudjester renaudjester deleted the use-s3-product-dataset-mapping branch September 30, 2024 09:57
renaudjester added a commit that referenced this pull request Oct 28, 2024
Use the s3 product dataset mapping and add errors in the doc and in the copernicus package
renaudjester added a commit that referenced this pull request Oct 28, 2024
Use the s3 product dataset mapping and add errors in the doc and in the copernicus package
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants