Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use generators/iterator for granule query #120

Closed
betolink opened this issue Aug 15, 2022 · 2 comments
Closed

Use generators/iterator for granule query #120

betolink opened this issue Aug 15, 2022 · 2 comments
Labels
type: will not do This will not be worked on

Comments

@betolink
Copy link
Member

If we are querying hundred of thousands of granules, we need to handle them in a more efficient way, perhaps implementing an iterator like

links = []
for g in Datagranules().concept_id("c-some-datasets").items():
    links.append(g.data_links())

This idea is that we'll only load the current page into memory instead of the whole number of granules in a collection query.

@chuckwondo
Copy link
Collaborator

As we've recently discussed during an earthaccess hack day, I'm going to create an issue in the python_cmr library for implementing this functionality upstream, so we don't duplicate the functionality here.

@asteiker
Copy link
Member

Based on @chuckwondo 's comment above, we'll close this out since it will be worked over in python_cmr

@github-project-automation github-project-automation bot moved this from 🆕 New to ✅ Done in earthaccess project Oct 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: will not do This will not be worked on
Projects
Status: Done
Development

No branches or pull requests

3 participants