Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pull data from multiple surveys with just one function call #17

Open
kellijohnson-NOAA opened this issue Apr 9, 2019 · 2 comments
Open
Labels
enhancement warehouse Pertains to getting, documenting, or fixing data in the warehouse.

Comments

@kellijohnson-NOAA
Copy link
Contributor

In the future we should add the ability to query multiple surveys with one pull, e.g., SurveyName = c("Triennial", "AFSC.Slope")? https://github.com/nwfsc-assess/nwfscSurvey/blob/b3e36334d1b390e6020614076101f9122a52f70c/R/PullBio.fn.R#L9
Looks like it would require some restructuring of the if statements to something like

if (any(SurveyName == "Triennial")) {

allowing multiple if statements to be true and the data to be filtered for each survey.
Some additional data merging would be needed as well.

@chantelwetzel-noaa
Copy link
Contributor

I agree. I initially constructed the code to pull only a single survey at a time in order to prevent people from accidentally combining data across surveys when creating indices or composition data for assessment. I think with some careful thought, the code can be reworked in order to not only pull multiple surveys, but also have traps to ensure proper processing of the data. I will try to revisit this post-assessment season.

@ericward-noaa
Copy link
Contributor

I agree. I had added a function to just pull the haul data (including environmental variables) and allowed multiple surveys to be grabbed at the same time. Surveys can be NULL, e.g.

haul_dat = PullHaul.fn()

One general thing to think about is right now, the user inputs (years, surveys, species, variables) are used to write a text string for the JSON query. Another option would be to grab all the data initially, and then apply filtering (using dplyr or whatever).

I'm happy to help with the other functions after the assessment season is wrapped up.

@kellijohnson-NOAA kellijohnson-NOAA added the warehouse Pertains to getting, documenting, or fixing data in the warehouse. label Dec 1, 2022
@kellijohnson-NOAA kellijohnson-NOAA changed the title Multiple surveys??? Pull data from multiple surveys with just one function call Dec 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement warehouse Pertains to getting, documenting, or fixing data in the warehouse.
Projects
None yet
Development

No branches or pull requests

3 participants