Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

R aws access #134

Open
2 tasks
wildintellect opened this issue Aug 16, 2024 · 2 comments
Open
2 tasks

R aws access #134

wildintellect opened this issue Aug 16, 2024 · 2 comments
Assignees

Comments

@wildintellect
Copy link
Contributor

Troubleshooting AWS S3 access from R we discovered that the current library we install aws.s3 (aka r-aws.s3) is unmaintained, and that other alternatives are equally challenging s3 (not in conda-forge).

Reading up, AWS recommends a package called paws (aka r-paws) which is an R SDK similar to the Python AWS SDK.
So you can do things very much like boto3, however it's not very R styled.

Ask

  • Update the R image to include paws, possibly remove aws.s3 <- some user code currently uses this though s3_get specifically (Platform Team)
  • Update Docs (Data Team) with examples

Example:

Get a list of files from s3

#bash
conda install -c conda-forge r-paws
#R
library(paws)
s3 <- paws::s3()
results <- s3$list_objects_v2(Bucket = "maap-ops-workspace", Prefix="shared/abarenblitt/GEDI_global_PA_v2/")
file_list <- sapply(results$Contents, function(x) {x$Key})

thanks to @abarenblitt for reporting issue
cc: @smk0033

@grallewellyn
Copy link
Contributor

How do we want to determine if we should remove r-aws.s3 from the R image?

@wildintellect
Copy link
Contributor Author

wildintellect commented Aug 16, 2024

For now it's not a conflict to also install paws so I would start with that. We have an open ticket to write best practices with R, we'll make sure not to use the likely to be removed library.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants