Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Download size reporting #102

Merged
merged 7 commits into from
Mar 13, 2023
Merged

Download size reporting #102

merged 7 commits into from
Mar 13, 2023

Conversation

multimeric
Copy link
Collaborator

@multimeric multimeric commented Mar 13, 2023

  • Added un-exported functions url_file_size and report_file_sizes, for calculating and reporting the file sizes of URLs, with a test.
  • Added a call to report_file_sizes before we download anything, which reports to the user how much data is about to be downloaded (closes Prepare users for 36GB download #60). This is the only user facing change. See below for an example.
  • Added zzz.R which was supposed to be in Persistent connections #44. This closes all cached connections when the package is unloaded.
  • Minor cleanup of imports and code style
> metadata <- get_metadata()
ℹ Downloading 1 file, totalling 0.48 GBDownloading https://object-store.rc.nectar.org.au/v1/AUTH_06d6e008e3e642da99d806ba3ea629c5/metadata/metadata.0.2.3.parquet to /stornext/Home/data/allstaff/m/milton.m/.cache/R/CuratedAtlasQueryR/metadata.0.2.3.parquet
  |===================================================================================================================================================================| 100%
> single_cell_counts = 
+ 	metadata |>
+     dplyr::filter(
+         ethnicity == "African" &
+         stringr::str_like(assay, "%10x%") &
+         tissue == "lung parenchyma" &
+         stringr::str_like(cell_type, "%CD4%")
+     ) |>
+     get_SingleCellExperiment()
ℹ Realising metadata.Synchronising filesDownloading 2 files, totalling 0.17 GBDownloading https://swift.rc.nectar.org.au/v1/AUTH_06d6e008e3e642da99d806ba3ea629c5/harmonised-human-atlas/original/bc380dae8b14313a870973697842878b/assays.h5 to /vast/scratch/users/milton.m/cache/R/CuratedAtlasQueryR/0.2/original/bc380dae8b14313a870973697842878b/assays.h5Downloading https://swift.rc.nectar.org.au/v1/AUTH_06d6e008e3e642da99d806ba3ea629c5/harmonised-human-atlas/original/bc380dae8b14313a870973697842878b/se.rds to /vast/scratch/users/milton.m/cache/R/CuratedAtlasQueryR/0.2/original/bc380dae8b14313a870973697842878b/se.rdsReading files.Compiling Single Cell Experiment.

@multimeric multimeric marked this pull request as ready for review March 13, 2023 06:22
@multimeric multimeric requested a review from stemangiola March 13, 2023 06:22
@stemangiola stemangiola merged commit 212be10 into master Mar 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Prepare users for 36GB download
2 participants