From 59da6a8475523a27c122a3f6a2ca7f80ee0b1f9b Mon Sep 17 00:00:00 2001 From: Tom Schwarzl <62601256+tschwarzl@users.noreply.github.com> Date: Tue, 22 Oct 2024 23:04:00 +0200 Subject: [PATCH 1/9] Added reload to {{ stage_name }}.qmd Added the dso-r reload() functionality to the stage template and added additional beginner friendly explanation and more user friendly layout to the template. --- .../stage/quarto/src/{{ stage_name }}.qmd | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd b/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd index 1ec9b35..8da43a1 100644 --- a/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd +++ b/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd @@ -5,18 +5,24 @@ require(conflicted) require(dso) ``` +`read_params(..)` +1. sets the stage directory +2. runs 'dso compile-config', which hierachically compiles config files (params.in.yaml and params.yaml) and +3. loads parameters defined in the file defining stage dependencies ('params' section in the 'dvc.yaml' file) from the compiled stage config file (params.yaml) + ```{r} #| label: read_params -# read_params sets the stage directory, runs 'dso compile-config' -# and loads params defined in the 'dvc.yaml' 'params' section of the stage params <- read_params("{{ stage_path }}") ``` +Once `params <- read_params(..)` is executed, use 'reload(params)' to quickly rerun read_params and reload the params while developing + + +To locate your files relative to the stage path use `stage_here(..)`. This command uses the stage path set in read_params. + ```{r} #| label: obtain_files_relative_to_stage_dir -# stage_here locates your files relative to the stage path set in read_params -# e.g. samplesheet <- readr::read_csv(stage_here(params$samplesheet)) ``` From c08f93541a5c3cf86387c85e0126248613b6af46 Mon Sep 17 00:00:00 2001 From: tschwarzl Date: Tue, 22 Oct 2024 21:04:47 +0000 Subject: [PATCH 2/9] pre-commit autofixes --- src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd b/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd index 8da43a1..83f9f5d 100644 --- a/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd +++ b/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd @@ -5,9 +5,9 @@ require(conflicted) require(dso) ``` -`read_params(..)` +`read_params(..)` 1. sets the stage directory -2. runs 'dso compile-config', which hierachically compiles config files (params.in.yaml and params.yaml) and +2. runs 'dso compile-config', which hierachically compiles config files (params.in.yaml and params.yaml) and 3. loads parameters defined in the file defining stage dependencies ('params' section in the 'dvc.yaml' file) from the compiled stage config file (params.yaml) ```{r} From 2c2ab227326597aa57de829b8e3f889d2ea2b163 Mon Sep 17 00:00:00 2001 From: Tom Schwarzl <62601256+tschwarzl@users.noreply.github.com> Date: Wed, 23 Oct 2024 15:54:45 +0200 Subject: [PATCH 3/9] Modified description --- .../stage/quarto/src/{{ stage_name }}.qmd | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd b/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd index 83f9f5d..3953f20 100644 --- a/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd +++ b/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd @@ -5,10 +5,8 @@ require(conflicted) require(dso) ``` -`read_params(..)` -1. sets the stage directory -2. runs 'dso compile-config', which hierachically compiles config files (params.in.yaml and params.yaml) and -3. loads parameters defined in the file defining stage dependencies ('params' section in the 'dvc.yaml' file) from the compiled stage config file (params.yaml) +Load the stage-specific 'params.yaml' config using the `read_params(..)` function. This function specifically loads +only the stage-dependent parameters that are defined in the 'params' section of the 'dvc.yaml' file. ```{r} #| label: read_params @@ -16,13 +14,20 @@ require(dso) params <- read_params("{{ stage_path }}") ``` -Once `params <- read_params(..)` is executed, use 'reload(params)' to quickly rerun read_params and reload the params while developing +When modifying the 'dvc.yaml', 'params.in.yaml', or 'params.yaml' files during development, use the 'reload(params)' +function to ensure proper application of the changes by rebuilding and reloading the configuration. +```{r} +#| label: reload_params + +reload(params) +``` -To locate your files relative to the stage path use `stage_here(..)`. This command uses the stage path set in read_params. +To locate your files relative to the stage path use `stage_here(..)`. ```{r} #| label: obtain_files_relative_to_stage_dir +# e.g. samplesheet <- readr::read_csv(stage_here(params$samplesheet)) ``` From 5b4f751a7d4f1272cc5b8b0c7116c642561ea5b9 Mon Sep 17 00:00:00 2001 From: tschwarzl Date: Wed, 23 Oct 2024 13:55:13 +0000 Subject: [PATCH 4/9] pre-commit autofixes --- src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd b/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd index 3953f20..6280fd0 100644 --- a/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd +++ b/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd @@ -5,7 +5,7 @@ require(conflicted) require(dso) ``` -Load the stage-specific 'params.yaml' config using the `read_params(..)` function. This function specifically loads +Load the stage-specific 'params.yaml' config using the `read_params(..)` function. This function specifically loads only the stage-dependent parameters that are defined in the 'params' section of the 'dvc.yaml' file. ```{r} @@ -23,7 +23,7 @@ function to ensure proper application of the changes by rebuilding and reloading reload(params) ``` -To locate your files relative to the stage path use `stage_here(..)`. +To locate your files relative to the stage path use `stage_here(..)`. ```{r} #| label: obtain_files_relative_to_stage_dir From 67051ca2de359823c4882393cc3ee7c23b1772b5 Mon Sep 17 00:00:00 2001 From: Tom Schwarzl <62601256+tschwarzl@users.noreply.github.com> Date: Fri, 25 Oct 2024 09:19:04 +0200 Subject: [PATCH 5/9] Modified {{ stage_name }}.qmd This changes incorporate @grst comments to have a functional template with comments, but a compromise to cover the most important key learning aspects for beginners in the comments. --- .../stage/quarto/src/{{ stage_name }}.qmd | 38 ++++++++++++------- 1 file changed, 25 insertions(+), 13 deletions(-) diff --git a/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd b/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd index 6280fd0..9f1c67e 100644 --- a/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd +++ b/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd @@ -1,33 +1,45 @@ +--- +title: "{{ stage_name }}" +description: "{{ stage_description }}" +--- + +# Setup + +## Load libraries + ```{r} -#| label: load_libraries +#| label: load_required_libraries require(conflicted) require(dso) ``` -Load the stage-specific 'params.yaml' config using the `read_params(..)` function. This function specifically loads -only the stage-dependent parameters that are defined in the 'params' section of the 'dvc.yaml' file. +## Load stage specific config ```{r} #| label: read_params +# Load the stage-specific 'params.yaml' config using the `read_params(..)` function. +# This function specifically loads only the stage-dependent parameters that are defined +# in the 'params', 'deps' and 'outs' section of the 'dvc.yaml' file. + params <- read_params("{{ stage_path }}") ``` -When modifying the 'dvc.yaml', 'params.in.yaml', or 'params.yaml' files during development, use the 'reload(params)' -function to ensure proper application of the changes by rebuilding and reloading the configuration. + -```{r} -#| label: reload_params - -reload(params) -``` - -To locate your files relative to the stage path use `stage_here(..)`. +## Input files ```{r} #| label: obtain_files_relative_to_stage_dir -# e.g. +# To locate your files relative to the stage path use `stage_here(..)`. +# e.g. when samplesheet is defined with `samplesheet = !path input/samplesheet.csv` in +# params.in.yaml and `samplesheet` is listed in 'deps' in 'dvc.yaml' + samplesheet <- readr::read_csv(stage_here(params$samplesheet)) ``` From 8aa245d36952fdd437b05df0fec87eb556107dc4 Mon Sep 17 00:00:00 2001 From: tschwarzl Date: Fri, 25 Oct 2024 07:19:22 +0000 Subject: [PATCH 6/9] pre-commit autofixes --- .../templates/stage/quarto/src/{{ stage_name }}.qmd | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd b/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd index 9f1c67e..950295c 100644 --- a/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd +++ b/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd @@ -1,6 +1,6 @@ --- title: "{{ stage_name }}" -description: "{{ stage_description }}" +description: "{{ stage_description }}" --- # Setup @@ -19,7 +19,7 @@ require(dso) ```{r} #| label: read_params -# Load the stage-specific 'params.yaml' config using the `read_params(..)` function. +# Load the stage-specific 'params.yaml' config using the `read_params(..)` function. # This function specifically loads only the stage-dependent parameters that are defined # in the 'params', 'deps' and 'outs' section of the 'dvc.yaml' file. @@ -28,7 +28,7 @@ params <- read_params("{{ stage_path }}") @@ -37,9 +37,9 @@ by rebuilding and reloading the configuration. ```{r} #| label: obtain_files_relative_to_stage_dir -# To locate your files relative to the stage path use `stage_here(..)`. -# e.g. when samplesheet is defined with `samplesheet = !path input/samplesheet.csv` in -# params.in.yaml and `samplesheet` is listed in 'deps' in 'dvc.yaml' +# To locate your files relative to the stage path use `stage_here(..)`. +# e.g. when samplesheet is defined with `samplesheet = !path input/samplesheet.csv` in +# params.in.yaml and `samplesheet` is listed in 'deps' in 'dvc.yaml' samplesheet <- readr::read_csv(stage_here(params$samplesheet)) ``` From b39554daf9ec70658395495919a42034be1f85fc Mon Sep 17 00:00:00 2001 From: Gregor Sturm Date: Mon, 4 Nov 2024 10:28:19 +0100 Subject: [PATCH 7/9] Revert "pre-commit autofixes" This reverts commit 8aa245d36952fdd437b05df0fec87eb556107dc4. --- .../templates/stage/quarto/src/{{ stage_name }}.qmd | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd b/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd index 950295c..9f1c67e 100644 --- a/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd +++ b/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd @@ -1,6 +1,6 @@ --- title: "{{ stage_name }}" -description: "{{ stage_description }}" +description: "{{ stage_description }}" --- # Setup @@ -19,7 +19,7 @@ require(dso) ```{r} #| label: read_params -# Load the stage-specific 'params.yaml' config using the `read_params(..)` function. +# Load the stage-specific 'params.yaml' config using the `read_params(..)` function. # This function specifically loads only the stage-dependent parameters that are defined # in the 'params', 'deps' and 'outs' section of the 'dvc.yaml' file. @@ -28,7 +28,7 @@ params <- read_params("{{ stage_path }}") @@ -37,9 +37,9 @@ by rebuilding and reloading the configuration. ```{r} #| label: obtain_files_relative_to_stage_dir -# To locate your files relative to the stage path use `stage_here(..)`. -# e.g. when samplesheet is defined with `samplesheet = !path input/samplesheet.csv` in -# params.in.yaml and `samplesheet` is listed in 'deps' in 'dvc.yaml' +# To locate your files relative to the stage path use `stage_here(..)`. +# e.g. when samplesheet is defined with `samplesheet = !path input/samplesheet.csv` in +# params.in.yaml and `samplesheet` is listed in 'deps' in 'dvc.yaml' samplesheet <- readr::read_csv(stage_here(params$samplesheet)) ``` From 0e9198f3900dd48021382c1332c35f012183b925 Mon Sep 17 00:00:00 2001 From: Gregor Sturm Date: Mon, 4 Nov 2024 10:28:20 +0100 Subject: [PATCH 8/9] Revert "Modified {{ stage_name }}.qmd" This reverts commit 67051ca2de359823c4882393cc3ee7c23b1772b5. --- .../stage/quarto/src/{{ stage_name }}.qmd | 38 +++++++------------ 1 file changed, 13 insertions(+), 25 deletions(-) diff --git a/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd b/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd index 9f1c67e..6280fd0 100644 --- a/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd +++ b/src/dso/templates/stage/quarto/src/{{ stage_name }}.qmd @@ -1,45 +1,33 @@ ---- -title: "{{ stage_name }}" -description: "{{ stage_description }}" ---- - -# Setup - -## Load libraries - ```{r} -#| label: load_required_libraries +#| label: load_libraries require(conflicted) require(dso) ``` -## Load stage specific config +Load the stage-specific 'params.yaml' config using the `read_params(..)` function. This function specifically loads +only the stage-dependent parameters that are defined in the 'params' section of the 'dvc.yaml' file. ```{r} #| label: read_params -# Load the stage-specific 'params.yaml' config using the `read_params(..)` function. -# This function specifically loads only the stage-dependent parameters that are defined -# in the 'params', 'deps' and 'outs' section of the 'dvc.yaml' file. - params <- read_params("{{ stage_path }}") ``` - +When modifying the 'dvc.yaml', 'params.in.yaml', or 'params.yaml' files during development, use the 'reload(params)' +function to ensure proper application of the changes by rebuilding and reloading the configuration. -## Input files +```{r} +#| label: reload_params + +reload(params) +``` + +To locate your files relative to the stage path use `stage_here(..)`. ```{r} #| label: obtain_files_relative_to_stage_dir -# To locate your files relative to the stage path use `stage_here(..)`. -# e.g. when samplesheet is defined with `samplesheet = !path input/samplesheet.csv` in -# params.in.yaml and `samplesheet` is listed in 'deps' in 'dvc.yaml' - +# e.g. samplesheet <- readr::read_csv(stage_here(params$samplesheet)) ``` From 9fe0a902afe32c3d795fbbf33763b76bebb891a5 Mon Sep 17 00:00:00 2001 From: Gregor Sturm Date: Mon, 4 Nov 2024 10:30:42 +0100 Subject: [PATCH 9/9] Update CHANGELOG --- CHANGELOG.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 09c3e64..dcf3131 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,10 @@ and this project adheres to [Semantic Versioning][]. [keep a changelog]: https://keepachangelog.com/en/1.0.0/ [semantic versioning]: https://semver.org/spec/v2.0.0.html +## [Unreleased] + +- Improve instruction text in quarto template to get users started more quickly ([#40](https://github.com/Boehringer-Ingelheim/dso/pull/40)) + ## v0.9.0 ### New Features