Merge pull request #1 from cgrudz/master

Update c-merchant branch

docs/make.jl

@@ -28,7 +28,7 @@ makedocs(
                     "FilterExps" => "submodules/experiments/FilterExps.md",
                     "SmootherExps" => "submodules/experiments/SmootherExps.md",
                     "SingleExperimentDriver" => "submodules/experiments/SingleExperimentDriver.md",
-                    "Slurm" => "submodules/experiments/Slurm.md",
+                    "ParallelExperimentDriver" => "submodules/experiments/ParallelExperimentDriver.md",
                    ],
                  "Analysis" => Any[
                     "ProcessExperimentData" => "submodules/analysis/ProcessExperimentData.md",
@@ -38,7 +38,6 @@ makedocs(
             ]
 )
 
-
 deploydocs(
     repo = "github.com:cgrudz/DataAssimilationBenchmarks.jl.git",
 )

docs/src/index.md

@@ -5,7 +5,7 @@ validating sequential filters and smoothers in toy model twin experiments.
 This code is meant to be performant in the sense that large hyper-parameter discretizations
 can be explored to determine hyper-parameter sensitivity and reliability of results across
 different experimental regimes, with parallel implementations in native Julia distributed
-computing and using workload managers such as Slurm.
+computing.
 
 This package currently includes code for developing and testing data assimilation schemes in
 the [L96-s model](https://gmd.copernicus.org/articles/13/1903/2020/) and the IEEE 39 bus test

docs/src/submodules/experiments/ParallelExperimentDriver.md

@@ -2,12 +2,14 @@
 
 ## ParallelExperimentDriver API
 
-The `ParallelExperimentDriver.jl` is a simple parallel implementation, though currently lacks a soft-fail when numerical
-instability is encountered.  This means that if a single experiment configuration in the collection fails due to overflow, the entire
-collection will cancel.  A fix for this is being explored, but the recommendation is to use the slurm submit
-scripts below as templates for generating large parameter grid configurations and running them on servers.
+The `ParallelExperimentDriver.jl` is a simple parallel implementation of calling experiment parameter arrays
+with `pmap` and Julia's native distributed computing.  This defines argumentless functions to construct
+the parameter array and input data necessary to generate a sensitivity test, and implements a soft-fail for
+experiments instability is encountered causing a crash of an experiment. This means that if a single experiment
+configuration in the parameter array fails due to overflow, the remaining configurations will continue their
+own course unaffected.
 
-## Docstrings
-#```@autodocs
-#Modules = [DataAssimilationBenchmarks.ParallelExperimentDriver]
-#```
+## Methods
+```@autodocs
+Modules = [DataAssimilationBenchmarks.ParallelExperimentDriver]
+```

docs/src/submodules/experiments/SmootherExps.md

@@ -1,38 +1,38 @@
 # SmootherExps 
 
-
 ## SmootherExps API
-
-The `SmootherExps.jl` sub-module configures twin experiments using a stored time series as generated by 
-[GenerateTimeSeries](@ref) to generate hyper-parameter configurations for the estimtor with the same
-underlying observation generating process.
-Experiment configurations are generated by function calls as with the filter experiments, but with the additional
-options of how the outer-loop is configured with a classic, single-iteration or the fully iterative Gauss-Newton style smoother.
-The parameters `lag` and `shift` specify how the data assimilation windows are translated in over the observation
-and analysis times.  The `mda` parameter is only applicable to the single-iteration and Gauss-Newton style smoothers,
-utlizing sequential multiple data assimilation.  Note, the single-iteration and fully iterative Gauss-Newton style smoothers are
-only defined for MDA compatible values of lag and shift where the lag is an integer multiple of the shift.
-
-Currently debugged and validated smoother experiment configurations include
+The `SmootherExps.jl` sub-module contains methods to configure filter twin experiments,
+using a stored time series as generated by [GenerateTimeSeries](@ref) as the underlying
+observation generating process.  The frequency of observations in continuous time is defined
+by the frequency of data saved in the time series and is inferred by the experiment
+when reading in the data.
+
+```@raw html
+<div sytle="float:left; width:100%;">
+<img style="width:95%" src="../cyclingSDA.png" alt="Observation analysis forecast cycle over multiple data assimilation windows">
+</div>
 ```
-classic_state          -- classic EnKS style state estimation
-classic_param          -- classic EnKS style state-parameter estimation
-single_iteration_state -- single-iteration EnKS state estimation
-single_iteration_param -- single-iteration EnKS state-parameter estimation
-iterative_state        -- Gauss-Newton style state estimation
-iterative_param        -- Gauss-Newton style state-parameter estimation
-```
-Other techniques are still in debugging and validation.  Each of these takes an analysis type as used in the
-`transform` function in the `EnsembleKalmanSchemes.jl` sub-module, like the filter analyses in the filter experiments.
-
+Smoother experiment configurations are generated by supplying a 
+[NamedTuple](https://docs.julialang.org/en/v1/base/base/#Core.NamedTuple)
+with the required fields as specified in the experiment method.  Conventions for
+these arguments are the same as with the [FilterExps](@ref), with the additional options
+that configure  the data assimilation window (DAW) and how this is shifted in time:
+  * `lag` - the number of past observation / analysis times to reanalyze in a DAW, corresponding to ``L`` in the figure above;
+  * `shift`- the nunber of observation / analysis times to shift the DAW, corresponding to ``S`` in the figure above;
+  * `mda` - (__Multiple Data Assimilation__), type `Bool`, determines whether the technique of multiple data assimilation is used (only compatible with `single_iteration` and `iterative` smoothers.
 
-All experiments are funcitonalized so that they can be called from an array of parameter
-values which will typically be varied with naive parallelism.  Relevant arguments and
-experimental results are dumped as a side effect to a dictionary in a JLD2.
-Experiments return their runtime in minutes.
+Currently debugged and validated smoother experiment configurations include
+  * `classic_state` - classic ETKS style state estimation
+  * `classic_param` - classic ETKS style state-parameter estimation
+  * `single_iteration_state` - SIEnKS state estimation
+  * `single_iteration_param` - SIEnKS state-parameter estimation
+  * `iterative_state` - IEnKS Gauss-Newton style state estimation
+  * `iterative_param` - IEnKS Gauss-Newton style state-parameter estimation
 
+Note, the single-iteration and fully iterative Gauss-Newton style smoothers are only defined
+for MDA compatible values of lag and shift where the lag is an integer multiple of the shift.
 
-## Docstrings
+## Smoother Experiment Methods
 ```@autodocs
 Modules = [DataAssimilationBenchmarks.SmootherExps]
 ```

docs/src/submodules/methods/EnsembleKalmanSchemes.md

@@ -4,12 +4,8 @@
 There are currently four families of ensemble Kalman estimators available in this package,
 which define the outer-loop of the data assimilation cycle.  Particularly, these define
 how the sequential data assimilation cycle will pass over a time series of observations,
-conceptually pictured in the figure below.
-```@raw html
-<div sytle="float:left; width:100%;">
-<img style="width:95%" src="../cyclingSDA.png" alt="Observation analysis forecast cycle over multiple data assimilation windows">
-</div>
-```
+with more details in the [SmootherExps](@ref) documents.
+
 Ensemble filters only produce analyses forward-in-time.  The classic lag-shift smoother runsi
 identically to the filter in its forecast and filter steps, but includes an additional retrospective
 analysis to past ensemble states stored in memory.  The single iteration smoother follows

src/DataAssimilationBenchmarks.jl

@@ -151,6 +151,7 @@ using .D3VARExps
 export DeSolvers, EnsembleKalmanSchemes, XdVAR, L96, IEEE39bus, ObsOperators,
        GenerateTimeSeries, FilterExps, SingleExperimentDriver, ParallelExperimentDriver,
        D3VARExps
+
 ##############################################################################################
 # info
 

src/experiments/FilterExps.jl

@@ -40,6 +40,10 @@ Output from the experiment is saved in a dictionary of the form,
                             "s_infl" => round(s_infl, digits=2)
                            )
 
+    if haskey(ts, "diff_mat")
+        data["diff_mat"] = ts["diff_mat"]::Array{Float64,2}
+    end
+
 Experiment output is written to a directory defined by
 
     path = pkgdir(DataAssimilationBenchmarks) * "/src/data/" * method * "/"
@@ -256,6 +260,11 @@ Output from the experiment is saved in a dictionary of the form,
                             "s_infl" => round(s_infl, digits=2),
                             "p_infl" => round(p_infl, digits=2)
                            )
+
+    if haskey(ts, "diff_mat")
+        data["diff_mat"] = ts["diff_mat"]::Array{Float64,2}
+    end
+
 Experiment output is written to a directory defined by
 
     path = pkgdir(DataAssimilationBenchmarks) * "/src/data/" * method * "/"
@@ -465,7 +474,6 @@ function ensemble_filter_param((time_series, method, seed, nanl, obs_un, obs_dim
                             "p_infl" => round(p_infl, digits=2)
                            )
 
-    # check if there is a diffusion structure matrix
     if haskey(ts, "diff_mat")
         data["diff_mat"] = ts["diff_mat"]::Array{Float64,2}
     end