Workflow Example compare two model runs

Workflow Example - How to compare two model runs

Example of an ad-hoc workflow to compare two model runs. In this case I wanted to compare the two model runs from nearby pixels of the same vegetation type. The difference between the pixels would basically be in the climate as well as the soil type/texture.

Start by making sure you have the latest code with something like this:

	$ git stash
	$ git checkout devel
	$ git pull upstream devel
	$ scons -j4

Note will all ncview examples, I have turned turned the interpolation off so that I can clearly see the pixel boundaries, and in some cases, changed the color scheme. Note depending on where you choose to keep files, you may have to adjust paths for these commands accordingly.

Look at run-mask.nc with ncview to check which pixel we are running (lower left, 0,0)
```
 tobeycarman@lituya:dvm-dos-tem (devel)$ ncview DATA/Toolik_10x10_allyrs/run_mask.nc
```
images/screenshot
Look at vegetation.nc with ncview to check which other pixels might be the same veg type, but ever so slightly different climate (most cells in this 10x10 region near Toolik are CMT05, or Tussock Tundra)
```
 tobeycarman@lituya:dvm-dos-tem (devel)$ ncview DATA/Toolik_10x10_allyrs/vegetation.nc
```
screenshot
Look at historic-climate.nc and soil_texture.nc files with ncview to find pixels that might be different than the one we have been using.
```
 tobeycarman@lituya:dvm-dos-tem (devel)$ ncview DATA/Toolik_10x10_allyrs/soil_texture.nc

 [[screenshot]]
```

Generally, it looks like the upper right area of the 10x10 window are going to be most different from the test pixel we've been using. Lets start with px (9,9)

Create a new run_mask.nc file that uses a different pixel. We haven't really done much of this yet, so there are not special command line arguments - you have to modify the create_region_input.py script by hand. So you modify the script, run it, and then copy the resulting file into place. Or run the script with enough arguments that the run_mask.nc file gets put in the right place automatically. I modified it to use pixel (9,9)

 tobeycarman@lituya:dvm-dos-tem (devel)$ vim scripts/create_region_input.py
 ...
 44   print " --> NOTE: Turning off all pixels except 0,0."
 45   run[:] = np.zeros((sizey, sizex))
 46   run[9,9] = 1
 ...

 tobeycarman@lituya:dvm-dos-tem (devel)$ ./scripts/create_region_input.py --outdir DATA/ --tag Toolik --which run_mask --xsize 10 --ysize 10
 Parsing command line arguments
 args:  Namespace(crtf_only=False, outdir='DATA/', start_year=0, tag='Toolik', tifs='../../snap-data', which='run_mask', xoff=915, xsize=10, years=10, yoff=292, ysize=10)
 Will be looking for files in:       ../../snap-data
 Will be (over)writing files to:     DATA/Toolik_10x10
 Creating a run_mask file, 10 by 10 pixels.
  --> NOTE: Turning off all pixels except 0,0.
 DONE

Note that the messages coming out of the script are incorrect regarding which pixel was turned on. Also note that there are not enough command line args to the script to get the file created in exactly the right place - it ends up in a DATA/Toolik_10x10/ directory instead of the DATA/Toolik_10x10_allyrs/ directory. While annoying, this is not a problem as we can just modify the config file to look for the new file

Check that the file got created correctly with ncview:

 tobeycarman@lituya:dvm-dos-tem (devel)$ ncview DATA/Toolik_10x10_allyrs/run_mask.nc

 [[screenshot]]

Now we try to run the model, so we can save the outputs for this "default parameters, different pixel" experiment.
```
 tobeycarman@lituya:dvm-dos-tem (devel)$ ./dvmdostem -p 100 -e 1000 -s 250 -t 109 -n 100 -c
```

But this fails! Try again with the --log-level=debug and see that we are missing some fire related input files. This must be a result of one of the recent merge's of Ruth's work with the dsb branch - looks like some of the newer input files she is using for fire/disturbance need to be added to the repository so that this isn't an issue in the future and we can still provide "out of the box" functionality. First we need to create the files, which is easy, because Ruth added functions to the input creation script:

Note that the file doesn't get created in quite the right spot, so we have to move it into the final directory.

	tobeycarman@lituya:dvm-dos-tem (devel)$ ./scripts/create_region_input.py --outdir DATA/ --tag Toolik --which fri_fire --xsize 10 --ysize 10
	tobeycarman@lituya:dvm-dos-tem (devel)$ mv DATA/Toolik_10x10/fri_fire.nc DATA/Toolik_10x10_allyrs/
	tobeycarman@lituya:dvm-dos-tem (devel)$ ./scripts/create_region_input.py --outdir DATA/ --tag Toolik --which explicit_fire --xsize 10 --ysize 10
	tobeycarman@lituya:dvm-dos-tem (devel)$ mv DATA/Toolik_10x10/explicit_fire.nc DATA/Toolik_10x10_allyrs/

Next modify the config file to use the different run mask, and to turn inter-stage pauses on:

 tobeycarman@lituya:dvm-dos-tem (devel)$ vim config/config.js
  14 
  15     "runmask_file":       "DATA/Toolik_10x10/run-mask.nc",
  16     ...
  ...
  50   "stage_settings": {
  51     "restart_mode": "restart",   // other options??
  52     "inter_stage_pause": true

Then finally run the model:

 tobeycarman@lituya:dvm-dos-tem (devel)$ ./dvmdostem -p 100 -e 1000 -s 250 -t 109 -n 100 -c

At each stage, tar the json outputs and copy them to the auto-post-process directory (in a separate terminal window)

	tobeycarman@lituya:10-14-default-params-diff-px (devel)$ mkdir -p auto-post-process/10-14-default-params-diff-px
	tobeycarman@lituya:10-14-default-params-diff-px (devel)$ cd auto-post-process/10-14-default-params-diff-px

	tobeycarman@lituya:10-14-default-params-diff-px (devel)$ tar -czf pr-data.tar.gz /tmp/dvmdostem/
	tobeycarman@lituya:10-14-default-params-diff-px (devel)$ tar -czf eq-data.tar.gz /tmp/dvmdostem/
	tobeycarman@lituya:10-14-default-params-diff-px (devel)$ tar -czf sp-data.tar.gz /tmp/dvmdostem/
	tobeycarman@lituya:10-14-default-params-diff-px (devel)$ tar -czf tr-data.tar.gz /tmp/dvmdostem/
	tobeycarman@lituya:10-14-default-params-diff-px (devel)$ tar -czf sc-data.tar.gz /tmp/dvmdostem/

Now, generate the images.

Note: On my computer, because of the way I have Python installed, I have to activate my particular python environment:

	tobeycarman@lituya:dvm-dos-tem (devel)$ pyenv activate del-plotting

	(sel-plotting)tobeycarman@lituya:10-14-default-params-default-px (devel)$ ../../calibration/calibration-viewer.py --from-archive pr-data.tar.gz --archive-series eq-data.tar.gz sp-data.tar.gz tr-data.tar.gz sc-data.tar.gz --bulk --no-show

I have then been manually deleting the extraneous Environment plots. Environmental plots are not PFT specific, but the crude mechanism that the calibration-viewer.py script uses to bulk generate plots ends up creating 9 redundant Environment plots. Same issue with Fire. The plots are blank because we don't have the disturbance module turned on. I have been leaving one Fire plot around for some reason.
Then modify the config file back to using the original pixel, and run the model again, also archiving the json files:
```
$ ./dvmdostem -p 100 -e 1000 -s 250 -t 109 -n 100 -c
```

Again, in another window:

	$ tar -czf pr-data.tar.gz /tmp/dvmdostem/
	$ tar -czf eq-data.tar.gz /tmp/dvmdostem/
	$ tar -czf sp-data.tar.gz /tmp/dvmdostem/
	$ tar -czf tr-data.tar.gz /tmp/dvmdostem/
	$ tar -czf sc-data.tar.gz /tmp/dvmdostem/

Then create the plots and manually delete the extra Environment and Fire plots:

$ ../../calibration/calibration-viewer.py --from-archive pr-data.tar.gz --archive-series eq-data.tar.gz sp-data.tar.gz tr-data.tar.gz sc-data.tar.gz --bulk --no-show

Finally, create the html page. Because I have the data from the most recent run using the "pest optimal" parameters, I will include them too:

(sel-plotting)tobeycarman@lituya:dvm-dos-tem (devel)$ ./scripts/generate-html-viewer.py -l auto-post-process/10-14-default-params-default-px/ -c auto-post-process/10-14-default-params-diff-px/ -r ~/Downloads/oct-10-pestpp-posterior-results/
(sel-plotting)tobeycarman@lituya:dvm-dos-tem (devel)$ ./scripts/generate-html-viewer.py -l auto-post-process/10-14-default-params-default-px/ -c auto-post-process/10-14-default-params-diff-px/ -r ~/Downloads/oct-10-pestpp-posterior-results/

Open the resulting "three-view.html" in your web browser and take a look:
```
[[screenshot]]
```

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Workflow Example compare two model runs

Workflow Example - How to compare two model runs

Clone this wiki locally