Basin Order with slurm and p6/om

[rburghol]

Tasks

• Get code from Gopal
• test if it can be automated
• Create a script named make_stream_order using code in Script 1 omitted in favor of watershed dependency approach in Run with slurm meta_model#3
• See https://ubccr.freshdesk.com/support/solutions/articles/5000686909-how-to-retrieve-job-history-and-accounting#:~:text=The%20SLURM%20job%20accounting%20records,use%20srun%20to%20launch%20tasks.

Get Stream Order and Tribs

• Need to obtain a list for any set that we run.
• Could use the old framework with _order files, or could embed elsewhere?
• Old framework is a set of files.

Existing Tools & Method

• Gopal emailed me a script with some code that I streamlined to generate orders and copy to the appropriate script directory (See Script 1`
• This works well, but the group of commands below should be turned into a script"
  • name make_stream_order
  • type = bash script
  • use: cbp make_stream_order JL4_6710_6740
  • note: need to extract the middle ID columns when calling basingen.csh
  • also basingen.csh is freaky since it requires the scenario argument but doesn't do anything with it
  • perhaps we should create a new basingen (no file extension) script to run with full segment name, and no scenario?

Script 1: Generate the order files for Rivanna basin and move to seglists directory.

cbp basingen.csh vadeq_2021 6710
cbp csh divide_rseg_by_stream_order.csh JL4_6710_6740 JL4_6710_6740
mkdir config/seglists/JL4_6710_6740_order
mv config/seglists/JL4_6710_6740_order_* config/seglists/JL4_6710_6740_order/
onames=""
files=`ls config/seglists/JL4_6710_6740_order/`
for i in $files;do j=`basename $i .riv`; onames="$onames $j"; done
echo "set order = ( $onames )" > config/seglists/JL4_6710_6740_order.riv

Example - Hand Built

• Example (Choptank, 1st order only):
  • config/seglists/CHOP_order.riv: set order = ( CHOP_order_01 )
  • config/seglists/CHOP_order/CHOP_order_01.riv: set segments = ( EM2_3980_0001 )
• My hand assembled one (using R below to grab orders for individual streams):
  • Basin: JL3_7090_7150
    • cbp get_config vadeq_2021 script BASINS (in case you need to find what outlet it is)
      • JL3_7090_7150
    • `cbp get_riversegs JL3_7090_7150
      • JL1_7080_7190 JL2_7240_7350 JL1_6940_7200 JL1_7190_7250 JL2_7350_7090 JL1_7200_7250 JL2_7250_7090 JL3_7090_7150
  • config/seglists/JL3_7090_7150_order.riv: set order = ( JL3_7090_7150_order_01, JL3_7090_7150_order_02, JL3_7090_7150_order_03, JL3_7090_7150_order_04 )
  • Make the directory: mkdir config/seglists/JL3_7090_7150_order
  • config/seglists/JL3_7090_7150_order/JL3_7090_7150_order_01.riv: set segments = (JL1_7080_7190, JL2_7240_7350, JL1_6940_7200)
  • config/seglists/JL3_7090_7150_order/JL3_7090_7150_order_02.riv: set segments = (JL1_7190_7250, JL2_7350_7090)
  • config/seglists/JL3_7090_7150_order/JL3_7090_7150_order_03.riv: set segments = (JL2_7250_7090)
  • config/seglists/JL3_7090_7150_order/JL3_7090_7150_order_04.riv: set segments = (JL3_7090_7150)

segs=`cbp get_riversegs JL3_7090_7150`
segs=${segs//" "/\',\'}
segs="'$segs'"
echo "AllSegList <- c($segs)"


# R Code
orders = list()
fn_stream_order <- function (seg, AllSegList) {
  itribs <- fn_upstream(seg,AllSegList)
  if (itribs[1] == "NA") {
    stream_order = 1
  } else {
    trib_order = c()
    for (t in itribs) {
      trib_order[t] = fn_stream_order(t, AllSegList)
    }
     stream_order = 1 + max(trib_order)
  }
  return(stream_order)
}

for (k in AllSegList) {
  korder <- fn_stream_order(k,AllSegList)
  print(paste(k, korder))
}

There is some evidence that the slurm order attempts when LACKING an _order file are amiss:

• see ./run_bhatt/OneCommandWSM_P6_STRLOAD_SCENCF/bhatt_one_command_wsm.csh
• for example, when running the basin JL6_6890_6990, the ETM part of the script appeared to shuffle the order
  • Monitored this with the following: fgrep JL /opt/model/p6/vadeq/tmp/rob-slurm/vadeq_2021_2022-09-08-13-09-08.out |grep -i FINISH
  • Note that JL6_7320_7150 runs ETM (or finishes etm) before JL6_7430_7320, which is supposed to be flowing into it.
  • Now, ETM doesn't NEED to run in order since it only handle local runoff, but the code should do this if I understand correctly?

FINISHETMLOG JL1_6770_6850
FINISHETMLOG JL6_7160_7440
FINISHETMLOG JL2_6850_6890
FINISHETMLOG JL1_7530_7430
FINISHETMLOG JL2_7350_7090
FINISHETMLOG JL2_7240_7350
FINISHETMLOG JL6_7320_7150
FINISHETMLOG JL6_7430_7320
FINISHETMLOG JL1_7080_7190
FINISHETMLOG JL1_7200_7250
FINISHETMLOG JL2_7250_7090
FINISHETMLOG JL1_6940_7200
FINISHETMLOG JL6_7440_7430
FINISHETMLOG JL6_6890_6990
FINISHETMLOG JL3_7090_7150
FINISHETMLOG JL1_7190_7250

• Test:
  • JL3_7090_7150 - this is a segment with 7 tribs, hopefully big enough to r\un afoul