HARP Weekly 9/12/2022

[rburghol]

• Prototype Meeting format:
  • Lightning Round Check-in -- each person goes 2ish minutes:
    • Brief of last weeks progress (only by the person who did last weeks summary email)
    • Personal and group priorities for the week.
    • Availability for the week.
  • Work session
    • Decide on topic focus and persons
    • Break-out rooms?
    • What advisors are needed?
• FTABLES FTABLEs for streams #331
  • plan session with scotty and joey or rob
• HYDRO etc Create script : script will read in hydr file and create csv [rovol_file_name] in the form . #482
  • Plan/Do R and sqldf briefing (Joey or Rob)
  • Model script callers Meta-model Runtime Components meta_model#1
• DEQ Stream Order code (for execution): Basin Order with slurm and p6/om cbp_wsm#86

[glenncampagna]

Using sqldf to add withdrawals and point sources to hydr

Example in R script:

hydr <- sqldf( 
  "select a.*, b.divr_cfs 
from hydr as a
left outer join divr as b
on (  
a.year = b.year
and a.month = b.month
and a.day = b.day
)
order by a.year,a.month,a.day,a.hour
")

Testing the hydr conversion script that creates aliases:
Use (new): hsp_hydr_conversion.R [riverseg] [output/scenario path]
Example: Rscript HARP-2022-Summer/AutomatedScripts/hsp_hydr_conversion.R OR1_7700_7980 /media/model/p532/out/river/hsp2_2022
Output: the hydr csv now includes columns for withdrawals and point sources

head(hydr)
                 index       DEP     IVOL O1 O2       O3 OVOL1 OVOL2     OVOL3
1: 1984-01-01 01:00:00 0.2415072 8.847264  0  0 2.055183     0     0 0.1229398
2: 1984-01-01 02:00:00 0.3002159 8.828485  0  0 3.175832     0     0 0.2161576
3: 1984-01-01 03:00:00 0.3486096 8.810900  0  0 4.282218     0     0 0.3081839
4: 1984-01-01 04:00:00 0.3905506 8.793962  0  0 5.374578     0     0 0.3990412
5: 1984-01-01 05:00:00 0.4279465 8.777404  0  0 6.453111     0     0 0.4887475
6: 1984-01-01 06:00:00 0.4619085 8.761090  0  0 7.517995     0     0 0.5773184
   PRSUPY       RO     ROVOL     SAREA        TAU     USTAR      VOL VOLEV
1:      0 2.055183 0.1229398  67.47366 0.02344255 0.1099862 15.79433     0
2:      0 3.175832 0.2161576  83.87602 0.02914127 0.1226281 24.40666     0
3:      0 4.282218 0.3081839  97.39652 0.03383873 0.1321425 32.90938     0
4:      0 5.374578 0.3990412 109.11423 0.03790982 0.1398658 41.30430     0
5:      0 6.453111 0.4887475 119.56212 0.04153978 0.1464090 49.59296     0
6:      0 7.517995 0.5773184 129.05061 0.04483640 0.1521076 57.77673     0
         date hour day month year     Qout divr_cfs ps_afd
1: 1984-01-01    1   1     1 1984 1.487572        0      0
2: 1984-01-01    2   1     1 1984 2.615507        0      0
3: 1984-01-01    3   1     1 1984 3.729025        0      0
4: 1984-01-01    4   1     1 1984 4.828398        0      0
5: 1984-01-01    5   1     1 1984 5.913844        0      0
6: 1984-01-01    6   1     1 1984 6.985553        0      0

• To-do:
• implement environment (CBP) variables into the hsp_hydr_conversion.R script (via arguments)
  Example in bash/terminal:

/opt/model/p53/p532c-sova$ . hspf_config
seg=OR1_7700_7980
scenario=hsp2_2022
Rscript ~/HARParchive/HARP-2022-Summer/AutomatedScripts/hsp_hydr_conversion.R $seg $CBP_EXPORT_DIR/river/$scenario

This syntax would probably be the easiest way to use the conversion script from the model script. It looks like $seg and $scenario are set by run_river.csh

• confirm units for withdrawals and point sources (from timeseries group in h5 files)
• provide syntax for using export_hsp_h5.R to export divr and ps csvs from h5s (in TIMESERIES). This is needed for use of the updates to conversion script and adding divr and ps to hydr table
  Use: export_hsp_h5.R [h5 path] [output csv path] [data source]
  Example:
  Rscript ~/HARParchive/HARP-2022-Summer/AutomatedScripts/export_hsp_h5.R /media/model/p532/out/river/hsp2_2022/h5/OR1_7700_7980.h5 /media/model/p532/out/river/hsp2_2022/divr/OR1_7700_7980_divr.csv /TIMESERIES/TS3007/table
  Note: the 3007 DSN used corresponds to only DIVR
  Important DSNs:

3000 | Point source 
3007 | DIVR  
3008 | DIVA

Should we add a step to our workflow where we sum the withdrawal data (DIVR + DIVA) @rburghol ?

[rburghol]

This is excellent @glenncampagna . Thanks!

[glenncampagna]

Confirming Units for diversions/withdrawals and point sources

From River UCI file:

Withdrawals/Diversions are given type OUTDGT

Point sources are given type IVOL
From HSPF manual:

OUTDGT has units of cfs
and from our Data Dictionary #237 : IVOL = sum of inflows to the RCHRES [ac.ft/ivld]
We found that both point sources and diversions are given as daily data, so:

Point sources = ac-ft/day
Withdrawals/diversions = cfs

[rburghol]

This is perfect @glenncampagna. Thanks!

[rburghol]

Follow up for @glenncampagna and @juliabruneau -- looking through the code today Joey and I realized that the rchres/hydr routines can reject some demand from OUTDGT, so the OUTDGT should be thought of as demand, while we need to think about the actual withdrawal as potentially something else. I'm gonna do a little re-org on the issue tomorrow to flesh this out but off the top of my head I am thinking we will do OUTDGT aliased as demand, and see if one (or more) of the ROVOLx/OVOLx variables contain just the amount withdrawn. We may put that into wd_mgd -- talk later on this.