Derecho transition: Move stuff from /glade/p

[samsrabin]

Originally part of #1995 from @slevis-lmwg:

/glade/p will be discontinued, so move stuff from there to Derecho replacement. A lot of this is CESM-wide stuff like inputdata, though we may have some CTSM-specific stuff, such as /glade/p/cgd/tss.

@ekluzek made this google document.

Also, see some updates there:

• Sam L. 2023-08-10
• Will 2023-08-10
• Erik 2023-08-10

[ekluzek]

We need to move:

• cgd/tss

  • ctsm_baselines, going to /glade/campaign/cgd/tss/ctsm_baselines , @ekluzek
  • datm forcing, see below, @slevis-lmwg , @olyson
  • CLM50_data_tools (user contributed scripts) and CropEvalData -> glade/campaign/cgd/tss/projects/, @olyson , DONE 11/16/23
  • @ekluzek move CTSM_tutorial2019 to a To_Be_Safely_Deleted directory
  • @ekluzek move historyfiles to a To_Be_Safely_Deleted directory
  • people/* - each person under this directory responsible for copying/moving their files: andywood, dll, erik, guoqiang, hydrofcst, jkshuman, lawrence, negins, oleson, slevis

• lmwg - In general @ekluzek suggested moving this to /campaign/cesm/community/lmwg
  In addition to the datm forcing directories mentioned below there are these directories:
  @slevis-lmwg DONE 11/16/23

  • cases
  • CLM2014_tutorial_space
  • CLM2016_tutorial_space
  • CLM-FATES2018_tutorial_space
  • CTSM2018_gittutorial_space
  • diag
  • npi_surfdata
  • ObsData

• lmwg_dev

  • each person under this directory responsible for copying/moving their files: dlawren, dll, swensosc, wwieder

• sdwg_dev

  • Dec 6 meeting @lawrencepj1 @ekluzek @slevis-lmwg to coordinate this data transfer.

• bgcwg_dev

• Move/glade/campaign/cesm/development/bgcwg/forcing/b.e20.B1850.f09_g17.pi_control.all.221.cplhist/cpl/hist.mon

• Change testmod to point to /glade/campaign/cesm/development/bgcwg/forcing/b.e20.B1850.f09_g17.pi_control.all.221.cplhist/cpl/hist.mon

• datm forcing directories @slevis-lmwg will do this w/ support from @olyson

  • some currently in /glade/p/cesm/lmwg and some in /glade/p/cgd/tss/CTSM_datm_forcing_data
  • all new data should be copied to /glade/campaign/cesm/cesmdata/cseg/inputdata/atm/datm7
  • @swensosc has updated NLDAS data that he'll move here too
  • Let's just copy the most recent: @slevis-lmwg DONE 11/16/23
    @slevis-lmwg copied atm_forcing.datm7.cruncep_qianFill.0.5d.v7.c160715 to atm_forcing.datm7.cruncep_qianFill.0.5d.v7.c160715_TEMPORARY. I was thinking that within 10 seconds I could remove the soft-linked copy and replace it with the "TEMPORARY" copy using the move command; instead of Erik having to do the following:
  • @ekluzek currently has softlinks in /glade/campaign/cesm/cesmdata/cseg/inputdata/atm/datm7 that point to /glade/p/cgd/tss/CTSM_datm_forcing_data. These data need to actually be moved (and suggested doing this Friday after Thanksgiving & letting people know).
  • @ekluzek is going to run a test that uses a soft link and then moves the data to see if an active case crashes.
  • Sean will keep the wfdei data in /project/tss/swensosc/input_data because it is unlikely to be used again.

@olyson recommends doing the file copying with globus.

[ekluzek]

I think for the datm forcing directories they just need to be moved over to:

$DIN_LOC_ROOT/atm/datm7
/glade/campaign/cesm/cesmdata/cseg/inputdata/atm/datm7
which should have enough room now.

We want to be careful replacing softlinks with local copies, because we may crash peoples' active jobs.
@mvdebolskiy suggests: rsync has --copy-links option that gets the target contents from a symbolic link

[swensosc]

@erik can you delete this directory, and I will copy over the newer nldas data in its place? /glade/campaign/cesm/cesmdata/cseg/inputdata/atm/datm7/NLDAS/

[wwieder]

@swensosc it looks like Tony Craig is the owner of that file.
/glade/campaign/cesm/cesmdata/cseg/inputdata/atm/datm7/NLDAS/

Can you coordinate with @ekluzek to make sure we have a single copy of the most up to date nldas data?

[ekluzek]

One part of this is Keith Lindsay did the following move for us

The contents of
/glade/p/cesm/bgcwg_dev/forcing/b.e20.B1850.f09_g17.pi_control.all.221.cplhist/cpl/hist.mon
are now at
/glade/campaign/cesm/development/bgcwg/forcing/b.e20.B1850.f09_g17.pi_control.all.221.cplhist/cpl/hist.mon

We need to update the code accordingly.