Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update to obsproc.v1.0.2 and prepobs.v1.0.1 in develop #1033

Closed
6 tasks done
KateFriedman-NOAA opened this issue Sep 27, 2022 · 0 comments · Fixed by #1049
Closed
6 tasks done

Update to obsproc.v1.0.2 and prepobs.v1.0.1 in develop #1033

KateFriedman-NOAA opened this issue Sep 27, 2022 · 0 comments · Fixed by #1049
Assignees
@KateFriedman-NOAA
Labels
maintenance Regular updates and maintenance work
Milestone
Port develop to W...

Comments

@KateFriedman-NOAA
Copy link
Member

KateFriedman-NOAA commented Sep 27, 2022 •
edited
Loading

Description

The develop branch is using an outdated version of the obsproc and prepobs packages. The obsproc and prepobs systems had major updates in the port to WCOSS2, including turning prepobs into a module that is loaded at runtime.

Refs #419

Make the following updates in develop:

  • 1. Replace HOMEobsproc_prep, HOMEobsproc_network with HOMEobsproc; remove HOMEobsproc_global variable.
  • 2. Set HOMEobsproc to the local installs of obsproc.v1.0.2 (depending on timing, may move to the newer obsproc.v1.1.0 which is being handed off with GFSv16.3).
  • 3. Add module loads for prepobs.v1.0.1 into module_base modulefiles.
  • 4. Replace $HOMEobsproc_network/jobs/JGLOBAL_PREP in jobs/rocoto/prep.sh with $HOMEobsproc/jobs/JOBSPROC_GLOBAL_PREP.
  • 5. Additionally...retire config.prepbufr. Upon examination, the config.prepbufr and its only use to set GESROOT in the prep job is no longer needed. Delete config.prepbufr and remove it from the config list in jobs/rocoto/prep.sh.

When version files and version variables are introduced into develop the HOMEobsproc variable will be set using obsproc_ver in version files and prepobs_ver used in the modulefiles will also be set in version files.

Acceptance Criteria (Definition of Done)

Prep jobs runs successfully with new obsproc/prepobs versions and produces expected files (e.g. prepbufr). Check if resulting prepbufr files differ from prepbufr produced from current obsproc package in develop.

Dependencies

  • Local installs of new obsproc and prepobs versions are already in place on supported platforms.
@KateFriedman-NOAA KateFriedman-NOAA added the maintenance Regular updates and maintenance work label Sep 27, 2022
@KateFriedman-NOAA KateFriedman-NOAA self-assigned this Sep 27, 2022
@KateFriedman-NOAA KateFriedman-NOAA changed the title Update obsproc and prepobs in develop Update to obsproc.v1.0.2 and prepobs.v1.0.1 in develop Sep 28, 2022
KateFriedman-NOAA added a commit to KateFriedman-NOAA/global-workflow that referenced this issue Sep 28, 2022
@KateFriedman-NOAA
Retire config.prepbufr
6f2db6c 
- The contents of config.prepbufr are only setting GESROOT;
which is no longer used in the GFS in operations.
- Don't need to set GESROOT so don't need config.prepbufr.
- Delete config.prepbufr.
- Remove "prepbufr" from config list in prep.sh.

Refs NOAA-EMC#1033
KateFriedman-NOAA added a commit to KateFriedman-NOAA/global-workflow that referenced this issue Sep 28, 2022
@KateFriedman-NOAA
Update primary obsproc JJOB call in prep.sh
97dba45 
The updated obsproc package renamed the primary global JJOB script
and the HOMEobsproc variable is the only home path for the package,
so update prep.sh to now call:
$HOMEobsproc/jobs/JOBSPROC_GLOBAL_PREP
instead of the old:
$HOMEobsproc_network/jobs/JGLOBAL_PREP

Refs NOAA-EMC#1033
KateFriedman-NOAA added a commit to KateFriedman-NOAA/global-workflow that referenced this issue Sep 28, 2022
@KateFriedman-NOAA
Add prepobs module load to R&D module_base modulefiles
7fbd166 
- Prepend MODULEPATH with local install of prepobs package on R&Ds.
- Load prepobs/1.0.1 module from local install.

The prepobs package is now a module that is loaded at runtime.

Refs NOAA-EMC#1033
KateFriedman-NOAA added a commit to KateFriedman-NOAA/global-workflow that referenced this issue Sep 28, 2022
@KateFriedman-NOAA
Update HOMEobsproc paths in config.base
44c0a9f 
- Consolidate HOMEobsproc_prep, HOMEobsproc_network, and HOMEobsproc_global
into a single HOMEobsproc path variable.
- Set HOMEobsproc to local install location and current version 1.0.2 in
config.base.emc.dyn.
- Set HOMEobsproc to operational package location and current version 1.0.2
in config.base.nco.static.

Refs NOAA-EMC#1033
KateFriedman-NOAA added a commit to KateFriedman-NOAA/global-workflow that referenced this issue Sep 30, 2022
@KateFriedman-NOAA
Add cdate10 setting to config.prep
428b438 
The cdate10 variable is ${PDY}${cyc} and is used by the obsproc package
to set CDATE10/CDATEMM.

Refs NOAA-EMC#1033
KateFriedman-NOAA added a commit to KateFriedman-NOAA/global-workflow that referenced this issue Sep 30, 2022
@KateFriedman-NOAA
Add launcher_PREP to HERA and ORION env files
684c426 
Refs NOAA-EMC#1033
KateFriedman-NOAA added a commit to KateFriedman-NOAA/global-workflow that referenced this issue Oct 3, 2022
@KateFriedman-NOAA
Retire prepbufr step from config.resources
3f8c53b 
Refs NOAA-EMC#1033
KateFriedman-NOAA added a commit to KateFriedman-NOAA/global-workflow that referenced this issue Oct 4, 2022
@KateFriedman-NOAA
Add needed COMINtcvital path to config.prep
24279f1 
- The prep job needs to know COMINtcvital so it can find the syndata.tcvitals file
from the global dump archive in developer testing; uses compath.py in ops.
- Add COMPONENT setting for use in COMINtcvital setting.

Refs NOAA-EMC#1033
@KateFriedman-NOAA KateFriedman-NOAA mentioned this issue Oct 4, 2022
Merged
8 tasks
KateFriedman-NOAA added a commit to KateFriedman-NOAA/global-workflow that referenced this issue Oct 7, 2022
@KateFriedman-NOAA
Remove COMPONENT use from config.prep
cdeedc5 
- Remove COMPONENT setting.
- Remove use of COMPONENT in setting COMINtcvital and hardcode "atmos" in path

Refs NOAA-EMC#1033
KateFriedman-NOAA added a commit that referenced this issue Oct 7, 2022
@KateFriedman-NOAA
Update to obsproc.v1.0.2 and prepobs.v1.0.1 (#1049)
b26a8ac 
* Update HOMEobsproc paths in config.base
* Update primary obsproc JJOB call in prep.sh
* Add prepobs module load to R&D module_base modulefiles
* Add cdate10 setting to config.prep
* Add launcher_PREP to HERA and ORION env files
* Add needed COMINtcvital path to config.prep
* Retire config.prepbufr and prepbufr step from config.resources

Refs #1033
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@KateFriedman-NOAA KateFriedman-NOAA

Labels
maintenance Regular updates and maintenance work
Projects
None yet
Milestone
Port develop to WCOSS2
Development

Successfully merging a pull request may close this issue.

Update to obsproc.v1.0.2 and prepobs.v1.0.1 KateFriedman-NOAA/global-workflow
1 participant
@KateFriedman-NOAA