Modify prepqc script to check for pre-qc data and skip if no data is present #184
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Modify code to check for presence of pre-qc files before running 'combfrd.x' and skip running prepqc if no pre-qc files are present
and skip combfrd.x and prepqc if no pre-qc files are present
gmao-msienkie
added
the
Non 0-diff
|
The test with unified radianceVarBC showed that the satbang rs was no longer needed. The test with the old bias correction (and $ANGLBC=0) required and acquired the satbang rs as needed |
rtodling
approved these changes
Jul 25, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Originally the prepqc script determined the templates for the 'pre-qc' files to be combined and passed through 'prepqc' by checking the output of the 'acquire_obsys' command. A few years ago the DAS was modified to obtain the templates for the 'pre-qc' files by checking the OBSCLASS and obtaining the templates from the obsys.rc. This was necessary because the 'acquire_obsys' was being run several times and the obsys.acq output from e.g. the acquire of aerosol data was overwriting the the acquire of the GSI observations.
The problem with this is the 'obsys.acq' had previously only had filename templates for data that was actually acquired for the segment while the newer method obtained templates for all of the files with pre-qc OBSCLASS entries even if the data was not being obtained.
This script adds a check that each pre-qc file exists before the filename is added to the input to the 'combfrd.x' program that combines the prepbufr file. If there are no matching pre-qc files present, the rest of the prepqc related processing is skipped.
Among other things, this allows a DAS run to transition between MERRA2/GEOSIT/R21c reanalysis data streams to the real-time GDAS stream (which does not need and should not have 'prepqc' processing performed on it) without having to stop and reconfigure. The obsys.rc file can be configured to use the 'pre-qc' files up to a certain date and the real-time files after that date.
The GEOSdas.csm modifications have been tested. There are similar modifications in 'prepqc_daemon.pl' that have not been tested since it doesn't look like that script is being used in current runs.
I have also made a small change that will hopefully remove the dependency on having the unneeded ana_satbang_rst file in the restart files. A test has been set up running with the old VarBC to verify that the ana_satbang_rst file will be handled correctly for that case.