Confusing BASELINE result: PASS but also "Could not interpret CPRNC output"

[billsacks]

I'm confused by the TestStatus I'm seeing in a test I just ran with baseline comparisons:

PASS SMS_Ld2.ne30pg3_t232.BLT1850.derecho_intel.allactive-defaultio BASELINE cesm3_0_alpha03c.derecho: Could not interpret CPRNC output

There are two things that I find confusing about this:

(1) It's not clear to me what CPRNC output is leading to this "Could not interpret" message. I'm attaching the TestStatus.log file here. At a glance, it looks like all cprnc output is showing typical "IDENTICAL" lines.

TestStatus.log

(2) If it's really the case that there's at least one CPRNC file for which the code can't interpret the output, I feel like it would be safest to mark the result as a FAIL rather than a PASS, so that this isn't overlooked.

It looks like @jasonb5 introduced this in e33bede, so Jason, I'm assigning this to you in hopes that you can help clarify this for me.

[billsacks]

I reran this and got the same message. I'm also seeing the same message in a lot of the tests that @fischer-ncar has been running recently.

[fischer-ncar]

I'm seeing these messages too. But didn't notice them since it was the BASELINE was passing.

[jedwards4b]

Comments passed to hist_utils:get_ts_synopsis are

Comparing hists for case 'SMS_Ld5.f09_f09_mg17.PC6.derecho_intel.cam-cam6_port_f09.C.20240927_080217_54xyj4' dir1='/glade/derecho/scratch/jedwards/SMS_Ld5.f09_f09_mg17.PC6.derecho_intel.cam-cam6_po\
rt_f09.C.20240927_080217_54xyj4/run', suffix1='',  dir2='/glade/campaign/cesm/cesmdata/cesm_baselines/cesm3_0_alpha03d/SMS_Ld5.f09_f09_mg17.PC6.derecho_intel.cam-cam6_port_f09' suffix2=''           
  comparing model 'cam'                                                                                                                                                                               
    SMS_Ld5.f09_f09_mg17.PC6.derecho_intel.cam-cam6_port_f09.C.20240927_080217_54xyj4.cam.h0a.0001-01-01-00000.nc matched cam.h0a.0001-01-01-00000.nc                                                 
    SMS_Ld5.f09_f09_mg17.PC6.derecho_intel.cam-cam6_port_f09.C.20240927_080217_54xyj4.cam.h0i.0001-01-01-00000.nc matched cam.h0i.0001-01-01-00000.nc                                                 
    SMS_Ld5.f09_f09_mg17.PC6.derecho_intel.cam-cam6_port_f09.C.20240927_080217_54xyj4.cam.h1i.0001-01-01-00000.nc matched cam.h1i.0001-01-01-00000.nc                                                 
PASS        

I see no reason for this to trigger the "Could not interpret CPRNC output" - except that as far as I can tell the comment should not have been passed in at all? @jasonb5

[jasonb5]

I see the issue the catch-all is Could not interpret CPRNC output so even if diff_test: the two files seem to be IDENTICAL is present it will always return a synopsis that is not empty. Not the intended behavior, I'll have a fix shortly.

[billsacks]

Thanks a lot @jasonb5 !

[peverwhee]

@jasonb5 While running our CAM regression tests, I'm encountering these ERROR Could not interpret CPRNC output errors with CIME tag cime6.1.49 (which includes your fix). This is the comment that it's trying to parse for one of the tests:

Comparing hists for case 'SMS_D_Ln9.f19_f19_mg17.QPC5M7.derecho_intel.cam-outfrq9s.GC.aux_cam_intel_20241218131727' dir1='/glade/derecho/scratch/courtneyp/aux_cam_intel_20241218131727/SMS_D_Ln9.f19_f19_mg17.QPC5M7.derecho_intel.cam-outfrq9s.GC.aux_cam_intel_20241218131727/run', suffix1='',  dir2='/glade/campaign/cesm/community/amwg/cam_baselines/cam6_4_047_intel/SMS_D_Ln9.f19_f19_mg17.QPC5M7.derecho_intel.cam-outfrq9s' suffix2=''
  comparing model 'cam'
    SMS_D_Ln9.f19_f19_mg17.QPC5M7.derecho_intel.cam-outfrq9s.GC.aux_cam_intel_20241218131727.cam.h0a.0001-01-01-16200.nc matched cam.h0a.0001-01-01-16200.nc
    SMS_D_Ln9.f19_f19_mg17.QPC5M7.derecho_intel.cam-outfrq9s.GC.aux_cam_intel_20241218131727.cam.h0i.0001-01-01-16200.nc matched cam.h0i.0001-01-01-16200.nc
    SMS_D_Ln9.f19_f19_mg17.QPC5M7.derecho_intel.cam-outfrq9s.GC.aux_cam_intel_20241218131727.cam.i.0001-01-01-16200.nc matched cam.i.0001-01-01-16200.nc
  comparing model 'slnd'
    no hist files found for model slnd
  comparing model 'sice'
    no hist files found for model sice
  comparing model 'docn'
    no hist files found for model docn
  comparing model 'srof'
    no hist files found for model srof
  comparing model 'sglc'
    no hist files found for model sglc
  comparing model 'swav'
    no hist files found for model swav
  comparing model 'sesp'
    no hist files found for model sesp
  comparing model 'cpl'
    SMS_D_Ln9.f19_f19_mg17.QPC5M7.derecho_intel.cam-outfrq9s.GC.aux_cam_intel_20241218131727.cpl.hi.0001-01-01-16200.nc matched cpl.hi.0001-01-01-16200.nc
PASS

I'm well outside my understanding of the code, but maybe we also need to check for "PASS"?