env_mach_pes.xml not being locked properly

[fischer-ncar]

I ran into this problem while doing testing on cheyenne. I was able to make changes in env_mach_pes.xml and submit to the queue without any error messages.

I was also able to change env_mach_pes.xml, then do a rebuild and submit without redoing case.setup.

[sarich]

I'm unable to reproduce this error locally (I don't have a cheyenne account), maybe it would help if I could get the git version and the exact commands and xml changes that were made.

And of course if this is only happening on cheyenne then someone else may have to look into it.

[fischer-ncar]

This is what I did using the current cime/master.

git version 1.8.5.6

./create_test SMS.f45_g37_rx1.A.cheyenne_intel
./xmlchange NTASKS=-2
./case.submit # run submits to queue, but fails

./case.build --clean
./case.build
./case.submit # run submits to queue, but fails

I wasn't able to get create_newcase to repeat this behavior. So, it's something specific with create_test.

[sarich]

Thanks, that was very helpful. I think I am getting the same behavior now and will investigate.

NOTES:
Running create_newcase and changing the NTASKS gets the expected behavior:

./create_newcase --case test1 --res f45_g37_rx1 --mach anlworkstation --compiler gnu --compset A
cd test1
./case.setup
./case.build
./xmlchange NTASKS=-2
./case.submit
File /nfs2/sarich/cime4/scripts/test1/LockedFiles/env_mach_pes.xml has been modified
found difference in NTASKS:{'component': 'ICE'} : case '-2' locked '-1'
found difference in NTASKS:{'component': 'ESP'} : case '-2' locked '-1'
found difference in NTASKS:{'component': 'CPL'} : case '-2' locked '-1'
found difference in NTASKS:{'component': 'OCN'} : case '-2' locked '-1'
found difference in NTASKS:{'component': 'LND'} : case '-2' locked '-1'
found difference in NTASKS:{'component': 'ATM'} : case '-2' locked '-1'
found difference in NTASKS:{'component': 'WAV'} : case '-2' locked '-1'
found difference in NTASKS:{'component': 'GLC'} : case '-2' locked '-1'
found difference in NTASKS:{'component': 'ROF'} : case '-2' locked '-1'

But using create_test for the same test case fails to catch the change:
./create_test SMS.f45_g37_rx1.A
cd ~/acme/scratch/SMS.f45_g37_rx1.A.anlworkstation_gnu.20170419_155053_r7bnhy
./xmlchange NTASKS=-2
./case.submit
...tries to run...

[sarich]

checked_lockfile() was counting the number of dots in full pathname and ignored any locked files that had more than one period in it. So if the case name had any dots, no locked files were checked.