cime nightly tests should test all supported compilers on the test machines

[billsacks]

My understanding is that the cime nightly tests - and scripts_regression_tests in general - just tests the default compiler on the given machine. This fails to catch problems like #1769 . We should run the cime nightly tests across all supported compilers on the test machines, and possibly extend scripts_regression_tests so that it always runs at least one test from each supported compiler on the current machine.

I personally don't feel we need to run all system tests on all compilers. But ideally we'd have at least one debug test and one non-debug test on each compiler on each tested machine.

cc @jedwards4b @jgfouca @mvertens @rljacob @fischer-ncar

[jedwards4b]

@billsacks I believe that any developer can add and maintain cdash tests - I would suggest that you add any additional testing you are interested in.

[rljacob]

I think what @billsacks wants is for jenkins to automatically run tests on different compilers on a given machine. That requires messing with jenkins options and that's not open to this developer.

[jedwards4b]

We don't use jenkins - we have a simple collection of cronjobs that run and submit results to cdash.

[jgfouca]

@jedwards4b yes, in that case the selection of compiler should be easy with an additional option to create_test. @billsacks is right though, the testing of non-default compilers is severely lacking in scripts_regression_tests.