Fix run tests buildbot

[aashish24]

No description provided.

[aashish24]

@doutriaux1 @chaosphere2112 @dlonie this is ready for review.

[alliepiper]

One of the ui tests is failing (missing env setup):

https://open.cdash.org/testDetails.php?test=332483462&build=3796225

Looks like the wrong baseline is being used or missing, or something in these:

https://open.cdash.org/testDetails.php?test=332676319&build=3796225
https://open.cdash.org/testDetails.php?test=332676324&build=3796225
https://open.cdash.org/testDetails.php?test=332676328&build=3796225

Widgets are drawn (but aren't in baseline), and no data:

https://open.cdash.org/testDetails.php?test=332676314&build=3796225

Not sure about this one:

https://open.cdash.org/testDetails.php?test=332617115&build=3796225

Diagnostics are having problems, too:

https://open.cdash.org/testDetails.php?test=332676353&build=3796225
https://open.cdash.org/testDetails.php?test=332676354&build=3796225
https://open.cdash.org/testDetails.php?test=332676355&build=3796225
https://open.cdash.org/testDetails.php?test=332676356&build=3796225

[alliepiper]

Looks like the wrapper approach did work out for most of the tests, though!

[aashish24]

thanks @dlonie. Looks like buildbot also reports 10 tests failing. I will see if they are also using the wrapper.

[jbeezley]

I think this one is related to #1250. Do any of these tests work elsewhere?

[jbeezley]

I have a few guesses.

Diagnostics are failing due to missing data. It is trying access data at /home/buildbot/uvcmetrics_test_data/cam_output, which doesn't exist. Possibly salix and einstein don't clean the CMakeCache containing some other prefix or CMake code to download the files was removed.

For the tests with images not matching, it looks to me like salix and einstein are failing in a way that isn't reported as a failure to ctest. Not images are are showing up at all for them and there is a suspicious error message before the output gets cropped.

[alliepiper]

It might be dirty state. UV-CDAT creates and makes heavy use of ~/.uvcdat for various things, perhaps something there is dirtying the test?

We should probably make sure that we remove ~/.uvcdat before the tests. This means we should also define the "Don't prompt me about automated feedback/usage stats" environment variable in runtest, too.

[jbeezley]

I am running cmake manually on the buidbot slave now and it seems to be downloading the test data. For some reason, when you configure as part of ctest, the data isn't downloaded.

[mathturtle]

When you figure out how the build needs to be changed, let me know.

[aashish24]

@dlonie the wrapper set the anonymous log variable.

thanks @jbeezley for looking into it. Please feel free to push any changes on this branch.

[aashish24]

LGTM. @doutriaux1 can we merge it when garant is done testing this branch?

[aashish24]

thanks @jbeezley

[jbeezley]

I enabled ccache on the builder so future builds should be faster.

[aashish24]

nice!

[jbeezley]

test-laptop looks okay, but I'm an getting odd black images from several tests on garant. I tried running some of them manually and they looked okay, so I don't know what is going on.

[aashish24]

may be some race condition? Are you running parallel build?

[jbeezley]

Yes, PARALLEL_LEVEL is set to 4.

[aashish24]

I am wondering if you set it to 1 do you still see black images?

[jbeezley]

@dlonie said this happens with NVidia drivers when another window is on top. I will turn of parallel testing and see what happens.

[aashish24]

right.. that's what I was thinking as well. So serial build should help. There is a way around it I think but I don't remember.

[aashish24]

@jbeezley can we merge this branch. This only fixes ctest so hopefully @doutriaux1 would be okay with you merging it.

[jbeezley]

Yeah, LGTM.

[doutriaux1]

I didn't get a chance to test this... So don't blame me if anything goes wrong 😉 but i trust you guys.

[aashish24]

thanks @doutriaux1. It worked on buildbot and some other systems. You will like this change. No need to
source setup_runtime.sh for running tests anymore even for dashboard machines. So your cron job script got lot simpler.

[durack1]

@doutriaux1 I think the key thing which has "fixed" things on travis is turning off the build against osx - if I do that for #1129, #1130 and #1131 I bet they also sail through the travis test for linux only..

[aashish24]

@durack1 feel free to update those branches with current master.

[durack1]

@aashish24 I just changed #1130 to build without osx on travis and it passes fine, I'm just not sure that removing the osx build test is a good idea..? Is there a branch which implements just the travis/buildbot fixes, I can't find the fix_run_tests_buildbot branch which appears to have been deleted

[durack1]

@aashish24 @doutriaux1 this might be the culprit http://stackoverflow.com/questions/7726649/library-fails-to-build-if-autoreconf-i-is-run-before-configure