Daily build fails because of modbus binding integration tests

[t2000]

I am opening this issue because I don't know what is wrong with it:

https://ci.openhab.org/job/openHAB-Addons/lastFailedBuild/org.openhab.addons.itests$org.openhab.binding.modbus.tests/console

I only see these warning from the code analysis:

[WARNING] Code Analysis Tool has found: 
 0 error(s)! 
 2 warning(s) 
 2 info(s)
[WARNING] OH-INF.thing.heatpump-types.xml:[0]
Missing the supported bridge with id tcp
[WARNING] OH-INF.thing.heatpump-types.xml:[0]
Missing the supported bridge with id serial

Unfortunately I am unable to simulate and build it locally, so I cannot fix it. Since its failing since 2 nights already it would be nice if someone could have a look, so we get a working openHAB distribution snapshot.

[Hilbrand]

It doesn't seem to be related to modbus, but fail randomly. build 229 failed on nest,tets, 230 failed on modbus.tests and 231 failed on maxcube.tests. But all with no clear error message in the log. Maybe it's a timeout issue or some other crash.

[daMihe]

I tried to analyse it. However the output of jenkins seems to be mixed between several tests (or more exactly: maven jobs). I wasn't able up to now to find the issues. Earlier (was also discussed somewhere here) we thought the issues of integration tests are timing issues.

Does maybe some know jenkins/maven better to get better output when doing parallel jobs?

[daMihe]

found the "discussion": #8820

[daMihe]

Maybe i've found a setting inside maven plugin of jenkins - but i'm still no specialist for jenkins. Who is the right person to talk to about these settings?

[Hilbrand]

@wborn or @kaikreuzer would know about jenkins settings.

[kaikreuzer]

@daMihe What setting do you have in mind?

[daMihe]

@kaikreuzer I'm experimenting with parallel modules build instead of adding cores to build:

That way jenkins executes each build in a separate job. The parallel executed job count is limited by the build processor count. This way every job has its own STDOUT and STDERR and the output is "clean". Afaik, maven does not seem to be able to prefix the output with a thread id or similar, so it must run single threaded to see a difference.

But my tiny build box is only a vm inside a small NUC, so it's quite hard to see from my sight, whether there is a performance impact (and how big it is). I'm also not sure if the PR-mechanism works like it does now, if "Block downstream trigger when building" is appropriate would have to be tested - i'm not sure whether this check is aware of branches.

[daMihe]

After adding a bit more ram to the vm i got more reproducible timing. It seems like starting maven for each job is inefficient. On my machine it takes around 2.5x as much time in comparison to -T 1C. Another issue is that the build of the main project is always successful because if a module fails, a sub job fails. So without further settings the overview about builds is lost.

I'm now not sure whether this is the right way. I've looked on the load of my build machine (dual-core). Interesting is, that, even if running single threaded (without -T 1C and multiple modules in parallel) the load is around 2 on my box. So maybe the server hosting the productive jenkins may be just overloaded. Just looking on what i see from the webinterface, the machine should have at least 48 real cores (6 build executors using 8 Threads each, no HT-Cores!) to not get overloaded. It that really the case? 48 cores is possible using DC hardware (though this is would be a high performance machine or something based on ARM/Cavium) - but the time a full build takes i would say the machine is overloaded.

Please don't feel demotivated, i'm trying to make things better and am learning especially about maven and jenkins now.

Beside the kind of build:

I had also luck getting one time a readable stacktrace from the integration test. The failing assert is inside a waitForAssert. The function repeats the lambda containing asserts until a timeout expires or the asserts are not failing. I'm pretty sure the timeout expires in those cases.

Here is the trace if you want to lookup yourself (only the relevant part):

TEST org.openhab.binding.modbus.tests.ModbusDataHandlerTest#testOnRegistersInt16StaticTransformation() <<< ERROR: DateTimeItem modbus_data_read1_lastReadSuccess_item, state null of type null
Expected: is is not null
     but: was null
java.lang.AssertionError: DateTimeItem modbus_data_read1_lastReadSuccess_item, state null of type null
Expected: is is not null
     but: was null
	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
	at org.openhab.binding.modbus.tests.ModbusDataHandlerTest.lambda$1(ModbusDataHandlerTest.java:248)
	at org.openhab.core.test.java.JavaTest.waitForAssert(JavaTest.java:152)
	at org.openhab.core.test.java.JavaTest.waitForAssert(JavaTest.java:120)
	at org.openhab.core.test.java.JavaTest.waitForAssert(JavaTest.java:73)
	at org.openhab.binding.modbus.tests.ModbusDataHandlerTest.assertSingleStateUpdate(ModbusDataHandlerTest.java:234)
	at org.openhab.binding.modbus.tests.ModbusDataHandlerTest.testOnRegistersInt16StaticTransformation(ModbusDataHandlerTest.java:526)

(Build on rev 572da80)

[wborn]

The modbus integration tests were fixed by #9668.