Ensure consistent use of system properties in tests run by Gradle as well IDEA

[ywelsch]

The Gradle test runner is using different system properties when running tests than what is used when tests are launched from the IDE (IntelliJ). This makes it tedious to run tests properly. Many security tests for example break because the system property es.set.netty.runtime.available.processors is not set (we set that for the Gradle test runner).

This provides a poor experience when running tests in the IDE, so I think that a fix should be prioritized accordingly.

[elasticmachine]

Pinging @elastic/es-delivery (Team:Delivery)

[mark-vieira]

This actually isn't a straitforward problem to solve. Unlike other settings which are applied to all tests, the system property you mention is configured explicitly on some test tasks for some projects. There's no mechanism in IntelliJ to convey this information to the IDE on a per-module basis. Realistically, the best way forward for stuff like this is to get to a position where we can delegate all test execution to Gradle vs using the platform test runner. Only that would guarantee a completely consistent experience.

[mark-vieira]

Alternatively we could just set this globally for all tests. Would it be problematic to do so?

[ywelsch]

I don't think it should be problematic, but I would rather have it the other way around (not having to specify this property in Gradle nor IDEA): Given that this property (es.set.netty.runtime.available.processors) needs to be set whenever tests with the corresponding code are run is an indication that we got this backwards. Running the corresponding code in a JUnit context should disable the corresponding check by default (e.g. using a trick similar to what org.elasticsearch.common.Randomness employs to detect execution in a JUnit test context).

[mark-vieira]

Running the corresponding code in a JUnit context should disable the corresponding check by default (e.g. using a trick similar to what org.elasticsearch.common.Randomness employs to detect execution in a JUnit test context).

I don't have super strong feelings on this but given this is "configuration" couldn't we just add something to ESTestCase to set this system property? Or something else that ES code subsequently looks for. This avoid hacky usages of reflection to detect a test execution context in production code.

[ywelsch]

That would work for me (there is already ESTestCase.setTestSysProps for this purpose).