improve plugin integration tests

[rmuir]

Currently some of these are very minimal:

• cloud plugins just check the plugin was installed
• most analyzer plugins just check the analyze api (ideally they have one that indexes a document too)

Improving these with additional rest tests would be great. The analyzer ones should be relatively simple tests.

[nik9000]

This issue itself isn't really low hanging fruit - but lots of the tests @rmuir is proposing certainly are. They would be a great place for any contributor to start.

Cloud plugins are hard - to really test them you need to run them against the cloud environment. I don't know how many of those environments have public test instances....

[rmuir]

Actually @dadoonet did a lot of work to improve the analysis tests that i described here recently. So they might be in good shape at the moment. But still might be worth looking at analysis/ and lang/ plugins to see if there are gaps.

I agree the cloud plugins are hard. Ideally we would setup a mock in pre-integration-test or something like that, even if its a limited mock of the service.

[nik9000]

I agree the cloud plugins are hard. Ideally we would setup a mock in pre-integration-test or something like that, even if its a limited mock of the service.

Yeah - mocks are fun because you can simulate failure modes.

[dadoonet]

Well about mock, I tried to do some mocking for azure. For example: https://github.com/elastic/elasticsearch-cloud-azure/blob/master/src/test/java/org/elasticsearch/cloud/azure/management/AzureComputeServiceTwoNodesMock.java

Would be easier may be if we could use something like Mockito ?

[nik9000]

Would be easier may be if we could use something like Mockito ?

I've used Mockito when testing other Elasticsearch plugins pretty successfully - but its not really as good as having a real mock service you can play with because you don't test the full communication paths. Mockito is one of those things that untethers you from reality a bit too easy.

[clintongormley]

I'll take this and work on rest tests for the non-cloud plugins. Cloud plugins will be dealt with by #12719

[clintongormley]

Had a look at the existing REST tests and we seem to be pretty well covered here. Closing