Elasticsearch based integration tests are failing due to authorization failures

[robbavey]

https://logstash-ci.elastic.co/job/elastic+logstash+master+multijob-integration-1/598/

Looks like certain actions performed by the integration test are now expecting authorization:

put_settings:

04:58:50     Failures:
04:58:50 
04:58:50       1) Test Elasticsearch output can ingest 37K log lines of sample apache logs
04:58:50          Failure/Error: es_allow_wildcard_deletes(@fixture.get_service("elasticsearch").get_client)
04:58:50          
04:58:50          Elasticsearch::Transport::Transport::Errors::Unauthorized:
04:58:50            [401] {"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication credentials for REST request [/_cluster/settings]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"missing authentication credentials for REST request [/_cluster/settings]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}
04:58:50          # /opt/logstash/build/qa/integration/vendor/jruby/2.5.0/gems/elasticsearch-transport-7.14.0/lib/elasticsearch/transport/transport/base.rb:218:in `__raise_transport_error'
04:58:50          # /opt/logstash/build/qa/integration/vendor/jruby/2.5.0/gems/elasticsearch-transport-7.14.0/lib/elasticsearch/transport/transport/base.rb:347:in `perform_request'
04:58:50          # /opt/logstash/build/qa/integration/vendor/jruby/2.5.0/gems/elasticsearch-transport-7.14.0/lib/elasticsearch/transport/transport/http/faraday.rb:36:in `perform_request'
04:58:50          # /opt/logstash/build/qa/integration/vendor/jruby/2.5.0/gems/elasticsearch-transport-7.14.0/lib/elasticsearch/transport/client.rb:191:in `perform_request'
04:58:50          # /opt/logstash/build/qa/integration/vendor/jruby/2.5.0/gems/elasticsearch-7.14.0/lib/elasticsearch.rb:41:in `method_missing'
04:58:50          # /opt/logstash/build/qa/integration/vendor/jruby/2.5.0/gems/elasticsearch-api-7.14.0/lib/elasticsearch/api/namespace/common.rb:38:in `perform_request'
04:58:50          # /opt/logstash/build/qa/integration/vendor/jruby/2.5.0/gems/elasticsearch-api-7.14.0/lib/elasticsearch/api/actions/cluster/put_settings.rb:44:in `put_settings'
04:58:50          # ./specs/spec_helper.rb:19:in `es_allow_wildcard_deletes'
04:58:50          # ./specs/es_output_how_spec.rb:27:in `block in <main>'
04:58:50          # ./rspec.rb:32:in `<main>'
04:58:50 

delete_template:

04:49:41 org.logstash.integration.RSpecTests > rspecTests STANDARD_ERROR
04:49:41     The client is unable to verify that the server is Elasticsearch due to security privileges on the server side. Some functionality may not be compatible if the server is running an unsupported product.
04:49:41 
04:49:41 org.logstash.integration.RSpecTests > rspecTests STANDARD_OUT
04:49:41       can ingest 37K log lines of sample apache logs (FAILED - 1)
04:49:42   
04:49:42     An error occurred in an `after(:context)` hook.
04:49:42     Failure/Error: clean_es(@fixture.get_service("elasticsearch").get_client)
04:49:42     
04:49:42     Elasticsearch::Transport::Transport::Errors::Unauthorized:
04:49:42       [401] {"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication credentials for REST request [/_template/*]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"missing authentication credentials for REST request [/_template/*]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}
04:49:42     # /opt/logstash/build/qa/integration/vendor/jruby/2.5.0/gems/elasticsearch-transport-7.14.0/lib/elasticsearch/transport/transport/base.rb:218:in `__raise_transport_error'
04:49:42     # /opt/logstash/build/qa/integration/vendor/jruby/2.5.0/gems/elasticsearch-transport-7.14.0/lib/elasticsearch/transport/transport/base.rb:347:in `perform_request'
04:49:42     # /opt/logstash/build/qa/integration/vendor/jruby/2.5.0/gems/elasticsearch-transport-7.14.0/lib/elasticsearch/transport/transport/http/faraday.rb:36:in `perform_request'
04:49:42     # /opt/logstash/build/qa/integration/vendor/jruby/2.5.0/gems/elasticsearch-transport-7.14.0/lib/elasticsearch/transport/client.rb:191:in `perform_request'
04:49:42     # /opt/logstash/build/qa/integration/vendor/jruby/2.5.0/gems/elasticsearch-7.14.0/lib/elasticsearch.rb:41:in `method_missing'
04:49:42     # /opt/logstash/build/qa/integration/vendor/jruby/2.5.0/gems/elasticsearch-api-7.14.0/lib/elasticsearch/api/namespace/common.rb:38:in `perform_request'
04:49:42     # /opt/logstash/build/qa/integration/vendor/jruby/2.5.0/gems/elasticsearch-api-7.14.0/lib/elasticsearch/api/actions/indices/delete_template.rb:48:in `delete_template'
04:49:42     # ./specs/spec_helper.rb:23:in `clean_es'
04:49:42     # ./specs/es_output_how_spec.rb:31:in `block in <main>'
04:49:42     # ./rspec.rb:32:in `<main>'

[kares]

this is due to elastic/elasticsearch-ruby#1428 we worked around the issue in plugins but not really in the integration specs (that directly use the elasticsearch gem) - believe we should just apply the same patch in integration tests ...