Wait for integration with elasticsearch queries never resolves

[stuartnelson3]

APM Server version (apm-server version):

apm-server version 7.15.0 (amd64), libbeat 7.15.0 [53fd4dce370efa8eef388bbee0f198e8f4f94eac built 2021-08-26 20:32:28 +0000 UTC]

Description of the problem including expected versus actual behavior:

Expected behavior:

If kibana is not enabled, apm-server should infer the apm-server integration has been installed by the presence of index templates in elasticsearch.

Actual behavior:

After installing the integration, apm-server continues to not find the index templates:

{"log.level":"error","@timestamp":"2021-08-30T16:14:46.238+0200","log.logger":"beater","log.origin":{"file.name":"beater/waitintegration.go","file.line":85},"message":"error querying Elasticsearch for integration index templates: unexpected HTTP status: 404 Not Found ({\"error\":{\"root_cause\":[{\"type\":\"resource_not_found_exception\",\"reason\":\"index template matching [traces-apm.sampled] not found\"}],\"type\":\"resource_not_found_exception\",\"reason\":\"index template matching [traces-apm.sampled] not found\"},\"status\":404})","service.name":"apm-server","ecs.version":"1.6.0"}

Steps to reproduce:

1. Start a fresh cluster: ./scripts/compose.py start 7.15 --no-apm-server --remove-orphans
2. Start apm-server with data streams enabled and kibana disabled: ./apm-server -c apm-server.yml -e -E apm-server.data_streams.enabled=true -E apm-server.kibana.enabled=false
3. Add the elastic apm integration via kibana
4. Confirm that the integration is installed: curl -s user:pass@localhost:5601/api/fleet/epm/packages/apm | jq .response.status
5. The server is failing on finding index template traces-apm.sampled

[stuartnelson3]

Note: The original PR has traces-sampled, which I see present in my index templates. This was changed to traces-apm.sampled in #5952.

[axw]

@stuartnelson3 did you install integration version 0.4.0, using a local registry? The latest package should install traces-apm.sampled.

[stuartnelson3]

I did not, I'll do that now

[stuartnelson3]

Issue was fixed using the apmpackage 0.4.0