From 11d763187b3c94ed5f9b6a7932f1d3f65f6133e6 Mon Sep 17 00:00:00 2001 From: gregoryfranklin <45693481+gregoryfranklin@users.noreply.github.com> Date: Thu, 22 Aug 2019 12:05:28 +0100 Subject: [PATCH] Add Elasticsearch 7 support (#1690) * Make it possible to run es-integration-test against different versions of elasticsearch A convenient way to run the elasticsearch integration test is to just run ./scripts/travis/es-integration-test.sh This change allows you to specify different elasticsearch versions eg ES_VERSION=5.6.12 scripts/travis/es-integration-test.sh ES_VERSION=6.8.1 scripts/travis/es-integration-test.sh ES_VERSION=7.2.0 scripts/travis/es-integration-test.sh The default version to use for the tests is currently 5.6.12 Signed-off-by: Greg Franklin * Update github.com/olivere/elastic to 6.2.21 Signed-off-by: Greg Franklin * Migrate to a single '_doc' elasticsearch document type Signed-off-by: Greg Franklin * Add IncludeTypeName for compatability with elasticsearch 7.x Signed-off-by: Greg Franklin * Remove deprecated __default__ field from es mappings Signed-off-by: Greg Franklin * Rebase on master. Fix for ES5 Signed-off-by: Greg Franklin * Update esRollover.py to use per-version templates Signed-off-by: Greg Franklin * Update github.com/olivere/elastic to 6.2.22 and use RestTotalHitsAsInt against ES7 Signed-off-by: Greg Franklin * esRollover.py should set '?include_type_name=true' when creating templates for ES7 Signed-off-by: Greg Franklin * Check loading of all versions of the ES mappings Signed-off-by: Greg Franklin * Run es-integration-test.sh against ES5, ES6 and ES7 Signed-off-by: Greg Franklin * Run 'make fmt' to add license text to mocks Signed-off-by: Greg Franklin * Update elasticsearch versions used for integration tests (and add --rm to docker commands) Signed-off-by: Greg Franklin * Do not use IncludeTypeName on ES7 Without IncludeTypeName, ES will use the type "_doc" all documents. Type is removed from all search queries so that it queries all types whether they be "_doc" or (eg) "span". Type should not be used when indexing documents on ES7 (so that the default "_doc" type is used). It should, however, be used on ES5 and ES6 so that the type matches that described by the mapping. Signed-off-by: Greg Franklin * Log elasticsearch version Signed-off-by: Greg Franklin * Make sure we can get the elasticsearch version on servers other than localhost Signed-off-by: Greg Franklin * Fix es dependency storage test Signed-off-by: Greg Franklin * Fix token propagation test for elasticsearch version detection The query service needs to detect the elasticsearch version to start up. Elasticsearch must therefore be started before the query service and must return a version number in response to a ping request Signed-off-by: Greg Franklin * Fix lint failure plugin/storage/es/dependencystore/schema.go:37:9: if block ends with a return statement, so drop this else and outdent its block Signed-off-by: Greg Franklin * Use the same file for elasticsearch mappings on ES5 and ES6. Only ES7 needs to be different Signed-off-by: Greg Franklin * Change log message to use structured logging Signed-off-by: Greg Franklin