From 0214cc01f03f7579ea7cdf462943a4a532a699e8 Mon Sep 17 00:00:00 2001 From: Mark Tozzi Date: Thu, 26 Mar 2020 15:01:07 -0400 Subject: [PATCH 01/15] Add ValuesSource Registry and associated logic (#54281) * Remove ValuesSourceType argument to ValuesSourceAggregationBuilder (#48638) * ValuesSourceRegistry Prototype (#48758) * Remove generics from ValuesSource related classes (#49606) * fix percentile aggregation tests (#50712) * Basic thread safety for ValuesSourceRegistry (#50340) * Remove target value type from ValuesSourceAggregationBuilder (#49943) * Cleanup default values source type (#50992) * CoreValuesSourceType no longer implements Writable (#51276) * Remove genereics & hard coded ValuesSource references from Matrix Stats (#51131) * Put values source types on fields (#51503) * Remove VST Any (#51539) * Rewire terms agg to use new VS registry (#51182) Also adds some basic AggTestCases for untested code paths (and boilerplate for future tests once the IT are converted over) * Wire Cardinality aggregation to work with the ValuesSourceRegistry (#51337) * Wire Percentiles aggregator into new VS framework (#51639) This required a bit of a refactor to percentiles itself. Before, the Builder would switch on the chosen algo to generate an algo-specific factory. This doesn't work (or at least, would be difficult) in the new VS framework. This refactor consolidates both factories together and introduces a PercentilesConfig object to act as a standardized way to pass algo-specific parameters through the factory. This object is then used when deciding which kind of aggregator to create Note: CoreValuesSourceType.HISTOGRAM still lives in core, and will be moved in a subsequent PR. * Remove generics and target value type from MultiVSAB (#51647) * fix checkstyle after merge (#52008) * Plumb ValuesSourceRegistry through to QuerySearchContext (#51710) * Convert RareTerms to new VS registry (#52166) * Wire up Value Count (#52225) * Wire up Max & Min aggregations (#52219) * ValuesSource refactoring: Wire up Sum aggregation (#52571) * ValuesSource refactoring: Wire up SigTerms aggregation (#52590) * Soft immutability for VSConfig (#52729) * Unmute testSupportedFieldTypes, fix Percentiles/Ranks/Terms tests (#52734) Also fixes Percentiles which was incorrectly specified to only accept numeric, but in fact also accepts Boolean and Date (because those are numeric on master - thanks `testSupportedFieldTypes` for catching it!) * VS refactoring: Wire up stats aggregation (#52891) * ValuesSource refactoring: Wire up string_stats aggregation (#52875) * VS refactoring: Wire up median (MAD) aggregation (#52945) * fix valuesourcetype issue with constant_keyword field (#53041)x-pack/plugin/rollup/src/main/java/org/elasticsearch/xpack/rollup/job/RollupIndexer.java this commit implements `getValuesSourceType` for the ConstantKeyword field type. master was merged into feature/extensible-values-source introducing a new field type that was not implementing `getValuesSourceType`. * ValuesSource refactoring: Wire up Avg aggregation (#52752) * Wire PercentileRanks aggregator into new VS framework (#51693) * Add a VSConfig resolver for aggregations not using the registry (#53038) * Vs refactor wire up ranges and date ranges (#52918) * Wire up geo_bounds aggregation to ValuesSourceRegistry (#53034) This commit updates the geo_bounds aggregation to depend on registering itself in the ValuesSourceRegistry relates #42949. * VS refactoring: convert Boxplot to new registry (#53132) * Wire-up geotile_grid and geohash_grid to ValuesSourceRegistry (#53037) This commit updates the geo*_grid aggregations to depend on registering itself in the ValuesSourceRegistry relates to the values-source refactoring meta issue #42949. * Wire-up geo_centroid agg to ValuesSourceRegistry (#53040) This commit updates the geo_centroid aggregation to depend on registering itself in the ValuesSourceRegistry. relates to the values-source refactoring meta issue #42949. * Fix type tests for Missing aggregation (#53501) * ValuesSource Refactor: move histo VSType into XPack module (#53298) - Introduces a new API (`getBareAggregatorRegistrar()`) which allows plugins to register aggregations against existing agg definitions defined in Core. - This moves the histogram VSType over to XPack where it belongs. `getHistogramValues()` still remains as a Core concept - Moves the histo-specific bits over to xpack (e.g. the actual aggregator logic). This requires extra boilerplate since we need to create a new "Analytics" Percentile/Rank aggregators to deal with the histo field. Doubly-so since percentiles/ranks are extra boiler-plate'y... should be much lighter for other aggs * Wire up DateHistogram to the ValuesSourceRegistry (#53484) * Vs refactor parser cleanup (#53198) Co-authored-by: Zachary Tong Co-authored-by: Zachary Tong Co-authored-by: Christos Soulios <1561376+csoulios@users.noreply.github.com> Co-authored-by: Tal Levy --- .../StringStatsAggregationBuilder.java | 15 +- .../client/RequestConvertersTests.java | 3 +- .../org/elasticsearch/client/SearchIT.java | 22 +- .../stats/MatrixStatsAggregationBuilder.java | 13 +- .../stats/MatrixStatsAggregatorFactory.java | 21 +- .../ArrayValuesSourceAggregationBuilder.java | 203 ++-------- .../ArrayValuesSourceAggregatorFactory.java | 16 +- .../support/ArrayValuesSourceParser.java | 19 +- .../index/mapper/ScaledFloatFieldMapper.java | 2 + .../ChildrenAggregationBuilder.java | 44 ++- .../ChildrenAggregatorFactory.java | 13 +- .../ParentAggregationBuilder.java | 44 ++- .../aggregations/ParentAggregatorFactory.java | 13 +- .../ChildrenToParentAggregatorTests.java | 7 +- .../org/elasticsearch/search/CCSDuelIT.java | 19 +- .../search.aggregation/280_rare_terms.yml | 2 +- .../test/search.aggregation/30_sig_terms.yml | 2 +- .../org/elasticsearch/index/IndexModule.java | 37 +- .../org/elasticsearch/index/IndexService.java | 8 +- .../index/mapper/BooleanFieldMapper.java | 2 +- .../index/mapper/DateFieldMapper.java | 2 +- .../index/mapper/IdFieldMapper.java | 2 + .../index/mapper/IpFieldMapper.java | 2 +- .../index/query/QueryShardContext.java | 17 +- .../elasticsearch/indices/IndicesService.java | 8 +- .../java/org/elasticsearch/node/Node.java | 3 +- .../elasticsearch/plugins/SearchPlugin.java | 27 ++ .../elasticsearch/search/SearchModule.java | 94 +++-- .../aggregations/AggregationBuilders.java | 10 +- .../CompositeValuesSourceBuilder.java | 7 +- .../CompositeValuesSourceParserHelper.java | 13 +- .../DateHistogramValuesSourceBuilder.java | 4 +- .../GeoTileGridValuesSourceBuilder.java | 4 +- .../HistogramValuesSourceBuilder.java | 4 +- .../composite/TermsValuesSourceBuilder.java | 4 +- .../geogrid/GeoGridAggregationBuilder.java | 29 +- .../GeoHashGridAggregationBuilder.java | 10 +- .../geogrid/GeoHashGridAggregatorFactory.java | 35 +- .../GeoTileGridAggregationBuilder.java | 12 +- .../geogrid/GeoTileGridAggregatorFactory.java | 35 +- .../AutoDateHistogramAggregationBuilder.java | 35 +- .../AutoDateHistogramAggregatorFactory.java | 15 +- .../DateHistogramAggregationBuilder.java | 35 +- .../DateHistogramAggregationSupplier.java | 53 +++ .../DateHistogramAggregatorFactory.java | 108 +++--- .../HistogramAggregationBuilder.java | 31 +- .../histogram/HistogramAggregatorFactory.java | 77 ++-- .../histogram/NumericHistogramAggregator.java | 4 +- .../histogram/RangeHistogramAggregator.java | 2 +- .../missing/MissingAggregationBuilder.java | 35 +- .../missing/MissingAggregatorFactory.java | 9 +- .../range/AbstractRangeAggregatorFactory.java | 65 +++- .../bucket/range/AbstractRangeBuilder.java | 15 +- .../range/BinaryRangeAggregatorFactory.java | 17 +- .../range/DateRangeAggregationBuilder.java | 19 +- .../range/DateRangeAggregatorFactory.java | 20 +- .../range/GeoDistanceAggregationBuilder.java | 32 +- .../GeoDistanceRangeAggregatorFactory.java | 23 +- .../range/IpRangeAggregationBuilder.java | 24 +- .../bucket/range/RangeAggregationBuilder.java | 12 +- .../bucket/range/RangeAggregatorFactory.java | 16 +- .../bucket/range/RangeAggregatorSupplier.java | 45 +++ .../DiversifiedAggregationBuilder.java | 28 +- .../sampler/DiversifiedAggregatorFactory.java | 4 +- .../SignificantTermsAggregationBuilder.java | 38 +- .../SignificantTermsAggregatorFactory.java | 200 ++++++---- .../SignificantTermsAggregatorSupplier.java | 50 +++ .../terms/RareTermsAggregationBuilder.java | 38 +- .../terms/RareTermsAggregatorFactory.java | 127 +++++-- .../terms/RareTermsAggregatorSupplier.java | 45 +++ .../bucket/terms/TermsAggregationBuilder.java | 39 +- .../bucket/terms/TermsAggregatorFactory.java | 243 +++++++----- .../bucket/terms/TermsAggregatorSupplier.java | 51 +++ .../AbstractHDRPercentilesAggregator.java | 42 +-- ...AbstractPercentilesAggregationBuilder.java | 9 +- .../AbstractTDigestPercentilesAggregator.java | 31 +- .../metrics/AvgAggregationBuilder.java | 22 +- .../metrics/AvgAggregatorFactory.java | 44 ++- .../CardinalityAggregationBuilder.java | 25 +- .../metrics/CardinalityAggregatorFactory.java | 38 +- .../CardinalityAggregatorSupplier.java | 41 +++ .../ExtendedStatsAggregationBuilder.java | 17 +- .../ExtendedStatsAggregatorFactory.java | 17 +- .../metrics/GeoBoundsAggregationBuilder.java | 24 +- .../metrics/GeoBoundsAggregatorFactory.java | 30 +- .../metrics/GeoBoundsAggregatorSupplier.java | 38 ++ .../GeoCentroidAggregationBuilder.java | 31 +- .../metrics/GeoCentroidAggregatorFactory.java | 28 +- .../GeoCentroidAggregatorSupplier.java | 38 ++ .../metrics/GeoGridAggregatorSupplier.java | 42 +++ .../metrics/InternalHDRPercentileRanks.java | 8 +- .../metrics/InternalHDRPercentiles.java | 6 +- .../InternalTDigestPercentileRanks.java | 8 +- .../metrics/InternalTDigestPercentiles.java | 6 +- .../metrics/MaxAggregationBuilder.java | 26 +- .../aggregations/metrics/MaxAggregator.java | 2 +- .../metrics/MaxAggregatorFactory.java | 44 ++- ...anAbsoluteDeviationAggregationBuilder.java | 28 +- ...ianAbsoluteDeviationAggregatorFactory.java | 74 ++-- ...anAbsoluteDeviationAggregatorSupplier.java | 41 +++ .../metrics/MetricAggregatorSupplier.java | 40 ++ .../metrics/MinAggregationBuilder.java | 22 +- .../aggregations/metrics/MinAggregator.java | 4 +- .../metrics/MinAggregatorFactory.java | 44 ++- .../metrics/MinMaxAggregatorSupplier.java | 40 ++ .../PercentileRanksAggregationBuilder.java | 17 +- .../PercentileRanksAggregatorFactory.java | 39 +- .../PercentilesAggregationBuilder.java | 17 +- .../metrics/PercentilesAggregatorFactory.java | 37 +- .../PercentilesAggregatorSupplier.java | 43 +++ .../metrics/PercentilesConfig.java | 32 +- .../metrics/StatsAggregationBuilder.java | 22 +- .../aggregations/metrics/StatsAggregator.java | 1 - .../metrics/StatsAggregatorFactory.java | 44 ++- .../metrics/SumAggregationBuilder.java | 22 +- .../metrics/SumAggregatorFactory.java | 39 +- .../metrics/ValueCountAggregationBuilder.java | 29 +- .../metrics/ValueCountAggregatorFactory.java | 38 +- .../metrics/ValueCountAggregatorSupplier.java | 38 ++ .../WeightedAvgAggregationBuilder.java | 26 +- .../metrics/WeightedAvgAggregatorFactory.java | 7 +- .../support/AggregatorSupplier.java | 22 ++ .../support/CoreValuesSourceType.java | 141 ++++--- .../support/HistogramAggregatorSupplier.java | 39 ++ .../support/MultiValuesSource.java | 12 +- .../MultiValuesSourceAggregationBuilder.java | 105 +++--- .../MultiValuesSourceAggregatorFactory.java | 9 +- .../support/MultiValuesSourceParseHelper.java | 18 +- .../aggregations/support/ValueType.java | 59 +-- .../aggregations/support/ValuesSource.java | 37 -- .../ValuesSourceAggregationBuilder.java | 205 +++++------ .../ValuesSourceAggregatorFactory.java | 32 +- .../support/ValuesSourceConfig.java | 348 +++++++++++------- .../support/ValuesSourceParserHelper.java | 112 ------ .../support/ValuesSourceRegistry.java | 193 ++++++++++ .../support/ValuesSourceType.java | 15 + .../TransportSearchActionSingleNodeTests.java | 2 +- .../MetadataCreateIndexServiceTests.java | 2 +- .../elasticsearch/index/IndexModuleTests.java | 2 +- .../index/mapper/DateFieldTypeTests.java | 4 +- .../mapper/FieldNamesFieldTypeTests.java | 2 +- .../index/mapper/IndexFieldTypeTests.java | 2 +- .../index/mapper/RangeFieldTypeTests.java | 2 +- .../query/IntervalQueryBuilderTests.java | 2 +- .../index/query/QueryShardContextTests.java | 2 +- .../index/query/RangeQueryRewriteTests.java | 6 +- .../search/SearchModuleTests.java | 23 +- .../search/SearchServiceTests.java | 8 +- .../aggregations/bucket/FiltersTests.java | 2 +- .../aggregations/bucket/GeoDistanceIT.java | 2 +- .../aggregations/bucket/MissingTests.java | 2 +- .../aggregations/bucket/RareTermsTests.java | 2 +- .../bucket/SignificantTermsTests.java | 2 +- .../aggregations/bucket/TermsTests.java | 2 +- .../composite/CompositeAggregatorTests.java | 4 +- .../bucket/histogram/ExtendedBoundsTests.java | 2 +- .../NumericHistogramAggregatorTests.java | 49 +++ .../missing/MissingAggregatorTests.java | 259 +++---------- .../bucket/nested/NestedAggregatorTests.java | 14 +- .../range/DateRangeAggregatorTests.java | 4 +- .../bucket/range/RangeAggregatorTests.java | 2 +- .../sampler/DiversifiedSamplerTests.java | 4 +- .../SignificantTermsAggregatorTests.java | 39 +- .../terms/BinaryTermsAggregatorTests.java | 176 +++++++++ .../terms/KeywordTermsAggregatorTests.java | 161 ++++++++ .../terms/NumericTermsAggregatorTests.java | 190 ++++++++++ .../terms/RareTermsAggregatorTests.java | 29 +- .../bucket/terms/TermsAggregatorTests.java | 83 +++-- .../metrics/AvgAggregatorTests.java | 2 +- .../metrics/CardinalityAggregatorTests.java | 8 +- .../metrics/CardinalityTests.java | 3 +- .../metrics/ExtendedStatsAggregatorTests.java | 2 + .../metrics/GeoCentroidAggregatorTests.java | 13 + .../HDRPercentileRanksAggregatorTests.java | 14 + .../HDRPercentilesAggregatorTests.java | 69 +++- .../metrics/MaxAggregatorTests.java | 9 +- .../metrics/MinAggregatorTests.java | 38 +- .../ScriptedMetricAggregatorTests.java | 5 +- .../metrics/StatsAggregatorTests.java | 2 + ...TDigestPercentileRanksAggregatorTests.java | 14 + .../TDigestPercentilesAggregatorTests.java | 14 + .../metrics/ValueCountAggregatorTests.java | 155 +++++++- .../aggregations/metrics/ValueCountTests.java | 2 +- .../aggregations/pipeline/AvgBucketTests.java | 2 +- .../pipeline/BucketScriptAggregatorTests.java | 2 +- .../pipeline/ExtendedStatsBucketTests.java | 2 +- .../aggregations/pipeline/MaxBucketTests.java | 2 +- .../aggregations/pipeline/MinBucketTests.java | 2 +- .../pipeline/PercentilesBucketTests.java | 2 +- .../PipelineAggregationHelperTests.java | 2 +- .../aggregations/pipeline/SerialDiffIT.java | 7 +- .../pipeline/StatsBucketTests.java | 2 +- .../aggregations/pipeline/SumBucketTests.java | 2 +- .../support/CoreValuesSourceTypeTests.java | 43 +-- .../aggregations/support/ValueTypeTests.java | 55 ++- .../support/ValuesSourceConfigTests.java | 89 ++--- .../highlight/HighlightBuilderTests.java | 2 +- .../rescore/QueryRescorerBuilderTests.java | 4 +- .../search/sort/AbstractSortTestCase.java | 2 +- .../AbstractSuggestionBuilderTestCase.java | 2 +- .../snapshots/SnapshotResiliencyTests.java | 3 +- .../aggregations/AggregatorTestCase.java | 92 +++-- .../aggregations/BaseAggregationTestCase.java | 2 +- .../test/AbstractBuilderTestCase.java | 2 +- .../search/MockSearchServiceTests.java | 2 +- .../xpack/analytics/AnalyticsPlugin.java | 15 +- ...ctHistoBackedHDRPercentilesAggregator.java | 121 ++++++ ...stoBackedTDigestPercentilesAggregator.java | 113 ++++++ ...AnalyticsPercentilesAggregatorFactory.java | 61 +++ ...stoBackedHDRPercentileRanksAggregator.java | 59 +++ .../HistoBackedHDRPercentilesAggregator.java | 60 +++ ...ackedTDigestPercentileRanksAggregator.java | 62 ++++ ...stoBackedTDigestPercentilesAggregator.java | 62 ++++ .../support/AnalyticsValuesSourceType.java | 59 +++ .../support/HistogramValuesSource.java | 51 +++ .../boxplot/BoxplotAggregationBuilder.java | 21 +- .../analytics/boxplot/BoxplotAggregator.java | 5 +- .../boxplot/BoxplotAggregatorFactory.java | 29 +- .../boxplot/BoxplotAggregatorSupplier.java | 31 ++ .../mapper/HistogramFieldMapper.java | 9 +- .../StringStatsAggregationBuilder.java | 29 +- .../StringStatsAggregatorFactory.java | 42 ++- .../StringStatsAggregatorSupplier.java | 29 ++ .../TopMetricsAggregatorFactory.java | 8 +- .../boxplot/BoxplotAggregatorTests.java | 40 +- .../CumulativeCardinalityAggregatorTests.java | 5 +- ...regatedPercentileRanksAggregatorTests.java | 31 +- ...eAggregatedPercentilesAggregatorTests.java | 33 +- ...regatedPercentileRanksAggregatorTests.java | 29 ++ ...eAggregatedPercentilesAggregatorTests.java | 33 +- .../StringStatsAggregatorTests.java | 15 +- .../topmetrics/TopMetricsAggregatorTests.java | 7 +- .../xpack/core/rollup/RollupField.java | 2 +- .../DocumentSubsetBitsetCacheTests.java | 2 +- ...ityIndexReaderWrapperIntegrationTests.java | 4 +- .../MockDeprecatedAggregationBuilder.java | 26 +- .../mapper/ConstantKeywordFieldMapper.java | 6 + .../mapper/FlatObjectFieldMapper.java | 9 +- .../RollupDataExtractorFactory.java | 8 +- .../xpack/rollup/RollupRequestTranslator.java | 5 +- .../xpack/rollup/job/RollupIndexer.java | 7 +- .../rollup/RollupJobIdentifierUtilTests.java | 2 +- .../rollup/RollupRequestTranslationTests.java | 4 +- .../RollupResponseTranslationTests.java | 20 +- .../job/RollupIndexerIndexingTests.java | 2 +- .../transforms/pivot/Aggregations.java | 2 +- .../wildcard/mapper/WildcardFieldMapper.java | 162 ++++---- .../mapper/WildcardFieldMapperTests.java | 2 +- 248 files changed, 5802 insertions(+), 2368 deletions(-) create mode 100644 server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/DateHistogramAggregationSupplier.java create mode 100644 server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/RangeAggregatorSupplier.java create mode 100644 server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorSupplier.java create mode 100644 server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregatorSupplier.java create mode 100644 server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorSupplier.java create mode 100644 server/src/main/java/org/elasticsearch/search/aggregations/metrics/CardinalityAggregatorSupplier.java create mode 100644 server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoBoundsAggregatorSupplier.java create mode 100644 server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregatorSupplier.java create mode 100644 server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoGridAggregatorSupplier.java create mode 100644 server/src/main/java/org/elasticsearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregatorSupplier.java create mode 100644 server/src/main/java/org/elasticsearch/search/aggregations/metrics/MetricAggregatorSupplier.java create mode 100644 server/src/main/java/org/elasticsearch/search/aggregations/metrics/MinMaxAggregatorSupplier.java create mode 100644 server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesAggregatorSupplier.java create mode 100644 server/src/main/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregatorSupplier.java create mode 100644 server/src/main/java/org/elasticsearch/search/aggregations/support/AggregatorSupplier.java create mode 100644 server/src/main/java/org/elasticsearch/search/aggregations/support/HistogramAggregatorSupplier.java delete mode 100644 server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceParserHelper.java create mode 100644 server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceRegistry.java create mode 100644 server/src/test/java/org/elasticsearch/search/aggregations/bucket/terms/BinaryTermsAggregatorTests.java create mode 100644 server/src/test/java/org/elasticsearch/search/aggregations/bucket/terms/KeywordTermsAggregatorTests.java create mode 100644 server/src/test/java/org/elasticsearch/search/aggregations/bucket/terms/NumericTermsAggregatorTests.java create mode 100644 x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/AbstractHistoBackedHDRPercentilesAggregator.java create mode 100644 x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/AbstractHistoBackedTDigestPercentilesAggregator.java create mode 100644 x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/AnalyticsPercentilesAggregatorFactory.java create mode 100644 x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedHDRPercentileRanksAggregator.java create mode 100644 x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedHDRPercentilesAggregator.java create mode 100644 x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedTDigestPercentileRanksAggregator.java create mode 100644 x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedTDigestPercentilesAggregator.java create mode 100644 x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/support/AnalyticsValuesSourceType.java create mode 100644 x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/support/HistogramValuesSource.java create mode 100644 x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregatorSupplier.java create mode 100644 x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregatorSupplier.java diff --git a/client/rest-high-level/src/main/java/org/elasticsearch/client/analytics/StringStatsAggregationBuilder.java b/client/rest-high-level/src/main/java/org/elasticsearch/client/analytics/StringStatsAggregationBuilder.java index 0122ebaf37c02..67a0c6ad2067f 100644 --- a/client/rest-high-level/src/main/java/org/elasticsearch/client/analytics/StringStatsAggregationBuilder.java +++ b/client/rest-high-level/src/main/java/org/elasticsearch/client/analytics/StringStatsAggregationBuilder.java @@ -30,12 +30,10 @@ import org.elasticsearch.search.aggregations.AggregatorFactories.Builder; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; -import org.elasticsearch.search.aggregations.support.ValueType; -import org.elasticsearch.search.aggregations.support.ValuesSource; -import org.elasticsearch.search.aggregations.support.ValuesSource.Bytes; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import org.elasticsearch.search.builder.SearchSourceBuilder; import java.io.IOException; @@ -52,14 +50,14 @@ * {@linkplain AggregationBuilder#rewrite(QueryRewriteContext)}, or * {@linkplain AbstractAggregationBuilder#build(QueryShardContext, AggregatorFactory)}. */ -public class StringStatsAggregationBuilder extends ValuesSourceAggregationBuilder { +public class StringStatsAggregationBuilder extends ValuesSourceAggregationBuilder { public static final String NAME = "string_stats"; private static final ParseField SHOW_DISTRIBUTION_FIELD = new ParseField("show_distribution"); private boolean showDistribution = false; public StringStatsAggregationBuilder(String name) { - super(name, CoreValuesSourceType.BYTES, ValueType.STRING); + super(name); } /** @@ -71,6 +69,11 @@ public StringStatsAggregationBuilder showDistribution(boolean showDistribution) return this; } + @Override + protected ValuesSourceType defaultValueSourceType() { + return CoreValuesSourceType.BYTES; + } + @Override public String getType() { return NAME; @@ -92,7 +95,7 @@ public BucketCardinality bucketCardinality() { } @Override - protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config, + protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config, AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException { throw new UnsupportedOperationException(); } diff --git a/client/rest-high-level/src/test/java/org/elasticsearch/client/RequestConvertersTests.java b/client/rest-high-level/src/test/java/org/elasticsearch/client/RequestConvertersTests.java index 28caf3364464a..2437531276424 100644 --- a/client/rest-high-level/src/test/java/org/elasticsearch/client/RequestConvertersTests.java +++ b/client/rest-high-level/src/test/java/org/elasticsearch/client/RequestConvertersTests.java @@ -1229,7 +1229,8 @@ public static SearchSourceBuilder createTestSearchSourceBuilder() { searchSourceBuilder.query(new TermQueryBuilder(randomAlphaOfLengthBetween(3, 10), randomAlphaOfLengthBetween(3, 10))); } if (randomBoolean()) { - searchSourceBuilder.aggregation(new TermsAggregationBuilder(randomAlphaOfLengthBetween(3, 10), ValueType.STRING) + searchSourceBuilder.aggregation(new TermsAggregationBuilder(randomAlphaOfLengthBetween(3, 10)) + .userValueTypeHint(ValueType.STRING) .field(randomAlphaOfLengthBetween(3, 10))); } if (randomBoolean()) { diff --git a/client/rest-high-level/src/test/java/org/elasticsearch/client/SearchIT.java b/client/rest-high-level/src/test/java/org/elasticsearch/client/SearchIT.java index ddccb957c4a5f..ab5d15db0f2c0 100644 --- a/client/rest-high-level/src/test/java/org/elasticsearch/client/SearchIT.java +++ b/client/rest-high-level/src/test/java/org/elasticsearch/client/SearchIT.java @@ -265,7 +265,8 @@ public void testSearchMatchQuery() throws IOException { public void testSearchWithTermsAgg() throws IOException { SearchRequest searchRequest = new SearchRequest(); SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); - searchSourceBuilder.aggregation(new TermsAggregationBuilder("agg1", ValueType.STRING).field("type.keyword")); + searchSourceBuilder.aggregation(new TermsAggregationBuilder("agg1").userValueTypeHint(ValueType.STRING) + .field("type.keyword")); searchSourceBuilder.size(0); searchRequest.source(searchSourceBuilder); SearchResponse searchResponse = execute(searchRequest, highLevelClient()::search, highLevelClient()::searchAsync); @@ -357,7 +358,7 @@ public void testSearchWithRangeAgg() throws IOException { public void testSearchWithTermsAndRangeAgg() throws IOException { SearchRequest searchRequest = new SearchRequest("index"); SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); - TermsAggregationBuilder agg = new TermsAggregationBuilder("agg1", ValueType.STRING).field("type.keyword"); + TermsAggregationBuilder agg = new TermsAggregationBuilder("agg1").userValueTypeHint(ValueType.STRING).field("type.keyword"); agg.subAggregation(new RangeAggregationBuilder("subagg").field("num") .addRange("first", 0, 30).addRange("second", 31, 200)); searchSourceBuilder.aggregation(agg); @@ -411,7 +412,7 @@ public void testSearchWithTermsAndRangeAgg() throws IOException { public void testSearchWithTermsAndWeightedAvg() throws IOException { SearchRequest searchRequest = new SearchRequest("index"); SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); - TermsAggregationBuilder agg = new TermsAggregationBuilder("agg1", ValueType.STRING).field("type.keyword"); + TermsAggregationBuilder agg = new TermsAggregationBuilder("agg1").userValueTypeHint(ValueType.STRING).field("type.keyword"); agg.subAggregation(new WeightedAvgAggregationBuilder("subagg") .value(new MultiValuesSourceFieldConfig.Builder().setFieldName("num").build()) .weight(new MultiValuesSourceFieldConfig.Builder().setFieldName("num2").build()) @@ -537,10 +538,12 @@ public void testSearchWithParentJoin() throws IOException { client().performRequest(answerDoc2); client().performRequest(new Request(HttpPost.METHOD_NAME, "/_refresh")); - TermsAggregationBuilder leafTermAgg = new TermsAggregationBuilder("top-names", ValueType.STRING) + TermsAggregationBuilder leafTermAgg = new TermsAggregationBuilder("top-names") + .userValueTypeHint(ValueType.STRING) .field("owner.display_name.keyword").size(10); ChildrenAggregationBuilder childrenAgg = new ChildrenAggregationBuilder("to-answers", "answer").subAggregation(leafTermAgg); - TermsAggregationBuilder termsAgg = new TermsAggregationBuilder("top-tags", ValueType.STRING).field("tags.keyword") + TermsAggregationBuilder termsAgg = new TermsAggregationBuilder("top-tags").userValueTypeHint(ValueType.STRING) + .field("tags.keyword") .size(10).subAggregation(childrenAgg); SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); searchSourceBuilder.size(0).aggregation(termsAgg); @@ -752,15 +755,18 @@ public void testMultiSearch() throws Exception { public void testMultiSearch_withAgg() throws Exception { MultiSearchRequest multiSearchRequest = new MultiSearchRequest(); SearchRequest searchRequest1 = new SearchRequest("index1"); - searchRequest1.source().size(0).aggregation(new TermsAggregationBuilder("name", ValueType.STRING).field("field.keyword") + searchRequest1.source().size(0).aggregation(new TermsAggregationBuilder("name").userValueTypeHint(ValueType.STRING) + .field("field.keyword") .order(BucketOrder.key(true))); multiSearchRequest.add(searchRequest1); SearchRequest searchRequest2 = new SearchRequest("index2"); - searchRequest2.source().size(0).aggregation(new TermsAggregationBuilder("name", ValueType.STRING).field("field.keyword") + searchRequest2.source().size(0).aggregation(new TermsAggregationBuilder("name").userValueTypeHint(ValueType.STRING) + .field("field.keyword") .order(BucketOrder.key(true))); multiSearchRequest.add(searchRequest2); SearchRequest searchRequest3 = new SearchRequest("index3"); - searchRequest3.source().size(0).aggregation(new TermsAggregationBuilder("name", ValueType.STRING).field("field.keyword") + searchRequest3.source().size(0).aggregation(new TermsAggregationBuilder("name").userValueTypeHint(ValueType.STRING) + .field("field.keyword") .order(BucketOrder.key(true))); multiSearchRequest.add(searchRequest3); diff --git a/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/matrix/stats/MatrixStatsAggregationBuilder.java b/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/matrix/stats/MatrixStatsAggregationBuilder.java index 43267a249c462..76003b947a740 100644 --- a/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/matrix/stats/MatrixStatsAggregationBuilder.java +++ b/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/matrix/stats/MatrixStatsAggregationBuilder.java @@ -28,23 +28,18 @@ import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.support.ArrayValuesSourceAggregationBuilder; -import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; -import org.elasticsearch.search.aggregations.support.ValueType; -import org.elasticsearch.search.aggregations.support.ValuesSource; -import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; import java.io.IOException; import java.util.Map; -public class MatrixStatsAggregationBuilder - extends ArrayValuesSourceAggregationBuilder.LeafOnly { +public class MatrixStatsAggregationBuilder extends ArrayValuesSourceAggregationBuilder.LeafOnly { public static final String NAME = "matrix_stats"; private MultiValueMode multiValueMode = MultiValueMode.AVG; public MatrixStatsAggregationBuilder(String name) { - super(name, CoreValuesSourceType.NUMERIC, ValueType.NUMERIC); + super(name); } protected MatrixStatsAggregationBuilder(MatrixStatsAggregationBuilder clone, @@ -62,7 +57,7 @@ protected AggregationBuilder shallowCopy(AggregatorFactories.Builder factoriesBu * Read from a stream. */ public MatrixStatsAggregationBuilder(StreamInput in) throws IOException { - super(in, CoreValuesSourceType.NUMERIC, ValueType.NUMERIC); + super(in); } @Override @@ -81,7 +76,7 @@ public MultiValueMode multiValueMode() { @Override protected MatrixStatsAggregatorFactory innerBuild(QueryShardContext queryShardContext, - Map> configs, + Map configs, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder) throws IOException { return new MatrixStatsAggregatorFactory(name, configs, multiValueMode, queryShardContext, parent, subFactoriesBuilder, metadata); diff --git a/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/matrix/stats/MatrixStatsAggregatorFactory.java b/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/matrix/stats/MatrixStatsAggregatorFactory.java index 57e199eb34f66..5afca9319c33e 100644 --- a/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/matrix/stats/MatrixStatsAggregatorFactory.java +++ b/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/matrix/stats/MatrixStatsAggregatorFactory.java @@ -20,6 +20,7 @@ import org.elasticsearch.index.query.QueryShardContext; import org.elasticsearch.search.MultiValueMode; +import org.elasticsearch.search.aggregations.AggregationExecutionException; import org.elasticsearch.search.aggregations.Aggregator; import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; @@ -30,15 +31,16 @@ import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; +import java.util.HashMap; import java.util.List; import java.util.Map; -final class MatrixStatsAggregatorFactory extends ArrayValuesSourceAggregatorFactory { +final class MatrixStatsAggregatorFactory extends ArrayValuesSourceAggregatorFactory { private final MultiValueMode multiValueMode; MatrixStatsAggregatorFactory(String name, - Map> configs, + Map configs, MultiValueMode multiValueMode, QueryShardContext queryShardContext, AggregatorFactory parent, @@ -58,12 +60,21 @@ protected Aggregator createUnmapped(SearchContext searchContext, } @Override - protected Aggregator doCreateInternal(Map valuesSources, + protected Aggregator doCreateInternal(Map valuesSources, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, List pipelineAggregators, - Map metadata) throws IOException { - return new MatrixStatsAggregator(name, valuesSources, searchContext, parent, multiValueMode, pipelineAggregators, metadata); + Map metaData) throws IOException { + Map typedValuesSources = new HashMap<>(valuesSources.size()); + for (Map.Entry entry : valuesSources.entrySet()) { + if (entry.getValue() instanceof ValuesSource.Numeric == false) { + throw new AggregationExecutionException("ValuesSource type " + entry.getValue().toString() + + "is not supported for aggregation " + this.name()); + } + // TODO: There must be a better option than this. + typedValuesSources.put(entry.getKey(), (ValuesSource.Numeric) entry.getValue()); + } + return new MatrixStatsAggregator(name, typedValuesSources, searchContext, parent, multiValueMode, pipelineAggregators, metaData); } } diff --git a/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/support/ArrayValuesSourceAggregationBuilder.java b/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/support/ArrayValuesSourceAggregationBuilder.java index 2017ae15ce468..ba23e689ecc67 100644 --- a/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/support/ArrayValuesSourceAggregationBuilder.java +++ b/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/support/ArrayValuesSourceAggregationBuilder.java @@ -18,18 +18,11 @@ */ package org.elasticsearch.search.aggregations.support; -import org.elasticsearch.common.Nullable; import org.elasticsearch.common.ParseField; import org.elasticsearch.common.io.stream.StreamInput; import org.elasticsearch.common.io.stream.StreamOutput; import org.elasticsearch.common.xcontent.XContentBuilder; -import org.elasticsearch.index.fielddata.IndexFieldData; -import org.elasticsearch.index.fielddata.IndexGeoPointFieldData; -import org.elasticsearch.index.fielddata.IndexNumericFieldData; -import org.elasticsearch.index.mapper.MappedFieldType; import org.elasticsearch.index.query.QueryShardContext; -import org.elasticsearch.script.Script; -import org.elasticsearch.search.DocValueFormat; import org.elasticsearch.search.aggregations.AbstractAggregationBuilder; import org.elasticsearch.search.aggregations.AggregationInitializationException; import org.elasticsearch.search.aggregations.AggregatorFactories; @@ -44,20 +37,20 @@ import java.util.Map; import java.util.Objects; -public abstract class ArrayValuesSourceAggregationBuilder> +public abstract class ArrayValuesSourceAggregationBuilder> extends AbstractAggregationBuilder { public static final ParseField MULTIVALUE_MODE_FIELD = new ParseField("mode"); - public abstract static class LeafOnly> - extends ArrayValuesSourceAggregationBuilder { + public abstract static class LeafOnly> + extends ArrayValuesSourceAggregationBuilder { - protected LeafOnly(String name, ValuesSourceType valuesSourceType, ValueType targetValueType) { - super(name, valuesSourceType, targetValueType); + protected LeafOnly(String name) { + super(name); } - protected LeafOnly(LeafOnly clone, Builder factoriesBuilder, Map metadata) { - super(clone, factoriesBuilder, metadata); + protected LeafOnly(LeafOnly clone, Builder factoriesBuilder, Map metaData) { + super(clone, factoriesBuilder, metaData); if (factoriesBuilder.count() > 0) { throw new AggregationInitializationException("Aggregator [" + name + "] of type [" + getType() + "] cannot accept sub-aggregations"); @@ -65,18 +58,10 @@ protected LeafOnly(LeafOnly clone, Builder factoriesBuilder, Map fields = Collections.emptyList(); - private ValueType valueType = null; + /* The parser doesn't support setting userValueTypeHint (aka valueType), but we do serialize and deserialize it, so keeping it around + for now so as to not break BWC. Future refactors should feel free to remove this field. --Tozzi 2020-01-16 + */ + private ValueType userValueTypeHint = null; private String format = null; private Object missing = null; private Map missingMap = Collections.emptyMap(); - protected ArrayValuesSourceAggregationBuilder(String name, ValuesSourceType valuesSourceType, ValueType targetValueType) { + protected ArrayValuesSourceAggregationBuilder(String name) { super(name); - if (valuesSourceType == null) { - throw new IllegalArgumentException("[valuesSourceType] must not be null: [" + name + "]"); - } - this.valuesSourceType = valuesSourceType; - this.targetValueType = targetValueType; } - protected ArrayValuesSourceAggregationBuilder(ArrayValuesSourceAggregationBuilder clone, - Builder factoriesBuilder, Map metadata) { - super(clone, factoriesBuilder, metadata); - this.valuesSourceType = clone.valuesSourceType; - this.targetValueType = clone.targetValueType; + protected ArrayValuesSourceAggregationBuilder(ArrayValuesSourceAggregationBuilder clone, + Builder factoriesBuilder, Map metaData) { + super(clone, factoriesBuilder, metaData); this.fields = new ArrayList<>(clone.fields); - this.valueType = clone.valueType; + this.userValueTypeHint = clone.userValueTypeHint; this.format = clone.format; this.missingMap = new HashMap<>(clone.missingMap); this.missing = clone.missing; } - protected ArrayValuesSourceAggregationBuilder(StreamInput in, ValuesSourceType valuesSourceType, ValueType targetValueType) + protected ArrayValuesSourceAggregationBuilder(StreamInput in) throws IOException { super(in); - assert false == serializeTargetValueType() : "Wrong read constructor called for subclass that provides its targetValueType"; - this.valuesSourceType = valuesSourceType; - this.targetValueType = targetValueType; - read(in); - } - - protected ArrayValuesSourceAggregationBuilder(StreamInput in, ValuesSourceType valuesSourceType) throws IOException { - super(in); - assert serializeTargetValueType() : "Wrong read constructor called for subclass that serializes its targetValueType"; - this.valuesSourceType = valuesSourceType; - this.targetValueType = in.readOptionalWriteable(ValueType::readFromStream); read(in); } @@ -143,18 +111,15 @@ protected ArrayValuesSourceAggregationBuilder(StreamInput in, ValuesSourceType v @SuppressWarnings("unchecked") private void read(StreamInput in) throws IOException { fields = (ArrayList)in.readGenericValue(); - valueType = in.readOptionalWriteable(ValueType::readFromStream); + userValueTypeHint = in.readOptionalWriteable(ValueType::readFromStream); format = in.readOptionalString(); missingMap = in.readMap(); } @Override protected final void doWriteTo(StreamOutput out) throws IOException { - if (serializeTargetValueType()) { - out.writeOptionalWriteable(targetValueType); - } out.writeGenericValue(fields); - out.writeOptionalWriteable(valueType); + out.writeOptionalWriteable(userValueTypeHint); out.writeOptionalString(format); out.writeMap(missingMap); innerWriteTo(out); @@ -184,25 +149,6 @@ public List fields() { return fields; } - /** - * Sets the {@link ValueType} for the value produced by this aggregation - */ - @SuppressWarnings("unchecked") - public AB valueType(ValueType valueType) { - if (valueType == null) { - throw new IllegalArgumentException("[valueType] must not be null: [" + name + "]"); - } - this.valueType = valueType; - return (AB) this; - } - - /** - * Gets the {@link ValueType} for the value produced by this aggregation - */ - public ValueType valueType() { - return valueType; - } - /** * Sets the format to use for the output of the aggregation. */ @@ -244,96 +190,27 @@ public Map missingMap() { } @Override - protected final ArrayValuesSourceAggregatorFactory doBuild(QueryShardContext queryShardContext, AggregatorFactory parent, - Builder subFactoriesBuilder) throws IOException { - Map> configs = resolveConfig(queryShardContext); - ArrayValuesSourceAggregatorFactory factory = innerBuild(queryShardContext, configs, parent, subFactoriesBuilder); + protected final ArrayValuesSourceAggregatorFactory doBuild(QueryShardContext queryShardContext, AggregatorFactory parent, + Builder subFactoriesBuilder) throws IOException { + Map configs = resolveConfig(queryShardContext); + ArrayValuesSourceAggregatorFactory factory = innerBuild(queryShardContext, configs, parent, subFactoriesBuilder); return factory; } - protected Map> resolveConfig(QueryShardContext queryShardContext) { - HashMap> configs = new HashMap<>(); + protected Map resolveConfig(QueryShardContext queryShardContext) { + HashMap configs = new HashMap<>(); for (String field : fields) { - ValuesSourceConfig config = config(queryShardContext, field, null); + ValuesSourceConfig config = ValuesSourceConfig.resolveUnregistered(queryShardContext, userValueTypeHint, field, null, + missingMap.get(field), null, format, CoreValuesSourceType.BYTES); configs.put(field, config); } return configs; } - protected abstract ArrayValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, - Map> configs, - AggregatorFactory parent, - AggregatorFactories.Builder subFactoriesBuilder) throws IOException; - - public ValuesSourceConfig config(QueryShardContext queryShardContext, String field, Script script) { - - ValueType valueType = this.valueType != null ? this.valueType : targetValueType; - - if (field == null) { - if (script == null) { - ValuesSourceConfig config = new ValuesSourceConfig<>(CoreValuesSourceType.ANY); - return config.format(resolveFormat(null, valueType)); - } - ValuesSourceType valuesSourceType = valueType != null ? valueType.getValuesSourceType() : this.valuesSourceType; - if (valuesSourceType == null || valuesSourceType == CoreValuesSourceType.ANY) { - // the specific value source type is undefined, but for scripts, - // we need to have a specific value source - // type to know how to handle the script values, so we fallback - // on Bytes - valuesSourceType = CoreValuesSourceType.BYTES; - } - ValuesSourceConfig config = new ValuesSourceConfig<>(valuesSourceType); - config.missing(missingMap.get(field)); - return config.format(resolveFormat(format, valueType)); - } - - MappedFieldType fieldType = queryShardContext.getMapperService().fieldType(field); - if (fieldType == null) { - ValuesSourceType valuesSourceType = valueType != null ? valueType.getValuesSourceType() : this.valuesSourceType; - ValuesSourceConfig config = new ValuesSourceConfig<>(valuesSourceType); - config.missing(missingMap.get(field)); - config.format(resolveFormat(format, valueType)); - return config.unmapped(true); - } - - IndexFieldData indexFieldData = queryShardContext.getForField(fieldType); - - ValuesSourceConfig config; - if (valuesSourceType == CoreValuesSourceType.ANY) { - if (indexFieldData instanceof IndexNumericFieldData) { - config = new ValuesSourceConfig<>(CoreValuesSourceType.NUMERIC); - } else if (indexFieldData instanceof IndexGeoPointFieldData) { - config = new ValuesSourceConfig<>(CoreValuesSourceType.GEOPOINT); - } else { - config = new ValuesSourceConfig<>(CoreValuesSourceType.BYTES); - } - } else { - config = new ValuesSourceConfig<>(valuesSourceType); - } - - config.fieldContext(new FieldContext(field, indexFieldData, fieldType)); - config.missing(missingMap.get(field)); - return config.format(fieldType.docValueFormat(format, null)); - } - - private static DocValueFormat resolveFormat(@Nullable String format, @Nullable ValueType valueType) { - if (valueType == null) { - return DocValueFormat.RAW; // we can't figure it out - } - DocValueFormat valueFormat = valueType.defaultFormat(); - if (valueFormat instanceof DocValueFormat.Decimal && format != null) { - valueFormat = new DocValueFormat.Decimal(format); - } - return valueFormat; - } - - /** - * Should this builder serialize its targetValueType? Defaults to false. All subclasses that override this to true - * should use the three argument read constructor rather than the four argument version. - */ - protected boolean serializeTargetValueType() { - return false; - } + protected abstract ArrayValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, + Map configs, + AggregatorFactory parent, + AggregatorFactories.Builder subFactoriesBuilder) throws IOException; @Override public final XContentBuilder internalXContent(XContentBuilder builder, Params params) throws IOException { @@ -348,8 +225,8 @@ public final XContentBuilder internalXContent(XContentBuilder builder, Params pa if (format != null) { builder.field(CommonFields.FORMAT.getPreferredName(), format); } - if (valueType != null) { - builder.field(CommonFields.VALUE_TYPE.getPreferredName(), valueType.getPreferredName()); + if (userValueTypeHint != null) { + builder.field(CommonFields.VALUE_TYPE.getPreferredName(), userValueTypeHint.getPreferredName()); } doXContentBody(builder, params); builder.endObject(); @@ -360,7 +237,7 @@ public final XContentBuilder internalXContent(XContentBuilder builder, Params pa @Override public int hashCode() { - return Objects.hash(super.hashCode(), fields, format, missing, targetValueType, valueType, valuesSourceType); + return Objects.hash(super.hashCode(), fields, format, missing, userValueTypeHint); } @Override @@ -368,12 +245,10 @@ public boolean equals(Object obj) { if (this == obj) return true; if (obj == null || getClass() != obj.getClass()) return false; if (super.equals(obj) == false) return false; - ArrayValuesSourceAggregationBuilder other = (ArrayValuesSourceAggregationBuilder) obj; + ArrayValuesSourceAggregationBuilder other = (ArrayValuesSourceAggregationBuilder) obj; return Objects.equals(fields, other.fields) && Objects.equals(format, other.format) && Objects.equals(missing, other.missing) - && Objects.equals(targetValueType, other.targetValueType) - && Objects.equals(valueType, other.valueType) - && Objects.equals(valuesSourceType, other.valuesSourceType); + && Objects.equals(userValueTypeHint, other.userValueTypeHint); } } diff --git a/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/support/ArrayValuesSourceAggregatorFactory.java b/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/support/ArrayValuesSourceAggregatorFactory.java index 5201513bf9975..ec42f7638f974 100644 --- a/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/support/ArrayValuesSourceAggregatorFactory.java +++ b/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/support/ArrayValuesSourceAggregatorFactory.java @@ -31,12 +31,12 @@ import java.util.List; import java.util.Map; -public abstract class ArrayValuesSourceAggregatorFactory +public abstract class ArrayValuesSourceAggregatorFactory extends AggregatorFactory { - protected Map> configs; + protected Map configs; - public ArrayValuesSourceAggregatorFactory(String name, Map> configs, + public ArrayValuesSourceAggregatorFactory(String name, Map configs, QueryShardContext queryShardContext, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, Map metadata) throws IOException { @@ -49,11 +49,11 @@ public Aggregator createInternal(SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, List pipelineAggregators, - Map metadata) throws IOException { - HashMap valuesSources = new HashMap<>(); + Map metaData) throws IOException { + HashMap valuesSources = new HashMap<>(); - for (Map.Entry> config : configs.entrySet()) { - VS vs = config.getValue().toValuesSource(queryShardContext); + for (Map.Entry config : configs.entrySet()) { + ValuesSource vs = config.getValue().toValuesSource(); if (vs != null) { valuesSources.put(config.getKey(), vs); } @@ -70,7 +70,7 @@ protected abstract Aggregator createUnmapped(SearchContext searchContext, List pipelineAggregators, Map metadata) throws IOException; - protected abstract Aggregator doCreateInternal(Map valuesSources, + protected abstract Aggregator doCreateInternal(Map valuesSources, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, diff --git a/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/support/ArrayValuesSourceParser.java b/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/support/ArrayValuesSourceParser.java index a0e7016291b17..4d935c92c2e1f 100644 --- a/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/support/ArrayValuesSourceParser.java +++ b/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/support/ArrayValuesSourceParser.java @@ -35,13 +35,6 @@ public abstract class ArrayValuesSourceParser implements Aggregator.Parser { - public abstract static class AnyValuesSourceParser extends ArrayValuesSourceParser { - - protected AnyValuesSourceParser(boolean formattable) { - super(formattable, CoreValuesSourceType.ANY, null); - } - } - public abstract static class NumericValuesSourceParser extends ArrayValuesSourceParser { protected NumericValuesSourceParser(boolean formattable) { @@ -74,7 +67,7 @@ private ArrayValuesSourceParser(boolean formattable, ValuesSourceType valuesSour } @Override - public final ArrayValuesSourceAggregationBuilder parse(String aggregationName, XContentParser parser) + public final ArrayValuesSourceAggregationBuilder parse(String aggregationName, XContentParser parser) throws IOException { List fields = null; @@ -139,7 +132,7 @@ private ArrayValuesSourceParser(boolean formattable, ValuesSourceType valuesSour } } - ArrayValuesSourceAggregationBuilder factory = createFactory(aggregationName, this.valuesSourceType, this.targetValueType, + ArrayValuesSourceAggregationBuilder factory = createFactory(aggregationName, this.valuesSourceType, this.targetValueType, otherOptions); if (fields != null) { factory.fields(fields); @@ -193,10 +186,10 @@ private void parseMissingAndAdd(final String aggregationName, final String curre * method * @return the created factory */ - protected abstract ArrayValuesSourceAggregationBuilder createFactory(String aggregationName, - ValuesSourceType valuesSourceType, - ValueType targetValueType, - Map otherOptions); + protected abstract ArrayValuesSourceAggregationBuilder createFactory(String aggregationName, + ValuesSourceType valuesSourceType, + ValueType targetValueType, + Map otherOptions); /** * Allows subclasses of {@link ArrayValuesSourceParser} to parse extra diff --git a/modules/mapper-extras/src/main/java/org/elasticsearch/index/mapper/ScaledFloatFieldMapper.java b/modules/mapper-extras/src/main/java/org/elasticsearch/index/mapper/ScaledFloatFieldMapper.java index 81c092f3a2de6..c969ee00d9fae 100644 --- a/modules/mapper-extras/src/main/java/org/elasticsearch/index/mapper/ScaledFloatFieldMapper.java +++ b/modules/mapper-extras/src/main/java/org/elasticsearch/index/mapper/ScaledFloatFieldMapper.java @@ -60,6 +60,8 @@ import org.elasticsearch.indices.breaker.CircuitBreakerService; import org.elasticsearch.search.DocValueFormat; import org.elasticsearch.search.MultiValueMode; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import org.elasticsearch.search.sort.BucketedSort; import org.elasticsearch.search.sort.SortOrder; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; diff --git a/modules/parent-join/src/main/java/org/elasticsearch/join/aggregations/ChildrenAggregationBuilder.java b/modules/parent-join/src/main/java/org/elasticsearch/join/aggregations/ChildrenAggregationBuilder.java index b298f4d7486ee..f965457209eaa 100644 --- a/modules/parent-join/src/main/java/org/elasticsearch/join/aggregations/ChildrenAggregationBuilder.java +++ b/modules/parent-join/src/main/java/org/elasticsearch/join/aggregations/ChildrenAggregationBuilder.java @@ -25,7 +25,6 @@ import org.elasticsearch.common.io.stream.StreamOutput; import org.elasticsearch.common.xcontent.XContentBuilder; import org.elasticsearch.common.xcontent.XContentParser; -import org.elasticsearch.index.fielddata.plain.SortedSetDVOrdinalsIndexFieldData; import org.elasticsearch.index.mapper.MappedFieldType; import org.elasticsearch.index.query.QueryShardContext; import org.elasticsearch.join.mapper.ParentIdFieldMapper; @@ -34,19 +33,17 @@ import org.elasticsearch.search.aggregations.AggregatorFactories.Builder; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; -import org.elasticsearch.search.aggregations.support.FieldContext; -import org.elasticsearch.search.aggregations.support.ValueType; -import org.elasticsearch.search.aggregations.support.ValuesSource.Bytes.WithOrdinals; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.util.Map; import java.util.Objects; public class ChildrenAggregationBuilder - extends ValuesSourceAggregationBuilder { + extends ValuesSourceAggregationBuilder { public static final String NAME = "children"; @@ -61,7 +58,7 @@ public class ChildrenAggregationBuilder * the type of children documents */ public ChildrenAggregationBuilder(String name, String childType) { - super(name, CoreValuesSourceType.BYTES, ValueType.STRING); + super(name); if (childType == null) { throw new IllegalArgumentException("[childType] must not be null: [" + name + "]"); } @@ -77,15 +74,20 @@ protected ChildrenAggregationBuilder(ChildrenAggregationBuilder clone, } @Override - protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metadata) { - return new ChildrenAggregationBuilder(this, factoriesBuilder, metadata); + protected ValuesSourceType defaultValueSourceType() { + return CoreValuesSourceType.BYTES; + } + + @Override + protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metaData) { + return new ChildrenAggregationBuilder(this, factoriesBuilder, metaData); } /** * Read from a stream. */ public ChildrenAggregationBuilder(StreamInput in) throws IOException { - super(in, CoreValuesSourceType.BYTES, ValueType.STRING); + super(in); childType = in.readString(); } @@ -100,33 +102,29 @@ public BucketCardinality bucketCardinality() { } @Override - protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, - ValuesSourceConfig config, - AggregatorFactory parent, - Builder subFactoriesBuilder) throws IOException { + protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, + ValuesSourceConfig config, + AggregatorFactory parent, + Builder subFactoriesBuilder) throws IOException { return new ChildrenAggregatorFactory(name, config, childFilter, parentFilter, queryShardContext, parent, subFactoriesBuilder, metadata); } @Override - protected ValuesSourceConfig resolveConfig(QueryShardContext queryShardContext) { - ValuesSourceConfig config = new ValuesSourceConfig<>(CoreValuesSourceType.BYTES); - joinFieldResolveConfig(queryShardContext, config); - return config; - } - - private void joinFieldResolveConfig(QueryShardContext queryShardContext, ValuesSourceConfig config) { + protected ValuesSourceConfig resolveConfig(QueryShardContext queryShardContext) { + ValuesSourceConfig config; ParentJoinFieldMapper parentJoinFieldMapper = ParentJoinFieldMapper.getMapper(queryShardContext.getMapperService()); ParentIdFieldMapper parentIdFieldMapper = parentJoinFieldMapper.getParentIdFieldMapper(childType, false); if (parentIdFieldMapper != null) { parentFilter = parentIdFieldMapper.getParentFilter(); childFilter = parentIdFieldMapper.getChildFilter(childType); MappedFieldType fieldType = parentIdFieldMapper.fieldType(); - final SortedSetDVOrdinalsIndexFieldData fieldData = queryShardContext.getForField(fieldType); - config.fieldContext(new FieldContext(fieldType.name(), fieldData, fieldType)); + config = ValuesSourceConfig.resolveFieldOnly(fieldType, queryShardContext); } else { - config.unmapped(true); + // Unmapped field case + config = ValuesSourceConfig.resolveUnmapped(defaultValueSourceType(), queryShardContext); } + return config; } @Override diff --git a/modules/parent-join/src/main/java/org/elasticsearch/join/aggregations/ChildrenAggregatorFactory.java b/modules/parent-join/src/main/java/org/elasticsearch/join/aggregations/ChildrenAggregatorFactory.java index 191da8142824c..45b5c30c6c370 100644 --- a/modules/parent-join/src/main/java/org/elasticsearch/join/aggregations/ChildrenAggregatorFactory.java +++ b/modules/parent-join/src/main/java/org/elasticsearch/join/aggregations/ChildrenAggregatorFactory.java @@ -21,12 +21,14 @@ import org.apache.lucene.search.Query; import org.elasticsearch.index.query.QueryShardContext; +import org.elasticsearch.search.aggregations.AggregationExecutionException; import org.elasticsearch.search.aggregations.Aggregator; import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.InternalAggregation; import org.elasticsearch.search.aggregations.NonCollectingAggregator; import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSource.Bytes.WithOrdinals; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; @@ -36,13 +38,13 @@ import java.util.List; import java.util.Map; -public class ChildrenAggregatorFactory extends ValuesSourceAggregatorFactory { +public class ChildrenAggregatorFactory extends ValuesSourceAggregatorFactory { private final Query parentFilter; private final Query childFilter; public ChildrenAggregatorFactory(String name, - ValuesSourceConfig config, + ValuesSourceConfig config, Query childFilter, Query parentFilter, QueryShardContext context, @@ -67,12 +69,17 @@ public InternalAggregation buildEmptyAggregation() { } @Override - protected Aggregator doCreateInternal(WithOrdinals valuesSource, + protected Aggregator doCreateInternal(ValuesSource rawValuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, List pipelineAggregators, Map metadata) throws IOException { + if (rawValuesSource instanceof WithOrdinals == false) { + throw new AggregationExecutionException("ValuesSource type " + rawValuesSource.toString() + + "is not supported for aggregation " + this.name()); + } + WithOrdinals valuesSource = (WithOrdinals) rawValuesSource; long maxOrd = valuesSource.globalMaxOrd(searchContext.searcher()); if (collectsFromSingleBucket) { return new ParentToChildrenAggregator(name, factories, searchContext, parent, childFilter, diff --git a/modules/parent-join/src/main/java/org/elasticsearch/join/aggregations/ParentAggregationBuilder.java b/modules/parent-join/src/main/java/org/elasticsearch/join/aggregations/ParentAggregationBuilder.java index 6d41524c8caf7..8c76abf7588ff 100644 --- a/modules/parent-join/src/main/java/org/elasticsearch/join/aggregations/ParentAggregationBuilder.java +++ b/modules/parent-join/src/main/java/org/elasticsearch/join/aggregations/ParentAggregationBuilder.java @@ -25,7 +25,6 @@ import org.elasticsearch.common.io.stream.StreamOutput; import org.elasticsearch.common.xcontent.XContentBuilder; import org.elasticsearch.common.xcontent.XContentParser; -import org.elasticsearch.index.fielddata.plain.SortedSetDVOrdinalsIndexFieldData; import org.elasticsearch.index.mapper.MappedFieldType; import org.elasticsearch.index.query.QueryShardContext; import org.elasticsearch.join.mapper.ParentIdFieldMapper; @@ -34,19 +33,17 @@ import org.elasticsearch.search.aggregations.AggregatorFactories.Builder; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; -import org.elasticsearch.search.aggregations.support.FieldContext; -import org.elasticsearch.search.aggregations.support.ValueType; -import org.elasticsearch.search.aggregations.support.ValuesSource.Bytes.WithOrdinals; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.util.Map; import java.util.Objects; public class ParentAggregationBuilder - extends ValuesSourceAggregationBuilder { + extends ValuesSourceAggregationBuilder { public static final String NAME = "parent"; @@ -61,7 +58,7 @@ public class ParentAggregationBuilder * the type of children documents */ public ParentAggregationBuilder(String name, String childType) { - super(name, CoreValuesSourceType.BYTES, ValueType.STRING); + super(name); if (childType == null) { throw new IllegalArgumentException("[childType] must not be null: [" + name + "]"); } @@ -77,15 +74,20 @@ protected ParentAggregationBuilder(ParentAggregationBuilder clone, } @Override - protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metadata) { - return new ParentAggregationBuilder(this, factoriesBuilder, metadata); + protected ValuesSourceType defaultValueSourceType() { + return CoreValuesSourceType.BYTES; + } + + @Override + protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metaData) { + return new ParentAggregationBuilder(this, factoriesBuilder, metaData); } /** * Read from a stream. */ public ParentAggregationBuilder(StreamInput in) throws IOException { - super(in, CoreValuesSourceType.BYTES, ValueType.STRING); + super(in); childType = in.readString(); } @@ -100,33 +102,29 @@ public BucketCardinality bucketCardinality() { } @Override - protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, - ValuesSourceConfig config, - AggregatorFactory parent, - Builder subFactoriesBuilder) throws IOException { + protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, + ValuesSourceConfig config, + AggregatorFactory parent, + Builder subFactoriesBuilder) throws IOException { return new ParentAggregatorFactory(name, config, childFilter, parentFilter, queryShardContext, parent, subFactoriesBuilder, metadata); } @Override - protected ValuesSourceConfig resolveConfig(QueryShardContext queryShardContext) { - ValuesSourceConfig config = new ValuesSourceConfig<>(CoreValuesSourceType.BYTES); - joinFieldResolveConfig(queryShardContext, config); - return config; - } - - private void joinFieldResolveConfig(QueryShardContext queryShardContext, ValuesSourceConfig config) { + protected ValuesSourceConfig resolveConfig(QueryShardContext queryShardContext) { + ValuesSourceConfig config; ParentJoinFieldMapper parentJoinFieldMapper = ParentJoinFieldMapper.getMapper(queryShardContext.getMapperService()); ParentIdFieldMapper parentIdFieldMapper = parentJoinFieldMapper.getParentIdFieldMapper(childType, false); if (parentIdFieldMapper != null) { parentFilter = parentIdFieldMapper.getParentFilter(); childFilter = parentIdFieldMapper.getChildFilter(childType); MappedFieldType fieldType = parentIdFieldMapper.fieldType(); - final SortedSetDVOrdinalsIndexFieldData fieldData = queryShardContext.getForField(fieldType); - config.fieldContext(new FieldContext(fieldType.name(), fieldData, fieldType)); + config = ValuesSourceConfig.resolveFieldOnly(fieldType, queryShardContext); } else { - config.unmapped(true); + // unmapped case + config = ValuesSourceConfig.resolveUnmapped(defaultValueSourceType(), queryShardContext); } + return config; } @Override diff --git a/modules/parent-join/src/main/java/org/elasticsearch/join/aggregations/ParentAggregatorFactory.java b/modules/parent-join/src/main/java/org/elasticsearch/join/aggregations/ParentAggregatorFactory.java index 1c932b1d53ea1..181c66089fe49 100644 --- a/modules/parent-join/src/main/java/org/elasticsearch/join/aggregations/ParentAggregatorFactory.java +++ b/modules/parent-join/src/main/java/org/elasticsearch/join/aggregations/ParentAggregatorFactory.java @@ -21,12 +21,14 @@ import org.apache.lucene.search.Query; import org.elasticsearch.index.query.QueryShardContext; +import org.elasticsearch.search.aggregations.AggregationExecutionException; import org.elasticsearch.search.aggregations.Aggregator; import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.InternalAggregation; import org.elasticsearch.search.aggregations.NonCollectingAggregator; import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSource.Bytes.WithOrdinals; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; @@ -36,13 +38,13 @@ import java.util.List; import java.util.Map; -public class ParentAggregatorFactory extends ValuesSourceAggregatorFactory { +public class ParentAggregatorFactory extends ValuesSourceAggregatorFactory { private final Query parentFilter; private final Query childFilter; public ParentAggregatorFactory(String name, - ValuesSourceConfig config, + ValuesSourceConfig config, Query childFilter, Query parentFilter, QueryShardContext queryShardContext, @@ -67,12 +69,17 @@ public InternalAggregation buildEmptyAggregation() { } @Override - protected Aggregator doCreateInternal(WithOrdinals valuesSource, + protected Aggregator doCreateInternal(ValuesSource rawValuesSource, SearchContext searchContext, Aggregator children, boolean collectsFromSingleBucket, List pipelineAggregators, Map metadata) throws IOException { + if (rawValuesSource instanceof WithOrdinals == false) { + throw new AggregationExecutionException("ValuesSource type " + rawValuesSource.toString() + + "is not supported for aggregation " + this.name()); + } + WithOrdinals valuesSource = (WithOrdinals) rawValuesSource; long maxOrd = valuesSource.globalMaxOrd(searchContext.searcher()); if (collectsFromSingleBucket) { return new ChildrenToParentAggregator(name, factories, searchContext, children, childFilter, diff --git a/modules/parent-join/src/test/java/org/elasticsearch/join/aggregations/ChildrenToParentAggregatorTests.java b/modules/parent-join/src/test/java/org/elasticsearch/join/aggregations/ChildrenToParentAggregatorTests.java index bbaa4e00ca9e5..3f2fed591b3d0 100644 --- a/modules/parent-join/src/test/java/org/elasticsearch/join/aggregations/ChildrenToParentAggregatorTests.java +++ b/modules/parent-join/src/test/java/org/elasticsearch/join/aggregations/ChildrenToParentAggregatorTests.java @@ -304,7 +304,8 @@ private void testCaseTerms(Query query, IndexSearcher indexSearcher, Consumer verify) throws IOException { AggregationBuilder aggregationBuilder = - new TermsAggregationBuilder("subvalue_terms", ValueType.LONG).field("subNumber"). + new TermsAggregationBuilder("subvalue_terms").userValueTypeHint(ValueType.LONG).field("subNumber"). subAggregation(new ParentAggregationBuilder("to_parent", CHILD_TYPE). - subAggregation(new TermsAggregationBuilder("value_terms", ValueType.LONG).field("number"))); + subAggregation(new TermsAggregationBuilder("value_terms").userValueTypeHint(ValueType.LONG).field("number"))); MappedFieldType fieldType = new NumberFieldMapper.NumberFieldType(NumberFieldMapper.NumberType.LONG); fieldType.setName("number"); diff --git a/qa/multi-cluster-search/src/test/java/org/elasticsearch/search/CCSDuelIT.java b/qa/multi-cluster-search/src/test/java/org/elasticsearch/search/CCSDuelIT.java index d2f876cd17848..0c16e539fcaaf 100644 --- a/qa/multi-cluster-search/src/test/java/org/elasticsearch/search/CCSDuelIT.java +++ b/qa/multi-cluster-search/src/test/java/org/elasticsearch/search/CCSDuelIT.java @@ -524,31 +524,32 @@ public void testTermsAggsWithProfile() throws Exception { private static SearchSourceBuilder buildTermsAggsSource() { SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); sourceBuilder.size(0); - TermsAggregationBuilder cluster = new TermsAggregationBuilder("cluster123", ValueType.STRING); + TermsAggregationBuilder cluster = new TermsAggregationBuilder("cluster123").userValueTypeHint(ValueType.STRING); cluster.field("_index"); - TermsAggregationBuilder type = new TermsAggregationBuilder("type", ValueType.STRING); + TermsAggregationBuilder type = new TermsAggregationBuilder("type").userValueTypeHint(ValueType.STRING); type.field("type.keyword"); type.showTermDocCountError(true); type.order(BucketOrder.key(true)); cluster.subAggregation(type); sourceBuilder.aggregation(cluster); - TermsAggregationBuilder tags = new TermsAggregationBuilder("tags", ValueType.STRING); + TermsAggregationBuilder tags = new TermsAggregationBuilder("tags").userValueTypeHint(ValueType.STRING); tags.field("tags.keyword"); tags.showTermDocCountError(true); tags.size(100); sourceBuilder.aggregation(tags); - TermsAggregationBuilder tags2 = new TermsAggregationBuilder("tags", ValueType.STRING); + TermsAggregationBuilder tags2 = new TermsAggregationBuilder("tags").userValueTypeHint(ValueType.STRING); tags2.field("tags.keyword"); tags.subAggregation(tags2); FilterAggregationBuilder answers = new FilterAggregationBuilder("answers", new TermQueryBuilder("type", "answer")); - TermsAggregationBuilder answerPerQuestion = new TermsAggregationBuilder("answer_per_question", ValueType.STRING); + TermsAggregationBuilder answerPerQuestion = new TermsAggregationBuilder("answer_per_question") + .userValueTypeHint(ValueType.STRING); answerPerQuestion.showTermDocCountError(true); answerPerQuestion.field("questionId.keyword"); answers.subAggregation(answerPerQuestion); - TermsAggregationBuilder answerPerUser = new TermsAggregationBuilder("answer_per_user", ValueType.STRING); + TermsAggregationBuilder answerPerUser = new TermsAggregationBuilder("answer_per_user").userValueTypeHint(ValueType.STRING); answerPerUser.field("user.keyword"); answerPerUser.size(30); answerPerUser.showTermDocCountError(true); @@ -563,7 +564,7 @@ public void testDateHistogram() throws Exception { SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); sourceBuilder.size(0); searchRequest.source(sourceBuilder); - TermsAggregationBuilder tags = new TermsAggregationBuilder("tags", ValueType.STRING); + TermsAggregationBuilder tags = new TermsAggregationBuilder("tags").userValueTypeHint(ValueType.STRING); tags.field("tags.keyword"); tags.showTermDocCountError(true); DateHistogramAggregationBuilder creation = new DateHistogramAggregationBuilder("creation"); @@ -580,7 +581,7 @@ public void testCardinalityAgg() throws Exception { SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); sourceBuilder.size(0); searchRequest.source(sourceBuilder); - CardinalityAggregationBuilder tags = new CardinalityAggregationBuilder("tags", ValueType.STRING); + CardinalityAggregationBuilder tags = new CardinalityAggregationBuilder("tags").userValueTypeHint(ValueType.STRING); tags.field("tags.keyword"); sourceBuilder.aggregation(tags); duelSearch(searchRequest, CCSDuelIT::assertAggs); @@ -619,7 +620,7 @@ public void testTopHits() throws Exception { topHits.size(10); topHits.sort("creationDate", SortOrder.DESC); topHits.sort("id", SortOrder.ASC); - TermsAggregationBuilder tags = new TermsAggregationBuilder("tags", ValueType.STRING); + TermsAggregationBuilder tags = new TermsAggregationBuilder("tags").userValueTypeHint(ValueType.STRING); tags.field("tags.keyword"); tags.size(10); tags.subAggregation(topHits); diff --git a/rest-api-spec/src/main/resources/rest-api-spec/test/search.aggregation/280_rare_terms.yml b/rest-api-spec/src/main/resources/rest-api-spec/test/search.aggregation/280_rare_terms.yml index 20d8bbf87c370..9048914174133 100644 --- a/rest-api-spec/src/main/resources/rest-api-spec/test/search.aggregation/280_rare_terms.yml +++ b/rest-api-spec/src/main/resources/rest-api-spec/test/search.aggregation/280_rare_terms.yml @@ -112,7 +112,7 @@ setup: - length: { aggregations.ip_terms.buckets: 0 } - do: - catch: request + catch: /Aggregation \[ip_terms\] cannot support regular expression style include\/exclude settings as they can only be applied to string fields\. Use an array of values for include\/exclude clauses/ search: index: test_1 body: { "size" : 0, "aggs" : { "ip_terms" : { "rare_terms" : { "field" : "ip", "exclude" : "127.*" } } } } diff --git a/rest-api-spec/src/main/resources/rest-api-spec/test/search.aggregation/30_sig_terms.yml b/rest-api-spec/src/main/resources/rest-api-spec/test/search.aggregation/30_sig_terms.yml index dabd4aebf815d..d7759afe4a907 100644 --- a/rest-api-spec/src/main/resources/rest-api-spec/test/search.aggregation/30_sig_terms.yml +++ b/rest-api-spec/src/main/resources/rest-api-spec/test/search.aggregation/30_sig_terms.yml @@ -133,7 +133,7 @@ - length: { aggregations.ip_terms.buckets: 0 } - do: - catch: request + catch: /Aggregation \[ip_terms\] cannot support regular expression style include\/exclude settings as they can only be applied to string fields\. Use an array of values for include\/exclude clauses/ search: rest_total_hits_as_int: true body: { "size" : 0, "aggs" : { "ip_terms" : { "significant_terms" : { "field" : "ip", "exclude" : "127.*" } } } } diff --git a/server/src/main/java/org/elasticsearch/index/IndexModule.java b/server/src/main/java/org/elasticsearch/index/IndexModule.java index 16b7c069bc5cd..30bd74e3edd2c 100644 --- a/server/src/main/java/org/elasticsearch/index/IndexModule.java +++ b/server/src/main/java/org/elasticsearch/index/IndexModule.java @@ -61,6 +61,7 @@ import org.elasticsearch.indices.mapper.MapperRegistry; import org.elasticsearch.plugins.IndexStorePlugin; import org.elasticsearch.script.ScriptService; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.threadpool.ThreadPool; import java.io.IOException; @@ -388,23 +389,22 @@ public static Type defaultStoreType(final boolean allowMmap) { } } - public IndexService newIndexService( - IndexService.IndexCreationContext indexCreationContext, - NodeEnvironment environment, - NamedXContentRegistry xContentRegistry, - IndexService.ShardStoreDeleter shardStoreDeleter, - CircuitBreakerService circuitBreakerService, - BigArrays bigArrays, - ThreadPool threadPool, - ScriptService scriptService, - ClusterService clusterService, - Client client, - IndicesQueryCache indicesQueryCache, - MapperRegistry mapperRegistry, - IndicesFieldDataCache indicesFieldDataCache, - NamedWriteableRegistry namedWriteableRegistry, - BooleanSupplier idFieldDataEnabled) - throws IOException { + public IndexService newIndexService(IndexService.IndexCreationContext indexCreationContext, + NodeEnvironment environment, + NamedXContentRegistry xContentRegistry, + IndexService.ShardStoreDeleter shardStoreDeleter, + CircuitBreakerService circuitBreakerService, + BigArrays bigArrays, + ThreadPool threadPool, + ScriptService scriptService, + ClusterService clusterService, + Client client, + IndicesQueryCache indicesQueryCache, + MapperRegistry mapperRegistry, + IndicesFieldDataCache indicesFieldDataCache, + NamedWriteableRegistry namedWriteableRegistry, + BooleanSupplier idFieldDataEnabled, + ValuesSourceRegistry valuesSourceRegistry) throws IOException { final IndexEventListener eventListener = freeze(); Function> readerWrapperFactory = indexReaderWrapper.get() == null ? (shard) -> null : indexReaderWrapper.get(); @@ -431,7 +431,8 @@ public IndexService newIndexService( new SimilarityService(indexSettings, scriptService, similarities), shardStoreDeleter, indexAnalyzers, engineFactory, circuitBreakerService, bigArrays, threadPool, scriptService, clusterService, client, queryCache, directoryFactory, eventListener, readerWrapperFactory, mapperRegistry, indicesFieldDataCache, searchOperationListeners, - indexOperationListeners, namedWriteableRegistry, idFieldDataEnabled, allowExpensiveQueries, expressionResolver); + indexOperationListeners, namedWriteableRegistry, idFieldDataEnabled, allowExpensiveQueries, expressionResolver, + valuesSourceRegistry); success = true; return indexService; } finally { diff --git a/server/src/main/java/org/elasticsearch/index/IndexService.java b/server/src/main/java/org/elasticsearch/index/IndexService.java index 0c8c66f18eae2..1546afcfa7851 100644 --- a/server/src/main/java/org/elasticsearch/index/IndexService.java +++ b/server/src/main/java/org/elasticsearch/index/IndexService.java @@ -81,6 +81,7 @@ import org.elasticsearch.indices.mapper.MapperRegistry; import org.elasticsearch.plugins.IndexStorePlugin; import org.elasticsearch.script.ScriptService; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.threadpool.ThreadPool; import java.io.Closeable; @@ -146,6 +147,7 @@ public class IndexService extends AbstractIndexComponent implements IndicesClust private final CircuitBreakerService circuitBreakerService; private final IndexNameExpressionResolver expressionResolver; private Supplier indexSortSupplier; + private ValuesSourceRegistry valuesSourceRegistry; public IndexService( IndexSettings indexSettings, @@ -172,7 +174,8 @@ public IndexService( NamedWriteableRegistry namedWriteableRegistry, BooleanSupplier idFieldDataEnabled, BooleanSupplier allowExpensiveQueries, - IndexNameExpressionResolver expressionResolver) { + IndexNameExpressionResolver expressionResolver, + ValuesSourceRegistry valuesSourceRegistry) { super(indexSettings); this.allowExpensiveQueries = allowExpensiveQueries; this.indexSettings = indexSettings; @@ -181,6 +184,7 @@ public IndexService( this.namedWriteableRegistry = namedWriteableRegistry; this.circuitBreakerService = circuitBreakerService; this.expressionResolver = expressionResolver; + this.valuesSourceRegistry = valuesSourceRegistry; if (needsMapperService(indexSettings, indexCreationContext)) { assert indexAnalyzers != null; this.mapperService = new MapperService(indexSettings, indexAnalyzers, xContentRegistry, similarityService, mapperRegistry, @@ -578,7 +582,7 @@ public QueryShardContext newQueryShardContext(int shardId, IndexSearcher searche return new QueryShardContext( shardId, indexSettings, bigArrays, indexCache.bitsetFilterCache(), indexFieldData::getForField, mapperService(), similarityService(), scriptService, xContentRegistry, namedWriteableRegistry, client, searcher, nowInMillis, clusterAlias, - indexNameMatcher, allowExpensiveQueries); + indexNameMatcher, allowExpensiveQueries, valuesSourceRegistry); } /** diff --git a/server/src/main/java/org/elasticsearch/index/mapper/BooleanFieldMapper.java b/server/src/main/java/org/elasticsearch/index/mapper/BooleanFieldMapper.java index 009848df1d8a9..3def7265d4c9c 100644 --- a/server/src/main/java/org/elasticsearch/index/mapper/BooleanFieldMapper.java +++ b/server/src/main/java/org/elasticsearch/index/mapper/BooleanFieldMapper.java @@ -193,7 +193,7 @@ public IndexFieldData.Builder fielddataBuilder(String fullyQualifiedIndexName) { @Override public ValuesSourceType getValuesSourceType() { - return CoreValuesSourceType.NUMERIC; + return CoreValuesSourceType.BOOLEAN; } @Override diff --git a/server/src/main/java/org/elasticsearch/index/mapper/DateFieldMapper.java b/server/src/main/java/org/elasticsearch/index/mapper/DateFieldMapper.java index 3fe1df76495d0..d3a781d2d65c4 100644 --- a/server/src/main/java/org/elasticsearch/index/mapper/DateFieldMapper.java +++ b/server/src/main/java/org/elasticsearch/index/mapper/DateFieldMapper.java @@ -545,7 +545,7 @@ public IndexFieldData.Builder fielddataBuilder(String fullyQualifiedIndexName) { @Override public ValuesSourceType getValuesSourceType() { - return CoreValuesSourceType.NUMERIC; + return CoreValuesSourceType.DATE; } @Override diff --git a/server/src/main/java/org/elasticsearch/index/mapper/IdFieldMapper.java b/server/src/main/java/org/elasticsearch/index/mapper/IdFieldMapper.java index 58c3b17714980..5cea22dc39387 100644 --- a/server/src/main/java/org/elasticsearch/index/mapper/IdFieldMapper.java +++ b/server/src/main/java/org/elasticsearch/index/mapper/IdFieldMapper.java @@ -48,6 +48,8 @@ import org.elasticsearch.indices.breaker.CircuitBreakerService; import org.elasticsearch.search.DocValueFormat; import org.elasticsearch.search.MultiValueMode; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import org.elasticsearch.search.sort.BucketedSort; import org.elasticsearch.search.sort.SortOrder; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; diff --git a/server/src/main/java/org/elasticsearch/index/mapper/IpFieldMapper.java b/server/src/main/java/org/elasticsearch/index/mapper/IpFieldMapper.java index c1cf50a56cde5..a3abf963104ae 100644 --- a/server/src/main/java/org/elasticsearch/index/mapper/IpFieldMapper.java +++ b/server/src/main/java/org/elasticsearch/index/mapper/IpFieldMapper.java @@ -298,7 +298,7 @@ public IndexFieldData.Builder fielddataBuilder(String fullyQualifiedIndexName) { @Override public ValuesSourceType getValuesSourceType() { - return CoreValuesSourceType.BYTES; + return CoreValuesSourceType.IP; } @Override diff --git a/server/src/main/java/org/elasticsearch/index/query/QueryShardContext.java b/server/src/main/java/org/elasticsearch/index/query/QueryShardContext.java index 6fbaedd0546a5..f6e263fb41fd2 100644 --- a/server/src/main/java/org/elasticsearch/index/query/QueryShardContext.java +++ b/server/src/main/java/org/elasticsearch/index/query/QueryShardContext.java @@ -58,6 +58,7 @@ import org.elasticsearch.script.ScriptContext; import org.elasticsearch.script.ScriptFactory; import org.elasticsearch.script.ScriptService; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.search.lookup.SearchLookup; import org.elasticsearch.transport.RemoteClusterAware; @@ -115,6 +116,7 @@ public String[] getTypes() { private boolean allowUnmappedFields; private boolean mapUnmappedFieldAsString; private NestedScope nestedScope; + private ValuesSourceRegistry valuesSourceRegistry; public QueryShardContext(int shardId, IndexSettings indexSettings, @@ -131,18 +133,19 @@ public QueryShardContext(int shardId, LongSupplier nowInMillis, String clusterAlias, Predicate indexNameMatcher, - BooleanSupplier allowExpensiveQueries) { + BooleanSupplier allowExpensiveQueries, + ValuesSourceRegistry valuesSourceRegistry) { this(shardId, indexSettings, bigArrays, bitsetFilterCache, indexFieldDataLookup, mapperService, similarityService, scriptService, xContentRegistry, namedWriteableRegistry, client, searcher, nowInMillis, indexNameMatcher, new Index(RemoteClusterAware.buildRemoteIndexName(clusterAlias, indexSettings.getIndex().getName()), - indexSettings.getIndex().getUUID()), allowExpensiveQueries); + indexSettings.getIndex().getUUID()), allowExpensiveQueries, valuesSourceRegistry); } public QueryShardContext(QueryShardContext source) { this(source.shardId, source.indexSettings, source.bigArrays, source.bitsetFilterCache, source.indexFieldDataService, source.mapperService, source.similarityService, source.scriptService, source.getXContentRegistry(), source.getWriteableRegistry(), source.client, source.searcher, source.nowInMillis, source.indexNameMatcher, - source.fullyQualifiedIndex, source.allowExpensiveQueries); + source.fullyQualifiedIndex, source.allowExpensiveQueries, source.valuesSourceRegistry); } private QueryShardContext(int shardId, @@ -160,7 +163,8 @@ private QueryShardContext(int shardId, LongSupplier nowInMillis, Predicate indexNameMatcher, Index fullyQualifiedIndex, - BooleanSupplier allowExpensiveQueries) { + BooleanSupplier allowExpensiveQueries, + ValuesSourceRegistry valuesSourceRegistry) { super(xContentRegistry, namedWriteableRegistry, client, nowInMillis); this.shardId = shardId; this.similarityService = similarityService; @@ -176,6 +180,7 @@ private QueryShardContext(int shardId, this.indexNameMatcher = indexNameMatcher; this.fullyQualifiedIndex = fullyQualifiedIndex; this.allowExpensiveQueries = allowExpensiveQueries; + this.valuesSourceRegistry = valuesSourceRegistry; } private void reset() { @@ -281,6 +286,10 @@ public Analyzer getSearchQuoteAnalyzer(MappedFieldType fieldType) { return getMapperService().searchQuoteAnalyzer(); } + public ValuesSourceRegistry getValuesSourceRegistry() { + return valuesSourceRegistry; + } + public void setAllowUnmappedFields(boolean allowUnmappedFields) { this.allowUnmappedFields = allowUnmappedFields; } diff --git a/server/src/main/java/org/elasticsearch/indices/IndicesService.java b/server/src/main/java/org/elasticsearch/indices/IndicesService.java index 05d816341d1ca..bd6cf727ec7a0 100644 --- a/server/src/main/java/org/elasticsearch/indices/IndicesService.java +++ b/server/src/main/java/org/elasticsearch/indices/IndicesService.java @@ -126,6 +126,7 @@ import org.elasticsearch.plugins.PluginsService; import org.elasticsearch.repositories.RepositoriesService; import org.elasticsearch.script.ScriptService; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.search.internal.AliasFilter; import org.elasticsearch.search.internal.SearchContext; import org.elasticsearch.search.internal.ShardSearchRequest; @@ -229,6 +230,7 @@ public class IndicesService extends AbstractLifecycleComponent private final EsThreadPoolExecutor danglingIndicesThreadPoolExecutor; private final Set danglingIndicesToWrite = Sets.newConcurrentHashSet(); private final boolean nodeWriteDanglingIndicesInfo; + private ValuesSourceRegistry valuesSourceRegistry; @Override @@ -243,12 +245,13 @@ public IndicesService(Settings settings, PluginsService pluginsService, NodeEnvi IndexScopedSettings indexScopedSettings, CircuitBreakerService circuitBreakerService, BigArrays bigArrays, ScriptService scriptService, ClusterService clusterService, Client client, MetaStateService metaStateService, Collection>> engineFactoryProviders, - Map directoryFactories) { + Map directoryFactories, ValuesSourceRegistry valuesSourceRegistry) { this.settings = settings; this.threadPool = threadPool; this.pluginsService = pluginsService; this.nodeEnv = nodeEnv; this.xContentRegistry = xContentRegistry; + this.valuesSourceRegistry = valuesSourceRegistry; this.shardsClosedTimeout = settings.getAsTime(INDICES_SHARDS_CLOSED_TIMEOUT, new TimeValue(1, TimeUnit.DAYS)); this.analysisRegistry = analysisRegistry; this.indexNameExpressionResolver = indexNameExpressionResolver; @@ -657,7 +660,8 @@ private synchronized IndexService createIndexService(IndexService.IndexCreationC mapperRegistry, indicesFieldDataCache, namedWriteableRegistry, - this::isIdFieldDataEnabled + this::isIdFieldDataEnabled, + valuesSourceRegistry ); } diff --git a/server/src/main/java/org/elasticsearch/node/Node.java b/server/src/main/java/org/elasticsearch/node/Node.java index 4c612e72279eb..3175f26502b2d 100644 --- a/server/src/main/java/org/elasticsearch/node/Node.java +++ b/server/src/main/java/org/elasticsearch/node/Node.java @@ -458,7 +458,8 @@ protected Node(final Environment initialEnvironment, new IndicesService(settings, pluginsService, nodeEnvironment, xContentRegistry, analysisModule.getAnalysisRegistry(), clusterModule.getIndexNameExpressionResolver(), indicesModule.getMapperRegistry(), namedWriteableRegistry, threadPool, settingsModule.getIndexScopedSettings(), circuitBreakerService, bigArrays, scriptService, - clusterService, client, metaStateService, engineFactoryProviders, indexStoreFactories); + clusterService, client, metaStateService, engineFactoryProviders, indexStoreFactories, + searchModule.getValuesSourceRegistry()); final AliasValidator aliasValidator = new AliasValidator(); diff --git a/server/src/main/java/org/elasticsearch/plugins/SearchPlugin.java b/server/src/main/java/org/elasticsearch/plugins/SearchPlugin.java index fe5c65d670468..2f6aef9dbbc1f 100644 --- a/server/src/main/java/org/elasticsearch/plugins/SearchPlugin.java +++ b/server/src/main/java/org/elasticsearch/plugins/SearchPlugin.java @@ -44,6 +44,7 @@ import org.elasticsearch.search.aggregations.pipeline.MovAvgModel; import org.elasticsearch.search.aggregations.pipeline.MovAvgPipelineAggregator; import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.search.fetch.FetchSubPhase; import org.elasticsearch.search.fetch.subphase.highlight.Highlighter; import org.elasticsearch.search.rescore.Rescorer; @@ -57,6 +58,7 @@ import java.util.Map; import java.util.TreeMap; import java.util.function.BiFunction; +import java.util.function.Consumer; import static java.util.Collections.emptyList; import static java.util.Collections.emptyMap; @@ -121,6 +123,13 @@ default List> getQueries() { default List getAggregations() { return emptyList(); } + /** + * Allows plugins to register new aggregations using aggregation names that are already defined + * in Core, as long as the new aggregations target different ValuesSourceTypes + */ + default List> getBareAggregatorRegistrar() { + return emptyList(); + } /** * The new {@link PipelineAggregator}s added by this plugin. */ @@ -259,6 +268,7 @@ public QuerySpec(String name, Writeable.Reader reader, QueryParser parser) */ class AggregationSpec extends SearchExtensionSpec> { private final Map> resultReaders = new TreeMap<>(); + private Consumer aggregatorRegistrar; /** * Specification for an {@link Aggregation}. @@ -341,6 +351,23 @@ public AggregationSpec addResultReader(String writeableName, Writeable.Reader> getResultReaders() { return resultReaders; } + + /** + * Get the function to register the {@link org.elasticsearch.search.aggregations.support.ValuesSource} to aggregator mappings for + * this aggregation + */ + public Consumer getAggregatorRegistrar() { + return aggregatorRegistrar; + } + + /** + * Set the function to register the {@link org.elasticsearch.search.aggregations.support.ValuesSource} to aggregator mappings for + * this aggregation + */ + public AggregationSpec setAggregatorRegistrar(Consumer aggregatorRegistrar) { + this.aggregatorRegistrar = aggregatorRegistrar; + return this; + } } /** diff --git a/server/src/main/java/org/elasticsearch/search/SearchModule.java b/server/src/main/java/org/elasticsearch/search/SearchModule.java index 9db914f70da82..e416e0e615a0a 100644 --- a/server/src/main/java/org/elasticsearch/search/SearchModule.java +++ b/server/src/main/java/org/elasticsearch/search/SearchModule.java @@ -241,6 +241,7 @@ import org.elasticsearch.search.aggregations.pipeline.StatsBucketPipelineAggregator; import org.elasticsearch.search.aggregations.pipeline.SumBucketPipelineAggregationBuilder; import org.elasticsearch.search.aggregations.pipeline.SumBucketPipelineAggregator; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.search.fetch.FetchPhase; import org.elasticsearch.search.fetch.FetchSubPhase; import org.elasticsearch.search.fetch.subphase.FetchDocValuesPhase; @@ -306,6 +307,7 @@ public class SearchModule { private final Settings settings; private final List namedWriteables = new ArrayList<>(); private final List namedXContents = new ArrayList<>(); + private ValuesSourceRegistry valuesSourceRegistry; /** * Constructs a new SearchModule object @@ -320,6 +322,7 @@ public class SearchModule { public SearchModule(Settings settings, boolean transportClient, List plugins) { this.settings = settings; this.transportClient = transportClient; + this.valuesSourceRegistry = new ValuesSourceRegistry(); registerSuggesters(plugins); highlighters = setupHighlighters(settings, plugins); registerScoreFunctions(plugins); @@ -346,6 +349,10 @@ public List getNamedXContents() { return namedXContents; } + public ValuesSourceRegistry getValuesSourceRegistry() { + return valuesSourceRegistry; + } + /** * Returns the {@link Highlighter} registry */ @@ -362,34 +369,45 @@ public ParseFieldRegistry getMovingAverageModel private void registerAggregations(List plugins) { registerAggregation(new AggregationSpec(AvgAggregationBuilder.NAME, AvgAggregationBuilder::new, AvgAggregationBuilder.PARSER) - .addResultReader(InternalAvg::new)); + .addResultReader(InternalAvg::new) + .setAggregatorRegistrar(AvgAggregationBuilder::registerAggregators)); registerAggregation(new AggregationSpec(WeightedAvgAggregationBuilder.NAME, WeightedAvgAggregationBuilder::new, WeightedAvgAggregationBuilder.PARSER).addResultReader(InternalWeightedAvg::new)); registerAggregation(new AggregationSpec(SumAggregationBuilder.NAME, SumAggregationBuilder::new, SumAggregationBuilder.PARSER) - .addResultReader(InternalSum::new)); + .addResultReader(InternalSum::new) + .setAggregatorRegistrar(SumAggregationBuilder::registerAggregators)); registerAggregation(new AggregationSpec(MinAggregationBuilder.NAME, MinAggregationBuilder::new, MinAggregationBuilder.PARSER) - .addResultReader(InternalMin::new)); + .addResultReader(InternalMin::new) + .setAggregatorRegistrar(MinAggregationBuilder::registerAggregators)); registerAggregation(new AggregationSpec(MaxAggregationBuilder.NAME, MaxAggregationBuilder::new, MaxAggregationBuilder.PARSER) - .addResultReader(InternalMax::new)); + .addResultReader(InternalMax::new) + .setAggregatorRegistrar(MaxAggregationBuilder::registerAggregators)); registerAggregation(new AggregationSpec(StatsAggregationBuilder.NAME, StatsAggregationBuilder::new, StatsAggregationBuilder.PARSER) - .addResultReader(InternalStats::new)); + .addResultReader(InternalStats::new) + .setAggregatorRegistrar(StatsAggregationBuilder::registerAggregators)); registerAggregation(new AggregationSpec(ExtendedStatsAggregationBuilder.NAME, ExtendedStatsAggregationBuilder::new, ExtendedStatsAggregationBuilder.PARSER).addResultReader(InternalExtendedStats::new)); registerAggregation(new AggregationSpec(ValueCountAggregationBuilder.NAME, ValueCountAggregationBuilder::new, - ValueCountAggregationBuilder.PARSER).addResultReader(InternalValueCount::new)); + ValueCountAggregationBuilder.PARSER) + .addResultReader(InternalValueCount::new) + .setAggregatorRegistrar(ValueCountAggregationBuilder::registerAggregators)); registerAggregation(new AggregationSpec(PercentilesAggregationBuilder.NAME, PercentilesAggregationBuilder::new, PercentilesAggregationBuilder::parse) .addResultReader(InternalTDigestPercentiles.NAME, InternalTDigestPercentiles::new) - .addResultReader(InternalHDRPercentiles.NAME, InternalHDRPercentiles::new)); + .addResultReader(InternalHDRPercentiles.NAME, InternalHDRPercentiles::new) + .setAggregatorRegistrar(PercentilesAggregationBuilder::registerAggregators)); registerAggregation(new AggregationSpec(PercentileRanksAggregationBuilder.NAME, PercentileRanksAggregationBuilder::new, PercentileRanksAggregationBuilder::parse) .addResultReader(InternalTDigestPercentileRanks.NAME, InternalTDigestPercentileRanks::new) - .addResultReader(InternalHDRPercentileRanks.NAME, InternalHDRPercentileRanks::new)); + .addResultReader(InternalHDRPercentileRanks.NAME, InternalHDRPercentileRanks::new) + .setAggregatorRegistrar(PercentileRanksAggregationBuilder::registerAggregators)); registerAggregation(new AggregationSpec(MedianAbsoluteDeviationAggregationBuilder.NAME, - MedianAbsoluteDeviationAggregationBuilder::new, MedianAbsoluteDeviationAggregationBuilder.PARSER) - .addResultReader(InternalMedianAbsoluteDeviation::new)); + MedianAbsoluteDeviationAggregationBuilder::new, MedianAbsoluteDeviationAggregationBuilder.PARSER) + .addResultReader(InternalMedianAbsoluteDeviation::new) + .setAggregatorRegistrar(MedianAbsoluteDeviationAggregationBuilder::registerAggregators)); registerAggregation(new AggregationSpec(CardinalityAggregationBuilder.NAME, CardinalityAggregationBuilder::new, - CardinalityAggregationBuilder.PARSER).addResultReader(InternalCardinality::new)); + CardinalityAggregationBuilder.PARSER).addResultReader(InternalCardinality::new) + .setAggregatorRegistrar(CardinalityAggregationBuilder::registerAggregators)); registerAggregation(new AggregationSpec(GlobalAggregationBuilder.NAME, GlobalAggregationBuilder::new, GlobalAggregationBuilder::parse).addResultReader(InternalGlobal::new)); registerAggregation(new AggregationSpec(MissingAggregationBuilder.NAME, MissingAggregationBuilder::new, @@ -412,37 +430,52 @@ private void registerAggregations(List plugins) { .addResultReader(StringTerms.NAME, StringTerms::new) .addResultReader(UnmappedTerms.NAME, UnmappedTerms::new) .addResultReader(LongTerms.NAME, LongTerms::new) - .addResultReader(DoubleTerms.NAME, DoubleTerms::new)); + .addResultReader(DoubleTerms.NAME, DoubleTerms::new) + .setAggregatorRegistrar(TermsAggregationBuilder::registerAggregators)); registerAggregation(new AggregationSpec(RareTermsAggregationBuilder.NAME, RareTermsAggregationBuilder::new, RareTermsAggregationBuilder.PARSER) .addResultReader(StringRareTerms.NAME, StringRareTerms::new) .addResultReader(UnmappedRareTerms.NAME, UnmappedRareTerms::new) - .addResultReader(LongRareTerms.NAME, LongRareTerms::new)); + .addResultReader(LongRareTerms.NAME, LongRareTerms::new) + .setAggregatorRegistrar(RareTermsAggregationBuilder::registerAggregators)); registerAggregation(new AggregationSpec(SignificantTermsAggregationBuilder.NAME, SignificantTermsAggregationBuilder::new, SignificantTermsAggregationBuilder::parse) .addResultReader(SignificantStringTerms.NAME, SignificantStringTerms::new) .addResultReader(SignificantLongTerms.NAME, SignificantLongTerms::new) - .addResultReader(UnmappedSignificantTerms.NAME, UnmappedSignificantTerms::new)); + .addResultReader(UnmappedSignificantTerms.NAME, UnmappedSignificantTerms::new) + .setAggregatorRegistrar(SignificantTermsAggregationBuilder::registerAggregators)); registerAggregation(new AggregationSpec(SignificantTextAggregationBuilder.NAME, SignificantTextAggregationBuilder::new, SignificantTextAggregationBuilder::parse)); registerAggregation(new AggregationSpec(RangeAggregationBuilder.NAME, RangeAggregationBuilder::new, - RangeAggregationBuilder.PARSER).addResultReader(InternalRange::new)); + RangeAggregationBuilder.PARSER) + .addResultReader(InternalRange::new) + .setAggregatorRegistrar(RangeAggregationBuilder::registerAggregators)); registerAggregation(new AggregationSpec(DateRangeAggregationBuilder.NAME, DateRangeAggregationBuilder::new, - DateRangeAggregationBuilder.PARSER).addResultReader(InternalDateRange::new)); + DateRangeAggregationBuilder.PARSER) + .addResultReader(InternalDateRange::new) + .setAggregatorRegistrar(DateRangeAggregationBuilder::registerAggregators)); registerAggregation(new AggregationSpec(IpRangeAggregationBuilder.NAME, IpRangeAggregationBuilder::new, IpRangeAggregationBuilder.PARSER).addResultReader(InternalBinaryRange::new)); registerAggregation(new AggregationSpec(HistogramAggregationBuilder.NAME, HistogramAggregationBuilder::new, - HistogramAggregationBuilder.PARSER).addResultReader(InternalHistogram::new)); + HistogramAggregationBuilder.PARSER) + .addResultReader(InternalHistogram::new) + .setAggregatorRegistrar(HistogramAggregationBuilder::registerAggregators)); registerAggregation(new AggregationSpec(DateHistogramAggregationBuilder.NAME, DateHistogramAggregationBuilder::new, - DateHistogramAggregationBuilder.PARSER).addResultReader(InternalDateHistogram::new)); + DateHistogramAggregationBuilder.PARSER) + .addResultReader(InternalDateHistogram::new) + .setAggregatorRegistrar(DateHistogramAggregationBuilder::registerAggregators)); registerAggregation(new AggregationSpec(AutoDateHistogramAggregationBuilder.NAME, AutoDateHistogramAggregationBuilder::new, AutoDateHistogramAggregationBuilder.PARSER).addResultReader(InternalAutoDateHistogram::new)); registerAggregation(new AggregationSpec(GeoDistanceAggregationBuilder.NAME, GeoDistanceAggregationBuilder::new, GeoDistanceAggregationBuilder::parse).addResultReader(InternalGeoDistance::new)); registerAggregation(new AggregationSpec(GeoHashGridAggregationBuilder.NAME, GeoHashGridAggregationBuilder::new, - GeoHashGridAggregationBuilder.PARSER).addResultReader(InternalGeoHashGrid::new)); + GeoHashGridAggregationBuilder.PARSER) + .addResultReader(InternalGeoHashGrid::new) + .setAggregatorRegistrar(GeoHashGridAggregationBuilder::registerAggregators)); registerAggregation(new AggregationSpec(GeoTileGridAggregationBuilder.NAME, GeoTileGridAggregationBuilder::new, - GeoTileGridAggregationBuilder.PARSER).addResultReader(InternalGeoTileGrid::new)); + GeoTileGridAggregationBuilder.PARSER) + .addResultReader(InternalGeoTileGrid::new) + .setAggregatorRegistrar(GeoTileGridAggregationBuilder::registerAggregators)); registerAggregation(new AggregationSpec(NestedAggregationBuilder.NAME, NestedAggregationBuilder::new, NestedAggregationBuilder::parse).addResultReader(InternalNested::new)); registerAggregation(new AggregationSpec(ReverseNestedAggregationBuilder.NAME, ReverseNestedAggregationBuilder::new, @@ -450,14 +483,21 @@ private void registerAggregations(List plugins) { registerAggregation(new AggregationSpec(TopHitsAggregationBuilder.NAME, TopHitsAggregationBuilder::new, TopHitsAggregationBuilder::parse).addResultReader(InternalTopHits::new)); registerAggregation(new AggregationSpec(GeoBoundsAggregationBuilder.NAME, GeoBoundsAggregationBuilder::new, - GeoBoundsAggregationBuilder.PARSER).addResultReader(InternalGeoBounds::new)); + GeoBoundsAggregationBuilder.PARSER) + .addResultReader(InternalGeoBounds::new) + .setAggregatorRegistrar(GeoBoundsAggregationBuilder::registerAggregators)); registerAggregation(new AggregationSpec(GeoCentroidAggregationBuilder.NAME, GeoCentroidAggregationBuilder::new, - GeoCentroidAggregationBuilder.PARSER).addResultReader(InternalGeoCentroid::new)); + GeoCentroidAggregationBuilder.PARSER) + .addResultReader(InternalGeoCentroid::new) + .setAggregatorRegistrar(GeoCentroidAggregationBuilder::registerAggregators)); registerAggregation(new AggregationSpec(ScriptedMetricAggregationBuilder.NAME, ScriptedMetricAggregationBuilder::new, ScriptedMetricAggregationBuilder.PARSER).addResultReader(InternalScriptedMetric::new)); registerAggregation((new AggregationSpec(CompositeAggregationBuilder.NAME, CompositeAggregationBuilder::new, CompositeAggregationBuilder.PARSER).addResultReader(InternalComposite::new))); registerFromPlugin(plugins, SearchPlugin::getAggregations, this::registerAggregation); + + // after aggs have been registered, see if there are any new VSTypes that need to be linked to core fields + registerFromPlugin(plugins, SearchPlugin::getBareAggregatorRegistrar, this::registerBareAggregatorRegistrar); } private void registerAggregation(AggregationSpec spec) { @@ -474,6 +514,16 @@ private void registerAggregation(AggregationSpec spec) { Writeable.Reader internalReader = t.getValue(); namedWriteables.add(new NamedWriteableRegistry.Entry(InternalAggregation.class, writeableName, internalReader)); } + Consumer register = spec.getAggregatorRegistrar(); + if (register != null) { + register.accept(this.valuesSourceRegistry); + } + } + + private void registerBareAggregatorRegistrar(Consumer registrar) { + if (registrar != null) { + registrar.accept(this.valuesSourceRegistry); + } } private void registerPipelineAggregations(List plugins) { diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/AggregationBuilders.java b/server/src/main/java/org/elasticsearch/search/aggregations/AggregationBuilders.java index 72ac99d94b951..3b658063d17b3 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/AggregationBuilders.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/AggregationBuilders.java @@ -105,7 +105,7 @@ private AggregationBuilders() { * Create a new {@link ValueCount} aggregation with the given name. */ public static ValueCountAggregationBuilder count(String name) { - return new ValueCountAggregationBuilder(name, null); + return new ValueCountAggregationBuilder(name); } /** @@ -217,7 +217,7 @@ public static GlobalAggregationBuilder global(String name) { * Create a new {@link Missing} aggregation with the given name. */ public static MissingAggregationBuilder missing(String name) { - return new MissingAggregationBuilder(name, null); + return new MissingAggregationBuilder(name); } /** @@ -266,7 +266,7 @@ public static GeoTileGridAggregationBuilder geotileGrid(String name) { * Create a new {@link SignificantTerms} aggregation with the given name. */ public static SignificantTermsAggregationBuilder significantTerms(String name) { - return new SignificantTermsAggregationBuilder(name, null); + return new SignificantTermsAggregationBuilder(name); } @@ -313,7 +313,7 @@ public static IpRangeAggregationBuilder ipRange(String name) { * Create a new {@link Terms} aggregation with the given name. */ public static TermsAggregationBuilder terms(String name) { - return new TermsAggregationBuilder(name, null); + return new TermsAggregationBuilder(name); } /** @@ -341,7 +341,7 @@ public static MedianAbsoluteDeviationAggregationBuilder medianAbsoluteDeviation( * Create a new {@link Cardinality} aggregation with the given name. */ public static CardinalityAggregationBuilder cardinality(String name) { - return new CardinalityAggregationBuilder(name, null); + return new CardinalityAggregationBuilder(name); } /** diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/composite/CompositeValuesSourceBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/composite/CompositeValuesSourceBuilder.java index 96f95eb171d2a..cb5d1290cc79f 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/composite/CompositeValuesSourceBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/composite/CompositeValuesSourceBuilder.java @@ -27,6 +27,7 @@ import org.elasticsearch.common.xcontent.XContentBuilder; import org.elasticsearch.index.query.QueryShardContext; import org.elasticsearch.script.Script; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; import org.elasticsearch.search.aggregations.support.ValueType; import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; @@ -294,11 +295,11 @@ public String format() { * @param config The {@link ValuesSourceConfig} for this source. */ protected abstract CompositeValuesSourceConfig innerBuild(QueryShardContext queryShardContext, - ValuesSourceConfig config) throws IOException; + ValuesSourceConfig config) throws IOException; public final CompositeValuesSourceConfig build(QueryShardContext queryShardContext) throws IOException { - ValuesSourceConfig config = ValuesSourceConfig.resolve(queryShardContext, - valueType, field, script, null, timeZone(), format); + ValuesSourceConfig config = ValuesSourceConfig.resolveUnregistered(queryShardContext, + valueType, field, script, null, timeZone(), format, CoreValuesSourceType.BYTES); return innerBuild(queryShardContext, config); } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/composite/CompositeValuesSourceParserHelper.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/composite/CompositeValuesSourceParserHelper.java index 60092f1d353c5..5ca9fcd2d3b80 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/composite/CompositeValuesSourceParserHelper.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/composite/CompositeValuesSourceParserHelper.java @@ -26,9 +26,9 @@ import org.elasticsearch.common.io.stream.StreamOutput; import org.elasticsearch.common.xcontent.AbstractObjectParser; import org.elasticsearch.common.xcontent.ObjectParser; +import org.elasticsearch.common.xcontent.ToXContent.Params; import org.elasticsearch.common.xcontent.XContentBuilder; import org.elasticsearch.common.xcontent.XContentParser; -import org.elasticsearch.common.xcontent.ToXContent.Params; import org.elasticsearch.script.Script; import org.elasticsearch.search.aggregations.support.ValueType; @@ -37,19 +37,20 @@ import static org.elasticsearch.common.xcontent.XContentParserUtils.ensureExpectedToken; public class CompositeValuesSourceParserHelper { + static , T> void declareValuesSourceFields(AbstractObjectParser objectParser, - ValueType targetValueType) { + ValueType expectedValueType) { objectParser.declareField(VB::field, XContentParser::text, new ParseField("field"), ObjectParser.ValueType.STRING); objectParser.declareBoolean(VB::missingBucket, new ParseField("missing_bucket")); objectParser.declareField(VB::valueType, p -> { - ValueType valueType = ValueType.resolveForScript(p.text()); - if (targetValueType != null && valueType.isNotA(targetValueType)) { + ValueType valueType = ValueType.lenientParse(p.text()); + if (expectedValueType != null && valueType.isNotA(expectedValueType)) { throw new ParsingException(p.getTokenLocation(), "Aggregation [" + objectParser.getName() + "] was configured with an incompatible value type [" - + valueType + "]. It can only work on value of type [" - + targetValueType + "]"); + + valueType + "]. It can only work on value of type [" + + expectedValueType + "]"); } return valueType; }, new ParseField("value_type"), ObjectParser.ValueType.STRING); diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/composite/DateHistogramValuesSourceBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/composite/DateHistogramValuesSourceBuilder.java index dfac9ee524c0b..805194b690ecb 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/composite/DateHistogramValuesSourceBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/composite/DateHistogramValuesSourceBuilder.java @@ -247,9 +247,9 @@ public DateHistogramValuesSourceBuilder offset(long offset) { } @Override - protected CompositeValuesSourceConfig innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config) throws IOException { + protected CompositeValuesSourceConfig innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config) throws IOException { Rounding rounding = dateHistogramInterval.createRounding(timeZone(), offset); - ValuesSource orig = config.toValuesSource(queryShardContext); + ValuesSource orig = config.toValuesSource(); if (orig == null) { orig = ValuesSource.Numeric.EMPTY; } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/composite/GeoTileGridValuesSourceBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/composite/GeoTileGridValuesSourceBuilder.java index 165a50db60d18..c74d62e623ebe 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/composite/GeoTileGridValuesSourceBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/composite/GeoTileGridValuesSourceBuilder.java @@ -128,8 +128,8 @@ public boolean equals(Object obj) { } @Override - protected CompositeValuesSourceConfig innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config) throws IOException { - ValuesSource orig = config.toValuesSource(queryShardContext); + protected CompositeValuesSourceConfig innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config) throws IOException { + ValuesSource orig = config.toValuesSource(); if (orig == null) { orig = ValuesSource.GeoPoint.EMPTY; } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/composite/HistogramValuesSourceBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/composite/HistogramValuesSourceBuilder.java index aa15d5a6947d9..4ef15908ddd7a 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/composite/HistogramValuesSourceBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/composite/HistogramValuesSourceBuilder.java @@ -110,8 +110,8 @@ public HistogramValuesSourceBuilder interval(double interval) { } @Override - protected CompositeValuesSourceConfig innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config) throws IOException { - ValuesSource orig = config.toValuesSource(queryShardContext); + protected CompositeValuesSourceConfig innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config) throws IOException { + ValuesSource orig = config.toValuesSource(); if (orig == null) { orig = ValuesSource.Numeric.EMPTY; } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/composite/TermsValuesSourceBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/composite/TermsValuesSourceBuilder.java index cd88a56614ec5..2954c155e0d7c 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/composite/TermsValuesSourceBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/composite/TermsValuesSourceBuilder.java @@ -70,8 +70,8 @@ public String type() { } @Override - protected CompositeValuesSourceConfig innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config) throws IOException { - ValuesSource vs = config.toValuesSource(queryShardContext); + protected CompositeValuesSourceConfig innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config) throws IOException { + ValuesSource vs = config.toValuesSource(); if (vs == null) { // The field is unmapped so we use a value source that can parse any type of values. // This is needed because the after values are parsed even when there are no values to process. diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoGridAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoGridAggregationBuilder.java index df8356e472f6f..7d872316caba6 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoGridAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoGridAggregationBuilder.java @@ -34,19 +34,18 @@ import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.bucket.BucketUtils; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; -import org.elasticsearch.search.aggregations.support.ValueType; -import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; -import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.util.Map; import java.util.Objects; import java.util.function.Function; -public abstract class GeoGridAggregationBuilder extends ValuesSourceAggregationBuilder { +public abstract class GeoGridAggregationBuilder extends ValuesSourceAggregationBuilder + implements MultiBucketAggregationBuilder { /* recognized field names in JSON */ static final ParseField FIELD_PRECISION = new ParseField("precision"); static final ParseField FIELD_SIZE = new ParseField("size"); @@ -66,7 +65,7 @@ protected interface PrecisionParser { public static ObjectParser createParser( String name, PrecisionParser precisionParser, Function ctor) { ObjectParser parser = ObjectParser.fromBuilder(name, ctor); - ValuesSourceParserHelper.declareGeoFields(parser, false, false); + ValuesSourceAggregationBuilder.declareFields(parser, false, false, false); parser.declareField((p, builder, context) -> builder.precision(precisionParser.parse(p)), FIELD_PRECISION, org.elasticsearch.common.xcontent.ObjectParser.ValueType.INT); parser.declareInt(GeoGridAggregationBuilder::size, FIELD_SIZE); @@ -79,7 +78,7 @@ public static ObjectParser crea } public GeoGridAggregationBuilder(String name) { - super(name, CoreValuesSourceType.GEOPOINT, ValueType.GEOPOINT); + super(name); } protected GeoGridAggregationBuilder(GeoGridAggregationBuilder clone, Builder factoriesBuilder, Map metadata) { @@ -94,7 +93,7 @@ protected GeoGridAggregationBuilder(GeoGridAggregationBuilder clone, Builder fac * Read from a stream. */ public GeoGridAggregationBuilder(StreamInput in) throws IOException { - super(in, CoreValuesSourceType.GEOPOINT, ValueType.GEOPOINT); + super(in); precision = in.readVInt(); requiredSize = in.readVInt(); shardSize = in.readVInt(); @@ -103,6 +102,11 @@ public GeoGridAggregationBuilder(StreamInput in) throws IOException { } } + @Override + protected ValuesSourceType defaultValueSourceType() { + return CoreValuesSourceType.GEOPOINT; + } + @Override protected void innerWriteTo(StreamOutput out) throws IOException { out.writeVInt(precision); @@ -123,8 +127,8 @@ protected void innerWriteTo(StreamOutput out) throws IOException { /** * Creates a new instance of the {@link ValuesSourceAggregatorFactory}-derived class specific to the geo aggregation. */ - protected abstract ValuesSourceAggregatorFactory createFactory( - String name, ValuesSourceConfig config, int precision, int requiredSize, int shardSize, + protected abstract ValuesSourceAggregatorFactory createFactory( + String name, ValuesSourceConfig config, int precision, int requiredSize, int shardSize, GeoBoundingBox geoBoundingBox, QueryShardContext queryShardContext, AggregatorFactory parent, Builder subFactoriesBuilder, Map metadata ) throws IOException; @@ -175,9 +179,10 @@ public final BucketCardinality bucketCardinality() { } @Override - protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, - ValuesSourceConfig config, - AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException { + protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, + ValuesSourceConfig config, + AggregatorFactory parent, Builder subFactoriesBuilder) + throws IOException { int shardSize = this.shardSize; int requiredSize = this.requiredSize; diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoHashGridAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoHashGridAggregationBuilder.java index 8fcd89a8bdcce..8b7415c67b5b8 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoHashGridAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoHashGridAggregationBuilder.java @@ -27,9 +27,9 @@ import org.elasticsearch.search.aggregations.AggregationBuilder; import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; -import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import java.io.IOException; import java.util.Map; @@ -53,6 +53,10 @@ public GeoHashGridAggregationBuilder(StreamInput in) throws IOException { super(in); } + public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + GeoHashGridAggregatorFactory.registerAggregators(valuesSourceRegistry); + } + @Override public GeoGridAggregationBuilder precision(int precision) { this.precision = GeoUtils.checkPrecisionRange(precision); @@ -60,8 +64,8 @@ public GeoGridAggregationBuilder precision(int precision) { } @Override - protected ValuesSourceAggregatorFactory createFactory( - String name, ValuesSourceConfig config, int precision, int requiredSize, int shardSize, + protected ValuesSourceAggregatorFactory createFactory( + String name, ValuesSourceConfig config, int precision, int requiredSize, int shardSize, GeoBoundingBox geoBoundingBox, QueryShardContext queryShardContext, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, Map metadata) throws IOException { diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoHashGridAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoHashGridAggregatorFactory.java index 3433ea6408b3c..8623a750b1c5a 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoHashGridAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoHashGridAggregatorFactory.java @@ -22,15 +22,20 @@ import org.elasticsearch.common.geo.GeoBoundingBox; import org.elasticsearch.geometry.utils.Geohash; import org.elasticsearch.index.query.QueryShardContext; +import org.elasticsearch.search.aggregations.AggregationExecutionException; import org.elasticsearch.search.aggregations.Aggregator; import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.InternalAggregation; import org.elasticsearch.search.aggregations.NonCollectingAggregator; +import org.elasticsearch.search.aggregations.metrics.GeoGridAggregatorSupplier; import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; @@ -39,14 +44,14 @@ import static java.util.Collections.emptyList; -public class GeoHashGridAggregatorFactory extends ValuesSourceAggregatorFactory { +public class GeoHashGridAggregatorFactory extends ValuesSourceAggregatorFactory { private final int precision; private final int requiredSize; private final int shardSize; private final GeoBoundingBox geoBoundingBox; - GeoHashGridAggregatorFactory(String name, ValuesSourceConfig config, int precision, int requiredSize, + GeoHashGridAggregatorFactory(String name, ValuesSourceConfig config, int precision, int requiredSize, int shardSize, GeoBoundingBox geoBoundingBox, QueryShardContext queryShardContext, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, Map metadata) throws IOException { @@ -72,17 +77,33 @@ public InternalAggregation buildEmptyAggregation() { } @Override - protected Aggregator doCreateInternal(final ValuesSource.GeoPoint valuesSource, + protected Aggregator doCreateInternal(final ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, List pipelineAggregators, - Map metadata) throws IOException { + Map metaData) throws IOException { + AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry() + .getAggregator(config.valueSourceType(), GeoHashGridAggregationBuilder.NAME); + if (aggregatorSupplier instanceof GeoGridAggregatorSupplier == false) { + throw new AggregationExecutionException("Registry miss-match - expected " + + GeoGridAggregatorSupplier.class.getName() + ", found [" + aggregatorSupplier.getClass().toString() + "]"); + } if (collectsFromSingleBucket == false) { return asMultiBucketAggregator(this, searchContext, parent); } - CellIdSource cellIdSource = new CellIdSource(valuesSource, precision, geoBoundingBox, Geohash::longEncode); - return new GeoHashGridAggregator(name, factories, cellIdSource, requiredSize, shardSize, - searchContext, parent, pipelineAggregators, metadata); + return ((GeoGridAggregatorSupplier) aggregatorSupplier).build(name, factories, valuesSource, precision, geoBoundingBox, + requiredSize, shardSize, searchContext, parent, pipelineAggregators, metaData); + } + + static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + valuesSourceRegistry.register(GeoHashGridAggregationBuilder.NAME, CoreValuesSourceType.GEOPOINT, + (GeoGridAggregatorSupplier) (name, factories, valuesSource, precision, geoBoundingBox, requiredSize, shardSize, + aggregationContext, parent, pipelineAggregators, metaData) -> { + CellIdSource cellIdSource = new CellIdSource((ValuesSource.GeoPoint) valuesSource, precision, geoBoundingBox, + Geohash::longEncode); + return new GeoHashGridAggregator(name, factories, cellIdSource, requiredSize, shardSize, aggregationContext, + parent, pipelineAggregators, metaData); + }); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoTileGridAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoTileGridAggregationBuilder.java index 63f9942ab0429..7026ab643cbfc 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoTileGridAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoTileGridAggregationBuilder.java @@ -29,9 +29,9 @@ import org.elasticsearch.search.aggregations.AggregationBuilder; import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; -import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; public class GeoTileGridAggregationBuilder extends GeoGridAggregationBuilder { public static final String NAME = "geotile_grid"; @@ -52,6 +52,10 @@ public GeoTileGridAggregationBuilder(StreamInput in) throws IOException { super(in); } + public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + GeoTileGridAggregatorFactory.registerAggregators(valuesSourceRegistry); + } + @Override public GeoGridAggregationBuilder precision(int precision) { this.precision = GeoTileUtils.checkPrecisionRange(precision); @@ -59,10 +63,10 @@ public GeoGridAggregationBuilder precision(int precision) { } @Override - protected ValuesSourceAggregatorFactory createFactory( - String name, ValuesSourceConfig config, int precision, int requiredSize, int shardSize, + protected ValuesSourceAggregatorFactory createFactory( + String name, ValuesSourceConfig config, int precision, int requiredSize, int shardSize, GeoBoundingBox geoBoundingBox, QueryShardContext queryShardContext, AggregatorFactory parent, - AggregatorFactories.Builder subFactoriesBuilder, Map metadata ) throws IOException { + AggregatorFactories.Builder subFactoriesBuilder, Map metaData) throws IOException { return new GeoTileGridAggregatorFactory(name, config, precision, requiredSize, shardSize, geoBoundingBox, queryShardContext, parent, subFactoriesBuilder, metadata); } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoTileGridAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoTileGridAggregatorFactory.java index f7123b7bbb85d..190ec0b0b3265 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoTileGridAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoTileGridAggregatorFactory.java @@ -21,15 +21,20 @@ import org.elasticsearch.common.geo.GeoBoundingBox; import org.elasticsearch.index.query.QueryShardContext; +import org.elasticsearch.search.aggregations.AggregationExecutionException; import org.elasticsearch.search.aggregations.Aggregator; import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.InternalAggregation; import org.elasticsearch.search.aggregations.NonCollectingAggregator; +import org.elasticsearch.search.aggregations.metrics.GeoGridAggregatorSupplier; import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; @@ -37,14 +42,14 @@ import java.util.List; import java.util.Map; -public class GeoTileGridAggregatorFactory extends ValuesSourceAggregatorFactory { +public class GeoTileGridAggregatorFactory extends ValuesSourceAggregatorFactory { private final int precision; private final int requiredSize; private final int shardSize; private final GeoBoundingBox geoBoundingBox; - GeoTileGridAggregatorFactory(String name, ValuesSourceConfig config, int precision, int requiredSize, + GeoTileGridAggregatorFactory(String name, ValuesSourceConfig config, int precision, int requiredSize, int shardSize, GeoBoundingBox geoBoundingBox, QueryShardContext queryShardContext, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, Map metadata) throws IOException { @@ -70,17 +75,33 @@ public InternalAggregation buildEmptyAggregation() { } @Override - protected Aggregator doCreateInternal(final ValuesSource.GeoPoint valuesSource, + protected Aggregator doCreateInternal(final ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, List pipelineAggregators, - Map metadata) throws IOException { + Map metaData) throws IOException { + AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry() + .getAggregator(config.valueSourceType(), GeoTileGridAggregationBuilder.NAME); + if (aggregatorSupplier instanceof GeoGridAggregatorSupplier == false) { + throw new AggregationExecutionException("Registry miss-match - expected " + + GeoGridAggregatorSupplier.class.getName() + ", found [" + aggregatorSupplier.getClass().toString() + "]"); + } if (collectsFromSingleBucket == false) { return asMultiBucketAggregator(this, searchContext, parent); } - CellIdSource cellIdSource = new CellIdSource(valuesSource, precision, geoBoundingBox, GeoTileUtils::longEncode); - return new GeoTileGridAggregator(name, factories, cellIdSource, requiredSize, shardSize, - searchContext, parent, pipelineAggregators, metadata); + return ((GeoGridAggregatorSupplier) aggregatorSupplier).build(name, factories, valuesSource, precision, geoBoundingBox, + requiredSize, shardSize, searchContext, parent, pipelineAggregators, metaData); + } + + static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + valuesSourceRegistry.register(GeoTileGridAggregationBuilder.NAME, CoreValuesSourceType.GEOPOINT, + (GeoGridAggregatorSupplier) (name, factories, valuesSource, precision, geoBoundingBox, requiredSize, shardSize, + aggregationContext, parent, pipelineAggregators, metaData) -> { + CellIdSource cellIdSource = new CellIdSource((ValuesSource.GeoPoint) valuesSource, precision, geoBoundingBox, + GeoTileUtils::longEncode); + return new GeoTileGridAggregator(name, factories, cellIdSource, requiredSize, shardSize, aggregationContext, + parent, pipelineAggregators, metaData); + }); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/AutoDateHistogramAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/AutoDateHistogramAggregationBuilder.java index a49840f3c80e7..be4f4c84d4c34 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/AutoDateHistogramAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/AutoDateHistogramAggregationBuilder.java @@ -34,13 +34,10 @@ import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.MultiBucketConsumerService; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; -import org.elasticsearch.search.aggregations.support.ValueType; -import org.elasticsearch.search.aggregations.support.ValuesSource; -import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; -import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.time.ZoneId; @@ -50,7 +47,7 @@ import java.util.Objects; public class AutoDateHistogramAggregationBuilder - extends ValuesSourceAggregationBuilder { + extends ValuesSourceAggregationBuilder { public static final String NAME = "auto_date_histogram"; @@ -60,7 +57,7 @@ public class AutoDateHistogramAggregationBuilder public static final ObjectParser PARSER = ObjectParser.fromBuilder(NAME, AutoDateHistogramAggregationBuilder::new); static { - ValuesSourceParserHelper.declareNumericFields(PARSER, true, true, true); + ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, true); PARSER.declareInt(AutoDateHistogramAggregationBuilder::setNumBuckets, NUM_BUCKETS_FIELD); PARSER.declareStringOrNull(AutoDateHistogramAggregationBuilder::setMinimumIntervalExpression, MINIMUM_INTERVAL_FIELD); } @@ -115,12 +112,12 @@ static RoundingInfo[] buildRoundings(ZoneId timeZone, String minimumInterval) { /** Create a new builder with the given name. */ public AutoDateHistogramAggregationBuilder(String name) { - super(name, CoreValuesSourceType.NUMERIC, ValueType.DATE); + super(name); } /** Read from a stream, for internal use only. */ public AutoDateHistogramAggregationBuilder(StreamInput in) throws IOException { - super(in, CoreValuesSourceType.NUMERIC, ValueType.DATE); + super(in); numBuckets = in.readVInt(); if (in.getVersion().onOrAfter(Version.V_7_3_0)) { minimumIntervalExpression = in.readOptionalString(); @@ -143,8 +140,22 @@ protected AutoDateHistogramAggregationBuilder(AutoDateHistogramAggregationBuilde } @Override - protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metadata) { - return new AutoDateHistogramAggregationBuilder(this, factoriesBuilder, metadata); + protected ValuesSourceType defaultValueSourceType() { + // TODO: This should probably be DATE, but we're not failing tests with BYTES, so needs more tests? + return CoreValuesSourceType.BYTES; + } + + @Override + protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metaData) { + return new AutoDateHistogramAggregationBuilder(this, factoriesBuilder, metaData); + } + + @Override + protected void innerWriteTo(StreamOutput out) throws IOException { + out.writeVInt(numBuckets); + if (out.getVersion().onOrAfter(Version.V_8_0_0)) { + out.writeOptionalString(minimumIntervalExpression); + } } @Override @@ -183,8 +194,8 @@ public BucketCardinality bucketCardinality() { } @Override - protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config, - AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException { + protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config, + AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException { RoundingInfo[] roundings = buildRoundings(timeZone(), getMinimumIntervalExpression()); int maxRoundingInterval = Arrays.stream(roundings,0, roundings.length-1) .map(rounding -> rounding.innerIntervals) diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/AutoDateHistogramAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/AutoDateHistogramAggregatorFactory.java index 7b09bd99bf25d..1046a2d6f8af0 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/AutoDateHistogramAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/AutoDateHistogramAggregatorFactory.java @@ -20,6 +20,7 @@ package org.elasticsearch.search.aggregations.bucket.histogram; import org.elasticsearch.index.query.QueryShardContext; +import org.elasticsearch.search.aggregations.AggregationExecutionException; import org.elasticsearch.search.aggregations.Aggregator; import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; @@ -36,13 +37,13 @@ import java.util.Map; public final class AutoDateHistogramAggregatorFactory - extends ValuesSourceAggregatorFactory { + extends ValuesSourceAggregatorFactory { private final int numBuckets; private RoundingInfo[] roundingInfos; public AutoDateHistogramAggregatorFactory(String name, - ValuesSourceConfig config, + ValuesSourceConfig config, int numBuckets, RoundingInfo[] roundingInfos, QueryShardContext queryShardContext, @@ -55,16 +56,20 @@ public AutoDateHistogramAggregatorFactory(String name, } @Override - protected Aggregator doCreateInternal(Numeric valuesSource, + protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, List pipelineAggregators, - Map metadata) throws IOException { + Map metaData) throws IOException { + if (valuesSource instanceof Numeric == false) { + throw new AggregationExecutionException("ValuesSource type " + valuesSource.toString() + "is not supported for aggregation " + + this.name()); + } if (collectsFromSingleBucket == false) { return asMultiBucketAggregator(this, searchContext, parent); } - return createAggregator(valuesSource, searchContext, parent, pipelineAggregators, metadata); + return createAggregator((Numeric) valuesSource, searchContext, parent, pipelineAggregators, metaData); } private Aggregator createAggregator(ValuesSource.Numeric valuesSource, diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/DateHistogramAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/DateHistogramAggregationBuilder.java index e2e06fcc97e71..38c6be10ee782 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/DateHistogramAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/DateHistogramAggregationBuilder.java @@ -36,7 +36,6 @@ import org.elasticsearch.index.mapper.MappedFieldType; import org.elasticsearch.index.mapper.MappedFieldType.Relation; import org.elasticsearch.index.query.QueryShardContext; -import org.elasticsearch.script.Script; import org.elasticsearch.search.aggregations.AggregationBuilder; import org.elasticsearch.search.aggregations.AggregatorFactories.Builder; import org.elasticsearch.search.aggregations.AggregatorFactory; @@ -44,12 +43,10 @@ import org.elasticsearch.search.aggregations.InternalOrder; import org.elasticsearch.search.aggregations.InternalOrder.CompoundOrder; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; -import org.elasticsearch.search.aggregations.support.ValueType; -import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; -import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; @@ -67,8 +64,9 @@ /** * A builder for histograms on date fields. */ -public class DateHistogramAggregationBuilder extends ValuesSourceAggregationBuilder - implements DateIntervalConsumer { +public class DateHistogramAggregationBuilder extends ValuesSourceAggregationBuilder + implements MultiBucketAggregationBuilder, DateIntervalConsumer { + public static final String NAME = "date_histogram"; public static final Map DATE_FIELD_UNITS; @@ -97,7 +95,7 @@ public class DateHistogramAggregationBuilder extends ValuesSourceAggregationBuil public static final ObjectParser PARSER = ObjectParser.fromBuilder(NAME, DateHistogramAggregationBuilder::new); static { - ValuesSourceParserHelper.declareAnyFields(PARSER, true, true, true); + ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, true); DateIntervalWrapper.declareIntervalFields(PARSER); PARSER.declareField(DateHistogramAggregationBuilder::offset, p -> { @@ -119,6 +117,10 @@ public class DateHistogramAggregationBuilder extends ValuesSourceAggregationBuil Histogram.ORDER_FIELD); } + public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + DateHistogramAggregatorFactory.registerAggregators(valuesSourceRegistry); + } + private DateIntervalWrapper dateHistogramInterval = new DateIntervalWrapper(); private long offset = 0; private ExtendedBounds extendedBounds; @@ -128,7 +130,7 @@ public class DateHistogramAggregationBuilder extends ValuesSourceAggregationBuil /** Create a new builder with the given name. */ public DateHistogramAggregationBuilder(String name) { - super(name, CoreValuesSourceType.ANY, ValueType.DATE); + super(name); } protected DateHistogramAggregationBuilder(DateHistogramAggregationBuilder clone, @@ -149,8 +151,8 @@ protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map innerBuild(QueryShardContext queryShardContext, - ValuesSourceConfig config, - AggregatorFactory parent, - Builder subFactoriesBuilder) throws IOException { + protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, + ValuesSourceConfig config, + AggregatorFactory parent, + Builder subFactoriesBuilder) throws IOException { final ZoneId tz = timeZone(); final Rounding rounding = dateHistogramInterval.createRounding(tz, offset); final ZoneId rewrittenTimeZone = rewriteTimeZone(queryShardContext); diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/DateHistogramAggregationSupplier.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/DateHistogramAggregationSupplier.java new file mode 100644 index 0000000000000..fde98b335d3cd --- /dev/null +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/DateHistogramAggregationSupplier.java @@ -0,0 +1,53 @@ +/* + * Licensed to Elasticsearch under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.elasticsearch.search.aggregations.bucket.histogram; + +import org.elasticsearch.common.Nullable; +import org.elasticsearch.common.Rounding; +import org.elasticsearch.search.DocValueFormat; +import org.elasticsearch.search.aggregations.Aggregator; +import org.elasticsearch.search.aggregations.AggregatorFactories; +import org.elasticsearch.search.aggregations.BucketOrder; +import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.ValuesSource; +import org.elasticsearch.search.internal.SearchContext; + +import java.io.IOException; +import java.util.List; +import java.util.Map; + +@FunctionalInterface +public interface DateHistogramAggregationSupplier extends AggregatorSupplier { + Aggregator build(String name, + AggregatorFactories factories, + Rounding rounding, + Rounding shardRounding, + BucketOrder order, + boolean keyed, + long minDocCount, + @Nullable ExtendedBounds extendedBounds, + @Nullable ValuesSource valuesSource, + DocValueFormat formatter, + SearchContext aggregationContext, + Aggregator parent, + List pipelineAggregators, + Map metaData) throws IOException; +} diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/DateHistogramAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/DateHistogramAggregatorFactory.java index 414b151f4bfa6..870313e7cf7b9 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/DateHistogramAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/DateHistogramAggregatorFactory.java @@ -19,25 +19,77 @@ package org.elasticsearch.search.aggregations.bucket.histogram; +import org.elasticsearch.common.Nullable; import org.elasticsearch.common.Rounding; import org.elasticsearch.index.mapper.RangeType; import org.elasticsearch.index.query.QueryShardContext; +import org.elasticsearch.search.DocValueFormat; +import org.elasticsearch.search.aggregations.AggregationExecutionException; import org.elasticsearch.search.aggregations.Aggregator; import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.BucketOrder; import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; import java.util.List; import java.util.Map; -public final class DateHistogramAggregatorFactory - extends ValuesSourceAggregatorFactory { +public final class DateHistogramAggregatorFactory extends ValuesSourceAggregatorFactory { + + public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + valuesSourceRegistry.register(DateHistogramAggregationBuilder.NAME, + List.of(CoreValuesSourceType.DATE, CoreValuesSourceType.NUMERIC, CoreValuesSourceType.BOOLEAN), + (DateHistogramAggregationSupplier) (String name, + AggregatorFactories factories, + Rounding rounding, + Rounding shardRounding, + BucketOrder order, + boolean keyed, + long minDocCount, + @Nullable ExtendedBounds extendedBounds, + @Nullable ValuesSource valuesSource, + DocValueFormat formatter, + SearchContext aggregationContext, + Aggregator parent, + List pipelineAggregators, + Map metaData) -> new DateHistogramAggregator(name, + factories, rounding, shardRounding, order, keyed, minDocCount, extendedBounds, (ValuesSource.Numeric) valuesSource, + formatter, aggregationContext, parent, pipelineAggregators, metaData)); + + valuesSourceRegistry.register(DateHistogramAggregationBuilder.NAME, + CoreValuesSourceType.RANGE, + (DateHistogramAggregationSupplier) (String name, + AggregatorFactories factories, + Rounding rounding, + Rounding shardRounding, + BucketOrder order, + boolean keyed, + long minDocCount, + @Nullable ExtendedBounds extendedBounds, + @Nullable ValuesSource valuesSource, + DocValueFormat formatter, + SearchContext aggregationContext, + Aggregator parent, + List pipelineAggregators, + Map metaData) -> { + + ValuesSource.Range rangeValueSource = (ValuesSource.Range) valuesSource; + if (rangeValueSource.rangeType() != RangeType.DATE) { + throw new IllegalArgumentException("Expected date range type but found range type [" + rangeValueSource.rangeType().name + + "]"); + } + return new DateRangeHistogramAggregator(name, + factories, rounding, shardRounding, order, keyed, minDocCount, extendedBounds, rangeValueSource, formatter, + aggregationContext, parent, pipelineAggregators, metaData); }); + } private final BucketOrder order; private final boolean keyed; @@ -46,7 +98,7 @@ public final class DateHistogramAggregatorFactory private final Rounding rounding; private final Rounding shardRounding; - public DateHistogramAggregatorFactory(String name, ValuesSourceConfig config, + public DateHistogramAggregatorFactory(String name, ValuesSourceConfig config, BucketOrder order, boolean keyed, long minDocCount, Rounding rounding, Rounding shardRounding, ExtendedBounds extendedBounds, QueryShardContext queryShardContext, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, @@ -64,15 +116,6 @@ public long minDocCount() { return minDocCount; } - @Override - protected ValuesSource resolveMissingAny(Object missing) { - if (missing instanceof Number) { - return ValuesSource.Numeric.EMPTY; - } - throw new IllegalArgumentException("Only numeric missing values are supported for date histogram aggregation, found [" - + missing + "]"); - } - @Override protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, @@ -83,43 +126,22 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, if (collectsFromSingleBucket == false) { return asMultiBucketAggregator(this, searchContext, parent); } - if (valuesSource instanceof ValuesSource.Numeric) { - return createAggregator((ValuesSource.Numeric) valuesSource, searchContext, parent, pipelineAggregators, metadata); - } else if (valuesSource instanceof ValuesSource.Range) { - ValuesSource.Range rangeValueSource = (ValuesSource.Range) valuesSource; - if (rangeValueSource.rangeType() != RangeType.DATE) { - throw new IllegalArgumentException("Expected date range type but found range type [" + rangeValueSource.rangeType().name - + "]"); - } - return createRangeAggregator((ValuesSource.Range) valuesSource, searchContext, parent, pipelineAggregators, metadata); - } - else { - throw new IllegalArgumentException("Expected one of [Date, Range] values source, found [" - + valuesSource.toString() + "]"); + AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), + DateHistogramAggregationBuilder.NAME); + if (aggregatorSupplier instanceof DateHistogramAggregationSupplier == false) { + throw new AggregationExecutionException("Registry miss-match - expected DateHistogramAggregationSupplier, found [" + + aggregatorSupplier.getClass().toString() + "]"); } - } - - private Aggregator createAggregator(ValuesSource.Numeric valuesSource, SearchContext searchContext, - Aggregator parent, List pipelineAggregators, - Map metadata) throws IOException { - return new DateHistogramAggregator(name, factories, rounding, shardRounding, order, keyed, minDocCount, extendedBounds, - valuesSource, config.format(), searchContext, parent, pipelineAggregators, metadata); - } - - private Aggregator createRangeAggregator(ValuesSource.Range valuesSource, - SearchContext searchContext, - Aggregator parent, - List pipelineAggregators, - Map metadata) throws IOException { - return new DateRangeHistogramAggregator(name, factories, rounding, shardRounding, order, keyed, minDocCount, extendedBounds, - valuesSource, config.format(), searchContext, parent, pipelineAggregators, metadata); + return ((DateHistogramAggregationSupplier) aggregatorSupplier).build(name, factories, rounding, shardRounding, order, keyed, + minDocCount, extendedBounds, valuesSource, config.format(), searchContext, parent, pipelineAggregators, metaData); } @Override protected Aggregator createUnmapped(SearchContext searchContext, Aggregator parent, List pipelineAggregators, - Map metadata) throws IOException { - return createAggregator(null, searchContext, parent, pipelineAggregators, metadata); + Map metaData) throws IOException { + return new DateHistogramAggregator(name, factories, rounding, shardRounding, order, keyed, minDocCount, extendedBounds, + null, config.format(), searchContext, parent, pipelineAggregators, metaData); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/HistogramAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/HistogramAggregationBuilder.java index ecef15e453247..cfb34af61cb4f 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/HistogramAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/HistogramAggregationBuilder.java @@ -25,7 +25,6 @@ import org.elasticsearch.common.xcontent.ObjectParser; import org.elasticsearch.common.xcontent.XContentBuilder; import org.elasticsearch.index.query.QueryShardContext; -import org.elasticsearch.script.Script; import org.elasticsearch.search.aggregations.AggregationBuilder; import org.elasticsearch.search.aggregations.AggregatorFactories.Builder; import org.elasticsearch.search.aggregations.AggregatorFactory; @@ -33,12 +32,10 @@ import org.elasticsearch.search.aggregations.InternalOrder; import org.elasticsearch.search.aggregations.InternalOrder.CompoundOrder; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; -import org.elasticsearch.search.aggregations.support.ValueType; -import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; -import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; @@ -50,7 +47,8 @@ * A builder for histograms on numeric fields. This builder can operate on either base numeric fields, or numeric range fields. IP range * fields are unsupported, and will throw at the factory layer. */ -public class HistogramAggregationBuilder extends ValuesSourceAggregationBuilder { +public class HistogramAggregationBuilder extends ValuesSourceAggregationBuilder + implements MultiBucketAggregationBuilder { public static final String NAME = "histogram"; private static final ObjectParser EXTENDED_BOUNDS_PARSER = new ObjectParser<>( @@ -64,7 +62,7 @@ public class HistogramAggregationBuilder extends ValuesSourceAggregationBuilder< public static final ObjectParser PARSER = ObjectParser.fromBuilder(NAME, HistogramAggregationBuilder::new); static { - ValuesSourceParserHelper.declareAnyFields(PARSER, true, true); + ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, false); PARSER.declareDouble(HistogramAggregationBuilder::interval, Histogram.INTERVAL_FIELD); @@ -82,6 +80,10 @@ public class HistogramAggregationBuilder extends ValuesSourceAggregationBuilder< Histogram.ORDER_FIELD); } + public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + HistogramAggregatorFactory.registerAggregators(valuesSourceRegistry); + } + private double interval; private double offset = 0; private double minBound = Double.POSITIVE_INFINITY; @@ -91,14 +93,13 @@ public class HistogramAggregationBuilder extends ValuesSourceAggregationBuilder< private long minDocCount = 0; @Override - protected ValuesSourceType resolveScriptAny(Script script) { - // TODO: No idea how we'd support Range scripts here. + protected ValuesSourceType defaultValueSourceType() { return CoreValuesSourceType.NUMERIC; } /** Create a new builder with the given name. */ public HistogramAggregationBuilder(String name) { - super(name, CoreValuesSourceType.ANY, ValueType.NUMERIC); + super(name); } protected HistogramAggregationBuilder(HistogramAggregationBuilder clone, Builder factoriesBuilder, Map metadata) { @@ -119,8 +120,8 @@ protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map innerBuild(QueryShardContext queryShardContext, - ValuesSourceConfig config, - AggregatorFactory parent, - Builder subFactoriesBuilder) throws IOException { + protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, + ValuesSourceConfig config, + AggregatorFactory parent, + Builder subFactoriesBuilder) throws IOException { return new HistogramAggregatorFactory(name, config, interval, offset, order, keyed, minDocCount, minBound, maxBound, queryShardContext, parent, subFactoriesBuilder, metadata); } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/HistogramAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/HistogramAggregatorFactory.java index 4d4b6fa68bce8..cfb1fb3c035e6 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/HistogramAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/HistogramAggregatorFactory.java @@ -20,14 +20,20 @@ package org.elasticsearch.search.aggregations.bucket.histogram; import org.elasticsearch.index.query.QueryShardContext; +import org.elasticsearch.search.DocValueFormat; +import org.elasticsearch.search.aggregations.AggregationExecutionException; import org.elasticsearch.search.aggregations.Aggregator; import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.BucketOrder; import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; +import org.elasticsearch.search.aggregations.support.HistogramAggregatorSupplier; import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; @@ -38,7 +44,7 @@ * Constructs the per-shard aggregator instance for histogram aggregation. Selects the numeric or range field implementation based on the * field type. */ -public final class HistogramAggregatorFactory extends ValuesSourceAggregatorFactory { +public final class HistogramAggregatorFactory extends ValuesSourceAggregatorFactory { private final double interval, offset; private final BucketOrder order; @@ -46,17 +52,45 @@ public final class HistogramAggregatorFactory extends ValuesSourceAggregatorFact private final long minDocCount; private final double minBound, maxBound; - @Override - protected ValuesSource resolveMissingAny(Object missing) { - if (missing instanceof Number) { - return ValuesSource.Numeric.EMPTY; - } - throw new IllegalArgumentException("Only numeric missing values are supported for histogram aggregation, found [" - + missing + "]"); + // TODO: Registration should happen on the actual aggregator classes, but I don't want to set up the whole dynamic loading thing yet + static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + valuesSourceRegistry.register(HistogramAggregationBuilder.NAME, CoreValuesSourceType.RANGE, + new HistogramAggregatorSupplier() { + @Override + public Aggregator build(String name, AggregatorFactories factories, double interval, double offset, + BucketOrder order, boolean keyed, long minDocCount, double minBound, double maxBound, + ValuesSource valuesSource, DocValueFormat formatter, SearchContext context, + Aggregator parent, List pipelineAggregators, + Map metaData) throws IOException { + ValuesSource.Range rangeValueSource = (ValuesSource.Range) valuesSource; + if (rangeValueSource.rangeType().isNumeric() == false) { + throw new IllegalArgumentException("Expected numeric range type but found non-numeric range [" + + rangeValueSource.rangeType().name + "]"); + } + return new RangeHistogramAggregator(name, factories, interval, offset, order, keyed, minDocCount, minBound, + maxBound, rangeValueSource, formatter, context, parent, pipelineAggregators, metaData); + } + } + ); + + valuesSourceRegistry.register(HistogramAggregationBuilder.NAME, + List.of(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN), + new HistogramAggregatorSupplier() { + @Override + public Aggregator build(String name, AggregatorFactories factories, double interval, double offset, + BucketOrder order, boolean keyed, long minDocCount, double minBound, double maxBound, + ValuesSource valuesSource, DocValueFormat formatter, SearchContext context, + Aggregator parent, List pipelineAggregators, + Map metaData) throws IOException { + return new NumericHistogramAggregator(name, factories, interval, offset, order, keyed, minDocCount, minBound, + maxBound, (ValuesSource.Numeric) valuesSource, formatter, context, parent, pipelineAggregators, metaData); + } + } + ); } public HistogramAggregatorFactory(String name, - ValuesSourceConfig config, + ValuesSourceConfig config, double interval, double offset, BucketOrder order, @@ -92,23 +126,16 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, if (collectsFromSingleBucket == false) { return asMultiBucketAggregator(this, searchContext, parent); } - if (valuesSource instanceof ValuesSource.Numeric) { - return new NumericHistogramAggregator(name, factories, interval, offset, order, keyed, minDocCount, minBound, maxBound, - (ValuesSource.Numeric) valuesSource, config.format(), searchContext, parent, pipelineAggregators, metadata); - } else if (valuesSource instanceof ValuesSource.Range) { - ValuesSource.Range rangeValueSource = (ValuesSource.Range) valuesSource; - if (rangeValueSource.rangeType().isNumeric() == false) { - throw new IllegalArgumentException("Expected numeric range type but found non-numeric range [" - + rangeValueSource.rangeType().name + "]"); - } - return new RangeHistogramAggregator(name, factories, interval, offset, order, keyed, minDocCount, minBound, maxBound, - (ValuesSource.Range) valuesSource, config.format(), searchContext, parent, pipelineAggregators, - metadata); - } - else { - throw new IllegalArgumentException("Expected one of [Numeric, Range] values source, found [" - + valuesSource.toString() + "]"); + + AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), + HistogramAggregationBuilder.NAME); + if (aggregatorSupplier instanceof HistogramAggregatorSupplier == false) { + throw new AggregationExecutionException("Registry miss-match - expected HistogramAggregatorSupplier, found [" + + aggregatorSupplier.getClass().toString() + "]"); } + HistogramAggregatorSupplier histogramAggregatorSupplier = (HistogramAggregatorSupplier) aggregatorSupplier; + return histogramAggregatorSupplier.build(name, factories, interval, offset, order, keyed, minDocCount, minBound, maxBound, + valuesSource, config.format(), searchContext, parent, pipelineAggregators, metaData); } @Override diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/NumericHistogramAggregator.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/NumericHistogramAggregator.java index 79eb40f9d45b5..bf6a85635686e 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/NumericHistogramAggregator.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/NumericHistogramAggregator.java @@ -51,7 +51,7 @@ * written as {@code interval * x + offset} and yet is less than or equal to * {@code value}. */ -class NumericHistogramAggregator extends BucketsAggregator { +public class NumericHistogramAggregator extends BucketsAggregator { private final ValuesSource.Numeric valuesSource; private final DocValueFormat formatter; @@ -63,7 +63,7 @@ class NumericHistogramAggregator extends BucketsAggregator { private final LongHash bucketOrds; - NumericHistogramAggregator(String name, AggregatorFactories factories, double interval, double offset, + public NumericHistogramAggregator(String name, AggregatorFactories factories, double interval, double offset, BucketOrder order, boolean keyed, long minDocCount, double minBound, double maxBound, @Nullable ValuesSource.Numeric valuesSource, DocValueFormat formatter, SearchContext context, Aggregator parent, diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/RangeHistogramAggregator.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/RangeHistogramAggregator.java index cae5c0aa59c93..f993e6f07a8db 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/RangeHistogramAggregator.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/RangeHistogramAggregator.java @@ -57,7 +57,7 @@ public class RangeHistogramAggregator extends BucketsAggregator { private final LongHash bucketOrds; - RangeHistogramAggregator(String name, AggregatorFactories factories, double interval, double offset, + public RangeHistogramAggregator(String name, AggregatorFactories factories, double interval, double offset, BucketOrder order, boolean keyed, long minDocCount, double minBound, double maxBound, @Nullable ValuesSource.Range valuesSource, DocValueFormat formatter, SearchContext context, Aggregator parent, diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/missing/MissingAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/missing/MissingAggregationBuilder.java index b93529803f842..62d76914fef29 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/missing/MissingAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/missing/MissingAggregationBuilder.java @@ -29,27 +29,25 @@ import org.elasticsearch.search.aggregations.AggregatorFactories.Builder; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; -import org.elasticsearch.search.aggregations.support.ValueType; -import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; -import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.util.Map; -public class MissingAggregationBuilder extends ValuesSourceAggregationBuilder { +public class MissingAggregationBuilder extends ValuesSourceAggregationBuilder { public static final String NAME = "missing"; public static final ObjectParser PARSER = - ObjectParser.fromBuilder(NAME, name -> new MissingAggregationBuilder(name, null)); + ObjectParser.fromBuilder(NAME, MissingAggregationBuilder::new); static { - ValuesSourceParserHelper.declareAnyFields(PARSER, true, true); + ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, false); } - public MissingAggregationBuilder(String name, ValueType targetValueType) { - super(name, CoreValuesSourceType.ANY, targetValueType); + public MissingAggregationBuilder(String name) { + super(name); } protected MissingAggregationBuilder(MissingAggregationBuilder clone, Builder factoriesBuilder, Map metadata) { @@ -57,15 +55,20 @@ protected MissingAggregationBuilder(MissingAggregationBuilder clone, Builder fac } @Override - protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metadata) { - return new MissingAggregationBuilder(this, factoriesBuilder, metadata); + protected ValuesSourceType defaultValueSourceType() { + return CoreValuesSourceType.BYTES; + } + + @Override + protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metaData) { + return new MissingAggregationBuilder(this, factoriesBuilder, metaData); } /** * Read from a stream. */ public MissingAggregationBuilder(StreamInput in) throws IOException { - super(in, CoreValuesSourceType.ANY); + super(in); } @Override @@ -84,11 +87,11 @@ public BucketCardinality bucketCardinality() { } @Override - protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, - ValuesSourceConfig config, - AggregatorFactory parent, - Builder subFactoriesBuilder) throws IOException { - return new MissingAggregatorFactory(name, config, queryShardContext, parent, subFactoriesBuilder, metadata); + protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, + ValuesSourceConfig config, + AggregatorFactory parent, + Builder subFactoriesBuilder) throws IOException { + return new MissingAggregatorFactory(name, config, queryShardContext, parent, subFactoriesBuilder, metaData); } @Override diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/missing/MissingAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/missing/MissingAggregatorFactory.java index d10a37fb771d1..6324ae4b749fe 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/missing/MissingAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/missing/MissingAggregatorFactory.java @@ -33,11 +33,12 @@ import java.util.List; import java.util.Map; -public class MissingAggregatorFactory extends ValuesSourceAggregatorFactory { +public class MissingAggregatorFactory extends ValuesSourceAggregatorFactory { - public MissingAggregatorFactory(String name, ValuesSourceConfig config, QueryShardContext queryShardContext, - AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, Map metadata) throws IOException { - super(name, config, queryShardContext, parent, subFactoriesBuilder, metadata); + public MissingAggregatorFactory(String name, ValuesSourceConfig config, QueryShardContext queryShardContext, + AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, + Map metaData) throws IOException { + super(name, config, queryShardContext, parent, subFactoriesBuilder, metaData); } @Override diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/AbstractRangeAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/AbstractRangeAggregatorFactory.java index 153f4072fc6f4..93f9a670de9f0 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/AbstractRangeAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/AbstractRangeAggregatorFactory.java @@ -20,41 +20,70 @@ package org.elasticsearch.search.aggregations.bucket.range; import org.elasticsearch.index.query.QueryShardContext; +import org.elasticsearch.search.DocValueFormat; +import org.elasticsearch.search.aggregations.AggregationExecutionException; import org.elasticsearch.search.aggregations.Aggregator; import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.bucket.range.RangeAggregator.Range; import org.elasticsearch.search.aggregations.bucket.range.RangeAggregator.Unmapped; import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; import java.util.List; import java.util.Map; -public class AbstractRangeAggregatorFactory extends ValuesSourceAggregatorFactory { +public class AbstractRangeAggregatorFactory extends ValuesSourceAggregatorFactory { private final InternalRange.Factory rangeFactory; private final R[] ranges; private final boolean keyed; + private final String aggregationTypeName; + public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry, String aggregationName) { + valuesSourceRegistry.register(aggregationName, + List.of(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN), + new RangeAggregatorSupplier() { + @Override + public Aggregator build(String name, + AggregatorFactories factories, + Numeric valuesSource, + DocValueFormat format, + InternalRange.Factory rangeFactory, + Range[] ranges, + boolean keyed, + SearchContext context, + Aggregator parent, + List pipelineAggregators, + Map metaData) throws IOException { + return new RangeAggregator(name, factories, valuesSource, format, rangeFactory, ranges, keyed, context, parent, + pipelineAggregators, metaData); + } + }); + } public AbstractRangeAggregatorFactory(String name, - ValuesSourceConfig config, - R[] ranges, - boolean keyed, - InternalRange.Factory rangeFactory, - QueryShardContext queryShardContext, - AggregatorFactory parent, - AggregatorFactories.Builder subFactoriesBuilder, - Map metadata) throws IOException { - super(name, config, queryShardContext, parent, subFactoriesBuilder, metadata); + String aggregationTypeName, + ValuesSourceConfig config, + R[] ranges, + boolean keyed, + InternalRange.Factory rangeFactory, + QueryShardContext queryShardContext, + AggregatorFactory parent, + AggregatorFactories.Builder subFactoriesBuilder, + Map metaData) throws IOException { + super(name, config, queryShardContext, parent, subFactoriesBuilder, metaData); this.ranges = ranges; this.keyed = keyed; this.rangeFactory = rangeFactory; + this.aggregationTypeName = aggregationTypeName; } @Override @@ -66,15 +95,19 @@ protected Aggregator createUnmapped(SearchContext searchContext, } @Override - protected Aggregator doCreateInternal(Numeric valuesSource, + protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, List pipelineAggregators, - Map metadata) throws IOException { - return new RangeAggregator(name, factories, valuesSource, config.format(), rangeFactory, ranges, keyed, searchContext, parent, - pipelineAggregators, metadata); + Map metaData) throws IOException { + AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), + aggregationTypeName); + if (aggregatorSupplier instanceof RangeAggregatorSupplier == false) { + throw new AggregationExecutionException("Registry miss-match - expected RangeAggregatorSupplier, found [" + + aggregatorSupplier.getClass().toString() + "]"); + } + return ((RangeAggregatorSupplier)aggregatorSupplier).build(name, factories, (Numeric) valuesSource, config.format(), rangeFactory, + ranges, keyed, searchContext, parent, pipelineAggregators, metaData); } - - } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/AbstractRangeBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/AbstractRangeBuilder.java index 63845b698b8b5..6cd43a271fade 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/AbstractRangeBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/AbstractRangeBuilder.java @@ -26,8 +26,8 @@ import org.elasticsearch.common.xcontent.XContentBuilder; import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.bucket.range.RangeAggregator.Range; -import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.util.ArrayList; @@ -37,13 +37,14 @@ import java.util.function.Function; public abstract class AbstractRangeBuilder, R extends Range> - extends ValuesSourceAggregationBuilder { + extends ValuesSourceAggregationBuilder implements MultiBucketAggregationBuilder { + protected final InternalRange.Factory rangeFactory; protected List ranges = new ArrayList<>(); protected boolean keyed = false; protected AbstractRangeBuilder(String name, InternalRange.Factory rangeFactory) { - super(name, rangeFactory.getValueSourceType(), rangeFactory.getValueType()); + super(name); this.rangeFactory = rangeFactory; } @@ -60,12 +61,18 @@ protected AbstractRangeBuilder(AbstractRangeBuilder clone, */ protected AbstractRangeBuilder(StreamInput in, InternalRange.Factory rangeFactory, Writeable.Reader rangeReader) throws IOException { - super(in, rangeFactory.getValueSourceType(), rangeFactory.getValueType()); + super(in); this.rangeFactory = rangeFactory; ranges = in.readList(rangeReader); keyed = in.readBoolean(); } + @Override + protected ValuesSourceType defaultValueSourceType() { + // Copied over from the old targetValueType setting. Not sure what cases this is still relevant for. --Tozzi 2020-01-13 + return rangeFactory.getValueSourceType(); + } + /** * Resolve any strings in the ranges so we have a number value for the from * and to of each range. The ranges are also sorted before being returned. diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/BinaryRangeAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/BinaryRangeAggregatorFactory.java index 787e307fd0b6d..2d0f6a479367f 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/BinaryRangeAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/BinaryRangeAggregatorFactory.java @@ -19,6 +19,7 @@ package org.elasticsearch.search.aggregations.bucket.range; import org.elasticsearch.index.query.QueryShardContext; +import org.elasticsearch.search.aggregations.AggregationExecutionException; import org.elasticsearch.search.aggregations.Aggregator; import org.elasticsearch.search.aggregations.AggregatorFactories.Builder; import org.elasticsearch.search.aggregations.AggregatorFactory; @@ -33,13 +34,13 @@ import java.util.Map; public class BinaryRangeAggregatorFactory - extends ValuesSourceAggregatorFactory { + extends ValuesSourceAggregatorFactory { private final List ranges; private final boolean keyed; public BinaryRangeAggregatorFactory(String name, - ValuesSourceConfig config, + ValuesSourceConfig config, List ranges, boolean keyed, QueryShardContext queryShardContext, AggregatorFactory parent, Builder subFactoriesBuilder, @@ -58,13 +59,17 @@ protected Aggregator createUnmapped(SearchContext searchContext, Aggregator pare } @Override - protected Aggregator doCreateInternal(ValuesSource.Bytes valuesSource, + protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, List pipelineAggregators, - Map metadata) throws IOException { - return new BinaryRangeAggregator(name, factories, valuesSource, config.format(), - ranges, keyed, searchContext, parent, pipelineAggregators, metadata); + Map metaData) throws IOException { + if (valuesSource instanceof ValuesSource.Bytes == false) { + throw new AggregationExecutionException("ValuesSource type " + valuesSource.toString() + "is not supported for aggregation " + + this.name()); + } + return new BinaryRangeAggregator(name, factories, (ValuesSource.Bytes) valuesSource, config.format(), + ranges, keyed, searchContext, parent, pipelineAggregators, metaData); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/DateRangeAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/DateRangeAggregationBuilder.java index 30fcc8a181c63..e92140eab4de8 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/DateRangeAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/DateRangeAggregationBuilder.java @@ -27,9 +27,11 @@ import org.elasticsearch.search.aggregations.AggregationBuilder; import org.elasticsearch.search.aggregations.AggregatorFactories.Builder; import org.elasticsearch.search.aggregations.AggregatorFactory; -import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; +import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; -import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.time.ZonedDateTime; @@ -41,7 +43,7 @@ public class DateRangeAggregationBuilder extends AbstractRangeBuilder PARSER = ObjectParser.fromBuilder(NAME, DateRangeAggregationBuilder::new); static { - ValuesSourceParserHelper.declareNumericFields(PARSER, true, true, true); + ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, true); PARSER.declareBoolean(DateRangeAggregationBuilder::keyed, RangeAggregator.KEYED_FIELD); PARSER.declareObjectArray((agg, ranges) -> { @@ -56,6 +58,10 @@ private static RangeAggregator.Range parseRange(XContentParser parser) throws IO return RangeAggregator.Range.fromXContent(parser); } + public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + AbstractRangeAggregatorFactory.registerAggregators(valuesSourceRegistry, NAME); + } + public DateRangeAggregationBuilder(String name) { super(name, InternalDateRange.FACTORY); } @@ -81,6 +87,11 @@ public String getType() { return NAME; } + @Override + protected ValuesSourceType defaultValueSourceType() { + return CoreValuesSourceType.DATE; + } + /** * Add a new range to this aggregation. * @@ -285,7 +296,7 @@ public DateRangeAggregationBuilder addUnboundedFrom(ZonedDateTime from) { } @Override - protected DateRangeAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config, + protected DateRangeAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config, AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException { // We need to call processRanges here so they are parsed and we know whether `now` has been used before we make // the decision of whether to cache the request diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/DateRangeAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/DateRangeAggregatorFactory.java index cf4ea92e3e03d..c981c13c839a9 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/DateRangeAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/DateRangeAggregatorFactory.java @@ -22,7 +22,6 @@ import org.elasticsearch.index.query.QueryShardContext; import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; -import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; import java.io.IOException; @@ -31,15 +30,16 @@ public class DateRangeAggregatorFactory extends AbstractRangeAggregatorFactory { public DateRangeAggregatorFactory(String name, - ValuesSourceConfig config, - RangeAggregator.Range[] ranges, - boolean keyed, - InternalRange.Factory rangeFactory, - QueryShardContext queryShardContext, - AggregatorFactory parent, - AggregatorFactories.Builder subFactoriesBuilder, - Map metadata) throws IOException { - super(name, config, ranges, keyed, rangeFactory, queryShardContext, parent, subFactoriesBuilder, metadata); + ValuesSourceConfig config, + RangeAggregator.Range[] ranges, + boolean keyed, + InternalRange.Factory rangeFactory, + QueryShardContext queryShardContext, + AggregatorFactory parent, + AggregatorFactories.Builder subFactoriesBuilder, + Map metaData) throws IOException { + super(name, DateRangeAggregationBuilder.NAME, config, ranges, keyed, rangeFactory, queryShardContext, parent, subFactoriesBuilder, + metaData); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/GeoDistanceAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/GeoDistanceAggregationBuilder.java index 9fb897da0891b..569eb10be0991 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/GeoDistanceAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/GeoDistanceAggregationBuilder.java @@ -35,11 +35,11 @@ import org.elasticsearch.search.aggregations.AggregationBuilder; import org.elasticsearch.search.aggregations.AggregatorFactories.Builder; import org.elasticsearch.search.aggregations.AggregatorFactory; -import org.elasticsearch.search.aggregations.support.ValuesSource; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; -import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.util.ArrayList; @@ -51,7 +51,7 @@ import static org.elasticsearch.search.aggregations.bucket.range.RangeAggregator.Range.KEY_FIELD; import static org.elasticsearch.search.aggregations.bucket.range.RangeAggregator.Range.TO_FIELD; -public class GeoDistanceAggregationBuilder extends ValuesSourceAggregationBuilder { +public class GeoDistanceAggregationBuilder extends ValuesSourceAggregationBuilder { public static final String NAME = "geo_distance"; static final ParseField ORIGIN_FIELD = new ParseField("origin", "center", "point", "por"); static final ParseField UNIT_FIELD = new ParseField("unit"); @@ -60,7 +60,7 @@ public class GeoDistanceAggregationBuilder extends ValuesSourceAggregationBuilde private static final ObjectParser PARSER; static { PARSER = new ObjectParser<>(GeoDistanceAggregationBuilder.NAME); - ValuesSourceParserHelper.declareGeoFields(PARSER, true, false); + ValuesSourceAggregationBuilder.declareFields(PARSER, true, false, false); PARSER.declareBoolean(GeoDistanceAggregationBuilder::keyed, RangeAggregator.KEYED_FIELD); @@ -230,15 +230,16 @@ public GeoDistanceAggregationBuilder(String name, GeoPoint origin) { private GeoDistanceAggregationBuilder(String name, GeoPoint origin, InternalRange.Factory rangeFactory) { - super(name, rangeFactory.getValueSourceType(), rangeFactory.getValueType()); + super(name); this.origin = origin; } + /** * Read from a stream. */ public GeoDistanceAggregationBuilder(StreamInput in) throws IOException { - super(in, InternalGeoDistance.FACTORY.getValueSourceType(), InternalGeoDistance.FACTORY.getValueType()); + super(in); origin = new GeoPoint(in.readDouble(), in.readDouble()); int size = in.readVInt(); ranges = new ArrayList<>(size); @@ -265,8 +266,15 @@ protected GeoDistanceAggregationBuilder(GeoDistanceAggregationBuilder clone, Bui } @Override - protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metadata) { - return new GeoDistanceAggregationBuilder(this, factoriesBuilder, metadata); + protected ValuesSourceType defaultValueSourceType() { + // TODO: This should probably not be BYTES, but we're not failing tests with BYTES, so needs more tests? + // TODO: this should set defaultValuesSourceType to GEOPOINT + return CoreValuesSourceType.BYTES; + } + + @Override + protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metaData) { + return new GeoDistanceAggregationBuilder(this, factoriesBuilder, metaData); } GeoDistanceAggregationBuilder origin(GeoPoint origin) { @@ -416,10 +424,10 @@ public BucketCardinality bucketCardinality() { } @Override - protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, - ValuesSourceConfig config, - AggregatorFactory parent, - Builder subFactoriesBuilder) throws IOException { + protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, + ValuesSourceConfig config, + AggregatorFactory parent, + Builder subFactoriesBuilder) throws IOException { Range[] ranges = this.ranges.toArray(new Range[this.range().size()]); if (ranges.length == 0) { throw new IllegalArgumentException("No [ranges] specified for the [" + this.getName() + "] aggregation"); diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/GeoDistanceRangeAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/GeoDistanceRangeAggregatorFactory.java index d4ce9b581c5a8..c3ae6be15a6ea 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/GeoDistanceRangeAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/GeoDistanceRangeAggregatorFactory.java @@ -29,6 +29,7 @@ import org.elasticsearch.index.fielddata.SortedBinaryDocValues; import org.elasticsearch.index.fielddata.SortedNumericDoubleValues; import org.elasticsearch.index.query.QueryShardContext; +import org.elasticsearch.search.aggregations.AggregationExecutionException; import org.elasticsearch.search.aggregations.Aggregator; import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; @@ -44,7 +45,7 @@ import java.util.Map; public class GeoDistanceRangeAggregatorFactory - extends ValuesSourceAggregatorFactory { + extends ValuesSourceAggregatorFactory { private final InternalRange.Factory rangeFactory = InternalGeoDistance.FACTORY; private final GeoPoint origin; @@ -53,10 +54,12 @@ public class GeoDistanceRangeAggregatorFactory private final GeoDistance distanceType; private final boolean keyed; - public GeoDistanceRangeAggregatorFactory(String name, ValuesSourceConfig config, GeoPoint origin, - Range[] ranges, DistanceUnit unit, GeoDistance distanceType, boolean keyed, QueryShardContext queryShardContext, - AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, Map metadata) throws IOException { - super(name, config, queryShardContext, parent, subFactoriesBuilder, metadata); + public GeoDistanceRangeAggregatorFactory(String name, ValuesSourceConfig config, GeoPoint origin, + Range[] ranges, DistanceUnit unit, GeoDistance distanceType, boolean keyed, + QueryShardContext queryShardContext, AggregatorFactory parent, + AggregatorFactories.Builder subFactoriesBuilder, + Map metaData) throws IOException { + super(name, config, queryShardContext, parent, subFactoriesBuilder, metaData); this.origin = origin; this.ranges = ranges; this.unit = unit; @@ -74,13 +77,17 @@ protected Aggregator createUnmapped(SearchContext searchContext, } @Override - protected Aggregator doCreateInternal(final ValuesSource.GeoPoint valuesSource, + protected Aggregator doCreateInternal(final ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, List pipelineAggregators, - Map metadata) throws IOException { - DistanceSource distanceSource = new DistanceSource(valuesSource, distanceType, origin, unit); + Map metaData) throws IOException { + if (valuesSource instanceof ValuesSource.GeoPoint == false) { + throw new AggregationExecutionException("ValuesSource type " + valuesSource.toString() + "is not supported for aggregation " + + this.name()); + } + DistanceSource distanceSource = new DistanceSource((ValuesSource.GeoPoint) valuesSource, distanceType, origin, unit); return new RangeAggregator(name, factories, distanceSource, config.format(), rangeFactory, ranges, keyed, searchContext, parent, pipelineAggregators, metadata); diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/IpRangeAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/IpRangeAggregationBuilder.java index 819e7e99e52cc..5014144135224 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/IpRangeAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/IpRangeAggregationBuilder.java @@ -37,12 +37,10 @@ import org.elasticsearch.search.aggregations.AggregatorFactories.Builder; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; -import org.elasticsearch.search.aggregations.support.ValueType; -import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; -import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.net.InetAddress; @@ -55,14 +53,14 @@ public final class IpRangeAggregationBuilder - extends ValuesSourceAggregationBuilder { + extends ValuesSourceAggregationBuilder { public static final String NAME = "ip_range"; private static final ParseField MASK_FIELD = new ParseField("mask"); public static final ObjectParser PARSER = ObjectParser.fromBuilder(NAME, IpRangeAggregationBuilder::new); static { - ValuesSourceParserHelper.declareBytesFields(PARSER, false, false); + ValuesSourceAggregationBuilder.declareFields(PARSER, false, false, false); PARSER.declareBoolean(IpRangeAggregationBuilder::keyed, RangeAggregator.KEYED_FIELD); @@ -216,7 +214,7 @@ public XContentBuilder toXContent(XContentBuilder builder, Params params) throws private List ranges = new ArrayList<>(); public IpRangeAggregationBuilder(String name) { - super(name, CoreValuesSourceType.BYTES, ValueType.IP); + super(name); } protected IpRangeAggregationBuilder(IpRangeAggregationBuilder clone, Builder factoriesBuilder, Map metadata) { @@ -333,7 +331,7 @@ public IpRangeAggregationBuilder addUnboundedFrom(String from) { } public IpRangeAggregationBuilder(StreamInput in) throws IOException { - super(in, CoreValuesSourceType.BYTES, ValueType.IP); + super(in); final int numRanges = in.readVInt(); for (int i = 0; i < numRanges; ++i) { addRange(new Range(in)); @@ -341,6 +339,11 @@ public IpRangeAggregationBuilder(StreamInput in) throws IOException { keyed = in.readBoolean(); } + @Override + protected ValuesSourceType defaultValueSourceType() { + return CoreValuesSourceType.IP; + } + @Override protected void innerWriteTo(StreamOutput out) throws IOException { out.writeVInt(ranges.size()); @@ -365,9 +368,10 @@ public BucketCardinality bucketCardinality() { } @Override - protected ValuesSourceAggregatorFactory innerBuild( - QueryShardContext queryShardContext, ValuesSourceConfig config, - AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException { + protected ValuesSourceAggregatorFactory innerBuild( + QueryShardContext queryShardContext, ValuesSourceConfig config, + AggregatorFactory parent, Builder subFactoriesBuilder) + throws IOException { List ranges = new ArrayList<>(); if(this.ranges.size() == 0){ throw new IllegalArgumentException("No [ranges] specified for the [" + this.getName() + "] aggregation"); diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/RangeAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/RangeAggregationBuilder.java index f3f680d0874ec..463cef8e8aff3 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/RangeAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/RangeAggregationBuilder.java @@ -28,9 +28,9 @@ import org.elasticsearch.search.aggregations.AggregatorFactories.Builder; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.bucket.range.RangeAggregator.Range; -import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric; +import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; -import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import java.io.IOException; import java.util.Map; @@ -41,7 +41,7 @@ public class RangeAggregationBuilder extends AbstractRangeBuilder PARSER = ObjectParser.fromBuilder(NAME, RangeAggregationBuilder::new); static { - ValuesSourceParserHelper.declareNumericFields(PARSER, true, true, false); + ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, false); PARSER.declareBoolean(RangeAggregationBuilder::keyed, RangeAggregator.KEYED_FIELD); PARSER.declareObjectArray((agg, ranges) -> { @@ -55,6 +55,10 @@ private static Range parseRange(XContentParser parser) throws IOException { return Range.fromXContent(parser); } + public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + AbstractRangeAggregatorFactory.registerAggregators(valuesSourceRegistry, NAME); + } + public RangeAggregationBuilder(String name) { super(name, InternalRange.FACTORY); } @@ -142,7 +146,7 @@ public RangeAggregationBuilder addUnboundedFrom(double from) { } @Override - protected RangeAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config, + protected RangeAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config, AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException { // We need to call processRanges here so they are parsed before we make the decision of whether to cache the request Range[] ranges = processRanges(range -> { diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/RangeAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/RangeAggregatorFactory.java index 65c6cefe0d6cc..7d30605d19646 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/RangeAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/RangeAggregatorFactory.java @@ -24,7 +24,6 @@ import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.bucket.range.InternalRange.Factory; import org.elasticsearch.search.aggregations.bucket.range.RangeAggregator.Range; -import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; import java.io.IOException; @@ -32,10 +31,17 @@ public class RangeAggregatorFactory extends AbstractRangeAggregatorFactory { - public RangeAggregatorFactory(String name, ValuesSourceConfig config, Range[] ranges, boolean keyed, - Factory rangeFactory, QueryShardContext queryShardContext, AggregatorFactory parent, - AggregatorFactories.Builder subFactoriesBuilder, Map metadata) throws IOException { - super(name, config, ranges, keyed, rangeFactory, queryShardContext, parent, subFactoriesBuilder, metadata); + public RangeAggregatorFactory(String name, + ValuesSourceConfig config, + Range[] ranges, + boolean keyed, + Factory rangeFactory, + QueryShardContext queryShardContext, + AggregatorFactory parent, + AggregatorFactories.Builder subFactoriesBuilder, + Map metaData) throws IOException { + super(name, RangeAggregationBuilder.NAME, config, ranges, keyed, rangeFactory, queryShardContext, parent, subFactoriesBuilder, + metaData); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/RangeAggregatorSupplier.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/RangeAggregatorSupplier.java new file mode 100644 index 0000000000000..2594396e33a3f --- /dev/null +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/RangeAggregatorSupplier.java @@ -0,0 +1,45 @@ +/* + * Licensed to Elasticsearch under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.elasticsearch.search.aggregations.bucket.range; + +import org.elasticsearch.search.DocValueFormat; +import org.elasticsearch.search.aggregations.Aggregator; +import org.elasticsearch.search.aggregations.AggregatorFactories; +import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.ValuesSource; +import org.elasticsearch.search.internal.SearchContext; + +import java.io.IOException; +import java.util.List; +import java.util.Map; + +public interface RangeAggregatorSupplier extends AggregatorSupplier { + Aggregator build(String name, + AggregatorFactories factories, + ValuesSource.Numeric valuesSource, + DocValueFormat format, + InternalRange.Factory rangeFactory, + RangeAggregator.Range[] ranges, + boolean keyed, + SearchContext context, + Aggregator parent, + List pipelineAggregators, + Map metaData) throws IOException; +} diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/sampler/DiversifiedAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/sampler/DiversifiedAggregationBuilder.java index 94f97d777c9e2..6c5e1d5cd4e47 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/sampler/DiversifiedAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/sampler/DiversifiedAggregationBuilder.java @@ -28,17 +28,16 @@ import org.elasticsearch.search.aggregations.AggregatorFactories.Builder; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; -import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; -import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.util.Map; import java.util.Objects; -public class DiversifiedAggregationBuilder extends ValuesSourceAggregationBuilder { +public class DiversifiedAggregationBuilder extends ValuesSourceAggregationBuilder { public static final String NAME = "diversified_sampler"; public static final int MAX_DOCS_PER_VALUE_DEFAULT = 1; @@ -46,7 +45,7 @@ public class DiversifiedAggregationBuilder extends ValuesSourceAggregationBuilde public static final ObjectParser PARSER = ObjectParser.fromBuilder(NAME, DiversifiedAggregationBuilder::new); static { - ValuesSourceParserHelper.declareAnyFields(PARSER, true, false); + ValuesSourceAggregationBuilder.declareFields(PARSER, true, false, false); PARSER.declareInt(DiversifiedAggregationBuilder::shardSize, SamplerAggregator.SHARD_SIZE_FIELD); PARSER.declareInt(DiversifiedAggregationBuilder::maxDocsPerValue, SamplerAggregator.MAX_DOCS_PER_VALUE_FIELD); PARSER.declareString(DiversifiedAggregationBuilder::executionHint, SamplerAggregator.EXECUTION_HINT_FIELD); @@ -57,7 +56,7 @@ public class DiversifiedAggregationBuilder extends ValuesSourceAggregationBuilde private String executionHint = null; public DiversifiedAggregationBuilder(String name) { - super(name, CoreValuesSourceType.ANY, null); + super(name); } protected DiversifiedAggregationBuilder(DiversifiedAggregationBuilder clone, Builder factoriesBuilder, Map metadata) { @@ -68,15 +67,20 @@ protected DiversifiedAggregationBuilder(DiversifiedAggregationBuilder clone, Bui } @Override - protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metadata) { - return new DiversifiedAggregationBuilder(this, factoriesBuilder, metadata); + protected ValuesSourceType defaultValueSourceType() { + return CoreValuesSourceType.BYTES; + } + + @Override + protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metaData) { + return new DiversifiedAggregationBuilder(this, factoriesBuilder, metaData); } /** * Read from a stream. */ public DiversifiedAggregationBuilder(StreamInput in) throws IOException { - super(in, CoreValuesSourceType.ANY, null); + super(in); shardSize = in.readVInt(); maxDocsPerValue = in.readVInt(); executionHint = in.readOptionalString(); @@ -148,10 +152,10 @@ public BucketCardinality bucketCardinality() { } @Override - protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, - ValuesSourceConfig config, - AggregatorFactory parent, - Builder subFactoriesBuilder) throws IOException { + protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, + ValuesSourceConfig config, + AggregatorFactory parent, + Builder subFactoriesBuilder) throws IOException { return new DiversifiedAggregatorFactory(name, config, shardSize, maxDocsPerValue, executionHint, queryShardContext, parent, subFactoriesBuilder, metadata); } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/sampler/DiversifiedAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/sampler/DiversifiedAggregatorFactory.java index d1b1422378b74..c7808de590bbe 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/sampler/DiversifiedAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/sampler/DiversifiedAggregatorFactory.java @@ -38,13 +38,13 @@ import java.util.List; import java.util.Map; -public class DiversifiedAggregatorFactory extends ValuesSourceAggregatorFactory { +public class DiversifiedAggregatorFactory extends ValuesSourceAggregatorFactory { private final int shardSize; private final int maxDocsPerValue; private final String executionHint; - DiversifiedAggregatorFactory(String name, ValuesSourceConfig config, int shardSize, int maxDocsPerValue, + DiversifiedAggregatorFactory(String name, ValuesSourceConfig config, int shardSize, int maxDocsPerValue, String executionHint, QueryShardContext queryShardContext, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, Map metadata) throws IOException { super(name, config, queryShardContext, parent, subFactoriesBuilder, metadata); diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregationBuilder.java index 412622e05b21f..f8ebc4686773a 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregationBuilder.java @@ -38,12 +38,11 @@ import org.elasticsearch.search.aggregations.bucket.terms.TermsAggregator; import org.elasticsearch.search.aggregations.bucket.terms.TermsAggregator.BucketCountThresholds; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; -import org.elasticsearch.search.aggregations.support.ValueType; -import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; -import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.util.Map; @@ -51,7 +50,8 @@ import static org.elasticsearch.index.query.AbstractQueryBuilder.parseInnerQueryBuilder; -public class SignificantTermsAggregationBuilder extends ValuesSourceAggregationBuilder { +public class SignificantTermsAggregationBuilder extends ValuesSourceAggregationBuilder + implements MultiBucketAggregationBuilder { public static final String NAME = "significant_terms"; static final ParseField BACKGROUND_FILTER = new ParseField("background_filter"); @@ -65,7 +65,7 @@ public class SignificantTermsAggregationBuilder extends ValuesSourceAggregationB SignificantTermsAggregationBuilder.NAME, SignificanceHeuristic.class, SignificantTermsAggregationBuilder::significanceHeuristic, null); static { - ValuesSourceParserHelper.declareAnyFields(PARSER, true, true); + ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, false); PARSER.declareInt(SignificantTermsAggregationBuilder::shardSize, TermsAggregationBuilder.SHARD_SIZE_FIELD_NAME); @@ -90,7 +90,11 @@ public class SignificantTermsAggregationBuilder extends ValuesSourceAggregationB IncludeExclude::parseExclude, IncludeExclude.EXCLUDE_FIELD, ObjectParser.ValueType.STRING_ARRAY); } public static SignificantTermsAggregationBuilder parse(String aggregationName, XContentParser parser) throws IOException { - return PARSER.parse(parser, new SignificantTermsAggregationBuilder(aggregationName, null), null); + return PARSER.parse(parser, new SignificantTermsAggregationBuilder(aggregationName), null); + } + + public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + SignificantTermsAggregatorFactory.registerAggregators(valuesSourceRegistry); } private IncludeExclude includeExclude = null; @@ -99,15 +103,15 @@ public static SignificantTermsAggregationBuilder parse(String aggregationName, X private TermsAggregator.BucketCountThresholds bucketCountThresholds = new BucketCountThresholds(DEFAULT_BUCKET_COUNT_THRESHOLDS); private SignificanceHeuristic significanceHeuristic = DEFAULT_SIGNIFICANCE_HEURISTIC; - public SignificantTermsAggregationBuilder(String name, ValueType valueType) { - super(name, CoreValuesSourceType.ANY, valueType); + public SignificantTermsAggregationBuilder(String name) { + super(name); } /** * Read from a Stream. */ public SignificantTermsAggregationBuilder(StreamInput in) throws IOException { - super(in, CoreValuesSourceType.ANY); + super(in); bucketCountThresholds = new BucketCountThresholds(in); executionHint = in.readOptionalString(); filterBuilder = in.readOptionalNamedWriteable(QueryBuilder.class); @@ -126,11 +130,15 @@ protected SignificantTermsAggregationBuilder(SignificantTermsAggregationBuilder } @Override - protected SignificantTermsAggregationBuilder shallowCopy(Builder factoriesBuilder, Map metadata) { - return new SignificantTermsAggregationBuilder(this, factoriesBuilder, metadata); + protected ValuesSourceType defaultValueSourceType() { + return CoreValuesSourceType.BYTES; } @Override + protected SignificantTermsAggregationBuilder shallowCopy(Builder factoriesBuilder, Map metaData) { + return new SignificantTermsAggregationBuilder(this, factoriesBuilder, metaData); + } + protected AggregationBuilder doRewrite(QueryRewriteContext queryShardContext) throws IOException { if (filterBuilder != null) { QueryBuilder rewrittenFilter = filterBuilder.rewrite(queryShardContext); @@ -286,10 +294,10 @@ public BucketCardinality bucketCardinality() { } @Override - protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, - ValuesSourceConfig config, - AggregatorFactory parent, - Builder subFactoriesBuilder) throws IOException { + protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, + ValuesSourceConfig config, + AggregatorFactory parent, + Builder subFactoriesBuilder) throws IOException { SignificanceHeuristic executionHeuristic = this.significanceHeuristic.rewrite(queryShardContext); return new SignificantTermsAggregatorFactory(name, config, includeExclude, executionHint, filterBuilder, bucketCountThresholds, executionHeuristic, queryShardContext, parent, subFactoriesBuilder, metadata); diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorFactory.java index e87414edd4a88..943fe5b065887 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorFactory.java @@ -50,17 +50,20 @@ import org.elasticsearch.search.aggregations.bucket.terms.TermsAggregator; import org.elasticsearch.search.aggregations.bucket.terms.TermsAggregator.BucketCountThresholds; import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; import java.util.List; import java.util.Map; -public class SignificantTermsAggregatorFactory extends ValuesSourceAggregatorFactory - implements Releasable { +public class SignificantTermsAggregatorFactory extends ValuesSourceAggregatorFactory + implements Releasable { private static final DeprecationLogger deprecationLogger = new DeprecationLogger( LogManager.getLogger(SignificantTermsAggregatorFactory.class)); @@ -75,18 +78,124 @@ public class SignificantTermsAggregatorFactory extends ValuesSourceAggregatorFac private final TermsAggregator.BucketCountThresholds bucketCountThresholds; private final SignificanceHeuristic significanceHeuristic; - public SignificantTermsAggregatorFactory(String name, - ValuesSourceConfig config, - IncludeExclude includeExclude, - String executionHint, - QueryBuilder filterBuilder, - TermsAggregator.BucketCountThresholds bucketCountThresholds, - SignificanceHeuristic significanceHeuristic, - QueryShardContext queryShardContext, - AggregatorFactory parent, - AggregatorFactories.Builder subFactoriesBuilder, - Map metadata) throws IOException { - super(name, config, queryShardContext, parent, subFactoriesBuilder, metadata); + static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + valuesSourceRegistry.register(SignificantTermsAggregationBuilder.NAME, + List.of(CoreValuesSourceType.BYTES, CoreValuesSourceType.IP), + SignificantTermsAggregatorFactory.bytesSupplier()); + + valuesSourceRegistry.register(SignificantTermsAggregationBuilder.NAME, + List.of(CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN, CoreValuesSourceType.NUMERIC), + SignificantTermsAggregatorFactory.numericSupplier()); + } + + /** + * This supplier is used for all the field types that should be aggregated as bytes/strings, + * including those that need global ordinals + */ + private static SignificantTermsAggregatorSupplier bytesSupplier() { + return new SignificantTermsAggregatorSupplier() { + @Override + public Aggregator build(String name, + AggregatorFactories factories, + ValuesSource valuesSource, + DocValueFormat format, + TermsAggregator.BucketCountThresholds bucketCountThresholds, + IncludeExclude includeExclude, + String executionHint, + SearchContext context, + Aggregator parent, + SignificanceHeuristic significanceHeuristic, + SignificantTermsAggregatorFactory sigTermsFactory, + List pipelineAggregators, + Map metaData) throws IOException { + + ExecutionMode execution = null; + if (executionHint != null) { + execution = ExecutionMode.fromString(executionHint, deprecationLogger); + } + if (valuesSource instanceof ValuesSource.Bytes.WithOrdinals == false) { + execution = ExecutionMode.MAP; + } + if (execution == null) { + execution = ExecutionMode.GLOBAL_ORDINALS; + } + + if ((includeExclude != null) && (includeExclude.isRegexBased()) && format != DocValueFormat.RAW) { + throw new IllegalArgumentException("Aggregation [" + name + "] cannot support regular expression style " + + "include/exclude settings as they can only be applied to string fields. Use an array of values for " + + "include/exclude clauses"); + } + + return execution.create(name, factories, valuesSource, format, bucketCountThresholds, includeExclude, context, parent, + significanceHeuristic, sigTermsFactory, pipelineAggregators, metaData); + + } + }; + } + + /** + * This supplier is used for all fields that expect to be aggregated as a numeric value. + * This includes floating points, and formatted types that use numerics internally for storage (date, boolean, etc) + */ + private static SignificantTermsAggregatorSupplier numericSupplier() { + return new SignificantTermsAggregatorSupplier() { + @Override + public Aggregator build(String name, + AggregatorFactories factories, + ValuesSource valuesSource, + DocValueFormat format, + TermsAggregator.BucketCountThresholds bucketCountThresholds, + IncludeExclude includeExclude, + String executionHint, + SearchContext context, + Aggregator parent, + SignificanceHeuristic significanceHeuristic, + SignificantTermsAggregatorFactory sigTermsFactory, + List pipelineAggregators, + Map metaData) throws IOException { + + if ((includeExclude != null) && (includeExclude.isRegexBased())) { + throw new IllegalArgumentException("Aggregation [" + name + "] cannot support regular expression style include/exclude " + + "settings as they can only be applied to string fields. Use an array of numeric " + + "values for include/exclude clauses used to filter numeric fields"); + } + + if (((ValuesSource.Numeric) valuesSource).isFloatingPoint()) { + throw new UnsupportedOperationException("No support for examining floating point numerics"); + } + + IncludeExclude.LongFilter longFilter = null; + if (includeExclude != null) { + longFilter = includeExclude.convertToLongFilter(format); + } + + return new SignificantLongTermsAggregator(name, factories, (ValuesSource.Numeric) valuesSource, format, + bucketCountThresholds, context, parent, significanceHeuristic, sigTermsFactory, longFilter, pipelineAggregators, + metaData); + + } + }; + } + + SignificantTermsAggregatorFactory(String name, + ValuesSourceConfig config, + IncludeExclude includeExclude, + String executionHint, + QueryBuilder filterBuilder, + TermsAggregator.BucketCountThresholds bucketCountThresholds, + SignificanceHeuristic significanceHeuristic, + QueryShardContext queryShardContext, + AggregatorFactory parent, + AggregatorFactories.Builder subFactoriesBuilder, + Map metaData) throws IOException { + super(name, config, queryShardContext, parent, subFactoriesBuilder, metaData); + + if (config.unmapped() == false) { + if (config.fieldContext().fieldType().isSearchable() == false) { + throw new IllegalArgumentException("SignificantText aggregation requires fields to be searchable, but [" + + config.fieldContext().fieldType().name() + "] is not"); + } + } if (!config.unmapped()) { this.fieldType = config.fieldContext().fieldType(); @@ -111,7 +220,7 @@ public SignificantTermsAggregatorFactory(String name, /** * Get the number of docs in the superset. */ - public long getSupersetNumDocs() { + long getSupersetNumDocs() { return supersetNumDocs; } @@ -151,12 +260,12 @@ private long getBackgroundFrequency(String value) throws IOException { return queryShardContext.searcher().count(query); } - public long getBackgroundFrequency(BytesRef termBytes) throws IOException { + long getBackgroundFrequency(BytesRef termBytes) throws IOException { String value = config.format().format(termBytes).toString(); return getBackgroundFrequency(value); } - public long getBackgroundFrequency(long termNum) throws IOException { + long getBackgroundFrequency(long termNum) throws IOException { String value = config.format().format(termNum).toString(); return getBackgroundFrequency(value); } @@ -187,6 +296,13 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, return asMultiBucketAggregator(this, searchContext, parent); } + AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), + SignificantTermsAggregationBuilder.NAME); + if (aggregatorSupplier instanceof SignificantTermsAggregatorSupplier == false) { + throw new AggregationExecutionException("Registry miss-match - expected SignificantTermsAggregatorSupplier, found [" + + aggregatorSupplier.getClass().toString() + "]"); + } + numberOfAggregatorsCreated++; BucketCountThresholds bucketCountThresholds = new BucketCountThresholds(this.bucketCountThresholds); if (bucketCountThresholds.getShardSize() == SignificantTermsAggregationBuilder.DEFAULT_BUCKET_COUNT_THRESHOLDS.getShardSize()) { @@ -205,52 +321,12 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, bucketCountThresholds.setShardSize(2 * BucketUtils.suggestShardSideQueueSize(bucketCountThresholds.getRequiredSize())); } - if (valuesSource instanceof ValuesSource.Bytes) { - ExecutionMode execution = null; - if (executionHint != null) { - execution = ExecutionMode.fromString(executionHint, deprecationLogger); - } - if (valuesSource instanceof ValuesSource.Bytes.WithOrdinals == false) { - execution = ExecutionMode.MAP; - } - if (execution == null) { - execution = ExecutionMode.GLOBAL_ORDINALS; - } - assert execution != null; - - DocValueFormat format = config.format(); - if ((includeExclude != null) && (includeExclude.isRegexBased()) && format != DocValueFormat.RAW) { - throw new AggregationExecutionException("Aggregation [" + name + "] cannot support regular expression style " - + "include/exclude settings as they can only be applied to string fields. Use an array of values for " - + "include/exclude clauses"); - } - - return execution.create(name, factories, valuesSource, format, bucketCountThresholds, includeExclude, searchContext, parent, - significanceHeuristic, this, pipelineAggregators, metadata); - } - - if ((includeExclude != null) && (includeExclude.isRegexBased())) { - throw new AggregationExecutionException("Aggregation [" + name + "] cannot support regular expression style include/exclude " - + "settings as they can only be applied to string fields. Use an array of numeric values for include/exclude clauses " - + "used to filter numeric fields"); - } - - if (valuesSource instanceof ValuesSource.Numeric) { - - if (((ValuesSource.Numeric) valuesSource).isFloatingPoint()) { - throw new UnsupportedOperationException("No support for examining floating point numerics"); - } - IncludeExclude.LongFilter longFilter = null; - if (includeExclude != null) { - longFilter = includeExclude.convertToLongFilter(config.format()); - } - return new SignificantLongTermsAggregator(name, factories, (ValuesSource.Numeric) valuesSource, config.format(), - bucketCountThresholds, searchContext, parent, significanceHeuristic, this, longFilter, pipelineAggregators, - metadata); - } + SignificantTermsAggregatorSupplier sigTermsAggregatorSupplier = (SignificantTermsAggregatorSupplier) aggregatorSupplier; - throw new AggregationExecutionException("significant_terms aggregation cannot be applied to field [" - + config.fieldContext().field() + "]. It can only be applied to numeric or string fields."); + // TODO we should refactor so that we don't need to use this Factory as a singleton (e.g. stop passing `this` to the aggregators) + return sigTermsAggregatorSupplier.build(name, factories, valuesSource, config.format(), + bucketCountThresholds, includeExclude, executionHint, searchContext, parent, + significanceHeuristic, this, pipelineAggregators, metaData); } public enum ExecutionMode { diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorSupplier.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorSupplier.java new file mode 100644 index 0000000000000..e9c494ffd1b48 --- /dev/null +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorSupplier.java @@ -0,0 +1,50 @@ +/* + * Licensed to Elasticsearch under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.elasticsearch.search.aggregations.bucket.significant; + +import org.elasticsearch.search.DocValueFormat; +import org.elasticsearch.search.aggregations.Aggregator; +import org.elasticsearch.search.aggregations.AggregatorFactories; +import org.elasticsearch.search.aggregations.bucket.significant.heuristics.SignificanceHeuristic; +import org.elasticsearch.search.aggregations.bucket.terms.IncludeExclude; +import org.elasticsearch.search.aggregations.bucket.terms.TermsAggregator; +import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.ValuesSource; +import org.elasticsearch.search.internal.SearchContext; + +import java.io.IOException; +import java.util.List; +import java.util.Map; + +interface SignificantTermsAggregatorSupplier extends AggregatorSupplier { + Aggregator build(String name, + AggregatorFactories factories, + ValuesSource valuesSource, + DocValueFormat format, + TermsAggregator.BucketCountThresholds bucketCountThresholds, + IncludeExclude includeExclude, + String executionHint, + SearchContext context, + Aggregator parent, + SignificanceHeuristic significanceHeuristic, + SignificantTermsAggregatorFactory sigTermsFactory, + List pipelineAggregators, + Map metaData) throws IOException; +} diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregationBuilder.java index 48c72619901bb..e11edec07845e 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregationBuilder.java @@ -29,18 +29,17 @@ import org.elasticsearch.search.aggregations.AggregatorFactories.Builder; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; -import org.elasticsearch.search.aggregations.support.ValueType; -import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; -import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.util.Map; import java.util.Objects; -public class RareTermsAggregationBuilder extends ValuesSourceAggregationBuilder { +public class RareTermsAggregationBuilder extends ValuesSourceAggregationBuilder { public static final String NAME = "rare_terms"; private static final ParseField MAX_DOC_COUNT_FIELD_NAME = new ParseField("max_doc_count"); @@ -48,9 +47,9 @@ public class RareTermsAggregationBuilder extends ValuesSourceAggregationBuilder< private static final int MAX_MAX_DOC_COUNT = 100; public static final ObjectParser PARSER = - ObjectParser.fromBuilder(NAME, name -> new RareTermsAggregationBuilder(name, null)); + ObjectParser.fromBuilder(NAME, RareTermsAggregationBuilder::new); static { - ValuesSourceParserHelper.declareAnyFields(PARSER, true, true); + ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, false); PARSER.declareLong(RareTermsAggregationBuilder::maxDocCount, MAX_DOC_COUNT_FIELD_NAME); PARSER.declareField((b, v) -> b.includeExclude(IncludeExclude.merge(v, b.includeExclude())), @@ -62,12 +61,16 @@ public class RareTermsAggregationBuilder extends ValuesSourceAggregationBuilder< PARSER.declareDouble(RareTermsAggregationBuilder::setPrecision, PRECISION); } + public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + RareTermsAggregatorFactory.registerAggregators(valuesSourceRegistry); + } + private IncludeExclude includeExclude = null; private int maxDocCount = 1; private double precision = 0.001; - public RareTermsAggregationBuilder(String name, ValueType valueType) { - super(name, CoreValuesSourceType.ANY, valueType); + public RareTermsAggregationBuilder(String name) { + super(name); } private RareTermsAggregationBuilder(RareTermsAggregationBuilder clone, Builder factoriesBuilder, Map metadata) { @@ -76,15 +79,20 @@ private RareTermsAggregationBuilder(RareTermsAggregationBuilder clone, Builder f } @Override - protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metadata) { - return new RareTermsAggregationBuilder(this, factoriesBuilder, metadata); + protected ValuesSourceType defaultValueSourceType() { + return CoreValuesSourceType.BYTES; + } + + @Override + protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metaData) { + return new RareTermsAggregationBuilder(this, factoriesBuilder, metaData); } /** * Read from a stream. */ public RareTermsAggregationBuilder(StreamInput in) throws IOException { - super(in, CoreValuesSourceType.ANY); + super(in); includeExclude = in.readOptionalWriteable(IncludeExclude::new); maxDocCount = in.readVInt(); } @@ -162,10 +170,10 @@ public BucketCardinality bucketCardinality() { } @Override - protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, - ValuesSourceConfig config, - AggregatorFactory parent, - Builder subFactoriesBuilder) throws IOException { + protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, + ValuesSourceConfig config, + AggregatorFactory parent, + Builder subFactoriesBuilder) throws IOException { return new RareTermsAggregatorFactory(name, config, includeExclude, queryShardContext, parent, subFactoriesBuilder, metadata, maxDocCount, precision); } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregatorFactory.java index 80a931ea5c450..84466b3e72d81 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregatorFactory.java @@ -30,21 +30,106 @@ import org.elasticsearch.search.aggregations.InternalAggregation; import org.elasticsearch.search.aggregations.NonCollectingAggregator; import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; import java.util.List; import java.util.Map; -public class RareTermsAggregatorFactory extends ValuesSourceAggregatorFactory { +public class RareTermsAggregatorFactory extends ValuesSourceAggregatorFactory { private final IncludeExclude includeExclude; private final int maxDocCount; private final double precision; - RareTermsAggregatorFactory(String name, ValuesSourceConfig config, + static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + valuesSourceRegistry.register(RareTermsAggregationBuilder.NAME, + List.of(CoreValuesSourceType.BYTES, CoreValuesSourceType.IP), + RareTermsAggregatorFactory.bytesSupplier()); + + valuesSourceRegistry.register(RareTermsAggregationBuilder.NAME, + List.of(CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN, CoreValuesSourceType.NUMERIC), + RareTermsAggregatorFactory.numericSupplier()); + } + + /** + * This supplier is used for all the field types that should be aggregated as bytes/strings, + * including those that need global ordinals + */ + private static RareTermsAggregatorSupplier bytesSupplier() { + return new RareTermsAggregatorSupplier() { + @Override + public Aggregator build(String name, + AggregatorFactories factories, + ValuesSource valuesSource, + DocValueFormat format, + int maxDocCount, + double precision, + IncludeExclude includeExclude, + SearchContext context, + Aggregator parent, + List pipelineAggregators, + Map metaData) throws IOException { + + ExecutionMode execution = ExecutionMode.MAP; //TODO global ords not implemented yet, only supports "map" + + if ((includeExclude != null) && (includeExclude.isRegexBased()) && format != DocValueFormat.RAW) { + throw new IllegalArgumentException("Aggregation [" + name + "] cannot support " + + "regular expression style include/exclude settings as they can only be applied to string fields. " + + "Use an array of values for include/exclude clauses"); + } + + return execution.create(name, factories, valuesSource, format, + includeExclude, context, parent, pipelineAggregators, metaData, maxDocCount, precision); + + } + }; + } + + /** + * This supplier is used for all fields that expect to be aggregated as a numeric value. + * This includes floating points, and formatted types that use numerics internally for storage (date, boolean, etc) + */ + private static RareTermsAggregatorSupplier numericSupplier() { + return new RareTermsAggregatorSupplier() { + @Override + public Aggregator build(String name, + AggregatorFactories factories, + ValuesSource valuesSource, + DocValueFormat format, + int maxDocCount, + double precision, + IncludeExclude includeExclude, + SearchContext context, + Aggregator parent, + List pipelineAggregators, + Map metaData) throws IOException { + + if ((includeExclude != null) && (includeExclude.isRegexBased())) { + throw new IllegalArgumentException("Aggregation [" + name + "] cannot support regular expression " + + "style include/exclude settings as they can only be applied to string fields. Use an array of numeric " + + "values for include/exclude clauses used to filter numeric fields"); + } + + IncludeExclude.LongFilter longFilter = null; + if (((ValuesSource.Numeric) valuesSource).isFloatingPoint()) { + throw new IllegalArgumentException("RareTerms aggregation does not support floating point fields."); + } + if (includeExclude != null) { + longFilter = includeExclude.convertToLongFilter(format); + } + return new LongRareTermsAggregator(name, factories, (ValuesSource.Numeric) valuesSource, format, + context, parent, longFilter, maxDocCount, precision, pipelineAggregators, metaData); + } + }; + } + + RareTermsAggregatorFactory(String name, ValuesSourceConfig config, IncludeExclude includeExclude, QueryShardContext queryShardContext, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, @@ -79,40 +164,16 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, if (collectsFromSingleBucket == false) { return asMultiBucketAggregator(this, searchContext, parent); } - if (valuesSource instanceof ValuesSource.Bytes) { - ExecutionMode execution = ExecutionMode.MAP; //TODO global ords not implemented yet, only supports "map" - - DocValueFormat format = config.format(); - if ((includeExclude != null) && (includeExclude.isRegexBased()) && format != DocValueFormat.RAW) { - throw new AggregationExecutionException("Aggregation [" + name + "] cannot support " + - "regular expression style include/exclude settings as they can only be applied to string fields. " + - "Use an array of values for include/exclude clauses"); - } - - return execution.create(name, factories, valuesSource, format, - includeExclude, searchContext, parent, pipelineAggregators, metadata, maxDocCount, precision); - } - - if ((includeExclude != null) && (includeExclude.isRegexBased())) { - throw new AggregationExecutionException("Aggregation [" + name + "] cannot support regular expression style include/exclude " - + "settings as they can only be applied to string fields. Use an array of numeric values for include/exclude clauses " + - "used to filter numeric fields"); - } - if (valuesSource instanceof ValuesSource.Numeric) { - IncludeExclude.LongFilter longFilter = null; - if (((ValuesSource.Numeric) valuesSource).isFloatingPoint()) { - throw new AggregationExecutionException("RareTerms aggregation does not support floating point fields."); - } - if (includeExclude != null) { - longFilter = includeExclude.convertToLongFilter(config.format()); - } - return new LongRareTermsAggregator(name, factories, (ValuesSource.Numeric) valuesSource, config.format(), - searchContext, parent, longFilter, maxDocCount, precision, pipelineAggregators, metadata); + AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), + RareTermsAggregationBuilder.NAME); + if (aggregatorSupplier instanceof RareTermsAggregatorSupplier == false) { + throw new AggregationExecutionException("Registry miss-match - expected RareTermsAggregatorSupplier, found [" + + aggregatorSupplier.getClass().toString() + "]"); } - throw new AggregationExecutionException("RareTerms aggregation cannot be applied to field [" + config.fieldContext().field() - + "]. It can only be applied to numeric or string fields."); + return ((RareTermsAggregatorSupplier) aggregatorSupplier).build(name, factories, valuesSource, config.format(), + maxDocCount, precision, includeExclude, searchContext, parent, pipelineAggregators, metaData); } public enum ExecutionMode { diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregatorSupplier.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregatorSupplier.java new file mode 100644 index 0000000000000..98e718d6e3b10 --- /dev/null +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregatorSupplier.java @@ -0,0 +1,45 @@ +/* + * Licensed to Elasticsearch under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.elasticsearch.search.aggregations.bucket.terms; + +import org.elasticsearch.search.DocValueFormat; +import org.elasticsearch.search.aggregations.Aggregator; +import org.elasticsearch.search.aggregations.AggregatorFactories; +import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.ValuesSource; +import org.elasticsearch.search.internal.SearchContext; + +import java.io.IOException; +import java.util.List; +import java.util.Map; + +interface RareTermsAggregatorSupplier extends AggregatorSupplier { + Aggregator build(String name, + AggregatorFactories factories, + ValuesSource valuesSource, + DocValueFormat format, + int maxDocCount, + double precision, + IncludeExclude includeExclude, + SearchContext context, + Aggregator parent, + List pipelineAggregators, + Map metaData) throws IOException; +} diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregationBuilder.java index cb139f21e565b..d2ad7e0359003 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregationBuilder.java @@ -36,19 +36,19 @@ import org.elasticsearch.search.aggregations.InternalOrder.CompoundOrder; import org.elasticsearch.search.aggregations.bucket.terms.TermsAggregator.BucketCountThresholds; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; -import org.elasticsearch.search.aggregations.support.ValueType; -import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; -import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.util.List; import java.util.Map; import java.util.Objects; -public class TermsAggregationBuilder extends ValuesSourceAggregationBuilder { +public class TermsAggregationBuilder extends ValuesSourceAggregationBuilder + implements MultiBucketAggregationBuilder { public static final String NAME = "terms"; public static final ParseField EXECUTION_HINT_FIELD_NAME = new ParseField("execution_hint"); @@ -63,9 +63,9 @@ public class TermsAggregationBuilder extends ValuesSourceAggregationBuilder PARSER = - ObjectParser.fromBuilder(NAME, name -> new TermsAggregationBuilder(name, null)); + ObjectParser.fromBuilder(NAME, TermsAggregationBuilder::new); static { - ValuesSourceParserHelper.declareAnyFields(PARSER, true, true); + ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, false); PARSER.declareBoolean(TermsAggregationBuilder::showTermDocCountError, TermsAggregationBuilder.SHOW_TERM_DOC_COUNT_ERROR); @@ -94,6 +94,10 @@ public class TermsAggregationBuilder extends ValuesSourceAggregationBuilder metadata) { @@ -117,15 +121,20 @@ protected TermsAggregationBuilder(TermsAggregationBuilder clone, Builder factori } @Override - protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metadata) { - return new TermsAggregationBuilder(this, factoriesBuilder, metadata); + protected ValuesSourceType defaultValueSourceType() { + return CoreValuesSourceType.BYTES; + } + + @Override + protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metaData) { + return new TermsAggregationBuilder(this, factoriesBuilder, metaData); } /** * Read from a stream. */ public TermsAggregationBuilder(StreamInput in) throws IOException { - super(in, CoreValuesSourceType.ANY); + super(in); bucketCountThresholds = new BucketCountThresholds(in); collectMode = in.readOptionalWriteable(SubAggCollectionMode::readFromStream); executionHint = in.readOptionalString(); @@ -333,10 +342,10 @@ public BucketCardinality bucketCardinality() { } @Override - protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, - ValuesSourceConfig config, - AggregatorFactory parent, - Builder subFactoriesBuilder) throws IOException { + protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, + ValuesSourceConfig config, + AggregatorFactory parent, + Builder subFactoriesBuilder) throws IOException { return new TermsAggregatorFactory(name, config, order, includeExclude, executionHint, collectMode, bucketCountThresholds, showTermDocCountError, queryShardContext, parent, subFactoriesBuilder, metadata); } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorFactory.java index 1118d6e9bb873..4eae7ad1fe249 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorFactory.java @@ -38,16 +38,19 @@ import org.elasticsearch.search.aggregations.bucket.BucketUtils; import org.elasticsearch.search.aggregations.bucket.terms.TermsAggregator.BucketCountThresholds; import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; import java.util.List; import java.util.Map; -public class TermsAggregatorFactory extends ValuesSourceAggregatorFactory { +public class TermsAggregatorFactory extends ValuesSourceAggregatorFactory { private static final DeprecationLogger deprecationLogger = new DeprecationLogger(LogManager.getLogger(TermsAggregatorFactory.class)); static Boolean REMAP_GLOBAL_ORDS, COLLECT_SEGMENT_ORDS; @@ -59,19 +62,142 @@ public class TermsAggregatorFactory extends ValuesSourceAggregatorFactory pipelineAggregators, + Map metaData) throws IOException { + + ExecutionMode execution = null; + if (executionHint != null) { + execution = ExecutionMode.fromString(executionHint, deprecationLogger); + } + // In some cases, using ordinals is just not supported: override it + if (valuesSource instanceof ValuesSource.Bytes.WithOrdinals == false) { + execution = ExecutionMode.MAP; + } + final long maxOrd = execution == ExecutionMode.GLOBAL_ORDINALS ? getMaxOrd(valuesSource, context.searcher()) : -1; + if (execution == null) { + execution = ExecutionMode.GLOBAL_ORDINALS; + } + if (subAggCollectMode == null) { + subAggCollectMode = SubAggCollectionMode.DEPTH_FIRST; + // TODO can we remove concept of AggregatorFactories.EMPTY? + if (factories != AggregatorFactories.EMPTY) { + subAggCollectMode = subAggCollectionMode(bucketCountThresholds.getShardSize(), maxOrd); + } + } + + if ((includeExclude != null) && (includeExclude.isRegexBased()) && format != DocValueFormat.RAW) { + // TODO this exception message is not really accurate for the string case. It's really disallowing regex + formatter + throw new AggregationExecutionException("Aggregation [" + name + "] cannot support regular expression style " + + "include/exclude settings as they can only be applied to string fields. Use an array of values for " + + "include/exclude clauses"); + } + + // TODO: [Zach] we might want refactor and remove ExecutionMode#create(), moving that logic outside the enum + return execution.create(name, factories, valuesSource, order, format, bucketCountThresholds, includeExclude, + context, parent, subAggCollectMode, showTermDocCountError, pipelineAggregators, metaData); + + } + }; + } + + /** + * This supplier is used for all fields that expect to be aggregated as a numeric value. + * This includes floating points, and formatted types that use numerics internally for storage (date, boolean, etc) + */ + private static TermsAggregatorSupplier numericSupplier() { + return new TermsAggregatorSupplier() { + @Override + public Aggregator build(String name, + AggregatorFactories factories, + ValuesSource valuesSource, + BucketOrder order, + DocValueFormat format, + TermsAggregator.BucketCountThresholds bucketCountThresholds, + IncludeExclude includeExclude, + String executionHint, + SearchContext context, + Aggregator parent, + SubAggCollectionMode subAggCollectMode, + boolean showTermDocCountError, + List pipelineAggregators, + Map metaData) throws IOException { + + if ((includeExclude != null) && (includeExclude.isRegexBased())) { + throw new AggregationExecutionException("Aggregation [" + name + "] cannot support regular expression style " + + "include/exclude settings as they can only be applied to string fields. Use an array of numeric values for " + + "include/exclude clauses used to filter numeric fields"); + } + + IncludeExclude.LongFilter longFilter = null; + if (subAggCollectMode == null) { + // TODO can we remove concept of AggregatorFactories.EMPTY? + if (factories != AggregatorFactories.EMPTY) { + subAggCollectMode = subAggCollectionMode(bucketCountThresholds.getShardSize(), -1); + } else { + subAggCollectMode = SubAggCollectionMode.DEPTH_FIRST; + } + } + if (((ValuesSource.Numeric) valuesSource).isFloatingPoint()) { + if (includeExclude != null) { + longFilter = includeExclude.convertToDoubleFilter(); + } + return new DoubleTermsAggregator(name, factories, (ValuesSource.Numeric) valuesSource, format, order, + bucketCountThresholds, context, parent, subAggCollectMode, showTermDocCountError, longFilter, + pipelineAggregators, metaData); + } + if (includeExclude != null) { + longFilter = includeExclude.convertToLongFilter(format); + } + return new LongTermsAggregator(name, factories, (ValuesSource.Numeric) valuesSource, format, order, + bucketCountThresholds, context, parent, subAggCollectMode, showTermDocCountError, longFilter, + pipelineAggregators, metaData); + } + }; + } + TermsAggregatorFactory(String name, - ValuesSourceConfig config, - BucketOrder order, - IncludeExclude includeExclude, - String executionHint, - SubAggCollectionMode collectMode, - TermsAggregator.BucketCountThresholds bucketCountThresholds, - boolean showTermDocCountError, - QueryShardContext queryShardContext, - AggregatorFactory parent, - AggregatorFactories.Builder subFactoriesBuilder, - Map metadata) throws IOException { - super(name, config, queryShardContext, parent, subFactoriesBuilder, metadata); + ValuesSourceConfig config, + BucketOrder order, + IncludeExclude includeExclude, + String executionHint, + SubAggCollectionMode collectMode, + TermsAggregator.BucketCountThresholds bucketCountThresholds, + boolean showTermDocCountError, + QueryShardContext queryShardContext, + AggregatorFactory parent, + AggregatorFactories.Builder subFactoriesBuilder, + Map metaData) throws IOException { + super(name, config, queryShardContext, parent, subFactoriesBuilder, metaData); this.order = order; this.includeExclude = includeExclude; this.executionHint = executionHint; @@ -111,89 +237,36 @@ private static boolean isAggregationSort(BucketOrder order) { @Override protected Aggregator doCreateInternal(ValuesSource valuesSource, - SearchContext searchContext, - Aggregator parent, - boolean collectsFromSingleBucket, - List pipelineAggregators, - Map metadata) throws IOException { + SearchContext searchContext, + Aggregator parent, + boolean collectsFromSingleBucket, + List pipelineAggregators, + Map metaData) throws IOException { if (collectsFromSingleBucket == false) { return asMultiBucketAggregator(this, searchContext, parent); } + + AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), + TermsAggregationBuilder.NAME); + if (aggregatorSupplier instanceof TermsAggregatorSupplier == false) { + throw new AggregationExecutionException("Registry miss-match - expected TermsAggregatorSupplier, found [" + + aggregatorSupplier.getClass().toString() + "]"); + } + + TermsAggregatorSupplier termsAggregatorSupplier = (TermsAggregatorSupplier) aggregatorSupplier; BucketCountThresholds bucketCountThresholds = new BucketCountThresholds(this.bucketCountThresholds); if (InternalOrder.isKeyOrder(order) == false - && bucketCountThresholds.getShardSize() == TermsAggregationBuilder.DEFAULT_BUCKET_COUNT_THRESHOLDS.getShardSize()) { + && bucketCountThresholds.getShardSize() == TermsAggregationBuilder.DEFAULT_BUCKET_COUNT_THRESHOLDS.getShardSize()) { // The user has not made a shardSize selection. Use default // heuristic to avoid any wrong-ranking caused by distributed // counting bucketCountThresholds.setShardSize(BucketUtils.suggestShardSideQueueSize(bucketCountThresholds.getRequiredSize())); } bucketCountThresholds.ensureValidity(); - if (valuesSource instanceof ValuesSource.Bytes) { - ExecutionMode execution = null; - if (executionHint != null) { - execution = ExecutionMode.fromString(executionHint, deprecationLogger); - } - // In some cases, using ordinals is just not supported: override it - if (valuesSource instanceof ValuesSource.Bytes.WithOrdinals == false) { - execution = ExecutionMode.MAP; - } - final long maxOrd = execution == ExecutionMode.GLOBAL_ORDINALS ? getMaxOrd(valuesSource, searchContext.searcher()) : -1; - if (execution == null) { - execution = ExecutionMode.GLOBAL_ORDINALS; - } - SubAggCollectionMode cm = collectMode; - if (cm == null) { - cm = SubAggCollectionMode.DEPTH_FIRST; - if (factories != AggregatorFactories.EMPTY) { - cm = subAggCollectionMode(bucketCountThresholds.getShardSize(), maxOrd); - } - } - - DocValueFormat format = config.format(); - if ((includeExclude != null) && (includeExclude.isRegexBased()) && format != DocValueFormat.RAW) { - throw new AggregationExecutionException("Aggregation [" + name + "] cannot support regular expression style " - + "include/exclude settings as they can only be applied to string fields. Use an array of values for " - + "include/exclude clauses"); - } - - return execution.create(name, factories, valuesSource, order, format, - bucketCountThresholds, includeExclude, searchContext, parent, cm, showTermDocCountError, pipelineAggregators, metadata); - } - - if ((includeExclude != null) && (includeExclude.isRegexBased())) { - throw new AggregationExecutionException("Aggregation [" + name + "] cannot support regular expression style " - + "include/exclude settings as they can only be applied to string fields. Use an array of numeric values for " - + "include/exclude clauses used to filter numeric fields"); - } - - if (valuesSource instanceof ValuesSource.Numeric) { - IncludeExclude.LongFilter longFilter = null; - SubAggCollectionMode cm = collectMode; - if (cm == null) { - if (factories != AggregatorFactories.EMPTY) { - cm = subAggCollectionMode(bucketCountThresholds.getShardSize(), -1); - } else { - cm = SubAggCollectionMode.DEPTH_FIRST; - } - } - if (((ValuesSource.Numeric) valuesSource).isFloatingPoint()) { - if (includeExclude != null) { - longFilter = includeExclude.convertToDoubleFilter(); - } - return new DoubleTermsAggregator(name, factories, (ValuesSource.Numeric) valuesSource, config.format(), order, - bucketCountThresholds, searchContext, parent, cm, showTermDocCountError, longFilter, - pipelineAggregators, metadata); - } - if (includeExclude != null) { - longFilter = includeExclude.convertToLongFilter(config.format()); - } - return new LongTermsAggregator(name, factories, (ValuesSource.Numeric) valuesSource, config.format(), order, - bucketCountThresholds, searchContext, parent, cm, showTermDocCountError, longFilter, pipelineAggregators, - metadata); - } - throw new AggregationExecutionException("terms aggregation cannot be applied to field [" + config.fieldContext().field() - + "]. It can only be applied to numeric or string fields."); + return termsAggregatorSupplier.build(name, factories, valuesSource, order, config.format(), + bucketCountThresholds, includeExclude, executionHint, searchContext, parent, collectMode, + showTermDocCountError, pipelineAggregators, metaData); } // return the SubAggCollectionMode that this aggregation should use based on the expected size @@ -214,7 +287,7 @@ static SubAggCollectionMode subAggCollectionMode(int expectedSize, long maxOrd) * Get the maximum global ordinal value for the provided {@link ValuesSource} or -1 * if the values source is not an instance of {@link ValuesSource.Bytes.WithOrdinals}. */ - static long getMaxOrd(ValuesSource source, IndexSearcher searcher) throws IOException { + private static long getMaxOrd(ValuesSource source, IndexSearcher searcher) throws IOException { if (source instanceof ValuesSource.Bytes.WithOrdinals) { ValuesSource.Bytes.WithOrdinals valueSourceWithOrdinals = (ValuesSource.Bytes.WithOrdinals) source; return valueSourceWithOrdinals.globalMaxOrd(searcher); diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorSupplier.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorSupplier.java new file mode 100644 index 0000000000000..ccfb082ac98ce --- /dev/null +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorSupplier.java @@ -0,0 +1,51 @@ +/* + * Licensed to Elasticsearch under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.elasticsearch.search.aggregations.bucket.terms; + +import org.elasticsearch.search.DocValueFormat; +import org.elasticsearch.search.aggregations.Aggregator; +import org.elasticsearch.search.aggregations.AggregatorFactories; +import org.elasticsearch.search.aggregations.BucketOrder; +import org.elasticsearch.search.aggregations.bucket.terms.IncludeExclude; +import org.elasticsearch.search.aggregations.bucket.terms.TermsAggregator; +import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.ValuesSource; +import org.elasticsearch.search.internal.SearchContext; + +import java.io.IOException; +import java.util.List; +import java.util.Map; + +interface TermsAggregatorSupplier extends AggregatorSupplier { + Aggregator build(String name, + AggregatorFactories factories, + ValuesSource valuesSource, + BucketOrder order, + DocValueFormat format, + TermsAggregator.BucketCountThresholds bucketCountThresholds, + IncludeExclude includeExclude, + String executionHint, + SearchContext context, + Aggregator parent, + Aggregator.SubAggCollectionMode subAggCollectMode, + boolean showTermDocCountError, + List pipelineAggregators, + Map metaData) throws IOException; +} diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/AbstractHDRPercentilesAggregator.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/AbstractHDRPercentilesAggregator.java index 593670b13ca9c..7ad2adda95abf 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/AbstractHDRPercentilesAggregator.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/AbstractHDRPercentilesAggregator.java @@ -26,8 +26,6 @@ import org.elasticsearch.common.util.ArrayUtils; import org.elasticsearch.common.util.BigArrays; import org.elasticsearch.common.util.ObjectArray; -import org.elasticsearch.index.fielddata.HistogramValue; -import org.elasticsearch.index.fielddata.HistogramValues; import org.elasticsearch.index.fielddata.SortedNumericDoubleValues; import org.elasticsearch.search.DocValueFormat; import org.elasticsearch.search.aggregations.Aggregator; @@ -78,18 +76,8 @@ public LeafBucketCollector getLeafCollector(LeafReaderContext ctx, return LeafBucketCollector.NO_OP_COLLECTOR; } final BigArrays bigArrays = context.bigArrays(); - if (valuesSource instanceof ValuesSource.Histogram) { - final HistogramValues values = ((ValuesSource.Histogram)valuesSource).getHistogramValues(ctx); - return collectHistogramValues(values, bigArrays, sub); - } else { - final SortedNumericDoubleValues values = ((ValuesSource.Numeric)valuesSource).doubleValues(ctx); - return collectNumeric(values, bigArrays, sub); - } - - } - private LeafBucketCollector collectNumeric(final SortedNumericDoubleValues values, - final BigArrays bigArrays, final LeafBucketCollector sub) { + final SortedNumericDoubleValues values = ((ValuesSource.Numeric)valuesSource).doubleValues(ctx); return new LeafBucketCollectorBase(sub, values) { @Override public void collect(int doc, long bucket) throws IOException { @@ -102,22 +90,7 @@ public void collect(int doc, long bucket) throws IOException { } } }; - } - private LeafBucketCollector collectHistogramValues(final HistogramValues values, - final BigArrays bigArrays, final LeafBucketCollector sub) { - return new LeafBucketCollectorBase(sub, values) { - @Override - public void collect(int doc, long bucket) throws IOException { - DoubleHistogram state = getExistingOrNewHistogram(bigArrays, bucket); - if (values.advanceExact(doc)) { - final HistogramValue sketch = values.histogram(); - while (sketch.next()) { - state.recordValueWithCount(sketch.value(), sketch.count()); - } - } - } - }; } private DoubleHistogram getExistingOrNewHistogram(final BigArrays bigArrays, long bucket) { @@ -125,12 +98,13 @@ private DoubleHistogram getExistingOrNewHistogram(final BigArrays bigArrays, lon DoubleHistogram state = states.get(bucket); if (state == null) { state = new DoubleHistogram(numberOfSignificantValueDigits); - // Set the histogram to autosize so it can resize itself as - // the data range increases. Resize operations should be - // rare as the histogram buckets are exponential (on the top - // level). In the future we could expose the range as an - // option on the request so the histogram can be fixed at - // initialisation and doesn't need resizing. + /* Set the histogram to autosize so it can resize itself as + the data range increases. Resize operations should be + rare as the histogram buckets are exponential (on the top + level). In the future we could expose the range as an + option on the request so the histogram can be fixed at + initialisation and doesn't need resizing. + */ state.setAutoResize(true); states.set(bucket, state); } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/AbstractPercentilesAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/AbstractPercentilesAggregationBuilder.java index b62f3c7dc7b70..7c44b27356355 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/AbstractPercentilesAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/AbstractPercentilesAggregationBuilder.java @@ -28,11 +28,8 @@ import org.elasticsearch.common.xcontent.ConstructingObjectParser; import org.elasticsearch.common.xcontent.XContentBuilder; import org.elasticsearch.search.aggregations.AggregatorFactories; -import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; -import org.elasticsearch.search.aggregations.support.ValueType; import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; -import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper; import java.io.IOException; import java.util.Arrays; @@ -107,7 +104,7 @@ public static > ConstructingO return ctor.apply(name, values, percentilesConfig); }); - ValuesSourceParserHelper.declareAnyFields(parser, true, true); + ValuesSourceAggregationBuilder.declareFields(parser, true, true, false); parser.declareDoubleArray(ConstructingObjectParser.optionalConstructorArg(), valuesField); parser.declareBoolean(T::keyed, KEYED_FIELD); parser.declareObject(ConstructingObjectParser.optionalConstructorArg(), PercentilesMethod.TDIGEST_PARSER, @@ -120,7 +117,7 @@ public static > ConstructingO AbstractPercentilesAggregationBuilder(String name, double[] values, PercentilesConfig percentilesConfig, ParseField valuesField) { - super(name, CoreValuesSourceType.NUMERIC, ValueType.NUMERIC); + super(name); if (values == null) { throw new IllegalArgumentException("[" + valuesField.getPreferredName() + "] must not be null: [" + name + "]"); } @@ -144,7 +141,7 @@ public static > ConstructingO } AbstractPercentilesAggregationBuilder(StreamInput in) throws IOException { - super(in, CoreValuesSourceType.NUMERIC, ValueType.NUMERIC); + super(in); values = in.readDoubleArray(); keyed = in.readBoolean(); if (in.getVersion().onOrAfter(Version.V_7_8_0)) { diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/AbstractTDigestPercentilesAggregator.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/AbstractTDigestPercentilesAggregator.java index 0de2665897f73..8c16e61fe61a7 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/AbstractTDigestPercentilesAggregator.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/AbstractTDigestPercentilesAggregator.java @@ -25,8 +25,6 @@ import org.elasticsearch.common.util.ArrayUtils; import org.elasticsearch.common.util.BigArrays; import org.elasticsearch.common.util.ObjectArray; -import org.elasticsearch.index.fielddata.HistogramValue; -import org.elasticsearch.index.fielddata.HistogramValues; import org.elasticsearch.index.fielddata.SortedNumericDoubleValues; import org.elasticsearch.search.DocValueFormat; import org.elasticsearch.search.aggregations.Aggregator; @@ -77,18 +75,7 @@ public LeafBucketCollector getLeafCollector(LeafReaderContext ctx, return LeafBucketCollector.NO_OP_COLLECTOR; } final BigArrays bigArrays = context.bigArrays(); - if (valuesSource instanceof ValuesSource.Histogram) { - final HistogramValues values = ((ValuesSource.Histogram)valuesSource).getHistogramValues(ctx); - return collectHistogramValues(values, bigArrays, sub); - } else { - final SortedNumericDoubleValues values = ((ValuesSource.Numeric)valuesSource).doubleValues(ctx); - return collectNumeric(values, bigArrays, sub); - } - - } - - private LeafBucketCollector collectNumeric(final SortedNumericDoubleValues values, - final BigArrays bigArrays, final LeafBucketCollector sub) { + final SortedNumericDoubleValues values = ((ValuesSource.Numeric)valuesSource).doubleValues(ctx); return new LeafBucketCollectorBase(sub, values) { @Override public void collect(int doc, long bucket) throws IOException { @@ -103,22 +90,6 @@ public void collect(int doc, long bucket) throws IOException { }; } - private LeafBucketCollector collectHistogramValues(final HistogramValues values, - final BigArrays bigArrays, final LeafBucketCollector sub) { - return new LeafBucketCollectorBase(sub, values) { - @Override - public void collect(int doc, long bucket) throws IOException { - TDigestState state = getExistingOrNewHistogram(bigArrays, bucket); - if (values.advanceExact(doc)) { - final HistogramValue sketch = values.histogram(); - while(sketch.next()) { - state.add(sketch.value(), sketch.count()); - } - } - } - }; - } - private TDigestState getExistingOrNewHistogram(final BigArrays bigArrays, long bucket) { states = bigArrays.grow(states, bucket + 1); TDigestState state = states.get(bucket); diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/AvgAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/AvgAggregationBuilder.java index e93672493efad..d7e36bc8dfcc5 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/AvgAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/AvgAggregationBuilder.java @@ -28,12 +28,11 @@ import org.elasticsearch.search.aggregations.AggregatorFactories.Builder; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; -import org.elasticsearch.search.aggregations.support.ValueType; import org.elasticsearch.search.aggregations.support.ValuesSource; -import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; -import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.util.Map; @@ -43,22 +42,31 @@ public class AvgAggregationBuilder extends ValuesSourceAggregationBuilder.LeafOn public static final ObjectParser PARSER = ObjectParser.fromBuilder(NAME, AvgAggregationBuilder::new); static { - ValuesSourceParserHelper.declareNumericFields(PARSER, true, true, false); + ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, false); + } + + public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + AvgAggregatorFactory.registerAggregators(valuesSourceRegistry); } public AvgAggregationBuilder(String name) { - super(name, CoreValuesSourceType.NUMERIC, ValueType.NUMERIC); + super(name); } public AvgAggregationBuilder(AvgAggregationBuilder clone, Builder factoriesBuilder, Map metadata) { super(clone, factoriesBuilder, metadata); } + @Override + protected ValuesSourceType defaultValueSourceType() { + return CoreValuesSourceType.NUMERIC; + } + /** * Read from a stream. */ public AvgAggregationBuilder(StreamInput in) throws IOException { - super(in, CoreValuesSourceType.NUMERIC, ValueType.NUMERIC); + super(in); } @Override @@ -72,7 +80,7 @@ protected void innerWriteTo(StreamOutput out) { } @Override - protected AvgAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config, + protected AvgAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config, AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException { return new AvgAggregatorFactory(name, config, queryShardContext, parent, subFactoriesBuilder, metadata); } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/AvgAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/AvgAggregatorFactory.java index 9c88f1aa72320..ca987fede2068 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/AvgAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/AvgAggregatorFactory.java @@ -20,25 +20,47 @@ package org.elasticsearch.search.aggregations.metrics; import org.elasticsearch.index.query.QueryShardContext; +import org.elasticsearch.search.DocValueFormat; +import org.elasticsearch.search.aggregations.AggregationExecutionException; import org.elasticsearch.search.aggregations.Aggregator; import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; import java.util.List; import java.util.Map; -class AvgAggregatorFactory extends ValuesSourceAggregatorFactory { +class AvgAggregatorFactory extends ValuesSourceAggregatorFactory { - AvgAggregatorFactory(String name, ValuesSourceConfig config, QueryShardContext queryShardContext, - AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, Map metadata) throws IOException { - super(name, config, queryShardContext, parent, subFactoriesBuilder, metadata); + AvgAggregatorFactory(String name, ValuesSourceConfig config, QueryShardContext queryShardContext, + AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, + Map metaData) throws IOException { + super(name, config, queryShardContext, parent, subFactoriesBuilder, metaData); + } + + static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + valuesSourceRegistry.register(AvgAggregationBuilder.NAME, + List.of(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN), + new MetricAggregatorSupplier() { + @Override + public Aggregator build(String name, + ValuesSource valuesSource, + DocValueFormat formatter, + SearchContext context, + Aggregator parent, + List pipelineAggregators, Map metaData) throws IOException { + return new AvgAggregator(name, (Numeric) valuesSource, formatter, context, parent, pipelineAggregators, metaData); + } + }); } @Override @@ -50,12 +72,20 @@ protected Aggregator createUnmapped(SearchContext searchContext, } @Override - protected Aggregator doCreateInternal(Numeric valuesSource, + protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, List pipelineAggregators, - Map metadata) throws IOException { - return new AvgAggregator(name, valuesSource, config.format(), searchContext, parent, pipelineAggregators, metadata); + Map metaData) throws IOException { + AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), + AvgAggregationBuilder.NAME); + + if (aggregatorSupplier instanceof MetricAggregatorSupplier == false) { + throw new AggregationExecutionException("Registry miss-match - expected MetricAggregatorSupplier, found [" + + aggregatorSupplier.getClass().toString() + "]"); + } + return ((MetricAggregatorSupplier) aggregatorSupplier).build(name, valuesSource, config.format(), searchContext, parent, + pipelineAggregators, metaData); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/CardinalityAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/CardinalityAggregationBuilder.java index 43002f846b2c1..269383f88ee2a 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/CardinalityAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/CardinalityAggregationBuilder.java @@ -30,11 +30,11 @@ import org.elasticsearch.search.aggregations.AggregatorFactories.Builder; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; -import org.elasticsearch.search.aggregations.support.ValueType; import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; -import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.util.Map; @@ -49,17 +49,21 @@ public final class CardinalityAggregationBuilder public static final ParseField PRECISION_THRESHOLD_FIELD = new ParseField("precision_threshold"); public static final ObjectParser PARSER = - ObjectParser.fromBuilder(NAME, name -> new CardinalityAggregationBuilder(name, null)); + ObjectParser.fromBuilder(NAME, CardinalityAggregationBuilder::new); static { - ValuesSourceParserHelper.declareAnyFields(PARSER, true, false); + ValuesSourceAggregationBuilder.declareFields(PARSER, true, false, false); PARSER.declareLong(CardinalityAggregationBuilder::precisionThreshold, CardinalityAggregationBuilder.PRECISION_THRESHOLD_FIELD); PARSER.declareLong((b, v) -> {/*ignore*/}, REHASH); } + public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + CardinalityAggregatorFactory.registerAggregators(valuesSourceRegistry); + } + private Long precisionThreshold = null; - public CardinalityAggregationBuilder(String name, ValueType targetValueType) { - super(name, CoreValuesSourceType.ANY, targetValueType); + public CardinalityAggregationBuilder(String name) { + super(name); } public CardinalityAggregationBuilder(CardinalityAggregationBuilder clone, Builder factoriesBuilder, Map metadata) { @@ -67,11 +71,16 @@ public CardinalityAggregationBuilder(CardinalityAggregationBuilder clone, Builde this.precisionThreshold = clone.precisionThreshold; } + @Override + protected ValuesSourceType defaultValueSourceType() { + return CoreValuesSourceType.BYTES; + } + /** * Read from a stream. */ public CardinalityAggregationBuilder(StreamInput in) throws IOException { - super(in, CoreValuesSourceType.ANY); + super(in); if (in.readBoolean()) { precisionThreshold = in.readLong(); } @@ -119,7 +128,7 @@ public Long precisionThreshold() { } @Override - protected CardinalityAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config, + protected CardinalityAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config, AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException { return new CardinalityAggregatorFactory(name, config, precisionThreshold, queryShardContext, parent, subFactoriesBuilder, metadata); } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/CardinalityAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/CardinalityAggregatorFactory.java index 64235e6d4bf53..78b06018aa076 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/CardinalityAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/CardinalityAggregatorFactory.java @@ -20,24 +20,27 @@ package org.elasticsearch.search.aggregations.metrics; import org.elasticsearch.index.query.QueryShardContext; +import org.elasticsearch.search.aggregations.AggregationExecutionException; import org.elasticsearch.search.aggregations.Aggregator; import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; import java.util.List; import java.util.Map; -class CardinalityAggregatorFactory extends ValuesSourceAggregatorFactory { +class CardinalityAggregatorFactory extends ValuesSourceAggregatorFactory { private final Long precisionThreshold; - CardinalityAggregatorFactory(String name, ValuesSourceConfig config, + CardinalityAggregatorFactory(String name, ValuesSourceConfig config, Long precisionThreshold, QueryShardContext queryShardContext, AggregatorFactory parent, @@ -47,6 +50,25 @@ class CardinalityAggregatorFactory extends ValuesSourceAggregatorFactory pipelineAggregators, + Map metaData) throws IOException { + return new CardinalityAggregator(name, valuesSource, precision, context, parent, pipelineAggregators, metaData); + } + }; + } + @Override protected Aggregator createUnmapped(SearchContext searchContext, Aggregator parent, @@ -61,9 +83,15 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, Aggregator parent, boolean collectsFromSingleBucket, List pipelineAggregators, - Map metadata) throws IOException { - return new CardinalityAggregator(name, valuesSource, precision(), searchContext, parent, pipelineAggregators, - metadata); + Map metaData) throws IOException { + AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), + CardinalityAggregationBuilder.NAME); + if (aggregatorSupplier instanceof CardinalityAggregatorSupplier == false) { + throw new AggregationExecutionException("Registry miss-match - expected CardinalityAggregatorSupplier, found [" + + aggregatorSupplier.getClass().toString() + "]"); + } + CardinalityAggregatorSupplier cardinalityAggregatorSupplier = (CardinalityAggregatorSupplier) aggregatorSupplier; + return cardinalityAggregatorSupplier.build(name, valuesSource, precision(), searchContext, parent, pipelineAggregators, metaData); } private int precision() { diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/CardinalityAggregatorSupplier.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/CardinalityAggregatorSupplier.java new file mode 100644 index 0000000000000..a27ae4d54c935 --- /dev/null +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/CardinalityAggregatorSupplier.java @@ -0,0 +1,41 @@ +/* + * Licensed to Elasticsearch under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.elasticsearch.search.aggregations.metrics; + +import org.elasticsearch.search.aggregations.Aggregator; +import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.ValuesSource; +import org.elasticsearch.search.internal.SearchContext; + +import java.io.IOException; +import java.util.List; +import java.util.Map; + +public interface CardinalityAggregatorSupplier extends AggregatorSupplier { + Aggregator build(String name, + ValuesSource valuesSource, + int precision, + SearchContext context, + Aggregator parent, + List pipelineAggregators, + Map metaData) throws IOException; + +} diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ExtendedStatsAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ExtendedStatsAggregationBuilder.java index f167942aec6b6..84ea7573c1a06 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ExtendedStatsAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ExtendedStatsAggregationBuilder.java @@ -28,12 +28,10 @@ import org.elasticsearch.search.aggregations.AggregatorFactories.Builder; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; -import org.elasticsearch.search.aggregations.support.ValueType; import org.elasticsearch.search.aggregations.support.ValuesSource; -import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; -import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.util.Map; @@ -46,14 +44,14 @@ public class ExtendedStatsAggregationBuilder public static final ObjectParser PARSER = ObjectParser.fromBuilder(NAME, ExtendedStatsAggregationBuilder::new); static { - ValuesSourceParserHelper.declareNumericFields(PARSER, true, true, false); + ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, false); PARSER.declareDouble(ExtendedStatsAggregationBuilder::sigma, ExtendedStatsAggregator.SIGMA_FIELD); } private double sigma = 2.0; public ExtendedStatsAggregationBuilder(String name) { - super(name, CoreValuesSourceType.NUMERIC, ValueType.NUMERIC); + super(name); } protected ExtendedStatsAggregationBuilder(ExtendedStatsAggregationBuilder clone, @@ -71,10 +69,15 @@ protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map config, + protected ExtendedStatsAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config, AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException { return new ExtendedStatsAggregatorFactory(name, config, sigma, queryShardContext, parent, subFactoriesBuilder, metadata); } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ExtendedStatsAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ExtendedStatsAggregatorFactory.java index 4126ef81fec2e..265ac8af63f40 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ExtendedStatsAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ExtendedStatsAggregatorFactory.java @@ -20,6 +20,7 @@ package org.elasticsearch.search.aggregations.metrics; import org.elasticsearch.index.query.QueryShardContext; +import org.elasticsearch.search.aggregations.AggregationExecutionException; import org.elasticsearch.search.aggregations.Aggregator; import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; @@ -34,12 +35,12 @@ import java.util.List; import java.util.Map; -class ExtendedStatsAggregatorFactory extends ValuesSourceAggregatorFactory { +class ExtendedStatsAggregatorFactory extends ValuesSourceAggregatorFactory { private final double sigma; ExtendedStatsAggregatorFactory(String name, - ValuesSourceConfig config, + ValuesSourceConfig config, double sigma, QueryShardContext queryShardContext, AggregatorFactory parent, @@ -59,13 +60,17 @@ protected Aggregator createUnmapped(SearchContext searchContext, } @Override - protected Aggregator doCreateInternal(Numeric valuesSource, + protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, List pipelineAggregators, - Map metadata) throws IOException { - return new ExtendedStatsAggregator(name, valuesSource, config.format(), searchContext, - parent, sigma, pipelineAggregators, metadata); + Map metaData) throws IOException { + if (valuesSource instanceof Numeric == false) { + throw new AggregationExecutionException("ValuesSource type " + valuesSource.toString() + "is not supported for aggregation " + + this.name()); + } + return new ExtendedStatsAggregator(name, (Numeric) valuesSource, config.format(), searchContext, + parent, sigma, pipelineAggregators, metaData); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoBoundsAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoBoundsAggregationBuilder.java index ac88a17378199..dc6942a4b9d9f 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoBoundsAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoBoundsAggregationBuilder.java @@ -28,30 +28,33 @@ import org.elasticsearch.search.aggregations.AggregatorFactories.Builder; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; -import org.elasticsearch.search.aggregations.support.ValueType; -import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; -import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.util.Map; import java.util.Objects; -public class GeoBoundsAggregationBuilder extends ValuesSourceAggregationBuilder { +public class GeoBoundsAggregationBuilder extends ValuesSourceAggregationBuilder { public static final String NAME = "geo_bounds"; public static final ObjectParser PARSER = ObjectParser.fromBuilder(NAME, GeoBoundsAggregationBuilder::new); static { - ValuesSourceParserHelper.declareGeoFields(PARSER, false, false); + ValuesSourceAggregationBuilder.declareFields(PARSER, false, false, false); PARSER.declareBoolean(GeoBoundsAggregationBuilder::wrapLongitude, GeoBoundsAggregator.WRAP_LONGITUDE_FIELD); } + public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + GeoBoundsAggregatorFactory.registerAggregators(valuesSourceRegistry); + } + private boolean wrapLongitude = true; public GeoBoundsAggregationBuilder(String name) { - super(name, CoreValuesSourceType.GEOPOINT, ValueType.GEOPOINT); + super(name); } protected GeoBoundsAggregationBuilder(GeoBoundsAggregationBuilder clone, Builder factoriesBuilder, Map metadata) { @@ -68,7 +71,7 @@ protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map config, + protected GeoBoundsAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config, AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException { return new GeoBoundsAggregatorFactory(name, config, wrapLongitude, queryShardContext, parent, subFactoriesBuilder, metadata); } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoBoundsAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoBoundsAggregatorFactory.java index 824410d1ad145..537be77211252 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoBoundsAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoBoundsAggregatorFactory.java @@ -20,25 +20,29 @@ package org.elasticsearch.search.aggregations.metrics; import org.elasticsearch.index.query.QueryShardContext; +import org.elasticsearch.search.aggregations.AggregationExecutionException; import org.elasticsearch.search.aggregations.Aggregator; import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; import java.util.List; import java.util.Map; -class GeoBoundsAggregatorFactory extends ValuesSourceAggregatorFactory { +class GeoBoundsAggregatorFactory extends ValuesSourceAggregatorFactory { private final boolean wrapLongitude; GeoBoundsAggregatorFactory(String name, - ValuesSourceConfig config, + ValuesSourceConfig config, boolean wrapLongitude, QueryShardContext queryShardContext, AggregatorFactory parent, @@ -57,12 +61,28 @@ protected Aggregator createUnmapped(SearchContext searchContext, } @Override - protected Aggregator doCreateInternal(ValuesSource.GeoPoint valuesSource, + protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, List pipelineAggregators, - Map metadata) throws IOException { - return new GeoBoundsAggregator(name, searchContext, parent, valuesSource, wrapLongitude, pipelineAggregators, metadata); + Map metaData) throws IOException { + AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry() + .getAggregator(config.valueSourceType(), GeoBoundsAggregationBuilder.NAME); + + if (aggregatorSupplier instanceof GeoBoundsAggregatorSupplier == false) { + throw new AggregationExecutionException("Registry miss-match - expected " + + GeoBoundsAggregatorSupplier.class.getName() + ", found [" + aggregatorSupplier.getClass().toString() + "]"); + } + + return ((GeoBoundsAggregatorSupplier) aggregatorSupplier).build(name, searchContext, parent, valuesSource, wrapLongitude, + pipelineAggregators, metaData); + } + + static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + valuesSourceRegistry.register(GeoBoundsAggregationBuilder.NAME, CoreValuesSourceType.GEOPOINT, + (GeoBoundsAggregatorSupplier) (name, aggregationContext, parent, valuesSource, wrapLongitude, pipelineAggregators, metaData) + -> new GeoBoundsAggregator(name, aggregationContext, parent, (ValuesSource.GeoPoint) valuesSource, + wrapLongitude, pipelineAggregators, metaData)); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoBoundsAggregatorSupplier.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoBoundsAggregatorSupplier.java new file mode 100644 index 0000000000000..4061f2022eb83 --- /dev/null +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoBoundsAggregatorSupplier.java @@ -0,0 +1,38 @@ +/* + * Licensed to Elasticsearch under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.elasticsearch.search.aggregations.metrics; + +import org.elasticsearch.search.aggregations.Aggregator; +import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.ValuesSource; +import org.elasticsearch.search.internal.SearchContext; + +import java.io.IOException; +import java.util.List; +import java.util.Map; + +@FunctionalInterface +public interface GeoBoundsAggregatorSupplier extends AggregatorSupplier { + + GeoBoundsAggregator build(String name, SearchContext aggregationContext, Aggregator parent, + ValuesSource valuesSource, boolean wrapLongitude, List pipelineAggregators, + Map metaData) throws IOException; +} diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregationBuilder.java index 0740b8505fb25..98722f96552e3 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregationBuilder.java @@ -19,9 +19,6 @@ package org.elasticsearch.search.aggregations.metrics; -import java.io.IOException; -import java.util.Map; - import org.elasticsearch.common.io.stream.StreamInput; import org.elasticsearch.common.io.stream.StreamOutput; import org.elasticsearch.common.xcontent.ObjectParser; @@ -31,11 +28,14 @@ import org.elasticsearch.search.aggregations.AggregatorFactories.Builder; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; -import org.elasticsearch.search.aggregations.support.ValueType; import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; -import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; + +import java.io.IOException; +import java.util.Map; public class GeoCentroidAggregationBuilder extends ValuesSourceAggregationBuilder.LeafOnly { @@ -44,11 +44,15 @@ public class GeoCentroidAggregationBuilder public static final ObjectParser PARSER = ObjectParser.fromBuilder(NAME, GeoCentroidAggregationBuilder::new); static { - ValuesSourceParserHelper.declareGeoFields(PARSER, true, false); + ValuesSourceAggregationBuilder.declareFields(PARSER, true, false, false); + } + + public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + GeoCentroidAggregatorFactory.registerAggregators(valuesSourceRegistry); } public GeoCentroidAggregationBuilder(String name) { - super(name, CoreValuesSourceType.GEOPOINT, ValueType.GEOPOINT); + super(name); } protected GeoCentroidAggregationBuilder(GeoCentroidAggregationBuilder clone, Builder factoriesBuilder, Map metadata) { @@ -56,15 +60,20 @@ protected GeoCentroidAggregationBuilder(GeoCentroidAggregationBuilder clone, Bui } @Override - protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metadata) { - return new GeoCentroidAggregationBuilder(this, factoriesBuilder, metadata); + protected ValuesSourceType defaultValueSourceType() { + return CoreValuesSourceType.GEOPOINT; + } + + @Override + protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metaData) { + return new GeoCentroidAggregationBuilder(this, factoriesBuilder, metaData); } /** * Read from a stream. */ public GeoCentroidAggregationBuilder(StreamInput in) throws IOException { - super(in, CoreValuesSourceType.GEOPOINT, ValueType.GEOPOINT); + super(in); } @Override @@ -73,7 +82,7 @@ protected void innerWriteTo(StreamOutput out) { } @Override - protected GeoCentroidAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config, + protected GeoCentroidAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config, AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException { return new GeoCentroidAggregatorFactory(name, config, queryShardContext, parent, subFactoriesBuilder, metadata); } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregatorFactory.java index bb50d93aa5677..61f98c79451ec 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregatorFactory.java @@ -20,23 +20,27 @@ package org.elasticsearch.search.aggregations.metrics; import org.elasticsearch.index.query.QueryShardContext; +import org.elasticsearch.search.aggregations.AggregationExecutionException; import org.elasticsearch.search.aggregations.Aggregator; import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; import java.util.List; import java.util.Map; -class GeoCentroidAggregatorFactory extends ValuesSourceAggregatorFactory { +class GeoCentroidAggregatorFactory extends ValuesSourceAggregatorFactory { GeoCentroidAggregatorFactory(String name, - ValuesSourceConfig config, + ValuesSourceConfig config, QueryShardContext queryShardContext, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, @@ -53,12 +57,26 @@ protected Aggregator createUnmapped(SearchContext searchContext, } @Override - protected Aggregator doCreateInternal(ValuesSource.GeoPoint valuesSource, + protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, List pipelineAggregators, - Map metadata) throws IOException { - return new GeoCentroidAggregator(name, searchContext, parent, valuesSource, pipelineAggregators, metadata); + Map metaData) throws IOException { + + AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), + GeoCentroidAggregationBuilder.NAME); + if (aggregatorSupplier instanceof GeoCentroidAggregatorSupplier == false) { + throw new AggregationExecutionException("Registry miss-match - expected " + + GeoCentroidAggregatorSupplier.class.getName() + ", found [" + aggregatorSupplier.getClass().toString() + "]"); + } + return ((GeoCentroidAggregatorSupplier) aggregatorSupplier).build(name, searchContext, parent, valuesSource, + pipelineAggregators, metaData); + } + + static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + valuesSourceRegistry.register(GeoCentroidAggregationBuilder.NAME, CoreValuesSourceType.GEOPOINT, + (GeoCentroidAggregatorSupplier) (name, context, parent, valuesSource, pipelineAggregators, metaData) -> + new GeoCentroidAggregator(name, context, parent, (ValuesSource.GeoPoint) valuesSource, pipelineAggregators, metaData)); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregatorSupplier.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregatorSupplier.java new file mode 100644 index 0000000000000..7c1d34ae20977 --- /dev/null +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregatorSupplier.java @@ -0,0 +1,38 @@ +/* + * Licensed to Elasticsearch under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.elasticsearch.search.aggregations.metrics; + +import org.elasticsearch.search.aggregations.Aggregator; +import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.ValuesSource; +import org.elasticsearch.search.internal.SearchContext; + +import java.io.IOException; +import java.util.List; +import java.util.Map; + +@FunctionalInterface +public interface GeoCentroidAggregatorSupplier extends AggregatorSupplier { + + GeoCentroidAggregator build(String name, SearchContext context, Aggregator parent, + ValuesSource valuesSource, List pipelineAggregators, + Map metaData) throws IOException; +} diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoGridAggregatorSupplier.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoGridAggregatorSupplier.java new file mode 100644 index 0000000000000..4b08afbc5977e --- /dev/null +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoGridAggregatorSupplier.java @@ -0,0 +1,42 @@ +/* + * Licensed to Elasticsearch under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +package org.elasticsearch.search.aggregations.metrics; + +import org.elasticsearch.common.geo.GeoBoundingBox; +import org.elasticsearch.search.aggregations.Aggregator; +import org.elasticsearch.search.aggregations.AggregatorFactories; +import org.elasticsearch.search.aggregations.bucket.geogrid.GeoGridAggregator; +import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.ValuesSource; +import org.elasticsearch.search.internal.SearchContext; + +import java.io.IOException; +import java.util.List; +import java.util.Map; + +@FunctionalInterface +public interface GeoGridAggregatorSupplier extends AggregatorSupplier { + + GeoGridAggregator build(String name, AggregatorFactories factories, ValuesSource valuesSource, + int precision, GeoBoundingBox geoBoundingBox, int requiredSize, int shardSize, + SearchContext aggregationContext, Aggregator parent, List pipelineAggregators, + Map metaData) throws IOException; +} diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalHDRPercentileRanks.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalHDRPercentileRanks.java index 2a5e5a97ac630..891a10c847e23 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalHDRPercentileRanks.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalHDRPercentileRanks.java @@ -29,9 +29,9 @@ public class InternalHDRPercentileRanks extends AbstractInternalHDRPercentiles implements PercentileRanks { public static final String NAME = "hdr_percentile_ranks"; - InternalHDRPercentileRanks(String name, double[] cdfValues, DoubleHistogram state, boolean keyed, DocValueFormat formatter, - Map metadata) { - super(name, cdfValues, state, keyed, formatter, metadata); + public InternalHDRPercentileRanks(String name, double[] cdfValues, DoubleHistogram state, boolean keyed, DocValueFormat formatter, + List pipelineAggregators, Map metaData) { + super(name, cdfValues, state, keyed, formatter, pipelineAggregators, metaData); } /** @@ -72,7 +72,7 @@ protected AbstractInternalHDRPercentiles createReduced(String name, double[] key return new InternalHDRPercentileRanks(name, keys, merged, keyed, format, metadata); } - static double percentileRank(DoubleHistogram state, double value) { + public static double percentileRank(DoubleHistogram state, double value) { if (state.getTotalCount() == 0) { return Double.NaN; } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalHDRPercentiles.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalHDRPercentiles.java index a53d162ef9a62..9b34c5c71e159 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalHDRPercentiles.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalHDRPercentiles.java @@ -29,9 +29,9 @@ public class InternalHDRPercentiles extends AbstractInternalHDRPercentiles implements Percentiles { public static final String NAME = "hdr_percentiles"; - InternalHDRPercentiles(String name, double[] percents, DoubleHistogram state, boolean keyed, DocValueFormat formatter, - Map metadata) { - super(name, percents, state, keyed, formatter, metadata); + public InternalHDRPercentiles(String name, double[] percents, DoubleHistogram state, boolean keyed, DocValueFormat formatter, + List pipelineAggregators, Map metaData) { + super(name, percents, state, keyed, formatter, pipelineAggregators, metaData); } /** diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalTDigestPercentileRanks.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalTDigestPercentileRanks.java index 87bc09c5588cf..9a1bd95720676 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalTDigestPercentileRanks.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalTDigestPercentileRanks.java @@ -30,9 +30,9 @@ public class InternalTDigestPercentileRanks extends AbstractInternalTDigestPercentiles implements PercentileRanks { public static final String NAME = "tdigest_percentile_ranks"; - InternalTDigestPercentileRanks(String name, double[] cdfValues, TDigestState state, boolean keyed, DocValueFormat formatter, - List pipelineAggregators, Map metadata) { - super(name, cdfValues, state, keyed, formatter, pipelineAggregators, metadata); + public InternalTDigestPercentileRanks(String name, double[] cdfValues, TDigestState state, boolean keyed, DocValueFormat formatter, + List pipelineAggregators, Map metaData) { + super(name, cdfValues, state, keyed, formatter, pipelineAggregators, metaData); } /** @@ -73,7 +73,7 @@ protected AbstractInternalTDigestPercentiles createReduced(String name, double[] return new InternalTDigestPercentileRanks(name, keys, merged, keyed, format, pipelineAggregators, metadata); } - static double percentileRank(TDigestState state, double value) { + public static double percentileRank(TDigestState state, double value) { double percentileRank = state.cdf(value); if (percentileRank < 0) { percentileRank = 0; diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalTDigestPercentiles.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalTDigestPercentiles.java index 8ec58ec43e800..3f8a75c6dd5cb 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalTDigestPercentiles.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalTDigestPercentiles.java @@ -30,9 +30,9 @@ public class InternalTDigestPercentiles extends AbstractInternalTDigestPercentiles implements Percentiles { public static final String NAME = "tdigest_percentiles"; - InternalTDigestPercentiles(String name, double[] percents, TDigestState state, boolean keyed, DocValueFormat formatter, - List pipelineAggregators, Map metadata) { - super(name, percents, state, keyed, formatter, pipelineAggregators, metadata); + public InternalTDigestPercentiles(String name, double[] percents, TDigestState state, boolean keyed, DocValueFormat formatter, + List pipelineAggregators, Map metaData) { + super(name, percents, state, keyed, formatter, pipelineAggregators, metaData); } /** diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MaxAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MaxAggregationBuilder.java index ada55e762329d..b8d8cb2799a7c 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MaxAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MaxAggregationBuilder.java @@ -28,12 +28,11 @@ import org.elasticsearch.search.aggregations.AggregatorFactories.Builder; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; -import org.elasticsearch.search.aggregations.support.ValueType; import org.elasticsearch.search.aggregations.support.ValuesSource; -import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; -import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.util.Map; @@ -43,11 +42,15 @@ public class MaxAggregationBuilder extends ValuesSourceAggregationBuilder.LeafOn public static final ObjectParser PARSER = ObjectParser.fromBuilder(NAME, MaxAggregationBuilder::new); static { - ValuesSourceParserHelper.declareNumericFields(PARSER, true, true, false); + ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, false); + } + + public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + MaxAggregatorFactory.registerAggregators(valuesSourceRegistry); } public MaxAggregationBuilder(String name) { - super(name, CoreValuesSourceType.NUMERIC, ValueType.NUMERIC); + super(name); } protected MaxAggregationBuilder(MaxAggregationBuilder clone, Builder factoriesBuilder, Map metadata) { @@ -55,15 +58,20 @@ protected MaxAggregationBuilder(MaxAggregationBuilder clone, Builder factoriesBu } @Override - protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metadata) { - return new MaxAggregationBuilder(this, factoriesBuilder, metadata); + protected ValuesSourceType defaultValueSourceType() { + return CoreValuesSourceType.NUMERIC; + } + + @Override + protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metaData) { + return new MaxAggregationBuilder(this, factoriesBuilder, metaData); } /** * Read from a stream. */ public MaxAggregationBuilder(StreamInput in) throws IOException { - super(in, CoreValuesSourceType.NUMERIC, ValueType.NUMERIC); + super(in); } @Override @@ -72,7 +80,7 @@ protected void innerWriteTo(StreamOutput out) { } @Override - protected MaxAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config, + protected MaxAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config, AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException { return new MaxAggregatorFactory(name, config, queryShardContext, parent, subFactoriesBuilder, metadata); } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MaxAggregator.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MaxAggregator.java index f2fb9cd081613..508d285a92675 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MaxAggregator.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MaxAggregator.java @@ -59,7 +59,7 @@ class MaxAggregator extends NumericMetricsAggregator.SingleValue { DoubleArray maxes; MaxAggregator(String name, - ValuesSourceConfig config, + ValuesSourceConfig config, ValuesSource.Numeric valuesSource, SearchContext context, Aggregator parent, List pipelineAggregators, diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MaxAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MaxAggregatorFactory.java index a84225bb75540..24073faf025fe 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MaxAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MaxAggregatorFactory.java @@ -20,25 +20,47 @@ package org.elasticsearch.search.aggregations.metrics; import org.elasticsearch.index.query.QueryShardContext; +import org.elasticsearch.search.aggregations.AggregationExecutionException; import org.elasticsearch.search.aggregations.Aggregator; import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; import java.util.List; import java.util.Map; -class MaxAggregatorFactory extends ValuesSourceAggregatorFactory { +class MaxAggregatorFactory extends ValuesSourceAggregatorFactory { - MaxAggregatorFactory(String name, ValuesSourceConfig config, QueryShardContext queryShardContext, - AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, Map metadata) throws IOException { - super(name, config, queryShardContext, parent, subFactoriesBuilder, metadata); + static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + valuesSourceRegistry.register(MaxAggregationBuilder.NAME, + List.of(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN), + new MinMaxAggregatorSupplier() { + @Override + public Aggregator build(String name, + ValuesSourceConfig config, + ValuesSource valuesSource, + SearchContext context, + Aggregator parent, + List pipelineAggregators, + Map metaData) throws IOException { + return new MaxAggregator(name, config, (Numeric) valuesSource, context, parent, pipelineAggregators, metaData); + } + }); + } + + MaxAggregatorFactory(String name, ValuesSourceConfig config, QueryShardContext queryShardContext, + AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, + Map metaData) throws IOException { + super(name, config, queryShardContext, parent, subFactoriesBuilder, metaData); } @Override @@ -50,12 +72,20 @@ protected Aggregator createUnmapped(SearchContext searchContext, } @Override - protected Aggregator doCreateInternal(Numeric valuesSource, + protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, List pipelineAggregators, - Map metadata) throws IOException { - return new MaxAggregator(name, config, valuesSource, searchContext, parent, pipelineAggregators, metadata); + Map metaData) throws IOException { + AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), + MaxAggregationBuilder.NAME); + + if (aggregatorSupplier instanceof MinMaxAggregatorSupplier == false) { + throw new AggregationExecutionException("Registry miss-match - expected MinMaxAggregatorSupplier, found [" + + aggregatorSupplier.getClass().toString() + "]"); + } + return ((MinMaxAggregatorSupplier) aggregatorSupplier).build(name, config, valuesSource, searchContext, parent, + pipelineAggregators, metaData); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregationBuilder.java index bf70a3726cbe9..a03042581378d 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregationBuilder.java @@ -29,12 +29,13 @@ import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; -import org.elasticsearch.search.aggregations.support.ValueType; import org.elasticsearch.search.aggregations.support.ValuesSource; +import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder.LeafOnly; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; -import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.util.Map; @@ -49,18 +50,22 @@ public class MedianAbsoluteDeviationAggregationBuilder extends LeafOnly PARSER = ObjectParser.fromBuilder(NAME, MedianAbsoluteDeviationAggregationBuilder::new); static { - ValuesSourceParserHelper.declareNumericFields(PARSER, true, true, false); + ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, false); PARSER.declareDouble(MedianAbsoluteDeviationAggregationBuilder::compression, COMPRESSION_FIELD); } + public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + MedianAbsoluteDeviationAggregatorFactory.registerAggregators(valuesSourceRegistry); + } + private double compression = 1000d; public MedianAbsoluteDeviationAggregationBuilder(String name) { - super(name, CoreValuesSourceType.NUMERIC, ValueType.NUMERIC); + super(name); } public MedianAbsoluteDeviationAggregationBuilder(StreamInput in) throws IOException { - super(in, CoreValuesSourceType.NUMERIC, ValueType.NUMERIC); + super(in); compression = in.readDouble(); } @@ -95,16 +100,21 @@ protected AggregationBuilder shallowCopy(AggregatorFactories.Builder factoriesBu return new MedianAbsoluteDeviationAggregationBuilder(this, factoriesBuilder, metadata); } + @Override + protected ValuesSourceType defaultValueSourceType() { + return CoreValuesSourceType.NUMERIC; + } + @Override protected void innerWriteTo(StreamOutput out) throws IOException { out.writeDouble(compression); } @Override - protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, - ValuesSourceConfig config, - AggregatorFactory parent, - AggregatorFactories.Builder subFactoriesBuilder) + protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, + ValuesSourceConfig config, + AggregatorFactory parent, + AggregatorFactories.Builder subFactoriesBuilder) throws IOException { return new MedianAbsoluteDeviationAggregatorFactory(name, config, queryShardContext, parent, subFactoriesBuilder, metadata, compression); diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregatorFactory.java index 90dcd4d7ab96a..4725a8db78b1c 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregatorFactory.java @@ -20,40 +20,72 @@ package org.elasticsearch.search.aggregations.metrics; import org.elasticsearch.index.query.QueryShardContext; +import org.elasticsearch.search.DocValueFormat; +import org.elasticsearch.search.aggregations.AggregationExecutionException; import org.elasticsearch.search.aggregations.Aggregator; import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; import java.util.List; import java.util.Map; -public class MedianAbsoluteDeviationAggregatorFactory extends ValuesSourceAggregatorFactory { +public class MedianAbsoluteDeviationAggregatorFactory extends ValuesSourceAggregatorFactory { private final double compression; MedianAbsoluteDeviationAggregatorFactory(String name, - ValuesSourceConfig config, - QueryShardContext queryShardContext, - AggregatorFactory parent, - AggregatorFactories.Builder subFactoriesBuilder, - Map metadata, - double compression) throws IOException { + ValuesSourceConfig config, + QueryShardContext queryShardContext, + AggregatorFactory parent, + AggregatorFactories.Builder subFactoriesBuilder, + Map metaData, + double compression) throws IOException { super(name, config, queryShardContext, parent, subFactoriesBuilder, metadata); this.compression = compression; } + static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + valuesSourceRegistry.register(MedianAbsoluteDeviationAggregationBuilder.NAME, + CoreValuesSourceType.NUMERIC, + new MedianAbsoluteDeviationAggregatorSupplier() { + @Override + public Aggregator build(String name, + ValuesSource valuesSource, + DocValueFormat format, + SearchContext context, + Aggregator parent, + List pipelineAggregators, + Map metaData, + double compression) throws IOException { + return new MedianAbsoluteDeviationAggregator( + name, + context, + parent, + pipelineAggregators, + metaData, + (ValuesSource.Numeric) valuesSource, + format, + compression + ); + } + }); + } + @Override protected Aggregator createUnmapped(SearchContext searchContext, - Aggregator parent, - List pipelineAggregators, - Map metadata) throws IOException { + Aggregator parent, + List pipelineAggregators, + Map metaData) throws IOException { return new MedianAbsoluteDeviationAggregator( name, @@ -68,22 +100,20 @@ protected Aggregator createUnmapped(SearchContext searchContext, } @Override - protected Aggregator doCreateInternal(ValuesSource.Numeric valuesSource, + protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, List pipelineAggregators, - Map metadata) throws IOException { + Map metaData) throws IOException { + AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), + MedianAbsoluteDeviationAggregationBuilder.NAME); - return new MedianAbsoluteDeviationAggregator( - name, - searchContext, - parent, - pipelineAggregators, - metadata, - valuesSource, - config.format(), - compression - ); + if (aggregatorSupplier instanceof MedianAbsoluteDeviationAggregatorSupplier == false) { + throw new AggregationExecutionException("Registry miss-match - expected MedianAbsoluteDeviationAggregatorSupplier, found [" + + aggregatorSupplier.getClass().toString() + "]"); + } + return ((MedianAbsoluteDeviationAggregatorSupplier) aggregatorSupplier).build(name, valuesSource, config.format(), + searchContext, parent, pipelineAggregators, metaData, compression); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregatorSupplier.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregatorSupplier.java new file mode 100644 index 0000000000000..4451f731487a8 --- /dev/null +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregatorSupplier.java @@ -0,0 +1,41 @@ +/* + * Licensed to Elasticsearch under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.elasticsearch.search.aggregations.metrics; + +import org.elasticsearch.search.DocValueFormat; +import org.elasticsearch.search.aggregations.Aggregator; +import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.ValuesSource; +import org.elasticsearch.search.internal.SearchContext; + +import java.io.IOException; +import java.util.List; +import java.util.Map; + +public interface MedianAbsoluteDeviationAggregatorSupplier extends AggregatorSupplier { + Aggregator build(String name, + ValuesSource valuesSource, + DocValueFormat format, + SearchContext context, + Aggregator parent, + List pipelineAggregators, + Map metaData, + double compression) throws IOException; +} diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MetricAggregatorSupplier.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MetricAggregatorSupplier.java new file mode 100644 index 0000000000000..4aa2136f03fbc --- /dev/null +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MetricAggregatorSupplier.java @@ -0,0 +1,40 @@ +/* + * Licensed to Elasticsearch under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.elasticsearch.search.aggregations.metrics; + +import org.elasticsearch.search.DocValueFormat; +import org.elasticsearch.search.aggregations.Aggregator; +import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.ValuesSource; +import org.elasticsearch.search.internal.SearchContext; + +import java.io.IOException; +import java.util.List; +import java.util.Map; + +public interface MetricAggregatorSupplier extends AggregatorSupplier { + Aggregator build(String name, + ValuesSource valuesSource, + DocValueFormat format, + SearchContext context, + Aggregator parent, + List pipelineAggregators, + Map metaData) throws IOException; +} diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MinAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MinAggregationBuilder.java index e3102b7b8af5d..aed10932f9f89 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MinAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MinAggregationBuilder.java @@ -28,12 +28,11 @@ import org.elasticsearch.search.aggregations.AggregatorFactories.Builder; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; -import org.elasticsearch.search.aggregations.support.ValueType; import org.elasticsearch.search.aggregations.support.ValuesSource; -import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; -import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.util.Map; @@ -43,11 +42,11 @@ public class MinAggregationBuilder extends ValuesSourceAggregationBuilder.LeafOn public static final ObjectParser PARSER = ObjectParser.fromBuilder(NAME, MinAggregationBuilder::new); static { - ValuesSourceParserHelper.declareNumericFields(PARSER, true, true, false); + ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, false); } public MinAggregationBuilder(String name) { - super(name, CoreValuesSourceType.NUMERIC, ValueType.NUMERIC); + super(name); } protected MinAggregationBuilder(MinAggregationBuilder clone, Builder factoriesBuilder, Map metadata) { @@ -59,11 +58,20 @@ protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map config, + protected MinAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config, AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException { return new MinAggregatorFactory(name, config, queryShardContext, parent, subFactoriesBuilder, metadata); } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MinAggregator.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MinAggregator.java index 8aff0ce230613..c289b06b35164 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MinAggregator.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MinAggregator.java @@ -62,7 +62,7 @@ class MinAggregator extends NumericMetricsAggregator.SingleValue { DoubleArray mins; MinAggregator(String name, - ValuesSourceConfig config, + ValuesSourceConfig config, ValuesSource.Numeric valuesSource, SearchContext context, Aggregator parent, @@ -172,7 +172,7 @@ public void doClose() { * @param config The config for the values source metric. */ static Function getPointReaderOrNull(SearchContext context, Aggregator parent, - ValuesSourceConfig config) { + ValuesSourceConfig config) { if (context.query() != null && context.query().getClass() != MatchAllDocsQuery.class) { return null; diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MinAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MinAggregatorFactory.java index 5c08b171c28a5..3b48768ad156a 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MinAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MinAggregatorFactory.java @@ -20,25 +20,47 @@ package org.elasticsearch.search.aggregations.metrics; import org.elasticsearch.index.query.QueryShardContext; +import org.elasticsearch.search.aggregations.AggregationExecutionException; import org.elasticsearch.search.aggregations.Aggregator; import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; import java.util.List; import java.util.Map; -class MinAggregatorFactory extends ValuesSourceAggregatorFactory { +class MinAggregatorFactory extends ValuesSourceAggregatorFactory { - MinAggregatorFactory(String name, ValuesSourceConfig config, QueryShardContext queryShardContext, - AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, Map metadata) throws IOException { - super(name, config, queryShardContext, parent, subFactoriesBuilder, metadata); + static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + valuesSourceRegistry.register(MinAggregationBuilder.NAME, + List.of(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN), + new MinMaxAggregatorSupplier() { + @Override + public Aggregator build(String name, + ValuesSourceConfig config, + ValuesSource valuesSource, + SearchContext context, + Aggregator parent, + List pipelineAggregators, + Map metaData) throws IOException { + return new MinAggregator(name, config, (Numeric) valuesSource, context, parent, pipelineAggregators, metaData); + } + }); + } + + MinAggregatorFactory(String name, ValuesSourceConfig config, QueryShardContext queryShardContext, + AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, + Map metaData) throws IOException { + super(name, config, queryShardContext, parent, subFactoriesBuilder, metaData); } @Override @@ -50,12 +72,20 @@ protected Aggregator createUnmapped(SearchContext searchContext, } @Override - protected Aggregator doCreateInternal(Numeric valuesSource, + protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, List pipelineAggregators, - Map metadata) throws IOException { - return new MinAggregator(name, config, valuesSource, searchContext, parent, pipelineAggregators, metadata); + Map metaData) throws IOException { + AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), + MinAggregationBuilder.NAME); + + if (aggregatorSupplier instanceof MinMaxAggregatorSupplier == false) { + throw new AggregationExecutionException("Registry miss-match - expected MinMaxAggregatorSupplier, found [" + + aggregatorSupplier.getClass().toString() + "]"); + } + return ((MinMaxAggregatorSupplier) aggregatorSupplier).build(name, config, valuesSource, searchContext, parent, + pipelineAggregators, metaData); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MinMaxAggregatorSupplier.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MinMaxAggregatorSupplier.java new file mode 100644 index 0000000000000..51e4c4d95bd1a --- /dev/null +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MinMaxAggregatorSupplier.java @@ -0,0 +1,40 @@ +/* + * Licensed to Elasticsearch under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.elasticsearch.search.aggregations.metrics; + +import org.elasticsearch.search.aggregations.Aggregator; +import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.ValuesSource; +import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.internal.SearchContext; + +import java.io.IOException; +import java.util.List; +import java.util.Map; + +public interface MinMaxAggregatorSupplier extends AggregatorSupplier { + Aggregator build(String name, + ValuesSourceConfig config, + ValuesSource valuesSource, + SearchContext context, + Aggregator parent, + List pipelineAggregators, + Map metaData) throws IOException; +} diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentileRanksAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentileRanksAggregationBuilder.java index 794ece846fcc6..a80f2a58d5f0a 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentileRanksAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentileRanksAggregationBuilder.java @@ -27,9 +27,11 @@ import org.elasticsearch.search.aggregations.AggregationBuilder; import org.elasticsearch.search.aggregations.AggregatorFactories.Builder; import org.elasticsearch.search.aggregations.AggregatorFactory; -import org.elasticsearch.search.aggregations.support.ValuesSource; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.util.Map; @@ -51,6 +53,10 @@ public static AggregationBuilder parse(String aggregationName, XContentParser pa return PARSER.parse(parser, aggregationName); } + public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + PercentileRanksAggregatorFactory.registerAggregators(valuesSourceRegistry); + } + public PercentileRanksAggregationBuilder(String name, double[] values) { this(name, values, null); } @@ -74,6 +80,11 @@ protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map innerBuild(QueryShardContext queryShardContext, - ValuesSourceConfig config, + protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, + ValuesSourceConfig config, AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException { return new PercentileRanksAggregatorFactory(name, config, values, configOrDefault(), keyed, queryShardContext, diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentileRanksAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentileRanksAggregatorFactory.java index 1886a9e44d3b6..748743093c969 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentileRanksAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentileRanksAggregatorFactory.java @@ -20,27 +20,48 @@ package org.elasticsearch.search.aggregations.metrics; import org.elasticsearch.index.query.QueryShardContext; +import org.elasticsearch.search.DocValueFormat; +import org.elasticsearch.search.aggregations.AggregationExecutionException; import org.elasticsearch.search.aggregations.Aggregator; import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; import java.util.List; import java.util.Map; -class PercentileRanksAggregatorFactory extends ValuesSourceAggregatorFactory { +class PercentileRanksAggregatorFactory extends ValuesSourceAggregatorFactory { private final double[] percents; private final PercentilesConfig percentilesConfig; private final boolean keyed; + static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + valuesSourceRegistry.register(PercentileRanksAggregationBuilder.NAME, + List.of(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN), + new PercentilesAggregatorSupplier() { + @Override + public Aggregator build(String name, ValuesSource valuesSource, SearchContext context, Aggregator parent, + double[] percents, PercentilesConfig percentilesConfig, boolean keyed, DocValueFormat formatter, + List pipelineAggregators, Map metaData) throws IOException { + + return percentilesConfig.createPercentileRanksAggregator(name, valuesSource, context, parent, percents, keyed, + formatter, pipelineAggregators, metaData); + } + } + ); + } + PercentileRanksAggregatorFactory(String name, - ValuesSourceConfig config, + ValuesSourceConfig config, double[] percents, PercentilesConfig percentilesConfig, boolean keyed, @@ -70,8 +91,16 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, Aggregator parent, boolean collectsFromSingleBucket, List pipelineAggregators, - Map metadata) throws IOException { - return percentilesConfig.createPercentileRanksAggregator(name, valuesSource, searchContext, parent, percents, keyed, - config.format(), pipelineAggregators, metadata); + Map metaData) throws IOException { + AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), + PercentileRanksAggregationBuilder.NAME); + + if (aggregatorSupplier instanceof PercentilesAggregatorSupplier == false) { + throw new AggregationExecutionException("Registry miss-match - expected PercentilesAggregatorSupplier, found [" + + aggregatorSupplier.getClass().toString() + "]"); + } + PercentilesAggregatorSupplier percentilesAggregatorSupplier = (PercentilesAggregatorSupplier) aggregatorSupplier; + return percentilesAggregatorSupplier.build(name, valuesSource, searchContext, parent, percents, percentilesConfig, keyed, + config.format(), pipelineAggregators, metaData); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesAggregationBuilder.java index 5f4aa7a25c67a..230d65ba7c04f 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesAggregationBuilder.java @@ -27,9 +27,11 @@ import org.elasticsearch.search.aggregations.AggregationBuilder; import org.elasticsearch.search.aggregations.AggregatorFactories.Builder; import org.elasticsearch.search.aggregations.AggregatorFactory; -import org.elasticsearch.search.aggregations.support.ValuesSource; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.util.Arrays; @@ -57,6 +59,10 @@ public class PercentilesAggregationBuilder extends AbstractPercentilesAggregatio PERCENTS_FIELD); } + public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + PercentilesAggregatorFactory.registerAggregators(valuesSourceRegistry); + } + public PercentilesAggregationBuilder(StreamInput in) throws IOException { super(in); } @@ -83,6 +89,11 @@ protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map innerBuild(QueryShardContext queryShardContext, - ValuesSourceConfig config, + protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, + ValuesSourceConfig config, AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException { return new PercentilesAggregatorFactory(name, config, values, configOrDefault(), keyed, diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesAggregatorFactory.java index f7e773c91e3f3..44aa22a325248 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesAggregatorFactory.java @@ -20,13 +20,18 @@ package org.elasticsearch.search.aggregations.metrics; import org.elasticsearch.index.query.QueryShardContext; +import org.elasticsearch.search.DocValueFormat; +import org.elasticsearch.search.aggregations.AggregationExecutionException; import org.elasticsearch.search.aggregations.Aggregator; import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; @@ -37,13 +42,29 @@ * This factory is used to generate both TDigest and HDRHisto aggregators, depending * on the selected method */ -class PercentilesAggregatorFactory extends ValuesSourceAggregatorFactory { +class PercentilesAggregatorFactory extends ValuesSourceAggregatorFactory { private final double[] percents; private final PercentilesConfig percentilesConfig; private final boolean keyed; - PercentilesAggregatorFactory(String name, ValuesSourceConfig config, double[] percents, + static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + valuesSourceRegistry.register(PercentilesAggregationBuilder.NAME, + List.of(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN), + new PercentilesAggregatorSupplier() { + @Override + public Aggregator build(String name, ValuesSource valuesSource, SearchContext context, Aggregator parent, + double[] percents, PercentilesConfig percentilesConfig, boolean keyed, DocValueFormat formatter, + List pipelineAggregators, Map metaData) throws IOException { + + return percentilesConfig.createPercentilesAggregator(name, valuesSource, context, parent, percents, keyed, + formatter, pipelineAggregators, metaData); + } + } + ); + } + + PercentilesAggregatorFactory(String name, ValuesSourceConfig config, double[] percents, PercentilesConfig percentilesConfig, boolean keyed, QueryShardContext queryShardContext, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, Map metadata) throws IOException { @@ -71,7 +92,15 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, List pipelineAggregators, Map metadata) throws IOException { - return percentilesConfig.createPercentilesAggregator(name, valuesSource, searchContext, parent, percents, keyed, - config.format(), pipelineAggregators, metadata); + AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), + PercentilesAggregationBuilder.NAME); + + if (aggregatorSupplier instanceof PercentilesAggregatorSupplier == false) { + throw new AggregationExecutionException("Registry miss-match - expected PercentilesAggregatorSupplier, found [" + + aggregatorSupplier.getClass().toString() + "]"); + } + PercentilesAggregatorSupplier percentilesAggregatorSupplier = (PercentilesAggregatorSupplier) aggregatorSupplier; + return percentilesAggregatorSupplier.build(name, valuesSource, searchContext, parent, percents, percentilesConfig, keyed, + config.format(), pipelineAggregators, metaData); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesAggregatorSupplier.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesAggregatorSupplier.java new file mode 100644 index 0000000000000..782d0b779da78 --- /dev/null +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesAggregatorSupplier.java @@ -0,0 +1,43 @@ +/* + * Licensed to Elasticsearch under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.elasticsearch.search.aggregations.metrics; + +import org.elasticsearch.search.DocValueFormat; +import org.elasticsearch.search.aggregations.Aggregator; +import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.ValuesSource; +import org.elasticsearch.search.internal.SearchContext; + +import java.io.IOException; +import java.util.List; +import java.util.Map; + +public interface PercentilesAggregatorSupplier extends AggregatorSupplier { + Aggregator build(String name, + ValuesSource valuesSource, + SearchContext context, + Aggregator parent, + double[] percents, + PercentilesConfig percentilesConfig, + boolean keyed, + DocValueFormat formatter, + List pipelineAggregators, + Map metaData) throws IOException; +} diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesConfig.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesConfig.java index e9519b8e69018..b1da934210135 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesConfig.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesConfig.java @@ -69,10 +69,10 @@ public PercentilesMethod getMethod() { return method; } - abstract Aggregator createPercentilesAggregator(String name, ValuesSource valuesSource, SearchContext context, Aggregator parent, - double[] values, boolean keyed, DocValueFormat formatter, - List pipelineAggregators, - Map metadata) throws IOException; + public abstract Aggregator createPercentilesAggregator(String name, ValuesSource valuesSource, SearchContext context, Aggregator parent, + double[] values, boolean keyed, DocValueFormat formatter, + List pipelineAggregators, + Map metaData) throws IOException; abstract Aggregator createPercentileRanksAggregator(String name, ValuesSource valuesSource, SearchContext context, Aggregator parent, double[] values, boolean keyed, @@ -102,11 +102,11 @@ public static class TDigest extends PercentilesConfig { static final double DEFAULT_COMPRESSION = 100.0; private double compression; - TDigest() { + public TDigest() { this(DEFAULT_COMPRESSION); } - TDigest(double compression) { + public TDigest(double compression) { super(PercentilesMethod.TDIGEST); setCompression(compression); } @@ -128,10 +128,10 @@ public double getCompression() { } @Override - Aggregator createPercentilesAggregator(String name, ValuesSource valuesSource, SearchContext context, Aggregator parent, - double[] values, boolean keyed, DocValueFormat formatter, - List pipelineAggregators, - Map metadata) throws IOException { + public Aggregator createPercentilesAggregator(String name, ValuesSource valuesSource, SearchContext context, Aggregator parent, + double[] values, boolean keyed, DocValueFormat formatter, + List pipelineAggregators, + Map metaData) throws IOException { return new TDigestPercentilesAggregator(name, valuesSource, context, parent, values, compression, keyed, formatter, pipelineAggregators, metadata); } @@ -179,11 +179,11 @@ public static class Hdr extends PercentilesConfig { static final int DEFAULT_NUMBER_SIG_FIGS = 3; private int numberOfSignificantValueDigits; - Hdr() { + public Hdr() { this(DEFAULT_NUMBER_SIG_FIGS); } - Hdr(int numberOfSignificantValueDigits) { + public Hdr(int numberOfSignificantValueDigits) { super(PercentilesMethod.HDR); setNumberOfSignificantValueDigits(numberOfSignificantValueDigits); } @@ -204,10 +204,10 @@ public int getNumberOfSignificantValueDigits() { } @Override - Aggregator createPercentilesAggregator(String name, ValuesSource valuesSource, SearchContext context, Aggregator parent, - double[] values, boolean keyed, DocValueFormat formatter, - List pipelineAggregators, - Map metadata) throws IOException { + public Aggregator createPercentilesAggregator(String name, ValuesSource valuesSource, SearchContext context, Aggregator parent, + double[] values, boolean keyed, DocValueFormat formatter, + List pipelineAggregators, + Map metaData) throws IOException { return new HDRPercentilesAggregator(name, valuesSource, context, parent, values, numberOfSignificantValueDigits, keyed, formatter, pipelineAggregators, metadata); } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/StatsAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/StatsAggregationBuilder.java index 847695450740d..e0e1c547abc83 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/StatsAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/StatsAggregationBuilder.java @@ -28,12 +28,11 @@ import org.elasticsearch.search.aggregations.AggregatorFactories.Builder; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; -import org.elasticsearch.search.aggregations.support.ValueType; import org.elasticsearch.search.aggregations.support.ValuesSource; -import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; -import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.util.Map; @@ -44,11 +43,11 @@ public class StatsAggregationBuilder extends ValuesSourceAggregationBuilder.Leaf public static final ObjectParser PARSER = ObjectParser.fromBuilder(NAME, StatsAggregationBuilder::new); static { - ValuesSourceParserHelper.declareNumericFields(PARSER, true, true, false); + ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, false); } public StatsAggregationBuilder(String name) { - super(name, CoreValuesSourceType.NUMERIC, ValueType.NUMERIC); + super(name); } protected StatsAggregationBuilder(StatsAggregationBuilder clone, @@ -56,6 +55,10 @@ protected StatsAggregationBuilder(StatsAggregationBuilder clone, super(clone, factoriesBuilder, metadata); } + public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + StatsAggregatorFactory.registerAggregators(valuesSourceRegistry); + } + @Override protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metadata) { return new StatsAggregationBuilder(this, factoriesBuilder, metadata); @@ -65,7 +68,12 @@ protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map config, + protected StatsAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config, AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException { return new StatsAggregatorFactory(name, config, queryShardContext, parent, subFactoriesBuilder, metadata); } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/StatsAggregator.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/StatsAggregator.java index 5da83c24b1822..8ce069398c5b9 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/StatsAggregator.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/StatsAggregator.java @@ -49,7 +49,6 @@ class StatsAggregator extends NumericMetricsAggregator.MultiValue { DoubleArray mins; DoubleArray maxes; - StatsAggregator(String name, ValuesSource.Numeric valuesSource, DocValueFormat format, SearchContext context, Aggregator parent, List pipelineAggregators, Map metadata) throws IOException { diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/StatsAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/StatsAggregatorFactory.java index 1d81a24a45880..19cb223a31f8b 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/StatsAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/StatsAggregatorFactory.java @@ -20,24 +20,29 @@ package org.elasticsearch.search.aggregations.metrics; import org.elasticsearch.index.query.QueryShardContext; +import org.elasticsearch.search.DocValueFormat; +import org.elasticsearch.search.aggregations.AggregationExecutionException; import org.elasticsearch.search.aggregations.Aggregator; import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; import java.util.List; import java.util.Map; -class StatsAggregatorFactory extends ValuesSourceAggregatorFactory { +class StatsAggregatorFactory extends ValuesSourceAggregatorFactory { StatsAggregatorFactory(String name, - ValuesSourceConfig config, + ValuesSourceConfig config, QueryShardContext queryShardContext, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, @@ -45,22 +50,45 @@ class StatsAggregatorFactory extends ValuesSourceAggregatorFactory pipelineAggregators, Map metaData) throws IOException { + return new StatsAggregator(name, (Numeric) valuesSource, formatter, context, parent, pipelineAggregators, metaData); + } + }); + } + @Override protected Aggregator createUnmapped(SearchContext searchContext, Aggregator parent, List pipelineAggregators, - Map metadata) - throws IOException { - return new StatsAggregator(name, null, config.format(), searchContext, parent, pipelineAggregators, metadata); + Map metaData) throws IOException { + return new StatsAggregator(name, null, config.format(), searchContext, parent, pipelineAggregators, metaData); } @Override - protected Aggregator doCreateInternal(Numeric valuesSource, + protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, List pipelineAggregators, - Map metadata) throws IOException { - return new StatsAggregator(name, valuesSource, config.format(), searchContext, parent, pipelineAggregators, metadata); + Map metaData) throws IOException { + AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), + StatsAggregationBuilder.NAME); + + if (aggregatorSupplier instanceof MetricAggregatorSupplier == false) { + throw new AggregationExecutionException("Registry miss-match - expected MetricAggregatorSupplier, found [" + + aggregatorSupplier.getClass().toString() + "]"); + } + return ((MetricAggregatorSupplier) aggregatorSupplier).build(name, valuesSource, config.format(), searchContext, parent, + pipelineAggregators, metaData); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/SumAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/SumAggregationBuilder.java index 7e17352b96289..c639e0e2b1fbe 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/SumAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/SumAggregationBuilder.java @@ -28,12 +28,11 @@ import org.elasticsearch.search.aggregations.AggregatorFactories.Builder; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; -import org.elasticsearch.search.aggregations.support.ValueType; import org.elasticsearch.search.aggregations.support.ValuesSource; -import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; -import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.util.Map; @@ -43,11 +42,15 @@ public class SumAggregationBuilder extends ValuesSourceAggregationBuilder.LeafOn public static final ObjectParser PARSER = ObjectParser.fromBuilder(NAME, SumAggregationBuilder::new); static { - ValuesSourceParserHelper.declareNumericFields(PARSER, true, true, false); + ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, false); + } + + public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + SumAggregatorFactory.registerAggregators(valuesSourceRegistry); } public SumAggregationBuilder(String name) { - super(name, CoreValuesSourceType.NUMERIC, ValueType.NUMERIC); + super(name); } protected SumAggregationBuilder(SumAggregationBuilder clone, Builder factoriesBuilder, Map metadata) { @@ -63,7 +66,12 @@ protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map config, + protected SumAggregatorFactory innerBuild(QueryShardContext queryShardContext, ValuesSourceConfig config, AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException { return new SumAggregatorFactory(name, config, queryShardContext, parent, subFactoriesBuilder, metadata); } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/SumAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/SumAggregatorFactory.java index 9267e8b81969c..38c8111a48801 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/SumAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/SumAggregatorFactory.java @@ -20,24 +20,29 @@ package org.elasticsearch.search.aggregations.metrics; import org.elasticsearch.index.query.QueryShardContext; +import org.elasticsearch.search.DocValueFormat; +import org.elasticsearch.search.aggregations.AggregationExecutionException; import org.elasticsearch.search.aggregations.Aggregator; import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; import java.util.List; import java.util.Map; -class SumAggregatorFactory extends ValuesSourceAggregatorFactory { +class SumAggregatorFactory extends ValuesSourceAggregatorFactory { SumAggregatorFactory(String name, - ValuesSourceConfig config, + ValuesSourceConfig config, QueryShardContext queryShardContext, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, @@ -45,6 +50,22 @@ class SumAggregatorFactory extends ValuesSourceAggregatorFactory pipelineAggregators, Map metaData) throws IOException { + return new SumAggregator(name, (Numeric) valuesSource, formatter, context, parent, pipelineAggregators, metaData); + } + }); + } + @Override protected Aggregator createUnmapped(SearchContext searchContext, Aggregator parent, @@ -55,12 +76,20 @@ protected Aggregator createUnmapped(SearchContext searchContext, } @Override - protected Aggregator doCreateInternal(Numeric valuesSource, + protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, List pipelineAggregators, - Map metadata) throws IOException { - return new SumAggregator(name, valuesSource, config.format(), searchContext, parent, pipelineAggregators, metadata); + Map metaData) throws IOException { + AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), + SumAggregationBuilder.NAME); + + if (aggregatorSupplier instanceof MetricAggregatorSupplier == false) { + throw new AggregationExecutionException("Registry miss-match - expected MetricAggregatorSupplier, found [" + + aggregatorSupplier.getClass().toString() + "]"); + } + return ((MetricAggregatorSupplier) aggregatorSupplier).build(name, valuesSource, config.format(), searchContext, parent, + pipelineAggregators, metaData); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregationBuilder.java index 381f5f480265a..8ed6e3f02698a 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregationBuilder.java @@ -29,11 +29,11 @@ import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; -import org.elasticsearch.search.aggregations.support.ValueType; import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; -import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.util.Map; @@ -42,13 +42,17 @@ public class ValueCountAggregationBuilder extends ValuesSourceAggregationBuilder public static final String NAME = "value_count"; public static final ObjectParser PARSER = - ObjectParser.fromBuilder(NAME, name -> new ValueCountAggregationBuilder(name, null)); + ObjectParser.fromBuilder(NAME, ValueCountAggregationBuilder::new); static { - ValuesSourceParserHelper.declareAnyFields(PARSER, true, true); + ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, false); } - public ValueCountAggregationBuilder(String name, ValueType targetValueType) { - super(name, CoreValuesSourceType.ANY, targetValueType); + public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + ValueCountAggregatorFactory.registerAggregators(valuesSourceRegistry); + } + + public ValueCountAggregationBuilder(String name) { + super(name); } protected ValueCountAggregationBuilder(ValueCountAggregationBuilder clone, @@ -57,15 +61,20 @@ protected ValueCountAggregationBuilder(ValueCountAggregationBuilder clone, } @Override - protected AggregationBuilder shallowCopy(AggregatorFactories.Builder factoriesBuilder, Map metadata) { - return new ValueCountAggregationBuilder(this, factoriesBuilder, metadata); + protected ValuesSourceType defaultValueSourceType() { + return CoreValuesSourceType.BYTES; + } + + @Override + protected AggregationBuilder shallowCopy(AggregatorFactories.Builder factoriesBuilder, Map metaData) { + return new ValueCountAggregationBuilder(this, factoriesBuilder, metaData); } /** * Read from a stream. */ public ValueCountAggregationBuilder(StreamInput in) throws IOException { - super(in, CoreValuesSourceType.ANY); + super(in); } @Override @@ -80,7 +89,7 @@ protected boolean serializeTargetValueType(Version version) { @Override protected ValueCountAggregatorFactory innerBuild(QueryShardContext queryShardContext, - ValuesSourceConfig config, + ValuesSourceConfig config, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder) throws IOException { return new ValueCountAggregatorFactory(name, config, queryShardContext, parent, subFactoriesBuilder, metadata); diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregatorFactory.java index 02deadb30a85f..377a489ff4093 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregatorFactory.java @@ -20,24 +20,43 @@ package org.elasticsearch.search.aggregations.metrics; import org.elasticsearch.index.query.QueryShardContext; +import org.elasticsearch.search.aggregations.AggregationExecutionException; import org.elasticsearch.search.aggregations.Aggregator; import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; import java.util.List; import java.util.Map; -class ValueCountAggregatorFactory extends ValuesSourceAggregatorFactory { +class ValueCountAggregatorFactory extends ValuesSourceAggregatorFactory { - ValueCountAggregatorFactory(String name, ValuesSourceConfig config, QueryShardContext queryShardContext, - AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, Map metadata) throws IOException { - super(name, config, queryShardContext, parent, subFactoriesBuilder, metadata); + public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + valuesSourceRegistry.registerAny(ValueCountAggregationBuilder.NAME, + new ValueCountAggregatorSupplier() { + @Override + public Aggregator build(String name, + ValuesSource valuesSource, + SearchContext aggregationContext, + Aggregator parent, + List pipelineAggregators, + Map metaData) throws IOException { + return new ValueCountAggregator(name, valuesSource, aggregationContext, parent, pipelineAggregators, metaData); + } + }); + } + + ValueCountAggregatorFactory(String name, ValuesSourceConfig config, QueryShardContext queryShardContext, + AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, + Map metaData) throws IOException { + super(name, config, queryShardContext, parent, subFactoriesBuilder, metaData); } @Override @@ -54,7 +73,14 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, Aggregator parent, boolean collectsFromSingleBucket, List pipelineAggregators, - Map metadata) throws IOException { - return new ValueCountAggregator(name, valuesSource, searchContext, parent, pipelineAggregators, metadata); + Map metaData) throws IOException { + AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), + ValueCountAggregationBuilder.NAME); + if (aggregatorSupplier instanceof ValueCountAggregatorSupplier == false) { + throw new AggregationExecutionException("Registry miss-match - expected ValueCountAggregatorSupplier, found [" + + aggregatorSupplier.getClass().toString() + "]"); + } + return ((ValueCountAggregatorSupplier) aggregatorSupplier) + .build(name, valuesSource, searchContext, parent, pipelineAggregators,metaData); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregatorSupplier.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregatorSupplier.java new file mode 100644 index 0000000000000..bbc47e6d56e84 --- /dev/null +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregatorSupplier.java @@ -0,0 +1,38 @@ +/* + * Licensed to Elasticsearch under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.elasticsearch.search.aggregations.metrics; + +import org.elasticsearch.search.aggregations.Aggregator; +import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.ValuesSource; +import org.elasticsearch.search.internal.SearchContext; + +import java.io.IOException; +import java.util.List; +import java.util.Map; + +public interface ValueCountAggregatorSupplier extends AggregatorSupplier { + Aggregator build(String name, + ValuesSource valuesSource, + SearchContext aggregationContext, + Aggregator parent, + List pipelineAggregators, + Map metaData) throws IOException; +} diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/WeightedAvgAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/WeightedAvgAggregationBuilder.java index 457cd2fb3755f..b6ec1c843376b 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/WeightedAvgAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/WeightedAvgAggregationBuilder.java @@ -30,19 +30,20 @@ import org.elasticsearch.search.aggregations.AggregationBuilder; import org.elasticsearch.search.aggregations.AggregatorFactories.Builder; import org.elasticsearch.search.aggregations.AggregatorFactory; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; import org.elasticsearch.search.aggregations.support.MultiValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.MultiValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.MultiValuesSourceFieldConfig; import org.elasticsearch.search.aggregations.support.MultiValuesSourceParseHelper; import org.elasticsearch.search.aggregations.support.ValueType; -import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.util.Map; import java.util.Objects; -public class WeightedAvgAggregationBuilder extends MultiValuesSourceAggregationBuilder.LeafOnly { +public class WeightedAvgAggregationBuilder extends MultiValuesSourceAggregationBuilder.LeafOnly { public static final String NAME = "weighted_avg"; public static final ParseField VALUE_FIELD = new ParseField("value"); public static final ParseField WEIGHT_FIELD = new ParseField("weight"); @@ -56,7 +57,7 @@ public class WeightedAvgAggregationBuilder extends MultiValuesSourceAggregationB } public WeightedAvgAggregationBuilder(String name) { - super(name, ValueType.NUMERIC); + super(name); } public WeightedAvgAggregationBuilder(WeightedAvgAggregationBuilder clone, Builder factoriesBuilder, Map metadata) { @@ -79,7 +80,7 @@ public WeightedAvgAggregationBuilder weight(MultiValuesSourceFieldConfig weightC * Read from a stream. */ public WeightedAvgAggregationBuilder(StreamInput in) throws IOException { - super(in, ValueType.NUMERIC); + super(in); } @Override @@ -87,6 +88,11 @@ protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map innerBuild(QueryShardContext queryShardContext, - Map> configs, - DocValueFormat format, - AggregatorFactory parent, - Builder subFactoriesBuilder) throws IOException { - return new WeightedAvgAggregatorFactory(name, configs, format, queryShardContext, parent, subFactoriesBuilder, metadata); + protected MultiValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, + Map configs, + DocValueFormat format, + AggregatorFactory parent, + Builder subFactoriesBuilder) throws IOException { + return new WeightedAvgAggregatorFactory(name, configs, format, queryShardContext, parent, subFactoriesBuilder, metaData); } @Override diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/WeightedAvgAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/WeightedAvgAggregatorFactory.java index 35e724d21c4ed..45bab4daf669b 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/WeightedAvgAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/WeightedAvgAggregatorFactory.java @@ -27,7 +27,6 @@ import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; import org.elasticsearch.search.aggregations.support.MultiValuesSource; import org.elasticsearch.search.aggregations.support.MultiValuesSourceAggregatorFactory; -import org.elasticsearch.search.aggregations.support.ValuesSource.Numeric; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; import org.elasticsearch.search.internal.SearchContext; @@ -35,9 +34,9 @@ import java.util.List; import java.util.Map; -class WeightedAvgAggregatorFactory extends MultiValuesSourceAggregatorFactory { +class WeightedAvgAggregatorFactory extends MultiValuesSourceAggregatorFactory { - WeightedAvgAggregatorFactory(String name, Map> configs, + WeightedAvgAggregatorFactory(String name, Map configs, DocValueFormat format, QueryShardContext queryShardContext, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, Map metadata) throws IOException { @@ -54,7 +53,7 @@ protected Aggregator createUnmapped(SearchContext searchContext, @Override protected Aggregator doCreateInternal(SearchContext searchContext, - Map> configs, + Map configs, DocValueFormat format, Aggregator parent, boolean collectsFromSingleBucket, diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/support/AggregatorSupplier.java b/server/src/main/java/org/elasticsearch/search/aggregations/support/AggregatorSupplier.java new file mode 100644 index 0000000000000..97ca793faedb3 --- /dev/null +++ b/server/src/main/java/org/elasticsearch/search/aggregations/support/AggregatorSupplier.java @@ -0,0 +1,22 @@ +/* + * Licensed to Elasticsearch under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.elasticsearch.search.aggregations.support; + +public interface AggregatorSupplier { +} diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/support/CoreValuesSourceType.java b/server/src/main/java/org/elasticsearch/search/aggregations/support/CoreValuesSourceType.java index 6df32b4deefa0..e6064cbd1f35d 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/support/CoreValuesSourceType.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/support/CoreValuesSourceType.java @@ -21,54 +21,28 @@ import org.apache.lucene.util.BytesRef; import org.elasticsearch.common.geo.GeoPoint; -import org.elasticsearch.common.io.stream.StreamInput; -import org.elasticsearch.common.io.stream.StreamOutput; -import org.elasticsearch.common.io.stream.Writeable; +import org.elasticsearch.common.time.DateFormatter; import org.elasticsearch.index.fielddata.IndexFieldData; import org.elasticsearch.index.fielddata.IndexGeoPointFieldData; -import org.elasticsearch.index.fielddata.IndexHistogramFieldData; import org.elasticsearch.index.fielddata.IndexNumericFieldData; import org.elasticsearch.index.fielddata.IndexOrdinalsFieldData; +import org.elasticsearch.index.mapper.DateFieldMapper; import org.elasticsearch.index.mapper.MappedFieldType; import org.elasticsearch.index.mapper.RangeFieldMapper; import org.elasticsearch.script.AggregationScript; import org.elasticsearch.search.DocValueFormat; import org.elasticsearch.search.aggregations.AggregationExecutionException; -import java.io.IOException; +import java.time.ZoneId; +import java.time.ZoneOffset; import java.util.Locale; import java.util.function.LongSupplier; /** * {@link CoreValuesSourceType} holds the {@link ValuesSourceType} implementations for the core aggregations package. */ -public enum CoreValuesSourceType implements Writeable, ValuesSourceType { - ANY { - // ANY still has a lot of special handling in ValuesSourceConfig, and as such doesn't adhere to this interface yet - @Override - public ValuesSource getEmpty() { - // TODO: Implement this or get rid of ANY - throw new UnsupportedOperationException("CoreValuesSourceType.ANY is still a special case"); - } - - @Override - public ValuesSource getScript(AggregationScript.LeafFactory script, ValueType scriptValueType) { - // TODO: Implement this or get rid of ANY - throw new UnsupportedOperationException("CoreValuesSourceType.ANY is still a special case"); - } - - @Override - public ValuesSource getField(FieldContext fieldContext, AggregationScript.LeafFactory script) { - // TODO: Implement this or get rid of ANY - throw new UnsupportedOperationException("CoreValuesSourceType.ANY is still a special case"); - } - - @Override - public ValuesSource replaceMissing(ValuesSource valuesSource, Object rawMissing, DocValueFormat docValueFormat, LongSupplier now) { - return BYTES.replaceMissing(valuesSource, rawMissing, docValueFormat, now); - } - }, - NUMERIC { +public enum CoreValuesSourceType implements ValuesSourceType { + NUMERIC() { @Override public ValuesSource getEmpty() { return ValuesSource.Numeric.EMPTY; @@ -88,7 +62,7 @@ public ValuesSource getField(FieldContext fieldContext, AggregationScript.LeafFa "], but got [" + fieldContext.fieldType().typeName() + "]"); } - ValuesSource.Numeric dataSource = new ValuesSource.Numeric.FieldData((IndexNumericFieldData)fieldContext.indexFieldData()); + ValuesSource.Numeric dataSource = new ValuesSource.Numeric.FieldData((IndexNumericFieldData) fieldContext.indexFieldData()); if (script != null) { // Value script case dataSource = new ValuesSource.Numeric.WithScript(dataSource, script); @@ -102,7 +76,7 @@ public ValuesSource replaceMissing(ValuesSource valuesSource, Object rawMissing, return MissingValues.replaceMissing((ValuesSource.Numeric) valuesSource, missing); } }, - BYTES { + BYTES() { @Override public ValuesSource getEmpty() { return ValuesSource.Bytes.WithOrdinals.EMPTY; @@ -139,7 +113,7 @@ public ValuesSource replaceMissing(ValuesSource valuesSource, Object rawMissing, } } }, - GEOPOINT { + GEOPOINT() { @Override public ValuesSource getEmpty() { return ValuesSource.GeoPoint.EMPTY; @@ -167,8 +141,13 @@ public ValuesSource replaceMissing(ValuesSource valuesSource, Object rawMissing, final GeoPoint missing = new GeoPoint(rawMissing.toString()); return MissingValues.replaceMissing((ValuesSource.GeoPoint) valuesSource, missing); } + + @Override + public DocValueFormat getFormatter(String format, ZoneId tz) { + return DocValueFormat.GEOHASH; + } }, - RANGE { + RANGE() { @Override public ValuesSource getEmpty() { // TODO: Is this the correct exception type here? @@ -188,7 +167,7 @@ public ValuesSource getField(FieldContext fieldContext, AggregationScript.LeafFa // TODO: Is this the correct exception type here? throw new IllegalStateException("Asked for range ValuesSource, but field is of type " + fieldType.name()); } - RangeFieldMapper.RangeFieldType rangeFieldType = (RangeFieldMapper.RangeFieldType)fieldType; + RangeFieldMapper.RangeFieldType rangeFieldType = (RangeFieldMapper.RangeFieldType) fieldType; return new ValuesSource.Range(fieldContext.indexFieldData(), rangeFieldType.rangeType()); } @@ -197,47 +176,93 @@ public ValuesSource replaceMissing(ValuesSource valuesSource, Object rawMissing, throw new IllegalArgumentException("Can't apply missing values on a " + valuesSource.getClass()); } }, - HISTOGRAM { + IP() { @Override public ValuesSource getEmpty() { - // TODO: Is this the correct exception type here? - throw new IllegalArgumentException("Can't deal with unmapped ValuesSource type " + this.value()); + return BYTES.getEmpty(); } @Override public ValuesSource getScript(AggregationScript.LeafFactory script, ValueType scriptValueType) { - throw new AggregationExecutionException("value source of type [" + this.value() + "] is not supported by scripts"); + return BYTES.getScript(script, scriptValueType); } @Override public ValuesSource getField(FieldContext fieldContext, AggregationScript.LeafFactory script) { - final IndexFieldData indexFieldData = fieldContext.indexFieldData(); + return BYTES.getField(fieldContext, script); + } - if (!(indexFieldData instanceof IndexHistogramFieldData)) { - throw new IllegalArgumentException("Expected histogram type on field [" + fieldContext.field() + - "], but got [" + fieldContext.fieldType().typeName() + "]"); - } - return new ValuesSource.Histogram.Fielddata((IndexHistogramFieldData) indexFieldData); + @Override + public ValuesSource replaceMissing(ValuesSource valuesSource, Object rawMissing, DocValueFormat docValueFormat, LongSupplier now) { + return BYTES.replaceMissing(valuesSource, rawMissing, docValueFormat, now); + } + + @Override + public DocValueFormat getFormatter(String format, ZoneId tz) { + return DocValueFormat.IP; + } + }, + DATE() { + @Override + public ValuesSource getEmpty() { + return NUMERIC.getEmpty(); + } + + @Override + public ValuesSource getScript(AggregationScript.LeafFactory script, ValueType scriptValueType) { + return NUMERIC.getScript(script, scriptValueType); + } + + @Override + public ValuesSource getField(FieldContext fieldContext, AggregationScript.LeafFactory script) { + return NUMERIC.getField(fieldContext, script); } @Override public ValuesSource replaceMissing(ValuesSource valuesSource, Object rawMissing, DocValueFormat docValueFormat, LongSupplier now) { - throw new IllegalArgumentException("Can't apply missing values on a " + valuesSource.getClass()); + return NUMERIC.replaceMissing(valuesSource, rawMissing, docValueFormat, now); } - }; - public static ValuesSourceType fromString(String name) { - return valueOf(name.trim().toUpperCase(Locale.ROOT)); - } + @Override + public DocValueFormat getFormatter(String format, ZoneId tz) { + return new DocValueFormat.DateTime( + format == null ? DateFieldMapper.DEFAULT_DATE_TIME_FORMATTER : DateFormatter.forPattern(format), + tz == null ? ZoneOffset.UTC : tz, + // If we were just looking at fields, we could read the resolution from the field settings, but we need to deal with script + // output, which has no way to indicate the resolution, so we need to default to something. Milliseconds is the standard. + DateFieldMapper.Resolution.MILLISECONDS); + } + }, + BOOLEAN() { + @Override + public ValuesSource getEmpty() { + return NUMERIC.getEmpty(); + } + + @Override + public ValuesSource getScript(AggregationScript.LeafFactory script, ValueType scriptValueType) { + return NUMERIC.getScript(script, scriptValueType); + } + + @Override + public ValuesSource getField(FieldContext fieldContext, AggregationScript.LeafFactory script) { + return NUMERIC.getField(fieldContext, script); + } - public static ValuesSourceType fromStream(StreamInput in) throws IOException { - return in.readEnum(CoreValuesSourceType.class); + @Override + public ValuesSource replaceMissing(ValuesSource valuesSource, Object rawMissing, DocValueFormat docValueFormat, LongSupplier now) { + return NUMERIC.replaceMissing(valuesSource, rawMissing, docValueFormat, now); + } + + @Override + public DocValueFormat getFormatter(String format, ZoneId tz) { + return DocValueFormat.BOOLEAN; + } } + ; - @Override - public void writeTo(StreamOutput out) throws IOException { - CoreValuesSourceType state = this; - out.writeEnum(state); + public static ValuesSourceType fromString(String name) { + return valueOf(name.trim().toUpperCase(Locale.ROOT)); } public String value() { diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/support/HistogramAggregatorSupplier.java b/server/src/main/java/org/elasticsearch/search/aggregations/support/HistogramAggregatorSupplier.java new file mode 100644 index 0000000000000..021577852c39e --- /dev/null +++ b/server/src/main/java/org/elasticsearch/search/aggregations/support/HistogramAggregatorSupplier.java @@ -0,0 +1,39 @@ +/* + * Licensed to Elasticsearch under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.elasticsearch.search.aggregations.support; + +import org.elasticsearch.common.Nullable; +import org.elasticsearch.search.DocValueFormat; +import org.elasticsearch.search.aggregations.Aggregator; +import org.elasticsearch.search.aggregations.AggregatorFactories; +import org.elasticsearch.search.aggregations.BucketOrder; +import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.internal.SearchContext; + +import java.io.IOException; +import java.util.List; +import java.util.Map; + +public interface HistogramAggregatorSupplier extends AggregatorSupplier { + Aggregator build(String name, AggregatorFactories factories, double interval, double offset, + BucketOrder order, boolean keyed, long minDocCount, double minBound, double maxBound, + @Nullable ValuesSource valuesSource, DocValueFormat formatter, + SearchContext context, Aggregator parent, + List pipelineAggregators, Map metaData) throws IOException; +} diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/support/MultiValuesSource.java b/server/src/main/java/org/elasticsearch/search/aggregations/support/MultiValuesSource.java index 3a8bd9e12feca..fe1d5be993209 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/support/MultiValuesSource.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/support/MultiValuesSource.java @@ -21,6 +21,7 @@ import org.apache.lucene.index.LeafReaderContext; import org.elasticsearch.index.fielddata.SortedNumericDoubleValues; import org.elasticsearch.index.query.QueryShardContext; +import org.elasticsearch.search.aggregations.AggregationExecutionException; import java.io.IOException; import java.util.HashMap; @@ -34,11 +35,16 @@ public abstract class MultiValuesSource { protected Map values; public static class NumericMultiValuesSource extends MultiValuesSource { - public NumericMultiValuesSource(Map> valuesSourceConfigs, + public NumericMultiValuesSource(Map valuesSourceConfigs, QueryShardContext context) { values = new HashMap<>(valuesSourceConfigs.size()); - for (Map.Entry> entry : valuesSourceConfigs.entrySet()) { - values.put(entry.getKey(), entry.getValue().toValuesSource(context)); + for (Map.Entry entry : valuesSourceConfigs.entrySet()) { + final ValuesSource valuesSource = entry.getValue().toValuesSource(); + if (valuesSource instanceof ValuesSource.Numeric == false) { + throw new AggregationExecutionException("ValuesSource type " + valuesSource.toString() + + "is not supported for multi-valued aggregation"); + } + values.put(entry.getKey(), (ValuesSource.Numeric) valuesSource); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/support/MultiValuesSourceAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/support/MultiValuesSourceAggregationBuilder.java index 989400f6a8a4f..1c36db7296472 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/support/MultiValuesSourceAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/support/MultiValuesSourceAggregationBuilder.java @@ -40,19 +40,19 @@ * * A limitation of this class is that all the ValuesSource's being refereenced must be of the same type. */ -public abstract class MultiValuesSourceAggregationBuilder> +public abstract class MultiValuesSourceAggregationBuilder> extends AbstractAggregationBuilder { - public abstract static class LeafOnly> - extends MultiValuesSourceAggregationBuilder { + public abstract static class LeafOnly> + extends MultiValuesSourceAggregationBuilder { - protected LeafOnly(String name, ValueType targetValueType) { - super(name, targetValueType); + protected LeafOnly(String name) { + super(name); } - protected LeafOnly(LeafOnly clone, Builder factoriesBuilder, Map metadata) { - super(clone, factoriesBuilder, metadata); + protected LeafOnly(LeafOnly clone, Builder factoriesBuilder, Map metaData) { + super(clone, factoriesBuilder, metaData); if (factoriesBuilder.count() > 0) { throw new AggregationInitializationException("Aggregator [" + name + "] of type [" + getType() + "] cannot accept sub-aggregations"); @@ -62,8 +62,8 @@ protected LeafOnly(LeafOnly clone, Builder factoriesBuilder, Map fields = new HashMap<>(); - private final ValueType targetValueType; - private ValueType valueType = null; + private ValueType userValueTypeHint = null; private String format = null; - protected MultiValuesSourceAggregationBuilder(String name, ValueType targetValueType) { + protected MultiValuesSourceAggregationBuilder(String name) { super(name); - this.targetValueType = targetValueType; } - protected MultiValuesSourceAggregationBuilder(MultiValuesSourceAggregationBuilder clone, - Builder factoriesBuilder, Map metadata) { - super(clone, factoriesBuilder, metadata); + protected MultiValuesSourceAggregationBuilder(MultiValuesSourceAggregationBuilder clone, + Builder factoriesBuilder, Map metaData) { + super(clone, factoriesBuilder, metaData); this.fields = new HashMap<>(clone.fields); - this.targetValueType = clone.targetValueType; - this.valueType = clone.valueType; + this.userValueTypeHint = clone.userValueTypeHint; this.format = clone.format; } - protected MultiValuesSourceAggregationBuilder(StreamInput in, ValueType targetValueType) + /** + * Read from a stream. + */ + protected MultiValuesSourceAggregationBuilder(StreamInput in) throws IOException { super(in); - assert false == serializeTargetValueType() : "Wrong read constructor called for subclass that provides its targetValueType"; - this.targetValueType = targetValueType; read(in); } @@ -109,17 +107,14 @@ protected MultiValuesSourceAggregationBuilder(StreamInput in, ValueType targetVa @SuppressWarnings("unchecked") private void read(StreamInput in) throws IOException { fields = in.readMap(StreamInput::readString, MultiValuesSourceFieldConfig::new); - valueType = in.readOptionalWriteable(ValueType::readFromStream); + userValueTypeHint = in.readOptionalWriteable(ValueType::readFromStream); format = in.readOptionalString(); } @Override protected final void doWriteTo(StreamOutput out) throws IOException { - if (serializeTargetValueType()) { - out.writeOptionalWriteable(targetValueType); - } out.writeMap(fields, StreamOutput::writeString, (o, value) -> value.writeTo(o)); - out.writeOptionalWriteable(valueType); + out.writeOptionalWriteable(userValueTypeHint); out.writeOptionalString(format); innerWriteTo(out); } @@ -142,11 +137,11 @@ protected AB field(String propertyName, MultiValuesSourceFieldConfig config) { * Sets the {@link ValueType} for the value produced by this aggregation */ @SuppressWarnings("unchecked") - public AB valueType(ValueType valueType) { + public AB userValueTypeHint(ValueType valueType) { if (valueType == null) { - throw new IllegalArgumentException("[valueType] must not be null: [" + name + "]"); + throw new IllegalArgumentException("[userValueTypeHint] must not be null: [" + name + "]"); } - this.valueType = valueType; + this.userValueTypeHint = valueType; return (AB) this; } @@ -162,25 +157,33 @@ public AB format(String format) { return (AB) this; } - @Override - protected final MultiValuesSourceAggregatorFactory doBuild(QueryShardContext queryShardContext, AggregatorFactory parent, - Builder subFactoriesBuilder) throws IOException { - ValueType finalValueType = this.valueType != null ? this.valueType : targetValueType; + /** + * Aggregations should use this method to define a {@link ValuesSourceType} of last resort. This will only be used when the resolver + * can't find a field and the user hasn't provided a value type hint. + * + * @return The CoreValuesSourceType we expect this script to yield. + */ + protected abstract ValuesSourceType defaultValueSourceType(); - Map> configs = new HashMap<>(fields.size()); + @Override + protected final MultiValuesSourceAggregatorFactory doBuild(QueryShardContext queryShardContext, AggregatorFactory parent, + Builder subFactoriesBuilder) throws IOException { + Map configs = new HashMap<>(fields.size()); fields.forEach((key, value) -> { - ValuesSourceConfig config = ValuesSourceConfig.resolve(queryShardContext, finalValueType, - value.getFieldName(), value.getScript(), value.getMissing(), value.getTimeZone(), format); + ValuesSourceConfig config = ValuesSourceConfig.resolveUnregistered(queryShardContext, userValueTypeHint, + value.getFieldName(), value.getScript(), value.getMissing(), value.getTimeZone(), format, defaultValueSourceType()); configs.put(key, config); }); - DocValueFormat docValueFormat = resolveFormat(format, finalValueType); + DocValueFormat docValueFormat = resolveFormat(format, userValueTypeHint, defaultValueSourceType()); return innerBuild(queryShardContext, configs, docValueFormat, parent, subFactoriesBuilder); } - private static DocValueFormat resolveFormat(@Nullable String format, @Nullable ValueType valueType) { + private static DocValueFormat resolveFormat(@Nullable String format, @Nullable ValueType valueType, + ValuesSourceType defaultValuesSourceType) { if (valueType == null) { - return DocValueFormat.RAW; // we can't figure it out + // If the user didn't send a hint, all we can do is fall back to the default + return defaultValuesSourceType.getFormatter(format, null); } DocValueFormat valueFormat = valueType.defaultFormat; if (valueFormat instanceof DocValueFormat.Decimal && format != null) { @@ -189,19 +192,11 @@ private static DocValueFormat resolveFormat(@Nullable String format, @Nullable V return valueFormat; } - protected abstract MultiValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, - Map> configs, - DocValueFormat format, AggregatorFactory parent, - Builder subFactoriesBuilder) throws IOException; - + protected abstract MultiValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, + Map configs, + DocValueFormat format, AggregatorFactory parent, + Builder subFactoriesBuilder) throws IOException; - /** - * Should this builder serialize its targetValueType? Defaults to false. All subclasses that override this to true - * should use the three argument read constructor rather than the four argument version. - */ - protected boolean serializeTargetValueType() { - return false; - } @Override public final XContentBuilder internalXContent(XContentBuilder builder, Params params) throws IOException { @@ -214,8 +209,8 @@ public final XContentBuilder internalXContent(XContentBuilder builder, Params pa if (format != null) { builder.field(CommonFields.FORMAT.getPreferredName(), format); } - if (valueType != null) { - builder.field(CommonFields.VALUE_TYPE.getPreferredName(), valueType.getPreferredName()); + if (userValueTypeHint != null) { + builder.field(CommonFields.VALUE_TYPE.getPreferredName(), userValueTypeHint.getPreferredName()); } doXContentBody(builder, params); builder.endObject(); @@ -226,7 +221,7 @@ public final XContentBuilder internalXContent(XContentBuilder builder, Params pa @Override public int hashCode() { - return Objects.hash(super.hashCode(), fields, format, targetValueType, valueType); + return Objects.hash(super.hashCode(), fields, format, userValueTypeHint); } @@ -239,6 +234,6 @@ public boolean equals(Object obj) { MultiValuesSourceAggregationBuilder other = (MultiValuesSourceAggregationBuilder) obj; return Objects.equals(this.fields, other.fields) && Objects.equals(this.format, other.format) - && Objects.equals(this.valueType, other.valueType); + && Objects.equals(this.userValueTypeHint, other.userValueTypeHint); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/support/MultiValuesSourceAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/support/MultiValuesSourceAggregatorFactory.java index 9605ca26c466e..5912850d58c02 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/support/MultiValuesSourceAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/support/MultiValuesSourceAggregatorFactory.java @@ -31,13 +31,12 @@ import java.util.List; import java.util.Map; -public abstract class MultiValuesSourceAggregatorFactory - extends AggregatorFactory { +public abstract class MultiValuesSourceAggregatorFactory extends AggregatorFactory { - protected final Map> configs; + protected final Map configs; protected final DocValueFormat format; - public MultiValuesSourceAggregatorFactory(String name, Map> configs, + public MultiValuesSourceAggregatorFactory(String name, Map configs, DocValueFormat format, QueryShardContext queryShardContext, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, Map metadata) throws IOException { @@ -62,7 +61,7 @@ protected abstract Aggregator createUnmapped(SearchContext searchContext, List pipelineAggregators, Map metadata) throws IOException; - protected abstract Aggregator doCreateInternal(SearchContext searchContext, Map> configs, + protected abstract Aggregator doCreateInternal(SearchContext searchContext, Map configs, DocValueFormat format, Aggregator parent, boolean collectsFromSingleBucket, List pipelineAggregators, Map metadata) throws IOException; diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/support/MultiValuesSourceParseHelper.java b/server/src/main/java/org/elasticsearch/search/aggregations/support/MultiValuesSourceParseHelper.java index 4888495f9d8da..bafa6426480be 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/support/MultiValuesSourceParseHelper.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/support/MultiValuesSourceParseHelper.java @@ -27,17 +27,17 @@ public final class MultiValuesSourceParseHelper { - public static void declareCommon( - AbstractObjectParser, T> objectParser, boolean formattable, - ValueType targetValueType) { + public static void declareCommon( + AbstractObjectParser, T> objectParser, boolean formattable, + ValueType expectedValueType) { - objectParser.declareField(MultiValuesSourceAggregationBuilder::valueType, p -> { - ValueType valueType = ValueType.resolveForScript(p.text()); - if (targetValueType != null && valueType.isNotA(targetValueType)) { + objectParser.declareField(MultiValuesSourceAggregationBuilder::userValueTypeHint, p -> { + ValueType valueType = ValueType.lenientParse(p.text()); + if (expectedValueType != null && valueType.isNotA(expectedValueType)) { throw new ParsingException(p.getTokenLocation(), "Aggregation [" + objectParser.getName() + "] was configured with an incompatible value type [" - + valueType + "]. It can only work on value of type [" - + targetValueType + "]"); + + valueType + "]. It can only work on value off type [" + + expectedValueType + "]"); } return valueType; }, ValueType.VALUE_TYPE, ObjectParser.ValueType.STRING); @@ -49,7 +49,7 @@ public static void declareCommon( } public static void declareField(String fieldName, - AbstractObjectParser, T> objectParser, + AbstractObjectParser, T> objectParser, boolean scriptable, boolean timezoneAware) { objectParser.declareField((o, fieldConfig) -> o.field(fieldName, fieldConfig.build()), diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/support/ValueType.java b/server/src/main/java/org/elasticsearch/search/aggregations/support/ValueType.java index a1e3237288fa9..068790030bd49 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/support/ValueType.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/support/ValueType.java @@ -23,38 +23,33 @@ import org.elasticsearch.common.io.stream.StreamInput; import org.elasticsearch.common.io.stream.StreamOutput; import org.elasticsearch.common.io.stream.Writeable; -import org.elasticsearch.index.fielddata.IndexFieldData; -import org.elasticsearch.index.fielddata.IndexGeoPointFieldData; -import org.elasticsearch.index.fielddata.IndexNumericFieldData; -import org.elasticsearch.index.fielddata.plain.BinaryDVIndexFieldData; import org.elasticsearch.index.mapper.DateFieldMapper; import org.elasticsearch.search.DocValueFormat; import java.io.IOException; import java.time.ZoneOffset; +import java.util.Set; public enum ValueType implements Writeable { STRING((byte) 1, "string", "string", CoreValuesSourceType.BYTES, - IndexFieldData.class, DocValueFormat.RAW), - LONG((byte) 2, "byte|short|integer|long", "long", - CoreValuesSourceType.NUMERIC, - IndexNumericFieldData.class, DocValueFormat.RAW), - DOUBLE((byte) 3, "float|double", "double", CoreValuesSourceType.NUMERIC, IndexNumericFieldData.class, DocValueFormat.RAW), - NUMBER((byte) 4, "number", "number", CoreValuesSourceType.NUMERIC, IndexNumericFieldData.class, DocValueFormat.RAW), - DATE((byte) 5, "date", "date", CoreValuesSourceType.NUMERIC, IndexNumericFieldData.class, - new DocValueFormat.DateTime(DateFieldMapper.DEFAULT_DATE_TIME_FORMATTER, ZoneOffset.UTC, + DocValueFormat.RAW), + + LONG((byte) 2, "byte|short|integer|long", "long", CoreValuesSourceType.NUMERIC, DocValueFormat.RAW), + DOUBLE((byte) 3, "float|double", "double", CoreValuesSourceType.NUMERIC, DocValueFormat.RAW), + NUMBER((byte) 4, "number", "number", CoreValuesSourceType.NUMERIC, DocValueFormat.RAW), + DATE((byte) 5, "date", "date", CoreValuesSourceType.DATE, + new DocValueFormat.DateTime(DateFieldMapper.DEFAULT_DATE_TIME_FORMATTER, ZoneOffset.UTC, DateFieldMapper.Resolution.MILLISECONDS)), - IP((byte) 6, "ip", "ip", CoreValuesSourceType.BYTES, IndexFieldData.class, DocValueFormat.IP), + IP((byte) 6, "ip", "ip", CoreValuesSourceType.IP, DocValueFormat.IP), // TODO: what is the difference between "number" and "numeric"? - NUMERIC((byte) 7, "numeric", "numeric", CoreValuesSourceType.NUMERIC, IndexNumericFieldData.class, DocValueFormat.RAW), - GEOPOINT((byte) 8, "geo_point", "geo_point", CoreValuesSourceType.GEOPOINT, IndexGeoPointFieldData.class, DocValueFormat.GEOHASH), - BOOLEAN((byte) 9, "boolean", "boolean", CoreValuesSourceType.NUMERIC, IndexNumericFieldData.class, DocValueFormat.BOOLEAN), - RANGE((byte) 10, "range", "range", CoreValuesSourceType.RANGE, BinaryDVIndexFieldData.class, DocValueFormat.RAW); + NUMERIC((byte) 7, "numeric", "numeric", CoreValuesSourceType.NUMERIC, DocValueFormat.RAW), + GEOPOINT((byte) 8, "geo_point", "geo_point", CoreValuesSourceType.GEOPOINT, DocValueFormat.GEOHASH), + BOOLEAN((byte) 9, "boolean", "boolean", CoreValuesSourceType.BOOLEAN, DocValueFormat.BOOLEAN), + RANGE((byte) 10, "range", "range", CoreValuesSourceType.RANGE, DocValueFormat.RAW); final String description; final ValuesSourceType valuesSourceType; - final Class fieldDataType; final DocValueFormat defaultFormat; private final byte id; private String preferredName; @@ -62,12 +57,11 @@ public enum ValueType implements Writeable { public static final ParseField VALUE_TYPE = new ParseField("value_type", "valueType"); ValueType(byte id, String description, String preferredName, ValuesSourceType valuesSourceType, - Class fieldDataType, DocValueFormat defaultFormat) { + DocValueFormat defaultFormat) { this.id = id; this.description = description; this.preferredName = preferredName; this.valuesSourceType = valuesSourceType; - this.fieldDataType = fieldDataType; this.defaultFormat = defaultFormat; } @@ -79,9 +73,26 @@ public ValuesSourceType getValuesSourceType() { return valuesSourceType; } + private static Set numericValueTypes = Set.of(ValueType.DOUBLE, ValueType.DATE, ValueType.LONG, ValueType.NUMBER, + ValueType.NUMERIC, ValueType.BOOLEAN); + private static Set stringValueTypes = Set.of(ValueType.STRING, ValueType.IP); + + /** + * This is a bit of a hack to mirror the old {@link ValueType} behavior, which would allow a rough compatibility between types. This + * behavior is being phased out in the aggregations framework, in favor of explicitly listing supported types, but we haven't gotten + * to fixing composite yet. + * + * @param valueType The value type the user suggested + * @return True iff the two value types are interchangeable + */ public boolean isA(ValueType valueType) { - return valueType.valuesSourceType == valuesSourceType && - valueType.fieldDataType.isAssignableFrom(fieldDataType); + if (numericValueTypes.contains(this)) { + return numericValueTypes.contains(valueType); + } + if (stringValueTypes.contains(this)) { + return stringValueTypes.contains(valueType); + } + return this.equals(valueType); } public boolean isNotA(ValueType valueType) { @@ -92,7 +103,7 @@ public DocValueFormat defaultFormat() { return defaultFormat; } - public static ValueType resolveForScript(String type) { + public static ValueType lenientParse(String type) { switch (type) { case "string": return STRING; case "double": @@ -122,7 +133,7 @@ public static ValueType readFromStream(StreamInput in) throws IOException { return valueType; } } - throw new IOException("No valueType found for id [" + id + "]"); + throw new IOException("No ValueType found for id [" + id + "]"); } @Override diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSource.java b/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSource.java index 483c3847c38d7..4bd3acfeeb08f 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSource.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSource.java @@ -33,10 +33,8 @@ import org.elasticsearch.index.fielddata.AbstractSortingNumericDocValues; import org.elasticsearch.index.fielddata.LeafOrdinalsFieldData; import org.elasticsearch.index.fielddata.DocValueBits; -import org.elasticsearch.index.fielddata.HistogramValues; import org.elasticsearch.index.fielddata.IndexFieldData; import org.elasticsearch.index.fielddata.IndexGeoPointFieldData; -import org.elasticsearch.index.fielddata.IndexHistogramFieldData; import org.elasticsearch.index.fielddata.IndexNumericFieldData; import org.elasticsearch.index.fielddata.IndexOrdinalsFieldData; import org.elasticsearch.index.fielddata.MultiGeoPointValues; @@ -565,39 +563,4 @@ public org.elasticsearch.index.fielddata.MultiGeoPointValues geoPointValues(Leaf } } } - - public abstract static class Histogram extends ValuesSource { - - public abstract HistogramValues getHistogramValues(LeafReaderContext context) throws IOException; - - public static class Fielddata extends Histogram { - - protected final IndexHistogramFieldData indexFieldData; - - public Fielddata(IndexHistogramFieldData indexFieldData) { - this.indexFieldData = indexFieldData; - } - - @Override - public SortedBinaryDocValues bytesValues(LeafReaderContext context) { - return indexFieldData.load(context).getBytesValues(); - } - - @Override - public DocValueBits docsWithValue(LeafReaderContext context) throws IOException { - HistogramValues values = getHistogramValues(context); - return new DocValueBits() { - @Override - public boolean advanceExact(int doc) throws IOException { - return values.advanceExact(doc); - } - }; - } - - public HistogramValues getHistogramValues(LeafReaderContext context) throws IOException { - return indexFieldData.load(context).getHistogramValues(); - } - } - } - } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceAggregationBuilder.java index b78c55b163635..a574b07ef8f2f 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceAggregationBuilder.java @@ -19,10 +19,14 @@ package org.elasticsearch.search.aggregations.support; import org.elasticsearch.Version; +import org.elasticsearch.common.ParseField; import org.elasticsearch.common.io.stream.StreamInput; import org.elasticsearch.common.io.stream.StreamOutput; import org.elasticsearch.common.time.DateUtils; +import org.elasticsearch.common.xcontent.AbstractObjectParser; +import org.elasticsearch.common.xcontent.ObjectParser; import org.elasticsearch.common.xcontent.XContentBuilder; +import org.elasticsearch.common.xcontent.XContentParser; import org.elasticsearch.index.query.QueryShardContext; import org.elasticsearch.script.Script; import org.elasticsearch.search.aggregations.AbstractAggregationBuilder; @@ -32,17 +36,54 @@ import java.io.IOException; import java.time.ZoneId; +import java.time.ZoneOffset; import java.util.Map; import java.util.Objects; -public abstract class ValuesSourceAggregationBuilder> +public abstract class ValuesSourceAggregationBuilder> extends AbstractAggregationBuilder { - public abstract static class LeafOnly> - extends ValuesSourceAggregationBuilder { + public static void declareFields( + AbstractObjectParser, T> objectParser, + boolean scriptable, boolean formattable, boolean timezoneAware) { - protected LeafOnly(String name, ValuesSourceType valuesSourceType, ValueType targetValueType) { - super(name, valuesSourceType, targetValueType); + + objectParser.declareField(ValuesSourceAggregationBuilder::field, XContentParser::text, + ParseField.CommonFields.FIELD, ObjectParser.ValueType.STRING); + + objectParser.declareField(ValuesSourceAggregationBuilder::missing, XContentParser::objectText, + ParseField.CommonFields.MISSING, ObjectParser.ValueType.VALUE); + + objectParser.declareField(ValuesSourceAggregationBuilder::userValueTypeHint, p -> ValueType.lenientParse(p.text()), + ValueType.VALUE_TYPE, ObjectParser.ValueType.STRING); + + if (formattable) { + objectParser.declareField(ValuesSourceAggregationBuilder::format, XContentParser::text, + ParseField.CommonFields.FORMAT, ObjectParser.ValueType.STRING); + } + + if (scriptable) { + objectParser.declareField(ValuesSourceAggregationBuilder::script, + (parser, context) -> Script.parse(parser), + Script.SCRIPT_PARSE_FIELD, ObjectParser.ValueType.OBJECT_OR_STRING); + } + + if (timezoneAware) { + objectParser.declareField(ValuesSourceAggregationBuilder::timeZone, p -> { + if (p.currentToken() == XContentParser.Token.VALUE_STRING) { + return ZoneId.of(p.text()); + } else { + return ZoneOffset.ofHours(p.intValue()); + } + }, ParseField.CommonFields.TIME_ZONE, ObjectParser.ValueType.LONG); + } + } + + public abstract static class LeafOnly> + extends ValuesSourceAggregationBuilder { + + protected LeafOnly(String name) { + super(name); } protected LeafOnly(LeafOnly clone, Builder factoriesBuilder, Map metadata) { @@ -54,18 +95,10 @@ protected LeafOnly(LeafOnly clone, Builder factoriesBuilder, Map config; + protected ValuesSourceConfig config; - protected ValuesSourceAggregationBuilder(String name, ValuesSourceType valuesSourceType, ValueType targetValueType) { + protected ValuesSourceAggregationBuilder(String name) { super(name); - if (valuesSourceType == null) { - throw new IllegalArgumentException("[valuesSourceType] must not be null: [" + name + "]"); - } - this.valuesSourceType = valuesSourceType; - this.targetValueType = targetValueType; } - protected ValuesSourceAggregationBuilder(ValuesSourceAggregationBuilder clone, - Builder factoriesBuilder, Map metadata) { - super(clone, factoriesBuilder, metadata); - this.valuesSourceType = clone.valuesSourceType; - this.targetValueType = clone.targetValueType; + protected ValuesSourceAggregationBuilder(ValuesSourceAggregationBuilder clone, + Builder factoriesBuilder, Map metaData) { + super(clone, factoriesBuilder, metaData); this.field = clone.field; - this.valueType = clone.valueType; + this.userValueTypeHint = clone.userValueTypeHint; this.format = clone.format; this.missing = clone.missing; this.timeZone = clone.timeZone; @@ -114,36 +138,18 @@ protected ValuesSourceAggregationBuilder(ValuesSourceAggregationBuilder } /** - * Read an aggregation from a stream that has a sensible default for TargetValueType. This should be used by most subclasses. - * Subclasses needing to maintain backward compatibility to a version that did not serialize TargetValueType should use this - * constructor, providing the old, constant value for TargetValueType and override {@link #serializeTargetValueType(Version)} to return - * true only for versions that support the serialization. + * Read from a stream. */ - protected ValuesSourceAggregationBuilder(StreamInput in, ValuesSourceType valuesSourceType, ValueType targetValueType) + protected ValuesSourceAggregationBuilder(StreamInput in) throws IOException { super(in); - this.valuesSourceType = valuesSourceType; if (serializeTargetValueType(in.getVersion())) { - this.targetValueType = in.readOptionalWriteable(ValueType::readFromStream); - } else { - this.targetValueType = targetValueType; + ValueType valueType = in.readOptionalWriteable(ValueType::readFromStream); + assert valueType == null; } read(in); } - /** - * Read an aggregation from a stream that serializes its targetValueType. This should only be used by subclasses that override - * {@link #serializeTargetValueType(Version)} to return true. - */ - protected ValuesSourceAggregationBuilder(StreamInput in, ValuesSourceType valuesSourceType) throws IOException { - super(in); - // TODO: Can we get rid of this constructor and always use the three value version? Does this assert provide any value? - assert serializeTargetValueType(in.getVersion()) : "Wrong read constructor called for subclass that serializes its targetValueType"; - this.valuesSourceType = valuesSourceType; - this.targetValueType = in.readOptionalWriteable(ValueType::readFromStream); - read(in); - } - /** * Read from a stream. */ @@ -153,7 +159,7 @@ private void read(StreamInput in) throws IOException { script = new Script(in); } if (in.readBoolean()) { - valueType = ValueType.readFromStream(in); + userValueTypeHint = ValueType.readFromStream(in); } format = in.readOptionalString(); missing = in.readGenericValue(); @@ -167,7 +173,8 @@ private void read(StreamInput in) throws IOException { @Override protected final void doWriteTo(StreamOutput out) throws IOException { if (serializeTargetValueType(out.getVersion())) { - out.writeOptionalWriteable(targetValueType); + // TODO: deprecate this so we don't need to carry around a useless null in the wire format + out.writeOptionalWriteable(null); } out.writeOptionalString(field); boolean hasScript = script != null; @@ -175,10 +182,10 @@ protected final void doWriteTo(StreamOutput out) throws IOException { if (hasScript) { script.writeTo(out); } - boolean hasValueType = valueType != null; + boolean hasValueType = userValueTypeHint != null; out.writeBoolean(hasValueType); if (hasValueType) { - valueType.writeTo(out); + userValueTypeHint.writeTo(out); } out.writeOptionalString(format); out.writeGenericValue(missing); @@ -196,8 +203,10 @@ protected final void doWriteTo(StreamOutput out) throws IOException { protected abstract void innerWriteTo(StreamOutput out) throws IOException; /** - * Should this builder serialize its targetValueType? Defaults to false. All subclasses that override this to true should use the three - * argument read constructor rather than the four argument version. + * DO NOT OVERRIDE THIS! + * + * This method only exists for legacy support. No new aggregations need this, nor should they override it. + * * @param version For backwards compatibility, subclasses can change behavior based on the version */ protected boolean serializeTargetValueType(Version version) { @@ -243,22 +252,25 @@ public Script script() { } /** - * Sets the {@link ValueType} for the value produced by this aggregation + * This setter should only be used during parsing, to set the userValueTypeHint. This is information the user provides in the json + * query to indicate the output type of a script or the type of the 'missing' replacement value. + * @param valueType - The parsed {@link ValueType} based on the string the user specified + * @return - The modified builder instance, for chaining. */ @SuppressWarnings("unchecked") - public AB valueType(ValueType valueType) { + public AB userValueTypeHint(ValueType valueType) { if (valueType == null) { - throw new IllegalArgumentException("[valueType] must not be null: [" + name + "]"); + // TODO: This is nonsense. We allow the value to be null (via constructor), but don't allow it to be set to null. This means + // thing looking to copy settings (like RollupRequestTranslator) need to check if userValueTypeHint is not null, and then + // set it if and only if it is non-null. + throw new IllegalArgumentException("[userValueTypeHint] must not be null: [" + name + "]"); } - this.valueType = valueType; + this.userValueTypeHint = valueType; return (AB) this; } - /** - * Gets the {@link ValueType} for the value produced by this aggregation - */ - public ValueType valueType() { - return valueType; + public ValueType userValueTypeHint() { + return userValueTypeHint; } /** @@ -321,44 +333,30 @@ public ZoneId timeZone() { } @Override - protected final ValuesSourceAggregatorFactory doBuild(QueryShardContext queryShardContext, AggregatorFactory parent, - Builder subFactoriesBuilder) throws IOException { - ValuesSourceConfig config = resolveConfig(queryShardContext); - ValuesSourceAggregatorFactory factory = innerBuild(queryShardContext, config, parent, subFactoriesBuilder); + protected final ValuesSourceAggregatorFactory doBuild(QueryShardContext queryShardContext, AggregatorFactory parent, + Builder subFactoriesBuilder) throws IOException { + ValuesSourceConfig config = resolveConfig(queryShardContext); + ValuesSourceAggregatorFactory factory = innerBuild(queryShardContext, config, parent, subFactoriesBuilder); return factory; } /** - * Provide a hook for aggregations to have finer grained control of the CoreValuesSourceType for script values. This will only be - * called if the user did not supply a type hint for the script. The script object is provided for reference. + * Aggregations should use this method to define a {@link ValuesSourceType} of last resort. This will only be used when the resolver + * can't find a field and the user hasn't provided a value type hint. * - * @param script - The user supplied script * @return The CoreValuesSourceType we expect this script to yield. */ - protected ValuesSourceType resolveScriptAny(Script script) { - return CoreValuesSourceType.BYTES; - } - - /** - * Provide a hook for aggregations to have finer grained control of the ValueType for script values. This will only be called if the - * user did not supply a type hint for the script. The script object is provided for reference - * @param script - the user supplied script - * @return The ValueType we expect this script to yield - */ - protected ValueType defaultValueType(Script script) { - return valueType; - } + protected abstract ValuesSourceType defaultValueSourceType(); - protected ValuesSourceConfig resolveConfig(QueryShardContext queryShardContext) { - ValueType valueType = this.valueType != null ? this.valueType : targetValueType; + protected ValuesSourceConfig resolveConfig(QueryShardContext queryShardContext) { return ValuesSourceConfig.resolve(queryShardContext, - valueType, field, script, missing, timeZone, format, this::resolveScriptAny); + this.userValueTypeHint, field, script, missing, timeZone, format, this.defaultValueSourceType(), this.getType()); } - protected abstract ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, - ValuesSourceConfig config, - AggregatorFactory parent, - Builder subFactoriesBuilder) throws IOException; + protected abstract ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, + ValuesSourceConfig config, + AggregatorFactory parent, + Builder subFactoriesBuilder) throws IOException; @Override public final XContentBuilder internalXContent(XContentBuilder builder, Params params) throws IOException { @@ -378,8 +376,8 @@ public final XContentBuilder internalXContent(XContentBuilder builder, Params pa if (timeZone != null) { builder.field("time_zone", timeZone.toString()); } - if (valueType != null) { - builder.field("value_type", valueType.getPreferredName()); + if (userValueTypeHint != null) { + builder.field("value_type", userValueTypeHint.getPreferredName()); } doXContentBody(builder, params); builder.endObject(); @@ -390,8 +388,7 @@ public final XContentBuilder internalXContent(XContentBuilder builder, Params pa @Override public int hashCode() { - return Objects.hash(super.hashCode(), field, format, missing, script, - targetValueType, timeZone, valueType, valuesSourceType); + return Objects.hash(super.hashCode(), field, format, missing, script, timeZone, userValueTypeHint); } @Override @@ -399,14 +396,12 @@ public boolean equals(Object obj) { if (this == obj) return true; if (obj == null || getClass() != obj.getClass()) return false; if (super.equals(obj) == false) return false; - ValuesSourceAggregationBuilder other = (ValuesSourceAggregationBuilder) obj; - return Objects.equals(valuesSourceType, other.valuesSourceType) - && Objects.equals(field, other.field) + ValuesSourceAggregationBuilder other = (ValuesSourceAggregationBuilder) obj; + return Objects.equals(field, other.field) && Objects.equals(format, other.format) && Objects.equals(missing, other.missing) && Objects.equals(script, other.script) - && Objects.equals(targetValueType, other.targetValueType) && Objects.equals(timeZone, other.timeZone) - && Objects.equals(valueType, other.valueType); + && Objects.equals(userValueTypeHint, other.userValueTypeHint); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceAggregatorFactory.java index 9bcccfb00bbd7..10c816f0579fd 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceAggregatorFactory.java @@ -30,47 +30,33 @@ import java.util.List; import java.util.Map; -public abstract class ValuesSourceAggregatorFactory extends AggregatorFactory { +public abstract class ValuesSourceAggregatorFactory extends AggregatorFactory { - protected ValuesSourceConfig config; + protected ValuesSourceConfig config; - public ValuesSourceAggregatorFactory(String name, ValuesSourceConfig config, QueryShardContext queryShardContext, - AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, Map metadata) throws IOException { - super(name, queryShardContext, parent, subFactoriesBuilder, metadata); + public ValuesSourceAggregatorFactory(String name, ValuesSourceConfig config, QueryShardContext queryShardContext, + AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, + Map metaData) throws IOException { + super(name, queryShardContext, parent, subFactoriesBuilder, metaData); this.config = config; } @Override public Aggregator createInternal(SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, - List pipelineAggregators, Map metadata) throws IOException { - VS vs = config.toValuesSource(queryShardContext, this::resolveMissingAny); + List pipelineAggregators, Map metaData) throws IOException { + ValuesSource vs = config.toValuesSource(); if (vs == null) { return createUnmapped(searchContext, parent, pipelineAggregators, metadata); } return doCreateInternal(vs, searchContext, parent, collectsFromSingleBucket, pipelineAggregators, metadata); } - /** - * This method provides a hook for aggregations that need finer grained control over the ValuesSource selected when the user supplies a - * missing value and there is no mapped field to infer the type from. This will only be called for aggregations that specify the - * CoreValuesSourceType.ANY in their constructors (On the builder class). The user supplied object is passed as a parameter, so its - * type * may be inspected as needed. - * - * Generally, only the type of the returned ValuesSource is used, so returning the EMPTY instance of the chosen type is recommended. - * - * @param missing The user supplied missing value - * @return A ValuesSource instance compatible with the supplied parameter - */ - protected ValuesSource resolveMissingAny(Object missing) { - return ValuesSource.Bytes.WithOrdinals.EMPTY; - } - protected abstract Aggregator createUnmapped(SearchContext searchContext, Aggregator parent, List pipelineAggregators, Map metadata) throws IOException; - protected abstract Aggregator doCreateInternal(VS valuesSource, + protected abstract Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceConfig.java b/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceConfig.java index ba599827b88af..6cd62a0d2a622 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceConfig.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceConfig.java @@ -19,12 +19,9 @@ package org.elasticsearch.search.aggregations.support; import org.elasticsearch.common.Nullable; -import org.elasticsearch.common.time.DateFormatter; import org.elasticsearch.index.fielddata.IndexFieldData; import org.elasticsearch.index.fielddata.IndexGeoPointFieldData; -import org.elasticsearch.index.fielddata.IndexHistogramFieldData; import org.elasticsearch.index.fielddata.IndexNumericFieldData; -import org.elasticsearch.index.mapper.DateFieldMapper; import org.elasticsearch.index.mapper.MappedFieldType; import org.elasticsearch.index.mapper.RangeFieldMapper; import org.elasticsearch.index.query.QueryShardContext; @@ -33,105 +30,183 @@ import org.elasticsearch.search.DocValueFormat; import java.time.ZoneId; -import java.time.ZoneOffset; -import java.util.function.Function; +import java.util.function.LongSupplier; /** * A configuration that tells aggregations how to retrieve data from the index * in order to run a specific aggregation. */ -public class ValuesSourceConfig { +public class ValuesSourceConfig { /** - * Resolve a {@link ValuesSourceConfig} given configuration parameters. + * Given the query context and other information, decide on the input {@link ValuesSource} for this aggretation run, and construct a new + * {@link ValuesSourceConfig} based on that {@link ValuesSourceType} + * + * @param context - the query context + * @param userValueTypeHint - User specified value type; used for missing values and scripts + * @param field - The field being aggregated over. At least one of field and script must not be null + * @param script - The script the user specified. At least one of field and script must not be null + * @param missing - A user specified value to apply when the field is missing. Should be of type userValueTypeHint + * @param timeZone - Used to generate a format for dates + * @param format - The format string to apply to this field. Confusingly, this is used for input parsing as well as output formatting + * See https://github.com/elastic/elasticsearch/issues/47469 + * @param defaultValueSourceType - per-aggregation {@link ValuesSource} of last resort. + * @param aggregationName - Name of the aggregation, generally from the aggregation builder. This is used as a lookup key in the + * {@link ValuesSourceRegistry} + * @return - An initialized {@link ValuesSourceConfig} that will yield the appropriate {@link ValuesSourceType} */ - public static ValuesSourceConfig resolve( - QueryShardContext context, - ValueType valueType, - String field, Script script, - Object missing, - ZoneId timeZone, - String format) { - return resolve(context, valueType, field, script, missing, timeZone, format, s -> CoreValuesSourceType.BYTES); + public static ValuesSourceConfig resolve(QueryShardContext context, + ValueType userValueTypeHint, + String field, + Script script, + Object missing, + ZoneId timeZone, + String format, + ValuesSourceType defaultValueSourceType, + String aggregationName) { + + return internalResolve(context, userValueTypeHint, field, script, missing, timeZone, format, defaultValueSourceType, + aggregationName, ValuesSourceConfig::getMappingFromRegistry); } /** - * Resolve a {@link ValuesSourceConfig} given configuration parameters. + * AKA legacy resolve. This method should be called by aggregations not supported by the {@link ValuesSourceRegistry}, to use the + * pre-registry logic to decide on the {@link ValuesSourceType}. New aggregations which extend from + * {@link ValuesSourceAggregationBuilder} should not use this method, preferring {@link ValuesSourceConfig#resolve} instead. + * + * @param context - the query context + * @param userValueTypeHint - User specified value type; used for missing values and scripts + * @param field - The field being aggregated over. At least one of field and script must not be null + * @param script - The script the user specified. At least one of field and script must not be null + * @param missing - A user specified value to apply when the field is missing. Should be of type userValueTypeHint + * @param timeZone - Used to generate a format for dates + * @param format - The format string to apply to this field. Confusingly, this is used for input parsing as well as output formatting + * See https://github.com/elastic/elasticsearch/issues/47469 + * @param defaultValueSourceType - per-aggregation {@link ValuesSource} of last resort. + * @return - An initialized {@link ValuesSourceConfig} that will yield the appropriate {@link ValuesSourceType} */ - public static ValuesSourceConfig resolve( - QueryShardContext context, - ValueType valueType, - String field, Script script, - Object missing, - ZoneId timeZone, - String format, - Function resolveScriptAny - ) { + public static ValuesSourceConfig resolveUnregistered(QueryShardContext context, + ValueType userValueTypeHint, + String field, + Script script, + Object missing, + ZoneId timeZone, + String format, + ValuesSourceType defaultValueSourceType) { + return internalResolve(context, userValueTypeHint, field, script, missing, timeZone, format, defaultValueSourceType, null, + ValuesSourceConfig::getLegacyMapping); + } + private static ValuesSourceConfig internalResolve(QueryShardContext context, + ValueType userValueTypeHint, + String field, + Script script, + Object missing, + ZoneId timeZone, + String format, + ValuesSourceType defaultValueSourceType, + String aggregationName, + FieldResolver fieldResolver + ) { + ValuesSourceConfig config; + MappedFieldType fieldType = null; + ValuesSourceType valuesSourceType; + ValueType scriptValueType = null; + AggregationScript.LeafFactory aggregationScript = null; + boolean unmapped = false; if (field == null) { + // Stand Alone Script Case if (script == null) { - ValuesSourceConfig config = new ValuesSourceConfig<>(CoreValuesSourceType.ANY); - config.format(resolveFormat(null, valueType, timeZone)); - return config; + throw new IllegalStateException( + "value source config is invalid; must have either a field context or a script or marked as unmapped"); } - ValuesSourceType valuesSourceType = valueType != null ? valueType.getValuesSourceType() : CoreValuesSourceType.ANY; - if (valuesSourceType == CoreValuesSourceType.ANY) { - // the specific value source type is undefined, but for scripts, - // we need to have a specific value source - // type to know how to handle the script values, so we fallback - // on Bytes - valuesSourceType = resolveScriptAny.apply(script); + /* + * This is the Stand Alone Script path. We should have a script that will produce a value independent of the presence or + * absence of any one field. The type of the script is given by the userValueTypeHint field, if the user specified a type, + * or the aggregation's default type if the user didn't. + */ + if (userValueTypeHint != null) { + valuesSourceType = userValueTypeHint.getValuesSourceType(); + } else { + valuesSourceType = defaultValueSourceType; } - ValuesSourceConfig config = new ValuesSourceConfig<>(valuesSourceType); - config.missing(missing); - config.timezone(timeZone); - config.format(resolveFormat(format, valueType, timeZone)); - config.script(createScript(script, context)); - config.scriptValueType(valueType); - return config; - } - - MappedFieldType fieldType = context.fieldMapper(field); - if (fieldType == null) { - ValuesSourceType valuesSourceType = valueType != null ? valueType.getValuesSourceType() : CoreValuesSourceType.ANY; - ValuesSourceConfig config = new ValuesSourceConfig<>(valuesSourceType); - config.missing(missing); - config.timezone(timeZone); - config.format(resolveFormat(format, valueType, timeZone)); - config.unmapped(true); - if (valueType != null) { - // todo do we really need this for unmapped? - config.scriptValueType(valueType); + aggregationScript = createScript(script, context); + scriptValueType = userValueTypeHint; + } else { + // Field case + fieldType = context.fieldMapper(field); + if (fieldType == null) { + /* Unmapped Field Case + * We got here because the user specified a field, but it doesn't exist on this index, possibly because of a wildcard index + * pattern. In this case, we're going to end up using the EMPTY variant of the ValuesSource, and possibly applying a user + * specified missing value. + */ + if (userValueTypeHint != null) { + valuesSourceType = userValueTypeHint.getValuesSourceType(); + } else { + valuesSourceType = defaultValueSourceType; + } + unmapped = true; + if (userValueTypeHint != null) { + // todo do we really need this for unmapped? + scriptValueType = userValueTypeHint; + } + } else { + valuesSourceType = fieldResolver.getValuesSourceType(context, fieldType, aggregationName, userValueTypeHint, + defaultValueSourceType); + aggregationScript = createScript(script, context); } - return config; } + config = new ValuesSourceConfig(valuesSourceType, fieldType, unmapped, aggregationScript, scriptValueType , context); + config.format(resolveFormat(format, valuesSourceType, timeZone, fieldType)); + config.missing(missing); + config.timezone(timeZone); + return config; + } + + @FunctionalInterface + private interface FieldResolver { + ValuesSourceType getValuesSourceType( + QueryShardContext context, + MappedFieldType fieldType, + String aggregationName, + ValueType userValueTypeHint, + ValuesSourceType defaultValuesSourceType); + + } + private static ValuesSourceType getMappingFromRegistry( + QueryShardContext context, + MappedFieldType fieldType, + String aggregationName, + ValueType userValueTypeHint, + ValuesSourceType defaultValuesSourceType) { IndexFieldData indexFieldData = context.getForField(fieldType); + return context.getValuesSourceRegistry().getValuesSourceType(fieldType, aggregationName, indexFieldData, + userValueTypeHint, defaultValuesSourceType); + } - ValuesSourceConfig config; + private static ValuesSourceType getLegacyMapping( + QueryShardContext context, + MappedFieldType fieldType, + String aggregationName, + ValueType userValueTypeHint, + ValuesSourceType defaultValuesSourceType) { + IndexFieldData indexFieldData = context.getForField(fieldType); if (indexFieldData instanceof IndexNumericFieldData) { - config = new ValuesSourceConfig<>(CoreValuesSourceType.NUMERIC); + return CoreValuesSourceType.NUMERIC; } else if (indexFieldData instanceof IndexGeoPointFieldData) { - config = new ValuesSourceConfig<>(CoreValuesSourceType.GEOPOINT); + return CoreValuesSourceType.GEOPOINT; } else if (fieldType instanceof RangeFieldMapper.RangeFieldType) { - config = new ValuesSourceConfig<>(CoreValuesSourceType.RANGE); - } else if (indexFieldData instanceof IndexHistogramFieldData) { - config = new ValuesSourceConfig<>(CoreValuesSourceType.HISTOGRAM); + return CoreValuesSourceType.RANGE; } else { - if (valueType == null) { - config = new ValuesSourceConfig<>(CoreValuesSourceType.BYTES); + if (userValueTypeHint == null) { + return defaultValuesSourceType; } else { - config = new ValuesSourceConfig<>(valueType.getValuesSourceType()); + return userValueTypeHint.getValuesSourceType(); } } - - config.fieldContext(new FieldContext(field, indexFieldData, fieldType)); - config.missing(missing); - config.timezone(timeZone); - config.script(createScript(script, context)); - config.format(fieldType.docValueFormat(format, timeZone)); - return config; } private static AggregationScript.LeafFactory createScript(Script script, QueryShardContext context) { @@ -143,36 +218,65 @@ private static AggregationScript.LeafFactory createScript(Script script, QuerySh } } - private static DocValueFormat resolveFormat(@Nullable String format, @Nullable ValueType valueType, @Nullable ZoneId tz) { - if (valueType == null) { - return DocValueFormat.RAW; // we can't figure it out - } - DocValueFormat valueFormat = valueType.defaultFormat; - if (valueFormat instanceof DocValueFormat.Decimal && format != null) { - valueFormat = new DocValueFormat.Decimal(format); - } - if (valueFormat instanceof DocValueFormat.DateTime && format != null) { - valueFormat = new DocValueFormat.DateTime(DateFormatter.forPattern(format), tz != null ? tz : ZoneOffset.UTC, - DateFieldMapper.Resolution.MILLISECONDS); + private static DocValueFormat resolveFormat(@Nullable String format, @Nullable ValuesSourceType valuesSourceType, @Nullable ZoneId tz, + MappedFieldType fieldType) { + if (fieldType != null) { + return fieldType.docValueFormat(format, tz); } - return valueFormat; + // Script or Unmapped case + return valuesSourceType.getFormatter(format, tz); } - private final ValuesSourceType valueSourceType; + /** + * Special case factory method, intended to be used by aggregations which have some specialized logic for figuring out what field they + * are operating on, for example Parent and Child join aggregations, which use the join relation to find the field they are reading from + * rather than a user specified field. + */ + public static ValuesSourceConfig resolveFieldOnly(MappedFieldType fieldType, + QueryShardContext queryShardContext) { + return new ValuesSourceConfig(fieldType.getValuesSourceType(), fieldType, false, null, null, queryShardContext); + } + + /** + * Convenience method for creating unmapped configs + */ + public static ValuesSourceConfig resolveUnmapped(ValuesSourceType valuesSourceType, QueryShardContext queryShardContext) { + return new ValuesSourceConfig(valuesSourceType, null, true, null, null, queryShardContext); + } + + private final ValuesSourceType valuesSourceType; private FieldContext fieldContext; private AggregationScript.LeafFactory script; private ValueType scriptValueType; - private boolean unmapped = false; + private boolean unmapped; private DocValueFormat format = DocValueFormat.RAW; private Object missing; private ZoneId timeZone; + private LongSupplier nowSupplier; + + + public ValuesSourceConfig(ValuesSourceType valuesSourceType, + MappedFieldType fieldType, + boolean unmapped, + AggregationScript.LeafFactory script, + ValueType scriptValueType, + QueryShardContext queryShardContext) { + if (unmapped && fieldType != null) { + throw new IllegalStateException("value source config is invalid; marked as unmapped but specified a mapped field"); + } + this.valuesSourceType = valuesSourceType; + if (fieldType != null) { + this.fieldContext = new FieldContext(fieldType.name(), queryShardContext.getForField(fieldType), fieldType); + } + this.unmapped = unmapped; + this.script = script; + this.scriptValueType = scriptValueType; + this.nowSupplier = queryShardContext::nowInMillis; - public ValuesSourceConfig(ValuesSourceType valueSourceType) { - this.valueSourceType = valueSourceType; } public ValuesSourceType valueSourceType() { - return valueSourceType; + return valuesSourceType; } public FieldContext fieldContext() { @@ -191,36 +295,16 @@ public boolean valid() { return fieldContext != null || script != null || unmapped; } - public ValuesSourceConfig fieldContext(FieldContext fieldContext) { - this.fieldContext = fieldContext; - return this; - } - - public ValuesSourceConfig script(AggregationScript.LeafFactory script) { - this.script = script; - return this; - } - - public ValuesSourceConfig scriptValueType(ValueType scriptValueType) { - this.scriptValueType = scriptValueType; - return this; - } - public ValueType scriptValueType() { return this.scriptValueType; } - public ValuesSourceConfig unmapped(boolean unmapped) { - this.unmapped = unmapped; - return this; - } - - public ValuesSourceConfig format(final DocValueFormat format) { + private ValuesSourceConfig format(final DocValueFormat format) { this.format = format; return this; } - public ValuesSourceConfig missing(final Object missing) { + private ValuesSourceConfig missing(final Object missing) { this.missing = missing; return this; } @@ -229,7 +313,7 @@ public Object missing() { return this.missing; } - public ValuesSourceConfig timezone(final ZoneId timeZone) { + private ValuesSourceConfig timezone(final ZoneId timeZone) { this.timeZone = timeZone; return this; } @@ -242,49 +326,41 @@ public DocValueFormat format() { return format; } + /** + * Transform the {@link ValuesSourceType} we selected in resolve into the specific {@link ValuesSource} instance to use for this shard + * @return - A {@link ValuesSource} ready to be read from by an aggregator + */ @Nullable - public VS toValuesSource(QueryShardContext context) { - return toValuesSource(context, value -> ValuesSource.Bytes.WithOrdinals.EMPTY); - } - - /** Get a value source given its configuration. A return value of null indicates that - * no value source could be built. */ - @Nullable - public VS toValuesSource(QueryShardContext context, Function resolveMissingAny) { + public ValuesSource toValuesSource() { if (!valid()) { + // TODO: resolve no longer generates invalid configs. Once VSConfig is immutable, we can drop this check throw new IllegalStateException( "value source config is invalid; must have either a field context or a script or marked as unwrapped"); } - final VS vs; + final ValuesSource vs; if (unmapped()) { if (missing() == null) { - // otherwise we will have values because of the missing value + /* Null values source signals to the AggregationBuilder to use the createUnmapped method, which aggregator factories can + * override to provide an aggregator optimized to return empty values + */ vs = null; - } else if (valueSourceType() == CoreValuesSourceType.ANY) { - // TODO: Clean up special cases around CoreValuesSourceType.ANY - vs = (VS) resolveMissingAny.apply(missing()); } else { - vs = (VS) valueSourceType().getEmpty(); + vs = valueSourceType().getEmpty(); } } else { if (fieldContext() == null) { - vs = (VS) valueSourceType().getScript(script(), scriptValueType()); + // Script case + vs = valueSourceType().getScript(script(), scriptValueType()); } else { - if (valueSourceType() == CoreValuesSourceType.ANY) { - // TODO: Clean up special cases around CoreValuesSourceType.ANY - // falling back to bytes values - vs = (VS) CoreValuesSourceType.BYTES.getField(fieldContext(), script()); - } else { - // TODO: Better docs for Scripts vs Scripted Fields - vs = (VS) valueSourceType().getField(fieldContext(), script()); - } + // Field or Value Script case + vs = valueSourceType().getField(fieldContext(), script()); } } if (missing() == null) { return vs; } - return (VS) valueSourceType().replaceMissing(vs, missing, format, context::nowInMillis); + return valueSourceType().replaceMissing(vs, missing, format, nowSupplier); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceParserHelper.java b/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceParserHelper.java deleted file mode 100644 index 567862ca92e3e..0000000000000 --- a/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceParserHelper.java +++ /dev/null @@ -1,112 +0,0 @@ -/* - * Licensed to Elasticsearch under one or more contributor - * license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright - * ownership. Elasticsearch licenses this file to you under - * the Apache License, Version 2.0 (the "License"); you may - * not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, - * software distributed under the License is distributed on an - * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - * KIND, either express or implied. See the License for the - * specific language governing permissions and limitations - * under the License. - */ - -package org.elasticsearch.search.aggregations.support; - -import org.elasticsearch.common.ParseField; -import org.elasticsearch.common.ParsingException; -import org.elasticsearch.common.xcontent.AbstractObjectParser; -import org.elasticsearch.common.xcontent.ObjectParser; -import org.elasticsearch.common.xcontent.XContentParser; -import org.elasticsearch.script.Script; - -import java.time.ZoneId; -import java.time.ZoneOffset; - -public final class ValuesSourceParserHelper { - - private ValuesSourceParserHelper() {} // utility class, no instantiation - - public static void declareAnyFields( - AbstractObjectParser, T> objectParser, - boolean scriptable, boolean formattable) { - declareAnyFields(objectParser, scriptable, formattable, false); - } - - public static void declareAnyFields( - AbstractObjectParser, T> objectParser, - boolean scriptable, boolean formattable, boolean timezoneAware) { - declareFields(objectParser, scriptable, formattable, timezoneAware, null); - } - - public static void declareNumericFields( - AbstractObjectParser, T> objectParser, - boolean scriptable, boolean formattable, boolean timezoneAware) { - declareFields(objectParser, scriptable, formattable, timezoneAware, ValueType.NUMERIC); - } - - public static void declareBytesFields( - AbstractObjectParser, T> objectParser, - boolean scriptable, boolean formattable) { - declareFields(objectParser, scriptable, formattable, false, ValueType.STRING); - } - - public static void declareGeoFields( - AbstractObjectParser, T> objectParser, - boolean scriptable, boolean formattable) { - declareFields(objectParser, scriptable, formattable, false, ValueType.GEOPOINT); - } - - private static void declareFields( - AbstractObjectParser, T> objectParser, - boolean scriptable, boolean formattable, boolean timezoneAware, ValueType targetValueType) { - - - objectParser.declareField(ValuesSourceAggregationBuilder::field, XContentParser::text, - ParseField.CommonFields.FIELD, ObjectParser.ValueType.STRING); - - objectParser.declareField(ValuesSourceAggregationBuilder::missing, XContentParser::objectText, - ParseField.CommonFields.MISSING, ObjectParser.ValueType.VALUE); - - objectParser.declareField(ValuesSourceAggregationBuilder::valueType, p -> { - ValueType valueType = ValueType.resolveForScript(p.text()); - if (targetValueType != null && valueType.isNotA(targetValueType)) { - throw new ParsingException(p.getTokenLocation(), - "Aggregation [" + objectParser.getName() + "] was configured with an incompatible value type [" - + valueType + "]. It can only work on value of type [" - + targetValueType + "]"); - } - return valueType; - }, ValueType.VALUE_TYPE, ObjectParser.ValueType.STRING); - - if (formattable) { - objectParser.declareField(ValuesSourceAggregationBuilder::format, XContentParser::text, - ParseField.CommonFields.FORMAT, ObjectParser.ValueType.STRING); - } - - if (scriptable) { - objectParser.declareField(ValuesSourceAggregationBuilder::script, - (parser, context) -> Script.parse(parser), - Script.SCRIPT_PARSE_FIELD, ObjectParser.ValueType.OBJECT_OR_STRING); - } - - if (timezoneAware) { - objectParser.declareField(ValuesSourceAggregationBuilder::timeZone, p -> { - if (p.currentToken() == XContentParser.Token.VALUE_STRING) { - return ZoneId.of(p.text()); - } else { - return ZoneOffset.ofHours(p.intValue()); - } - }, ParseField.CommonFields.TIME_ZONE, ObjectParser.ValueType.LONG); - } - } - - - -} diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceRegistry.java b/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceRegistry.java new file mode 100644 index 0000000000000..5356821c5f58f --- /dev/null +++ b/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceRegistry.java @@ -0,0 +1,193 @@ +/* + * Licensed to Elasticsearch under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.elasticsearch.search.aggregations.support; + +import org.elasticsearch.index.fielddata.IndexFieldData; +import org.elasticsearch.index.fielddata.IndexGeoPointFieldData; +import org.elasticsearch.index.fielddata.IndexNumericFieldData; +import org.elasticsearch.index.mapper.MappedFieldType; +import org.elasticsearch.index.mapper.RangeFieldMapper; +import org.elasticsearch.index.query.QueryShardContext; +import org.elasticsearch.search.SearchModule; +import org.elasticsearch.search.aggregations.AggregationExecutionException; + +import java.util.AbstractMap; +import java.util.List; +import java.util.Map; +import java.util.function.Predicate; + +/** + * {@link ValuesSourceRegistry} holds the mapping from {@link ValuesSourceType}s to {@link AggregatorSupplier}s. DO NOT directly + * instantiate this class, instead get an already-configured copy from {@link QueryShardContext#getValuesSourceRegistry()}, or (in the case + * of some test scenarios only) directly from {@link SearchModule#getValuesSourceRegistry()} + * + */ +public class ValuesSourceRegistry { + // Maps Aggregation names to (ValuesSourceType, Supplier) pairs, keyed by ValuesSourceType + private Map, AggregatorSupplier>>> aggregatorRegistry = Map.of(); + + /** + * Register a ValuesSource to Aggregator mapping. + * + * Threading behavior notes: This call is both synchronized and expensive. It copies the entire existing mapping structure each + * time it is invoked. We expect that register will be called a small number of times during startup only (as plugins are being + * registered) and we can tolerate the cost at that time. Once all plugins are registered, we should never need to call register + * again. Comparatively, we expect to do many reads from the registry data structures, and those reads may be interleaved on + * different worker threads. Thus we want to optimize the read case to be thread safe and fast, which the immutable + * collections do well. Using immutable collections requires a copy on write mechanic, thus the somewhat non-intuitive + * implementation of this method. + * @param aggregationName The name of the family of aggregations, typically found via {@link ValuesSourceAggregationBuilder#getType()} + * @param appliesTo A predicate which accepts the resolved {@link ValuesSourceType} and decides if the given aggregator can be applied + * to that type. + * @param aggregatorSupplier An Aggregation-specific specialization of AggregatorSupplier which will construct the mapped aggregator + */ + public synchronized void register(String aggregationName, Predicate appliesTo, + AggregatorSupplier aggregatorSupplier) { + AbstractMap.SimpleEntry[] mappings; + if (aggregatorRegistry.containsKey(aggregationName)) { + List currentMappings = aggregatorRegistry.get(aggregationName); + mappings = (AbstractMap.SimpleEntry[]) currentMappings.toArray(new AbstractMap.SimpleEntry[currentMappings.size() + 1]); + } else { + mappings = new AbstractMap.SimpleEntry[1]; + } + mappings[mappings.length - 1] = new AbstractMap.SimpleEntry<>(appliesTo, aggregatorSupplier); + aggregatorRegistry = copyAndAdd(aggregatorRegistry,new AbstractMap.SimpleEntry<>(aggregationName, List.of(mappings))); + } + + /** + * Register a ValuesSource to Aggregator mapping. This version provides a convenience method for mappings that only apply to a single + * {@link ValuesSourceType}, to allow passing in the type and auto-wrapping it in a predicate + * @param aggregationName The name of the family of aggregations, typically found via {@link ValuesSourceAggregationBuilder#getType()} + * @param valuesSourceType The ValuesSourceType this mapping applies to. + * @param aggregatorSupplier An Aggregation-specific specialization of AggregatorSupplier which will construct the mapped aggregator + * from the aggregation standard set of parameters + */ + public void register(String aggregationName, ValuesSourceType valuesSourceType, AggregatorSupplier aggregatorSupplier) { + register(aggregationName, (candidate) -> valuesSourceType.equals(candidate), aggregatorSupplier); + } + + /** + * Register a ValuesSource to Aggregator mapping. This version provides a convenience method for mappings that only apply to a known + * list of {@link ValuesSourceType}, to allow passing in the type and auto-wrapping it in a predicate + * @param aggregationName The name of the family of aggregations, typically found via {@link ValuesSourceAggregationBuilder#getType()} + * @param valuesSourceTypes The ValuesSourceTypes this mapping applies to. + * @param aggregatorSupplier An Aggregation-specific specialization of AggregatorSupplier which will construct the mapped aggregator + * from the aggregation standard set of parameters + */ + public void register(String aggregationName, List valuesSourceTypes, AggregatorSupplier aggregatorSupplier) { + register(aggregationName, (candidate) -> { + for (ValuesSourceType valuesSourceType : valuesSourceTypes) { + if (valuesSourceType.equals(candidate)) { + return true; + } + } + return false; + }, aggregatorSupplier); + } + + /** + * Register an aggregator that applies to any values source type. This is a convenience method for aggregations that do not care at all + * about the types of their inputs. Aggregations using this version of registration should not make any other registrations, as the + * aggregator registered using this function will be applied in all cases. + * + * @param aggregationName The name of the family of aggregations, typically found via {@link ValuesSourceAggregationBuilder#getType()} + * @param aggregatorSupplier An Aggregation-specific specialization of AggregatorSupplier which will construct the mapped aggregator + * from the aggregation standard set of parameters. + */ + public void registerAny(String aggregationName, AggregatorSupplier aggregatorSupplier) { + register(aggregationName, (ignored) -> true, aggregatorSupplier); + } + + private AggregatorSupplier findMatchingSuppier(ValuesSourceType valuesSourceType, + List, AggregatorSupplier>> supportedTypes) { + for (Map.Entry, AggregatorSupplier> candidate : supportedTypes) { + if (candidate.getKey().test(valuesSourceType)) { + return candidate.getValue(); + } + } + return null; + } + + public AggregatorSupplier getAggregator(ValuesSourceType valuesSourceType, String aggregationName) { + if (aggregationName != null && aggregatorRegistry.containsKey(aggregationName)) { + AggregatorSupplier supplier = findMatchingSuppier(valuesSourceType, aggregatorRegistry.get(aggregationName)); + if (supplier == null) { + throw new AggregationExecutionException("ValuesSource type " + valuesSourceType.toString() + + " is not supported for aggregation" + aggregationName); + } + return supplier; + } + throw new AggregationExecutionException("Unregistered Aggregation [" + aggregationName + "]"); + } + + public ValuesSourceType getValuesSourceType(MappedFieldType fieldType, String aggregationName, + // TODO: the following arguments are only needed for the legacy case + IndexFieldData indexFieldData, + ValueType valueType, + ValuesSourceType defaultValuesSourceType) { + if (aggregationName != null && aggregatorRegistry.containsKey(aggregationName)) { + // This will throw if the field doesn't support values sources, although really we probably threw much earlier in that case + ValuesSourceType valuesSourceType = fieldType.getValuesSourceType(); + if (aggregatorRegistry.get(aggregationName) != null + && findMatchingSuppier(valuesSourceType, aggregatorRegistry.get(aggregationName)) != null) { + return valuesSourceType; + } + String fieldDescription = fieldType.typeName() + "(" + fieldType.toString() + ")"; + throw new IllegalArgumentException("Field [" + fieldType.name() + "] of type [" + fieldDescription + + "] is not supported for aggregation [" + aggregationName + "]"); + } else { + // TODO: Legacy resolve logic; remove this after converting all aggregations to the new system + if (indexFieldData instanceof IndexNumericFieldData) { + return CoreValuesSourceType.NUMERIC; + } else if (indexFieldData instanceof IndexGeoPointFieldData) { + return CoreValuesSourceType.GEOPOINT; + } else if (fieldType instanceof RangeFieldMapper.RangeFieldType) { + return CoreValuesSourceType.RANGE; + } else { + if (valueType == null) { + return defaultValuesSourceType; + } else { + return valueType.getValuesSourceType(); + } + } + } + } + + private static Map copyAndAdd(Map source, Map.Entry newValue) { + Map.Entry[] entries; + if (source.containsKey(newValue.getKey())) { + // Replace with new value + entries = new Map.Entry[source.size()]; + int i = 0; + for (Map.Entry entry : source.entrySet()) { + if (entry.getKey() == newValue.getKey()) { + entries[i] = newValue; + } else { + entries[i] = entry; + } + i++; + } + } else { + entries = source.entrySet().toArray(new Map.Entry[source.size() + 1]); + entries[entries.length - 1] = newValue; + } + return Map.ofEntries(entries); + } + +} diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceType.java b/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceType.java index 52f39cfe9326f..040ede186c7ab 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceType.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceType.java @@ -22,6 +22,7 @@ import org.elasticsearch.script.AggregationScript; import org.elasticsearch.search.DocValueFormat; +import java.time.ZoneId; import java.util.function.LongSupplier; /** @@ -70,4 +71,18 @@ public interface ValuesSourceType { */ ValuesSource replaceMissing(ValuesSource valuesSource, Object rawMissing, DocValueFormat docValueFormat, LongSupplier now); + + /** + * This method provides a hook for specifying a type-specific formatter. When {@link ValuesSourceConfig} can resolve a + * {@link org.elasticsearch.index.mapper.MappedFieldType}, it prefers to get the formatter from there. Only when a field can't be + * resolved (which is to say script cases and unmapped field cases), it will fall back to calling this method on whatever + * {@link ValuesSourceType} it was able to resolve to. + * + * @param format - User supplied format string (Optional) + * @param tz - User supplied time zone (Optional) + * @return - A formatter object, configured with the passed in settings if appropriate. + */ + default DocValueFormat getFormatter(String format, ZoneId tz) { + return DocValueFormat.RAW; + } } diff --git a/server/src/test/java/org/elasticsearch/action/search/TransportSearchActionSingleNodeTests.java b/server/src/test/java/org/elasticsearch/action/search/TransportSearchActionSingleNodeTests.java index 10f252c30dc3b..7e27aaac59ecb 100644 --- a/server/src/test/java/org/elasticsearch/action/search/TransportSearchActionSingleNodeTests.java +++ b/server/src/test/java/org/elasticsearch/action/search/TransportSearchActionSingleNodeTests.java @@ -150,7 +150,7 @@ public void testFinalReduce() { SearchSourceBuilder source = new SearchSourceBuilder(); source.size(0); originalRequest.source(source); - TermsAggregationBuilder terms = new TermsAggregationBuilder("terms", ValueType.NUMERIC); + TermsAggregationBuilder terms = new TermsAggregationBuilder("terms").userValueTypeHint(ValueType.NUMERIC); terms.field("price"); terms.size(1); source.aggregation(terms); diff --git a/server/src/test/java/org/elasticsearch/cluster/metadata/MetadataCreateIndexServiceTests.java b/server/src/test/java/org/elasticsearch/cluster/metadata/MetadataCreateIndexServiceTests.java index 26d4560d6f19d..46d97532681c6 100644 --- a/server/src/test/java/org/elasticsearch/cluster/metadata/MetadataCreateIndexServiceTests.java +++ b/server/src/test/java/org/elasticsearch/cluster/metadata/MetadataCreateIndexServiceTests.java @@ -135,7 +135,7 @@ public void setupCreateIndexRequestAndAliasValidator() { queryShardContext = new QueryShardContext(0, new IndexSettings(IndexMetadata.builder("test").settings(indexSettings).build(), indexSettings), BigArrays.NON_RECYCLING_INSTANCE, null, null, null, null, null, xContentRegistry(), writableRegistry(), - null, null, () -> randomNonNegativeLong(), null, null, () -> true); + null, null, () -> randomNonNegativeLong(), null, null, () -> true, null); } private ClusterState createClusterState(String name, int numShards, int numReplicas, Settings settings) { diff --git a/server/src/test/java/org/elasticsearch/index/IndexModuleTests.java b/server/src/test/java/org/elasticsearch/index/IndexModuleTests.java index 83c2a28c362d5..d452cc7d44c99 100644 --- a/server/src/test/java/org/elasticsearch/index/IndexModuleTests.java +++ b/server/src/test/java/org/elasticsearch/index/IndexModuleTests.java @@ -162,7 +162,7 @@ public void tearDown() throws Exception { private IndexService newIndexService(IndexModule module) throws IOException { return module.newIndexService(CREATE_INDEX, nodeEnvironment, xContentRegistry(), deleter, circuitBreakerService, bigArrays, threadPool, scriptService, clusterService, null, indicesQueryCache, mapperRegistry, - new IndicesFieldDataCache(settings, listener), writableRegistry(), () -> false); + new IndicesFieldDataCache(settings, listener), writableRegistry(), () -> false, null); } public void testWrapperIsBound() throws IOException { diff --git a/server/src/test/java/org/elasticsearch/index/mapper/DateFieldTypeTests.java b/server/src/test/java/org/elasticsearch/index/mapper/DateFieldTypeTests.java index 863201709eb28..255e79a4f8929 100644 --- a/server/src/test/java/org/elasticsearch/index/mapper/DateFieldTypeTests.java +++ b/server/src/test/java/org/elasticsearch/index/mapper/DateFieldTypeTests.java @@ -232,7 +232,7 @@ public void testTermQuery() { QueryShardContext context = new QueryShardContext(0, new IndexSettings(IndexMetadata.builder("foo").settings(indexSettings).build(), indexSettings), BigArrays.NON_RECYCLING_INSTANCE, null, null, null, null, null, - xContentRegistry(), writableRegistry(), null, null, () -> nowInMillis, null, null, () -> true); + xContentRegistry(), writableRegistry(), null, null, () -> nowInMillis, null, null, () -> true, null); MappedFieldType ft = createDefaultFieldType(); ft.setName("field"); String date = "2015-10-12T14:10:55"; @@ -255,7 +255,7 @@ public void testRangeQuery() throws IOException { QueryShardContext context = new QueryShardContext(0, new IndexSettings(IndexMetadata.builder("foo").settings(indexSettings).build(), indexSettings), BigArrays.NON_RECYCLING_INSTANCE, null, null, null, null, null, xContentRegistry(), writableRegistry(), - null, null, () -> nowInMillis, null, null, () -> true); + null, null, () -> nowInMillis, null, null, () -> true, null); MappedFieldType ft = createDefaultFieldType(); ft.setName("field"); String date1 = "2015-10-12T14:10:55"; diff --git a/server/src/test/java/org/elasticsearch/index/mapper/FieldNamesFieldTypeTests.java b/server/src/test/java/org/elasticsearch/index/mapper/FieldNamesFieldTypeTests.java index 06025774e635e..f9ab7934b3ef7 100644 --- a/server/src/test/java/org/elasticsearch/index/mapper/FieldNamesFieldTypeTests.java +++ b/server/src/test/java/org/elasticsearch/index/mapper/FieldNamesFieldTypeTests.java @@ -68,7 +68,7 @@ public void testTermQuery() { QueryShardContext queryShardContext = new QueryShardContext(0, indexSettings, BigArrays.NON_RECYCLING_INSTANCE, null, null, mapperService, - null, null, null, null, null, null, () -> 0L, null, null, () -> true); + null, null, null, null, null, null, () -> 0L, null, null, () -> true, null); fieldNamesFieldType.setEnabled(true); Query termQuery = fieldNamesFieldType.termQuery("field_name", queryShardContext); assertEquals(new TermQuery(new Term(FieldNamesFieldMapper.CONTENT_TYPE, "field_name")), termQuery); diff --git a/server/src/test/java/org/elasticsearch/index/mapper/IndexFieldTypeTests.java b/server/src/test/java/org/elasticsearch/index/mapper/IndexFieldTypeTests.java index 99a7605ab51f3..aef0b793b981f 100644 --- a/server/src/test/java/org/elasticsearch/index/mapper/IndexFieldTypeTests.java +++ b/server/src/test/java/org/elasticsearch/index/mapper/IndexFieldTypeTests.java @@ -78,6 +78,6 @@ private QueryShardContext createContext() { Predicate indexNameMatcher = pattern -> Regex.simpleMatch(pattern, "index"); return new QueryShardContext(0, indexSettings, null, null, null, null, null, null, xContentRegistry(), writableRegistry(), - null, null, System::currentTimeMillis, null, indexNameMatcher, () -> true); + null, null, System::currentTimeMillis, null, indexNameMatcher, () -> true, null); } } diff --git a/server/src/test/java/org/elasticsearch/index/mapper/RangeFieldTypeTests.java b/server/src/test/java/org/elasticsearch/index/mapper/RangeFieldTypeTests.java index 632732b64b6aa..a4e3f55f9c28c 100644 --- a/server/src/test/java/org/elasticsearch/index/mapper/RangeFieldTypeTests.java +++ b/server/src/test/java/org/elasticsearch/index/mapper/RangeFieldTypeTests.java @@ -230,7 +230,7 @@ private QueryShardContext createContext() { .put(IndexMetadata.SETTING_VERSION_CREATED, Version.CURRENT).build(); IndexSettings idxSettings = IndexSettingsModule.newIndexSettings(randomAlphaOfLengthBetween(1, 10), indexSettings); return new QueryShardContext(0, idxSettings, BigArrays.NON_RECYCLING_INSTANCE, null, null, null, null, null, - xContentRegistry(), writableRegistry(), null, null, () -> nowInMillis, null, null, () -> true); + xContentRegistry(), writableRegistry(), null, null, () -> nowInMillis, null, null, () -> true, null); } public void testDateRangeQueryUsingMappingFormat() { diff --git a/server/src/test/java/org/elasticsearch/index/query/IntervalQueryBuilderTests.java b/server/src/test/java/org/elasticsearch/index/query/IntervalQueryBuilderTests.java index 3593c2bc4d8c8..63bd21ad8df1d 100644 --- a/server/src/test/java/org/elasticsearch/index/query/IntervalQueryBuilderTests.java +++ b/server/src/test/java/org/elasticsearch/index/query/IntervalQueryBuilderTests.java @@ -424,7 +424,7 @@ public FactoryType compile(Script script, ScriptContext true); + null, scriptService, null, null, null, null, null, null, null, () -> true, null); String json = "{ \"intervals\" : { \"" + TEXT_FIELD_NAME + "\": { " + "\"match\" : { " + diff --git a/server/src/test/java/org/elasticsearch/index/query/QueryShardContextTests.java b/server/src/test/java/org/elasticsearch/index/query/QueryShardContextTests.java index 03ca6fbf70074..147af47d41372 100644 --- a/server/src/test/java/org/elasticsearch/index/query/QueryShardContextTests.java +++ b/server/src/test/java/org/elasticsearch/index/query/QueryShardContextTests.java @@ -153,6 +153,6 @@ public static QueryShardContext createQueryShardContext(String indexUuid, String (mappedFieldType, idxName) -> mappedFieldType.fielddataBuilder(idxName).build(indexSettings, mappedFieldType, null, null, null), mapperService, null, null, NamedXContentRegistry.EMPTY, new NamedWriteableRegistry(Collections.emptyList()), - null, null, () -> nowInMillis, clusterAlias, null, () -> true); + null, null, () -> nowInMillis, clusterAlias, null, () -> true, null); } } diff --git a/server/src/test/java/org/elasticsearch/index/query/RangeQueryRewriteTests.java b/server/src/test/java/org/elasticsearch/index/query/RangeQueryRewriteTests.java index c43470ea3b21c..263a9eb95942f 100644 --- a/server/src/test/java/org/elasticsearch/index/query/RangeQueryRewriteTests.java +++ b/server/src/test/java/org/elasticsearch/index/query/RangeQueryRewriteTests.java @@ -41,7 +41,7 @@ public void testRewriteMissingField() throws Exception { IndexReader reader = new MultiReader(); QueryRewriteContext context = new QueryShardContext(0, indexService.getIndexSettings(), BigArrays.NON_RECYCLING_INSTANCE, null, null, indexService.mapperService(), null, null, xContentRegistry(), writableRegistry(), - null, new IndexSearcher(reader), null, null, null, () -> true); + null, new IndexSearcher(reader), null, null, null, () -> true, null); RangeQueryBuilder range = new RangeQueryBuilder("foo"); assertEquals(Relation.DISJOINT, range.getRelation(context)); } @@ -59,7 +59,7 @@ public void testRewriteMissingReader() throws Exception { new CompressedXContent(mapping), MergeReason.MAPPING_UPDATE); QueryRewriteContext context = new QueryShardContext(0, indexService.getIndexSettings(), null, null, null, indexService.mapperService(), null, null, xContentRegistry(), writableRegistry(), - null, null, null, null, null, () -> true); + null, null, null, null, null, () -> true, null); RangeQueryBuilder range = new RangeQueryBuilder("foo"); // can't make assumptions on a missing reader, so it must return INTERSECT assertEquals(Relation.INTERSECTS, range.getRelation(context)); @@ -79,7 +79,7 @@ public void testRewriteEmptyReader() throws Exception { IndexReader reader = new MultiReader(); QueryRewriteContext context = new QueryShardContext(0, indexService.getIndexSettings(), BigArrays.NON_RECYCLING_INSTANCE, null, null, indexService.mapperService(), null, null, xContentRegistry(), writableRegistry(), - null, new IndexSearcher(reader), null, null, null, () -> true); + null, new IndexSearcher(reader), null, null, null, () -> true, null); RangeQueryBuilder range = new RangeQueryBuilder("foo"); // no values -> DISJOINT assertEquals(Relation.DISJOINT, range.getRelation(context)); diff --git a/server/src/test/java/org/elasticsearch/search/SearchModuleTests.java b/server/src/test/java/org/elasticsearch/search/SearchModuleTests.java index 7c9fe82048aee..ab99e796ccc00 100644 --- a/server/src/test/java/org/elasticsearch/search/SearchModuleTests.java +++ b/server/src/test/java/org/elasticsearch/search/SearchModuleTests.java @@ -49,9 +49,11 @@ import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; import org.elasticsearch.search.aggregations.pipeline.SimpleModel; import org.elasticsearch.search.aggregations.support.ValuesSource; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import org.elasticsearch.search.fetch.FetchSubPhase; import org.elasticsearch.search.fetch.subphase.ExplainPhase; import org.elasticsearch.search.fetch.subphase.highlight.CustomHighlighter; @@ -370,21 +372,26 @@ public List> getRescorers() { /** * Dummy test {@link AggregationBuilder} used to test registering aggregation builders. */ - private static class TestAggregationBuilder extends ValuesSourceAggregationBuilder { + private static class TestAggregationBuilder extends ValuesSourceAggregationBuilder { protected TestAggregationBuilder(TestAggregationBuilder clone, Builder factoriesBuilder, Map metadata) { super(clone, factoriesBuilder, metadata); } @Override - protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metadata) { - return new TestAggregationBuilder(this, factoriesBuilder, metadata); + protected ValuesSourceType defaultValueSourceType() { + return CoreValuesSourceType.BYTES; + } + + @Override + protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metaData) { + return new TestAggregationBuilder(this, factoriesBuilder, metaData); } /** * Read from a stream. */ protected TestAggregationBuilder(StreamInput in) throws IOException { - super(in, null, null); + super(in); } @Override @@ -402,10 +409,10 @@ public BucketCardinality bucketCardinality() { } @Override - protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, - ValuesSourceConfig config, - AggregatorFactory parent, - Builder subFactoriesBuilder) throws IOException { + protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, + ValuesSourceConfig config, + AggregatorFactory parent, + Builder subFactoriesBuilder) throws IOException { return null; } diff --git a/server/src/test/java/org/elasticsearch/search/SearchServiceTests.java b/server/src/test/java/org/elasticsearch/search/SearchServiceTests.java index dae5416ecbc45..86299c2ef26ac 100644 --- a/server/src/test/java/org/elasticsearch/search/SearchServiceTests.java +++ b/server/src/test/java/org/elasticsearch/search/SearchServiceTests.java @@ -657,7 +657,7 @@ public void testCanMatch() throws IOException, InterruptedException { new AliasFilter(null, Strings.EMPTY_ARRAY), 1f, -1, null, null)).canMatch()); searchRequest.source(new SearchSourceBuilder().query(new MatchNoneQueryBuilder()) - .aggregation(new TermsAggregationBuilder("test", ValueType.STRING).minDocCount(0))); + .aggregation(new TermsAggregationBuilder("test").userValueTypeHint(ValueType.STRING).minDocCount(0))); assertTrue(service.canMatch(new ShardSearchRequest(OriginalIndices.NONE, searchRequest, indexShard.shardId(), 1, new AliasFilter(null, Strings.EMPTY_ARRAY), 1f, -1, null, null)).canMatch()); searchRequest.source(new SearchSourceBuilder().query(new MatchNoneQueryBuilder()) @@ -704,12 +704,12 @@ public void testCanRewriteToMatchNone() { assertFalse(SearchService.canRewriteToMatchNone(new SearchSourceBuilder())); assertFalse(SearchService.canRewriteToMatchNone(null)); assertFalse(SearchService.canRewriteToMatchNone(new SearchSourceBuilder().query(new MatchNoneQueryBuilder()) - .aggregation(new TermsAggregationBuilder("test", ValueType.STRING).minDocCount(0)))); + .aggregation(new TermsAggregationBuilder("test").userValueTypeHint(ValueType.STRING).minDocCount(0)))); assertTrue(SearchService.canRewriteToMatchNone(new SearchSourceBuilder().query(new TermQueryBuilder("foo", "bar")))); assertTrue(SearchService.canRewriteToMatchNone(new SearchSourceBuilder().query(new MatchNoneQueryBuilder()) - .aggregation(new TermsAggregationBuilder("test", ValueType.STRING).minDocCount(1)))); + .aggregation(new TermsAggregationBuilder("test").userValueTypeHint(ValueType.STRING).minDocCount(1)))); assertFalse(SearchService.canRewriteToMatchNone(new SearchSourceBuilder().query(new MatchNoneQueryBuilder()) - .aggregation(new TermsAggregationBuilder("test", ValueType.STRING).minDocCount(1)) + .aggregation(new TermsAggregationBuilder("test").userValueTypeHint(ValueType.STRING).minDocCount(1)) .suggest(new SuggestBuilder()))); assertFalse(SearchService.canRewriteToMatchNone(new SearchSourceBuilder().query(new TermQueryBuilder("foo", "bar")) .suggest(new SuggestBuilder()))); diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/FiltersTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/FiltersTests.java index c9ff5773cd6f7..0f93bf734b6ba 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/FiltersTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/FiltersTests.java @@ -168,7 +168,7 @@ public void testRewrite() throws IOException { assertTrue(((FiltersAggregationBuilder) rewritten).isKeyed()); // test sub-agg filter that does rewrite - original = new TermsAggregationBuilder("terms", ValueType.BOOLEAN) + original = new TermsAggregationBuilder("terms").userValueTypeHint(ValueType.BOOLEAN) .subAggregation( new FiltersAggregationBuilder("my-agg", new KeyedFilter("my-filter", new BoolQueryBuilder())) ); diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/GeoDistanceIT.java b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/GeoDistanceIT.java index 26e76bc0b7b33..b843b6d3725ba 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/GeoDistanceIT.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/GeoDistanceIT.java @@ -444,7 +444,7 @@ public void testEmptyAggregation() throws Exception { public void testNoRangesInQuery() { try { client().prepareSearch("idx") - .addAggregation(geoDistance("geo_dist", new GeoPoint(52.3760, 4.894))) + .addAggregation(geoDistance("geo_dist", new GeoPoint(52.3760, 4.894)).field("location")) .get(); fail(); } catch (SearchPhaseExecutionException spee){ diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/MissingTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/MissingTests.java index af8cfc01b3e97..9f63bb6c7a36d 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/MissingTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/MissingTests.java @@ -26,7 +26,7 @@ public class MissingTests extends BaseAggregationTestCase @Override protected TermsAggregationBuilder createTestAggregatorBuilder() { String name = randomAlphaOfLengthBetween(3, 20); - TermsAggregationBuilder factory = new TermsAggregationBuilder(name, null); + TermsAggregationBuilder factory = new TermsAggregationBuilder(name); String field = randomAlphaOfLengthBetween(3, 20); randomFieldOrScript(factory, field); if (randomBoolean()) { diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/composite/CompositeAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/composite/CompositeAggregatorTests.java index 4481069ee8c22..9c7218c457f83 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/composite/CompositeAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/composite/CompositeAggregatorTests.java @@ -1632,7 +1632,7 @@ public void testWithTermsSubAggExecutionMode() throws Exception { .field("keyword"); return new CompositeAggregationBuilder("name", Collections.singletonList(terms)) .subAggregation( - new TermsAggregationBuilder("terms", ValueType.STRING) + new TermsAggregationBuilder("terms").userValueTypeHint(ValueType.STRING) .field("terms") .collectMode(mode) .subAggregation(new MaxAggregationBuilder("max").field("long")) @@ -1660,7 +1660,7 @@ public void testWithTermsSubAggExecutionMode() throws Exception { .field("keyword"); return new CompositeAggregationBuilder("name", Collections.singletonList(terms)) .subAggregation( - new TermsAggregationBuilder("terms", ValueType.STRING) + new TermsAggregationBuilder("terms").userValueTypeHint(ValueType.STRING) .field("terms") .collectMode(mode) .subAggregation(new MaxAggregationBuilder("max").field("long")) diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/histogram/ExtendedBoundsTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/histogram/ExtendedBoundsTests.java index 1b13098eb0d89..be9c9947b6b37 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/histogram/ExtendedBoundsTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/histogram/ExtendedBoundsTests.java @@ -97,7 +97,7 @@ public void testParseAndValidate() { QueryShardContext qsc = new QueryShardContext(0, new IndexSettings(IndexMetadata.builder("foo").settings(indexSettings).build(), indexSettings), BigArrays.NON_RECYCLING_INSTANCE, null, null, null, null, null, xContentRegistry(), writableRegistry(), - null, null, () -> now, null, null, () -> true); + null, null, () -> now, null, null, () -> true, null); DateFormatter formatter = DateFormatter.forPattern("dateOptionalTime"); DocValueFormat format = new DocValueFormat.DateTime(formatter, ZoneOffset.UTC, DateFieldMapper.Resolution.MILLISECONDS); diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/histogram/NumericHistogramAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/histogram/NumericHistogramAggregatorTests.java index e3d1b931c71d5..1d49f871d2816 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/histogram/NumericHistogramAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/histogram/NumericHistogramAggregatorTests.java @@ -29,11 +29,16 @@ import org.apache.lucene.store.Directory; import org.apache.lucene.util.BytesRef; import org.apache.lucene.util.NumericUtils; +import org.elasticsearch.index.mapper.DateFieldMapper; import org.elasticsearch.index.mapper.KeywordFieldMapper; import org.elasticsearch.index.mapper.MappedFieldType; import org.elasticsearch.index.mapper.NumberFieldMapper; import org.elasticsearch.search.aggregations.AggregatorTestCase; import org.elasticsearch.search.aggregations.support.AggregationInspectionHelper; + +import java.util.Arrays; +import java.util.List; + import static org.hamcrest.Matchers.containsString; public class NumericHistogramAggregatorTests extends AggregatorTestCase { @@ -100,6 +105,50 @@ public void testDoubles() throws Exception { } } + public void testDates() throws Exception { + List dataset = Arrays.asList( + "2019-11-01T01:07:45", + "2019-11-02T03:43:34", + "2019-11-03T04:11:00", + "2019-11-04T05:11:31", + "2019-11-05T08:24:05", + "2019-11-06T13:09:32", + "2019-11-07T13:47:43", + "2019-11-08T16:14:34", + "2019-11-09T17:09:50", + "2019-11-10T22:55:46"); + + String fieldName = "date_field"; + DateFieldMapper.Builder builder = new DateFieldMapper.Builder(fieldName); + DateFieldMapper.DateFieldType fieldType = builder.fieldType(); + fieldType.setName(fieldName); + fieldType.setHasDocValues(true); + + try (Directory dir = newDirectory(); + RandomIndexWriter indexWriter = new RandomIndexWriter(random(), dir)) { + Document document = new Document(); + for (String date : dataset) { + if (frequently()) { + indexWriter.commit(); + } + + long instant = fieldType.parse(date); + document.add(new SortedNumericDocValuesField(fieldName, instant)); + indexWriter.addDocument(document); + document.clear(); + } + + HistogramAggregationBuilder aggBuilder = new HistogramAggregationBuilder("my_agg") + .field(fieldName) + .interval(1000 * 60 * 60 * 24); + try (IndexReader reader = indexWriter.getReader()) { + IndexSearcher searcher = new IndexSearcher(reader); + InternalHistogram histogram = search(searcher, new MatchAllDocsQuery(), aggBuilder, fieldType); + assertTrue(AggregationInspectionHelper.hasValue(histogram)); + } + } + } + public void testIrrationalInterval() throws Exception { try (Directory dir = newDirectory(); RandomIndexWriter w = new RandomIndexWriter(random(), dir)) { diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/missing/MissingAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/missing/MissingAggregatorTests.java index be9c4ab2042c8..aefe998660a97 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/missing/MissingAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/missing/MissingAggregatorTests.java @@ -20,12 +20,10 @@ package org.elasticsearch.search.aggregations.bucket.missing; import org.apache.lucene.document.BinaryDocValuesField; -import org.apache.lucene.document.SortedDocValuesField; import org.apache.lucene.document.SortedNumericDocValuesField; import org.apache.lucene.index.DirectoryReader; import org.apache.lucene.index.IndexReader; import org.apache.lucene.index.IndexableField; -import org.apache.lucene.index.MultiReader; import org.apache.lucene.index.RandomIndexWriter; import org.apache.lucene.search.IndexSearcher; import org.apache.lucene.search.Query; @@ -33,7 +31,6 @@ import org.apache.lucene.util.BytesRef; import org.elasticsearch.common.CheckedConsumer; import org.elasticsearch.common.settings.Settings; -import org.elasticsearch.index.mapper.KeywordFieldMapper; import org.elasticsearch.index.mapper.MappedFieldType; import org.elasticsearch.index.mapper.NumberFieldMapper; import org.elasticsearch.index.mapper.NumberFieldMapper.NumberType; @@ -47,11 +44,6 @@ import org.elasticsearch.script.ScriptType; import org.elasticsearch.search.aggregations.AggregationBuilder; import org.elasticsearch.search.aggregations.AggregatorTestCase; -import org.elasticsearch.search.aggregations.InternalAggregation; -import org.elasticsearch.search.aggregations.bucket.histogram.HistogramAggregationBuilder; -import org.elasticsearch.search.aggregations.bucket.histogram.InternalHistogram; -import org.elasticsearch.search.aggregations.metrics.InternalSum; -import org.elasticsearch.search.aggregations.metrics.SumAggregationBuilder; import org.elasticsearch.search.aggregations.support.AggregationInspectionHelper; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; import org.elasticsearch.search.aggregations.support.ValuesSourceType; @@ -61,7 +53,6 @@ import java.util.ArrayList; import java.util.Collection; import java.util.HashMap; -import java.util.HashSet; import java.util.List; import java.util.Map; import java.util.Set; @@ -73,10 +64,6 @@ import static java.util.Collections.singletonMap; import static java.util.stream.Collectors.toList; import static org.elasticsearch.common.lucene.search.Queries.newMatchAllQuery; -import static org.hamcrest.CoreMatchers.not; -import static org.hamcrest.CoreMatchers.notNullValue; -import static org.hamcrest.collection.IsCollectionWithSize.hasSize; -import static org.hamcrest.collection.IsEmptyCollection.empty; public class MissingAggregatorTests extends AggregatorTestCase { @@ -93,7 +80,7 @@ public void testMatchNoDocs() throws IOException { final MappedFieldType fieldType = new NumberFieldMapper.Builder("_name", NumberType.LONG).fieldType(); fieldType.setName("field"); - final MissingAggregationBuilder builder = new MissingAggregationBuilder("_name", null) + final MissingAggregationBuilder builder = new MissingAggregationBuilder("_name") .field(fieldType.name()); testCase( @@ -104,9 +91,9 @@ public void testMatchNoDocs() throws IOException { writer.addDocument(singleton(new SortedNumericDocValuesField(fieldType.name(), randomLong()))); } }, - (InternalMissing missing) -> { - assertEquals(0, missing.getDocCount()); - assertFalse(AggregationInspectionHelper.hasValue(missing)); + internalMissing -> { + assertEquals(0, internalMissing.getDocCount()); + assertFalse(AggregationInspectionHelper.hasValue(internalMissing)); }, singleton(fieldType) ); @@ -120,7 +107,7 @@ public void testMatchAllDocs() throws IOException { final MappedFieldType anotherFieldType = new NumberFieldMapper.Builder("_name", NumberType.LONG).fieldType(); anotherFieldType.setName("another_field"); - final MissingAggregationBuilder builder = new MissingAggregationBuilder("_name", null) + final MissingAggregationBuilder builder = new MissingAggregationBuilder("_name") .field(aggFieldType.name()); testCase( @@ -131,9 +118,9 @@ public void testMatchAllDocs() throws IOException { writer.addDocument(singleton(new SortedNumericDocValuesField(anotherFieldType.name(), randomLong()))); } }, - (InternalMissing missing) -> { - assertEquals(numDocs, missing.getDocCount()); - assertTrue(AggregationInspectionHelper.hasValue(missing)); + internalMissing -> { + assertEquals(numDocs, internalMissing.getDocCount()); + assertTrue(AggregationInspectionHelper.hasValue(internalMissing)); }, org.elasticsearch.common.collect.List.of(aggFieldType, anotherFieldType) ); @@ -145,7 +132,7 @@ public void testMatchSparse() throws IOException { final MappedFieldType anotherFieldType = new NumberFieldMapper.Builder("_name", NumberType.LONG).fieldType(); anotherFieldType.setName("another_field"); - final MissingAggregationBuilder builder = new MissingAggregationBuilder("_name", null) + final MissingAggregationBuilder builder = new MissingAggregationBuilder("_name") .field(aggFieldType.name()); final int numDocs = randomIntBetween(100, 200); @@ -165,9 +152,9 @@ public void testMatchSparse() throws IOException { newMatchAllQuery(), builder, writer -> writer.addDocuments(docs), - (InternalMissing missing) -> { - assertEquals(finalDocsMissingAggField, missing.getDocCount()); - assertTrue(AggregationInspectionHelper.hasValue(missing)); + internalMissing -> { + assertEquals(finalDocsMissingAggField, internalMissing.getDocCount()); + assertTrue(AggregationInspectionHelper.hasValue(internalMissing)); }, org.elasticsearch.common.collect.List.of(aggFieldType, anotherFieldType) ); @@ -184,7 +171,7 @@ public void testMatchSparseRangeField() throws IOException { final BytesRef encodedRange = rangeType.encodeRanges(singleton(range)); final BinaryDocValuesField encodedRangeField = new BinaryDocValuesField(aggFieldType.name(), encodedRange); - final MissingAggregationBuilder builder = new MissingAggregationBuilder("_name", null) + final MissingAggregationBuilder builder = new MissingAggregationBuilder("_name") .field(aggFieldType.name()); final int numDocs = randomIntBetween(100, 200); @@ -204,110 +191,11 @@ public void testMatchSparseRangeField() throws IOException { newMatchAllQuery(), builder, writer -> writer.addDocuments(docs), - (InternalMissing missing) -> { - assertEquals(finalDocsMissingAggField, missing.getDocCount()); - assertTrue(AggregationInspectionHelper.hasValue(missing)); - }, - org.elasticsearch.common.collect.List.of(aggFieldType, anotherFieldType) - ); - } - - public void testSubAggregation() throws IOException { - final MappedFieldType missingAggFieldType = new KeywordFieldMapper.Builder("_name").fieldType(); - missingAggFieldType.setName("missing_agg_field"); - missingAggFieldType.setHasDocValues(true); - final MappedFieldType sumAggFieldType = new NumberFieldMapper.Builder("_name", NumberType.LONG).fieldType(); - sumAggFieldType.setName("sum_agg_field"); - - final MissingAggregationBuilder builder = new MissingAggregationBuilder("missing", null) - .field(missingAggFieldType.name()) - .subAggregation( - new SumAggregationBuilder("sum") - .field(sumAggFieldType.name()) - ); - - final int numDocs = randomIntBetween(10, 20); - int docsMissingAggField = 0; - long expectedSum = 0; - final List> docs = new ArrayList<>(); - for (int i = 0; i < numDocs; i++) { - final Set doc = new HashSet<>(); - final long sumFieldValue = randomLongBetween(0, 1000); - doc.add(new SortedNumericDocValuesField(sumAggFieldType.name(), sumFieldValue)); - if (randomBoolean()) { - doc.add(new SortedDocValuesField(missingAggFieldType.name(), new BytesRef(randomUnicodeOfLengthBetween(2, 20)))); - } else { - docsMissingAggField++; - expectedSum += sumFieldValue; - } - docs.add(doc); - } - final int finalDocsMissingAggField = docsMissingAggField; - final long finalExpectedSum = expectedSum; - - testCase( - newMatchAllQuery(), - builder, - writer -> writer.addDocuments(docs), - (InternalMissing internalMissing) -> { + internalMissing -> { assertEquals(finalDocsMissingAggField, internalMissing.getDocCount()); assertTrue(AggregationInspectionHelper.hasValue(internalMissing)); - - assertThat(internalMissing.getAggregations().asList(), not(empty())); - final InternalSum internalSum = internalMissing.getAggregations().get("sum"); - assertEquals(finalExpectedSum, internalSum.getValue(), 0d); - assertTrue(AggregationInspectionHelper.hasValue(internalSum)); - }, - org.elasticsearch.common.collect.List.of(missingAggFieldType, sumAggFieldType) - ); - } - - public void testEmptyBucket() throws IOException { - final MappedFieldType histoAggFieldType = new NumberFieldMapper.Builder("_name", NumberType.LONG).fieldType(); - histoAggFieldType.setName("histo_agg_field"); - final MappedFieldType missingAggFieldType = new NumberFieldMapper.Builder("_name", NumberType.LONG).fieldType(); - missingAggFieldType.setName("missing_agg_field"); - - final HistogramAggregationBuilder builder = new HistogramAggregationBuilder("histo") - .field(histoAggFieldType.name()) - .interval(1) - .minDocCount(0) - .subAggregation( - new MissingAggregationBuilder("missing", null) - .field(missingAggFieldType.name()) - ); - - testCaseWithReduce( - newMatchAllQuery(), - builder, - writer -> { - writer.addDocument(singleton(new SortedNumericDocValuesField(histoAggFieldType.name(), 0))); - writer.addDocument(singleton(new SortedNumericDocValuesField(histoAggFieldType.name(), 2))); - }, - (InternalHistogram histogram) -> { - assertThat(histogram.getBuckets(), hasSize(3)); - - { - assertThat(histogram.getBuckets().get(0), notNullValue()); - final InternalMissing missing = histogram.getBuckets().get(0).getAggregations().get("missing"); - assertEquals(1, missing.getDocCount()); - assertTrue(AggregationInspectionHelper.hasValue(missing)); - } - { - assertThat(histogram.getBuckets().get(1), notNullValue()); - final InternalMissing missing = histogram.getBuckets().get(1).getAggregations().get("missing"); - assertEquals(0, missing.getDocCount()); - assertFalse(AggregationInspectionHelper.hasValue(missing)); - } - { - assertThat(histogram.getBuckets().get(2), notNullValue()); - final InternalMissing missing = histogram.getBuckets().get(2).getAggregations().get("missing"); - assertEquals(1, missing.getDocCount()); - assertTrue(AggregationInspectionHelper.hasValue(missing)); - } }, - org.elasticsearch.common.collect.List.of(histoAggFieldType, missingAggFieldType), - true + List.of(aggFieldType, anotherFieldType) ); } @@ -316,7 +204,7 @@ public void testUnmappedWithoutMissingParam() throws IOException { final MappedFieldType aggFieldType = new NumberFieldMapper.Builder("_name", NumberType.LONG).fieldType(); aggFieldType.setName("agg_field"); - final MissingAggregationBuilder builder = new MissingAggregationBuilder("_name", null) + final MissingAggregationBuilder builder = new MissingAggregationBuilder("_name") .field("unknown_field"); testCase( @@ -327,9 +215,9 @@ public void testUnmappedWithoutMissingParam() throws IOException { writer.addDocument(singleton(new SortedNumericDocValuesField(aggFieldType.name(), randomLong()))); } }, - (InternalMissing missing) -> { - assertEquals(numDocs, missing.getDocCount()); - assertTrue(AggregationInspectionHelper.hasValue(missing)); + internalMissing -> { + assertEquals(numDocs, internalMissing.getDocCount()); + assertTrue(AggregationInspectionHelper.hasValue(internalMissing)); }, singleton(aggFieldType) ); @@ -340,7 +228,7 @@ public void testUnmappedWithMissingParam() throws IOException { final MappedFieldType aggFieldType = new NumberFieldMapper.Builder("_name", NumberType.LONG).fieldType(); aggFieldType.setName("agg_field"); - final MissingAggregationBuilder builder = new MissingAggregationBuilder("_name", null) + final MissingAggregationBuilder builder = new MissingAggregationBuilder("_name") .field("unknown_field") .missing(randomLong()); @@ -352,44 +240,14 @@ public void testUnmappedWithMissingParam() throws IOException { writer.addDocument(singleton(new SortedNumericDocValuesField(aggFieldType.name(), randomLong()))); } }, - (InternalMissing missing) -> { - assertEquals(0, missing.getDocCount()); - assertFalse(AggregationInspectionHelper.hasValue(missing)); + internalMissing -> { + assertEquals(0, internalMissing.getDocCount()); + assertFalse(AggregationInspectionHelper.hasValue(internalMissing)); }, singleton(aggFieldType) ); } - public void testSingleValuedFieldPartiallyUnmapped() throws IOException { - final int numDocs = randomIntBetween(10, 20); - - final MappedFieldType fieldType = new NumberFieldMapper.Builder("_name", NumberType.LONG).fieldType(); - fieldType.setName("field"); - final MissingAggregationBuilder builder = new MissingAggregationBuilder("_name", null) - .field(fieldType.name()); - - try (Directory mappedDirectory = newDirectory(); Directory unmappedDirectory = newDirectory()) { - try (RandomIndexWriter mappedWriter = new RandomIndexWriter(random(), mappedDirectory)) { - for (int i = 0; i < numDocs; i++) { - mappedWriter.addDocument(singleton(new SortedNumericDocValuesField(fieldType.name(), randomLong()))); - } - } - - new RandomIndexWriter(random(), unmappedDirectory).close(); - - try (IndexReader mappedReader = DirectoryReader.open(mappedDirectory); - IndexReader unmappedReader = DirectoryReader.open(unmappedDirectory); - MultiReader multiReader = new MultiReader(mappedReader, unmappedReader)) { - - final IndexSearcher searcher = newSearcher(multiReader, true, true); - - final InternalMissing internalMissing = searchAndReduce(searcher, newMatchAllQuery(), builder, fieldType); - assertEquals(0, internalMissing.getDocCount()); - assertFalse(AggregationInspectionHelper.hasValue(internalMissing)); - } - } - } - public void testMissingParam() throws IOException { final int numDocs = randomIntBetween(10, 20); @@ -398,7 +256,7 @@ public void testMissingParam() throws IOException { final MappedFieldType anotherFieldType = new NumberFieldMapper.Builder("_name", NumberType.LONG).fieldType(); anotherFieldType.setName("another_field"); - final MissingAggregationBuilder builder = new MissingAggregationBuilder("_name", null) + final MissingAggregationBuilder builder = new MissingAggregationBuilder("_name") .field(aggFieldType.name()) .missing(randomLong()); @@ -410,9 +268,9 @@ public void testMissingParam() throws IOException { writer.addDocument(singleton(new SortedNumericDocValuesField(anotherFieldType.name(), randomLong()))); } }, - (InternalMissing missing) -> { - assertEquals(0, missing.getDocCount()); - assertFalse(AggregationInspectionHelper.hasValue(missing)); + internalMissing -> { + assertEquals(0, internalMissing.getDocCount()); + assertFalse(AggregationInspectionHelper.hasValue(internalMissing)); }, org.elasticsearch.common.collect.List.of(aggFieldType, anotherFieldType) ); @@ -424,7 +282,7 @@ public void testMultiValuedField() throws IOException { final MappedFieldType anotherFieldType = new NumberFieldMapper.Builder("_name", NumberType.LONG).fieldType(); anotherFieldType.setName("another_field"); - final MissingAggregationBuilder builder = new MissingAggregationBuilder("_name", null) + final MissingAggregationBuilder builder = new MissingAggregationBuilder("_name") .field(aggFieldType.name()); final int numDocs = randomIntBetween(100, 200); @@ -448,9 +306,9 @@ public void testMultiValuedField() throws IOException { newMatchAllQuery(), builder, writer -> writer.addDocuments(docs), - (InternalMissing missing) -> { - assertEquals(finalDocsMissingAggField, missing.getDocCount()); - assertTrue(AggregationInspectionHelper.hasValue(missing)); + internalMissing -> { + assertEquals(finalDocsMissingAggField, internalMissing.getDocCount()); + assertTrue(AggregationInspectionHelper.hasValue(internalMissing)); }, org.elasticsearch.common.collect.List.of(aggFieldType, anotherFieldType) ); @@ -470,7 +328,7 @@ private void valueScriptTestCase(Script script) throws IOException { final MappedFieldType anotherFieldType = new NumberFieldMapper.Builder("_name", NumberType.LONG).fieldType(); anotherFieldType.setName("another_field"); - final MissingAggregationBuilder builder = new MissingAggregationBuilder("_name", null) + final MissingAggregationBuilder builder = new MissingAggregationBuilder("_name") .field(aggFieldType.name()) .script(script); @@ -491,9 +349,9 @@ private void valueScriptTestCase(Script script) throws IOException { newMatchAllQuery(), builder, writer -> writer.addDocuments(docs), - (InternalMissing missing) -> { - assertEquals(finalDocsMissingField, missing.getDocCount()); - assertTrue(AggregationInspectionHelper.hasValue(missing)); + internalMissing -> { + assertEquals(finalDocsMissingField, internalMissing.getDocCount()); + assertTrue(AggregationInspectionHelper.hasValue(internalMissing)); }, org.elasticsearch.common.collect.List.of(aggFieldType, anotherFieldType) ); @@ -514,7 +372,7 @@ private void fieldScriptTestCase(Script script, long threshold) throws IOExcepti final MappedFieldType aggFieldType = new NumberFieldMapper.Builder("_name", NumberType.LONG).fieldType(); aggFieldType.setName("agg_field"); - final MissingAggregationBuilder builder = new MissingAggregationBuilder("_name", null) + final MissingAggregationBuilder builder = new MissingAggregationBuilder("_name") .script(script); final int numDocs = randomIntBetween(100, 200); @@ -537,32 +395,29 @@ private void fieldScriptTestCase(Script script, long threshold) throws IOExcepti newMatchAllQuery(), builder, writer -> writer.addDocuments(docs), - (InternalMissing missing) -> { - assertEquals(finalDocsBelowThreshold, missing.getDocCount()); - assertTrue(AggregationInspectionHelper.hasValue(missing)); + internalMissing -> { + assertEquals(finalDocsBelowThreshold, internalMissing.getDocCount()); + assertTrue(AggregationInspectionHelper.hasValue(internalMissing)); }, singleton(aggFieldType) ); } - private void testCase( - Query query, - B builder, - CheckedConsumer writeIndex, - Consumer verify, - Collection fieldTypes) throws IOException { - + private void testCase(Query query, + MissingAggregationBuilder builder, + CheckedConsumer writeIndex, + Consumer verify, + Collection fieldTypes) throws IOException { testCaseWithReduce(query, builder, writeIndex, verify, fieldTypes, false); testCaseWithReduce(query, builder, writeIndex, verify, fieldTypes, true); } - private void testCaseWithReduce( - Query query, - B builder, - CheckedConsumer writeIndex, - Consumer verify, - Collection fieldTypes, - boolean reduced) throws IOException { + private void testCaseWithReduce(Query query, + MissingAggregationBuilder builder, + CheckedConsumer writeIndex, + Consumer verify, + Collection fieldTypes, + boolean reduced) throws IOException { try (Directory directory = newDirectory()) { try (RandomIndexWriter indexWriter = new RandomIndexWriter(random(), directory)) { @@ -572,20 +427,20 @@ private void testC try (IndexReader indexReader = DirectoryReader.open(directory)) { final IndexSearcher indexSearcher = newSearcher(indexReader, true, true); final MappedFieldType[] fieldTypesArray = fieldTypes.toArray(new MappedFieldType[0]); - final I result; + final InternalMissing missing; if (reduced) { - result = searchAndReduce(indexSearcher, query, builder, fieldTypesArray); + missing = searchAndReduce(indexSearcher, query, builder, fieldTypesArray); } else { - result = search(indexSearcher, query, builder, fieldTypesArray); + missing = search(indexSearcher, query, builder, fieldTypesArray); } - verify.accept(result); + verify.accept(missing); } } } @Override protected AggregationBuilder createAggBuilderForTypeTest(MappedFieldType fieldType, String fieldName) { - return new MissingAggregationBuilder("_name", null) + return new MissingAggregationBuilder("_name") .field(fieldName); } @@ -596,7 +451,9 @@ protected List getSupportedValuesSourceTypes() { CoreValuesSourceType.BYTES, CoreValuesSourceType.GEOPOINT, CoreValuesSourceType.RANGE, - CoreValuesSourceType.HISTOGRAM + CoreValuesSourceType.IP, + CoreValuesSourceType.BOOLEAN, + CoreValuesSourceType.DATE ); } diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/nested/NestedAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/nested/NestedAggregatorTests.java index bf5e4b7f61f96..a91f965191735 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/nested/NestedAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/nested/NestedAggregatorTests.java @@ -418,7 +418,7 @@ public void testNestedOrdering() throws IOException { fieldType2.setHasDocValues(true); fieldType2.setName("author"); - TermsAggregationBuilder termsBuilder = new TermsAggregationBuilder("authors", ValueType.STRING) + TermsAggregationBuilder termsBuilder = new TermsAggregationBuilder("authors").userValueTypeHint(ValueType.STRING) .field("author").order(BucketOrder.aggregation("chapters>num_pages.value", true)); NestedAggregationBuilder nestedBuilder = new NestedAggregationBuilder("chapters", "nested_chapters"); MaxAggregationBuilder maxAgg = new MaxAggregationBuilder("num_pages").field("num_pages"); @@ -467,7 +467,7 @@ public void testNestedOrdering() throws IOException { assertEquals(70, (int) numPages.getValue()); // reverse order: - termsBuilder = new TermsAggregationBuilder("authors", ValueType.STRING) + termsBuilder = new TermsAggregationBuilder("authors").userValueTypeHint(ValueType.STRING) .field("author").order(BucketOrder.aggregation("chapters>num_pages.value", false)); nestedBuilder = new NestedAggregationBuilder("chapters", "nested_chapters"); maxAgg = new MaxAggregationBuilder("num_pages").field("num_pages"); @@ -556,7 +556,7 @@ public void testNestedOrdering_random() throws IOException { fieldType2.setHasDocValues(true); fieldType2.setName("author"); - TermsAggregationBuilder termsBuilder = new TermsAggregationBuilder("authors", ValueType.STRING) + TermsAggregationBuilder termsBuilder = new TermsAggregationBuilder("authors").userValueTypeHint(ValueType.STRING) .size(books.size()).field("author") .order(BucketOrder.compound(BucketOrder.aggregation("chapters>num_pages.value", true), BucketOrder.key(true))); NestedAggregationBuilder nestedBuilder = new NestedAggregationBuilder("chapters", "nested_chapters"); @@ -648,8 +648,10 @@ public void testPreGetChildLeafCollectors() throws IOException { iw.commit(); } try (IndexReader indexReader = wrap(DirectoryReader.open(directory))) { - TermsAggregationBuilder valueBuilder = new TermsAggregationBuilder("value", ValueType.STRING).field("value"); - TermsAggregationBuilder keyBuilder = new TermsAggregationBuilder("key", ValueType.STRING).field("key"); + TermsAggregationBuilder valueBuilder = new TermsAggregationBuilder("value").userValueTypeHint(ValueType.STRING) + .field("value"); + TermsAggregationBuilder keyBuilder = new TermsAggregationBuilder("key").userValueTypeHint(ValueType.STRING) + .field("key"); keyBuilder.subAggregation(valueBuilder); NestedAggregationBuilder nestedBuilder = new NestedAggregationBuilder(NESTED_AGG, "nested_field"); nestedBuilder.subAggregation(keyBuilder); @@ -765,7 +767,7 @@ public void testNestedWithPipeline() throws IOException { } try (IndexReader indexReader = wrap(DirectoryReader.open(directory))) { NestedAggregationBuilder nestedBuilder = new NestedAggregationBuilder(NESTED_AGG, NESTED_OBJECT) - .subAggregation(new TermsAggregationBuilder("terms", ValueType.NUMERIC).field(VALUE_FIELD_NAME) + .subAggregation(new TermsAggregationBuilder("terms").field(VALUE_FIELD_NAME).userValueTypeHint(ValueType.NUMERIC) .subAggregation(new MaxAggregationBuilder(MAX_AGG_NAME).field(VALUE_FIELD_NAME)) .subAggregation(new BucketScriptPipelineAggregationBuilder("bucketscript", Collections.singletonMap("_value", MAX_AGG_NAME), diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/range/DateRangeAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/range/DateRangeAggregatorTests.java index 2746e69be29de..2fb281170b732 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/range/DateRangeAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/range/DateRangeAggregatorTests.java @@ -224,8 +224,8 @@ public void testKeywordField() { () -> testCase(aggregationBuilder, new MatchAllDocsQuery(), iw -> { iw.addDocument(singleton(new SortedSetDocValuesField("string", new BytesRef("foo")))); }, range -> fail("Should have thrown exception"), fieldType)); - // I believe this error is coming from improperly parsing the range, not the field. - assertEquals("For input string: \"2015-11-13\"", e.getMessage()); + assertEquals("Field [not_a_number] of type [keyword(indexed,tokenized)] is not supported for aggregation [date_range]", + e.getMessage()); } public void testBadMissingField() { diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/range/RangeAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/range/RangeAggregatorTests.java index 2b3571ffaac85..092a0319c9c21 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/range/RangeAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/range/RangeAggregatorTests.java @@ -238,7 +238,7 @@ public void testUnsupportedType() { () -> testCase(aggregationBuilder, new MatchAllDocsQuery(), iw -> { iw.addDocument(singleton(new SortedSetDocValuesField("string", new BytesRef("foo")))); }, range -> fail("Should have thrown exception"), fieldType)); - assertEquals(e.getMessage(), "Expected numeric type on field [not_a_number], but got [keyword]"); + assertEquals("Field [not_a_number] of type [keyword(indexed,tokenized)] is not supported for aggregation [range]", e.getMessage()); } public void testBadMissingField() { diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/sampler/DiversifiedSamplerTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/sampler/DiversifiedSamplerTests.java index 5c29aabfa9dc4..d9ebbf633f242 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/sampler/DiversifiedSamplerTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/sampler/DiversifiedSamplerTests.java @@ -201,7 +201,7 @@ private void testCase(IndexSearcher indexSearcher, MappedFieldType genreFieldTyp .executionHint(executionHint) .maxDocsPerValue(maxDocsPerValue) .shardSize(shardSize) - .subAggregation(new TermsAggregationBuilder("terms", null).field("id")); + .subAggregation(new TermsAggregationBuilder("terms").field("id")); InternalSampler result = search(indexSearcher, query, builder, genreFieldType, idFieldType); verify.accept(result); @@ -224,7 +224,7 @@ public void testDiversifiedSampler_noDocs() throws Exception { DiversifiedAggregationBuilder builder = new DiversifiedAggregationBuilder("_name") .field(genreFieldType.name()) - .subAggregation(new TermsAggregationBuilder("terms", null).field("id")); + .subAggregation(new TermsAggregationBuilder("terms").field("id")); InternalSampler result = search(indexSearcher, new MatchAllDocsQuery(), builder, genreFieldType, idFieldType); Terms terms = result.getAggregations().get("terms"); diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorTests.java index 6cb4826ead2c4..e94ca73aa5797 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorTests.java @@ -50,6 +50,12 @@ import org.elasticsearch.search.aggregations.AggregatorTestCase; import org.elasticsearch.search.aggregations.bucket.significant.SignificantTermsAggregatorFactory.ExecutionMode; import org.elasticsearch.search.aggregations.bucket.terms.IncludeExclude; +import org.elasticsearch.search.aggregations.AggregationBuilder; +import org.elasticsearch.search.aggregations.AggregatorTestCase; +import org.elasticsearch.search.aggregations.bucket.significant.SignificantTermsAggregatorFactory.ExecutionMode; +import org.elasticsearch.search.aggregations.bucket.terms.IncludeExclude; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import org.junit.Before; import java.io.IOException; @@ -75,6 +81,27 @@ public void setUpTest() throws Exception { fieldType.setName("field"); } + @Override + protected AggregationBuilder createAggBuilderForTypeTest(MappedFieldType fieldType, String fieldName) { + return new SignificantTermsAggregationBuilder("foo").field(fieldName); + } + + @Override + protected List getSupportedValuesSourceTypes() { + return List.of(CoreValuesSourceType.NUMERIC, + CoreValuesSourceType.BYTES); + } + + @Override + protected List unsupportedMappedFieldTypes() { + return List.of( + NumberFieldMapper.NumberType.DOUBLE.typeName(), // floating points are not supported at all + NumberFieldMapper.NumberType.FLOAT.typeName(), + NumberFieldMapper.NumberType.HALF_FLOAT.typeName(), + BinaryFieldMapper.CONTENT_TYPE // binary fields are not supported because they cannot be searched + ); + } + /** * For each provided field type, we also register an alias with name {@code -alias}. */ @@ -101,14 +128,14 @@ public void testSignificance() throws IOException { try (Directory dir = newDirectory(); IndexWriter w = new IndexWriter(dir, indexWriterConfig)) { addMixedTextDocs(textFieldType, w); - SignificantTermsAggregationBuilder sigAgg = new SignificantTermsAggregationBuilder("sig_text", null).field("text"); + SignificantTermsAggregationBuilder sigAgg = new SignificantTermsAggregationBuilder("sig_text").field("text"); sigAgg.executionHint(randomExecutionHint()); if (randomBoolean()) { // Use a background filter which just happens to be same scope as whole-index. sigAgg.backgroundFilter(QueryBuilders.termsQuery("text", "common")); } - SignificantTermsAggregationBuilder sigNumAgg = new SignificantTermsAggregationBuilder("sig_number", null).field("long_field"); + SignificantTermsAggregationBuilder sigNumAgg = new SignificantTermsAggregationBuilder("sig_number").field("long_field"); sigNumAgg.executionHint(randomExecutionHint()); try (IndexReader reader = DirectoryReader.open(w)) { @@ -200,7 +227,7 @@ public void testNumericSignificance() throws IOException { w.addDocument(doc); } - SignificantTermsAggregationBuilder sigNumAgg = new SignificantTermsAggregationBuilder("sig_number", null).field("long_field"); + SignificantTermsAggregationBuilder sigNumAgg = new SignificantTermsAggregationBuilder("sig_number").field("long_field"); sigNumAgg.executionHint(randomExecutionHint()); try (IndexReader reader = DirectoryReader.open(w)) { @@ -242,7 +269,7 @@ public void testUnmapped() throws IOException { addMixedTextDocs(textFieldType, w); // Attempt aggregation on unmapped field - SignificantTermsAggregationBuilder sigAgg = new SignificantTermsAggregationBuilder("sig_text", null).field("unmapped_field"); + SignificantTermsAggregationBuilder sigAgg = new SignificantTermsAggregationBuilder("sig_text").field("unmapped_field"); sigAgg.executionHint(randomExecutionHint()); try (IndexReader reader = DirectoryReader.open(w)) { @@ -289,12 +316,12 @@ public void testRangeField() throws IOException { } // Attempt aggregation on range field - SignificantTermsAggregationBuilder sigAgg = new SignificantTermsAggregationBuilder("sig_text", null).field(fieldName); + SignificantTermsAggregationBuilder sigAgg = new SignificantTermsAggregationBuilder("sig_text").field(fieldName); sigAgg.executionHint(randomExecutionHint()); try (IndexReader reader = DirectoryReader.open(w)) { IndexSearcher indexSearcher = newIndexSearcher(reader); - expectThrows(AggregationExecutionException.class, () -> createAggregator(sigAgg, indexSearcher, fieldType)); + expectThrows(IllegalArgumentException.class, () -> createAggregator(sigAgg, indexSearcher, fieldType)); } } } diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/terms/BinaryTermsAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/terms/BinaryTermsAggregatorTests.java new file mode 100644 index 0000000000000..31859a7a4fa04 --- /dev/null +++ b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/terms/BinaryTermsAggregatorTests.java @@ -0,0 +1,176 @@ +/* + * Licensed to Elasticsearch under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.elasticsearch.search.aggregations.bucket.terms; + +import org.apache.lucene.document.Document; +import org.apache.lucene.index.DirectoryReader; +import org.apache.lucene.index.IndexReader; +import org.apache.lucene.index.RandomIndexWriter; +import org.apache.lucene.search.IndexSearcher; +import org.apache.lucene.search.MatchAllDocsQuery; +import org.apache.lucene.search.MatchNoDocsQuery; +import org.apache.lucene.search.Query; +import org.apache.lucene.store.Directory; +import org.apache.lucene.util.BytesRef; +import org.apache.lucene.util.automaton.RegExp; +import org.elasticsearch.common.Numbers; +import org.elasticsearch.index.mapper.BinaryFieldMapper; +import org.elasticsearch.index.mapper.MappedFieldType; +import org.elasticsearch.search.DocValueFormat; +import org.elasticsearch.search.aggregations.AggregationExecutionException; +import org.elasticsearch.search.aggregations.AggregatorTestCase; +import org.elasticsearch.search.aggregations.support.ValueType; + +import java.io.IOException; +import java.util.ArrayList; +import java.util.List; +import java.util.function.Consumer; + +import static org.hamcrest.Matchers.equalTo; + +public class BinaryTermsAggregatorTests extends AggregatorTestCase { + private static final String BINARY_FIELD = "binary"; + + private static final List dataset; + static { + List d = new ArrayList<>(45); + for (int i = 0; i < 10; i++) { + for (int j = 0; j < i; j++) { + d.add((long) i); + } + } + dataset = d; + } + + public void testMatchNoDocs() throws IOException { + testBothCases(new MatchNoDocsQuery(), dataset, + aggregation -> aggregation.field(BINARY_FIELD), + agg -> assertEquals(0, agg.getBuckets().size()), ValueType.STRING + ); + } + + public void testMatchAllDocs() throws IOException { + Query query = new MatchAllDocsQuery(); + + testBothCases(query, dataset, + aggregation -> aggregation.field(BINARY_FIELD), + agg -> { + assertEquals(9, agg.getBuckets().size()); + for (int i = 0; i < 9; i++) { + StringTerms.Bucket bucket = (StringTerms.Bucket) agg.getBuckets().get(i); + byte[] bytes = Numbers.longToBytes(9L - i); + String bytesAsString = (String) DocValueFormat.BINARY.format(new BytesRef(bytes)); + assertThat(bucket.getKey(), equalTo(bytesAsString)); + assertThat(bucket.getDocCount(), equalTo(9L - i)); + } + }, null); + } + + public void testBadIncludeExclude() throws IOException { + IncludeExclude includeExclude = new IncludeExclude(new RegExp("foo"), null); + + // Make sure the include/exclude fails regardless of how the user tries to type hint the agg + AggregationExecutionException e = expectThrows(AggregationExecutionException.class, + () -> testBothCases(new MatchNoDocsQuery(), dataset, + aggregation -> aggregation.field(BINARY_FIELD).includeExclude(includeExclude).format("yyyy-MM-dd"), + agg -> fail("test should have failed with exception"), null // default, no hint + )); + assertThat(e.getMessage(), equalTo("Aggregation [_name] cannot support regular expression style include/exclude settings as " + + "they can only be applied to string fields. Use an array of values for include/exclude clauses")); + + e = expectThrows(AggregationExecutionException.class, + () -> testBothCases(new MatchNoDocsQuery(), dataset, + aggregation -> aggregation.field(BINARY_FIELD).includeExclude(includeExclude).format("yyyy-MM-dd"), + agg -> fail("test should have failed with exception"), ValueType.STRING // string type hint + )); + assertThat(e.getMessage(), equalTo("Aggregation [_name] cannot support regular expression style include/exclude settings as " + + "they can only be applied to string fields. Use an array of values for include/exclude clauses")); + + e = expectThrows(AggregationExecutionException.class, () -> testBothCases(new MatchNoDocsQuery(), dataset, + aggregation -> aggregation.field(BINARY_FIELD).includeExclude(includeExclude), + agg -> fail("test should have failed with exception"), ValueType.NUMERIC // numeric type hint + )); + assertThat(e.getMessage(), equalTo("Aggregation [_name] cannot support regular expression style include/exclude settings as " + + "they can only be applied to string fields. Use an array of values for include/exclude clauses")); + } + + private void testSearchCase(Query query, List dataset, + Consumer configure, + Consumer verify, ValueType valueType) throws IOException { + executeTestCase(false, query, dataset, configure, verify, valueType); + } + + private void testSearchAndReduceCase(Query query, List dataset, + Consumer configure, + Consumer verify, ValueType valueType) throws IOException { + executeTestCase(true, query, dataset, configure, verify, valueType); + } + + private void testBothCases(Query query, List dataset, + Consumer configure, + Consumer verify, ValueType valueType) throws IOException { + testSearchCase(query, dataset, configure, verify, valueType); + testSearchAndReduceCase(query, dataset, configure, verify, valueType); + } + + private void executeTestCase(boolean reduced, Query query, List dataset, + Consumer configure, + Consumer verify, ValueType valueType) throws IOException { + + try (Directory directory = newDirectory()) { + try (RandomIndexWriter indexWriter = new RandomIndexWriter(random(), directory)) { + Document document = new Document(); + for (Long value : dataset) { + if (frequently()) { + indexWriter.commit(); + } + + document.add(new BinaryFieldMapper.CustomBinaryDocValuesField(BINARY_FIELD, Numbers.longToBytes(value))); + indexWriter.addDocument(document); + document.clear(); + } + } + + try (IndexReader indexReader = DirectoryReader.open(directory)) { + IndexSearcher indexSearcher = newIndexSearcher(indexReader); + + TermsAggregationBuilder aggregationBuilder = new TermsAggregationBuilder("_name"); + if (valueType != null) { + aggregationBuilder.userValueTypeHint(valueType); + } + if (configure != null) { + configure.accept(aggregationBuilder); + } + + MappedFieldType binaryFieldType = new BinaryFieldMapper.Builder(BINARY_FIELD).fieldType(); + binaryFieldType.setName(BINARY_FIELD); + binaryFieldType.setHasDocValues(true); + + InternalMappedTerms rareTerms; + if (reduced) { + rareTerms = searchAndReduce(indexSearcher, query, aggregationBuilder, binaryFieldType); + } else { + rareTerms = search(indexSearcher, query, aggregationBuilder, binaryFieldType); + } + verify.accept(rareTerms); + } + } + } + +} diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/terms/KeywordTermsAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/terms/KeywordTermsAggregatorTests.java new file mode 100644 index 0000000000000..098e94d3d4450 --- /dev/null +++ b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/terms/KeywordTermsAggregatorTests.java @@ -0,0 +1,161 @@ +/* + * Licensed to Elasticsearch under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.elasticsearch.search.aggregations.bucket.terms; + +import org.apache.lucene.document.Document; +import org.apache.lucene.document.SortedSetDocValuesField; +import org.apache.lucene.index.DirectoryReader; +import org.apache.lucene.index.IndexReader; +import org.apache.lucene.index.RandomIndexWriter; +import org.apache.lucene.search.IndexSearcher; +import org.apache.lucene.search.MatchAllDocsQuery; +import org.apache.lucene.search.MatchNoDocsQuery; +import org.apache.lucene.search.Query; +import org.apache.lucene.store.Directory; +import org.apache.lucene.util.BytesRef; +import org.elasticsearch.index.mapper.KeywordFieldMapper; +import org.elasticsearch.index.mapper.MappedFieldType; +import org.elasticsearch.search.aggregations.AggregatorTestCase; +import org.elasticsearch.search.aggregations.support.ValueType; + +import java.io.IOException; +import java.util.ArrayList; +import java.util.List; +import java.util.function.Consumer; + +import static org.hamcrest.Matchers.equalTo; + +public class KeywordTermsAggregatorTests extends AggregatorTestCase { + private static final String KEYWORD_FIELD = "keyword"; + + private static final List dataset; + static { + List d = new ArrayList<>(45); + for (int i = 0; i < 10; i++) { + for (int j = 0; j < i; j++) { + d.add(String.valueOf(i)); + } + } + dataset = d; + } + + public void testMatchNoDocs() throws IOException { + testBothCases(new MatchNoDocsQuery(), dataset, + aggregation -> aggregation.field(KEYWORD_FIELD), + agg -> assertEquals(0, agg.getBuckets().size()), null // without type hint + ); + + testBothCases(new MatchNoDocsQuery(), dataset, + aggregation -> aggregation.field(KEYWORD_FIELD), + agg -> assertEquals(0, agg.getBuckets().size()), ValueType.STRING // with type hint + ); + } + + public void testMatchAllDocs() throws IOException { + Query query = new MatchAllDocsQuery(); + + testBothCases(query, dataset, + aggregation -> aggregation.field(KEYWORD_FIELD), + agg -> { + assertEquals(9, agg.getBuckets().size()); + for (int i = 0; i < 9; i++) { + StringTerms.Bucket bucket = (StringTerms.Bucket) agg.getBuckets().get(i); + assertThat(bucket.getKey(), equalTo(String.valueOf(9L - i))); + assertThat(bucket.getDocCount(), equalTo(9L - i)); + } + }, null // without type hint + ); + + testBothCases(query, dataset, + aggregation -> aggregation.field(KEYWORD_FIELD), + agg -> { + assertEquals(9, agg.getBuckets().size()); + for (int i = 0; i < 9; i++) { + StringTerms.Bucket bucket = (StringTerms.Bucket) agg.getBuckets().get(i); + assertThat(bucket.getKey(), equalTo(String.valueOf(9L - i))); + assertThat(bucket.getDocCount(), equalTo(9L - i)); + } + }, ValueType.STRING // with type hint + ); + } + + private void testSearchCase(Query query, List dataset, + Consumer configure, + Consumer verify, ValueType valueType) throws IOException { + executeTestCase(false, query, dataset, configure, verify, valueType); + } + + private void testSearchAndReduceCase(Query query, List dataset, + Consumer configure, + Consumer verify, ValueType valueType) throws IOException { + executeTestCase(true, query, dataset, configure, verify, valueType); + } + + private void testBothCases(Query query, List dataset, + Consumer configure, + Consumer verify, ValueType valueType) throws IOException { + testSearchCase(query, dataset, configure, verify, valueType); + testSearchAndReduceCase(query, dataset, configure, verify, valueType); + } + + private void executeTestCase(boolean reduced, Query query, List dataset, + Consumer configure, + Consumer verify, ValueType valueType) throws IOException { + + try (Directory directory = newDirectory()) { + try (RandomIndexWriter indexWriter = new RandomIndexWriter(random(), directory)) { + Document document = new Document(); + for (String value : dataset) { + if (frequently()) { + indexWriter.commit(); + } + + document.add(new SortedSetDocValuesField(KEYWORD_FIELD, new BytesRef(value))); + indexWriter.addDocument(document); + document.clear(); + } + } + + try (IndexReader indexReader = DirectoryReader.open(directory)) { + IndexSearcher indexSearcher = newIndexSearcher(indexReader); + + TermsAggregationBuilder aggregationBuilder = new TermsAggregationBuilder("_name"); + if (valueType != null) { + aggregationBuilder.userValueTypeHint(valueType); + } + if (configure != null) { + configure.accept(aggregationBuilder); + } + + MappedFieldType keywordFieldType = new KeywordFieldMapper.KeywordFieldType(); + keywordFieldType.setName(KEYWORD_FIELD); + keywordFieldType.setHasDocValues(true); + + InternalMappedTerms rareTerms; + if (reduced) { + rareTerms = searchAndReduce(indexSearcher, query, aggregationBuilder, keywordFieldType); + } else { + rareTerms = search(indexSearcher, query, aggregationBuilder, keywordFieldType); + } + verify.accept(rareTerms); + } + } + } + +} diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/terms/NumericTermsAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/terms/NumericTermsAggregatorTests.java new file mode 100644 index 0000000000000..b0790b9fa0413 --- /dev/null +++ b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/terms/NumericTermsAggregatorTests.java @@ -0,0 +1,190 @@ +/* + * Licensed to Elasticsearch under one or more contributor + * license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright + * ownership. Elasticsearch licenses this file to you under + * the Apache License, Version 2.0 (the "License"); you may + * not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.elasticsearch.search.aggregations.bucket.terms; + +import org.apache.lucene.document.Document; +import org.apache.lucene.document.LongPoint; +import org.apache.lucene.document.SortedNumericDocValuesField; +import org.apache.lucene.index.DirectoryReader; +import org.apache.lucene.index.IndexReader; +import org.apache.lucene.index.RandomIndexWriter; +import org.apache.lucene.search.IndexSearcher; +import org.apache.lucene.search.MatchAllDocsQuery; +import org.apache.lucene.search.MatchNoDocsQuery; +import org.apache.lucene.search.Query; +import org.apache.lucene.store.Directory; +import org.apache.lucene.util.automaton.RegExp; +import org.elasticsearch.index.mapper.MappedFieldType; +import org.elasticsearch.index.mapper.NumberFieldMapper; +import org.elasticsearch.search.aggregations.AggregationExecutionException; +import org.elasticsearch.search.aggregations.AggregatorTestCase; +import org.elasticsearch.search.aggregations.support.ValueType; + +import java.io.IOException; +import java.util.ArrayList; +import java.util.List; +import java.util.function.Consumer; + +import static org.hamcrest.Matchers.equalTo; + +public class NumericTermsAggregatorTests extends AggregatorTestCase { + private static final String LONG_FIELD = "long"; + + private static final List dataset; + static { + List d = new ArrayList<>(45); + for (int i = 0; i < 10; i++) { + for (int j = 0; j < i; j++) { + d.add((long) i); + } + } + dataset = d; + } + + public void testMatchNoDocs() throws IOException { + + testBothCases(new MatchNoDocsQuery(), dataset, + aggregation -> aggregation.field(LONG_FIELD), + agg -> assertEquals(0, agg.getBuckets().size()), null // without type hint + ); + + testBothCases(new MatchNoDocsQuery(), dataset, + aggregation -> aggregation.field(LONG_FIELD), + agg -> assertEquals(0, agg.getBuckets().size()), ValueType.NUMERIC // with type hint + ); + } + + public void testMatchAllDocs() throws IOException { + Query query = new MatchAllDocsQuery(); + + testBothCases(query, dataset, + aggregation -> aggregation.field(LONG_FIELD), + agg -> { + assertEquals(9, agg.getBuckets().size()); + for (int i = 0; i < 9; i++) { + LongTerms.Bucket bucket = (LongTerms.Bucket) agg.getBuckets().get(i); + assertThat(bucket.getKey(), equalTo(9L - i)); + assertThat(bucket.getDocCount(), equalTo(9L - i)); + } + }, null //without type hint + ); + + testBothCases(query, dataset, + aggregation -> aggregation.field(LONG_FIELD), + agg -> { + assertEquals(9, agg.getBuckets().size()); + for (int i = 0; i < 9; i++) { + LongTerms.Bucket bucket = (LongTerms.Bucket) agg.getBuckets().get(i); + assertThat(bucket.getKey(), equalTo(9L - i)); + assertThat(bucket.getDocCount(), equalTo(9L - i)); + } + }, ValueType.NUMERIC //with type hint + ); + } + + public void testBadIncludeExclude() throws IOException { + IncludeExclude includeExclude = new IncludeExclude(new RegExp("foo"), null); + + // Numerics don't support any regex include/exclude, so should fail no matter what we do + + AggregationExecutionException e = expectThrows(AggregationExecutionException.class, + () -> testBothCases(new MatchNoDocsQuery(), dataset, + aggregation -> aggregation.field(LONG_FIELD).includeExclude(includeExclude).format("yyyy-MM-dd"), + agg -> fail("test should have failed with exception"), null + )); + assertThat(e.getMessage(), equalTo("Aggregation [_name] cannot support regular expression style " + + "include/exclude settings as they can only be applied to string fields. Use an array of numeric " + + "values for include/exclude clauses used to filter numeric fields")); + + e = expectThrows(AggregationExecutionException.class, + () -> testBothCases(new MatchNoDocsQuery(), dataset, + aggregation -> aggregation.field(LONG_FIELD).includeExclude(includeExclude).format("yyyy-MM-dd"), + agg -> fail("test should have failed with exception"), ValueType.NUMERIC // with type hint + )); + assertThat(e.getMessage(), equalTo("Aggregation [_name] cannot support regular expression style " + + "include/exclude settings as they can only be applied to string fields. Use an array of numeric " + + "values for include/exclude clauses used to filter numeric fields")); + + } + + private void testSearchCase(Query query, List dataset, + Consumer configure, + Consumer verify, ValueType valueType) throws IOException { + executeTestCase(false, query, dataset, configure, verify, valueType); + } + + private void testSearchAndReduceCase(Query query, List dataset, + Consumer configure, + Consumer verify, ValueType valueType) throws IOException { + executeTestCase(true, query, dataset, configure, verify, valueType); + } + + private void testBothCases(Query query, List dataset, + Consumer configure, + Consumer verify, ValueType valueType) throws IOException { + testSearchCase(query, dataset, configure, verify, valueType); + testSearchAndReduceCase(query, dataset, configure, verify, valueType); + } + + private void executeTestCase(boolean reduced, Query query, List dataset, + Consumer configure, + Consumer verify, ValueType valueType) throws IOException { + + try (Directory directory = newDirectory()) { + try (RandomIndexWriter indexWriter = new RandomIndexWriter(random(), directory)) { + Document document = new Document(); + for (Long value : dataset) { + if (frequently()) { + indexWriter.commit(); + } + + document.add(new SortedNumericDocValuesField(LONG_FIELD, value)); + document.add(new LongPoint(LONG_FIELD, value)); + indexWriter.addDocument(document); + document.clear(); + } + } + + try (IndexReader indexReader = DirectoryReader.open(directory)) { + IndexSearcher indexSearcher = newIndexSearcher(indexReader); + + TermsAggregationBuilder aggregationBuilder = new TermsAggregationBuilder("_name"); + if (valueType != null) { + aggregationBuilder.userValueTypeHint(valueType); + } + if (configure != null) { + configure.accept(aggregationBuilder); + } + + MappedFieldType longFieldType = new NumberFieldMapper.NumberFieldType(NumberFieldMapper.NumberType.LONG); + longFieldType.setName(LONG_FIELD); + longFieldType.setHasDocValues(true); + + InternalMappedTerms rareTerms; + if (reduced) { + rareTerms = searchAndReduce(indexSearcher, query, aggregationBuilder, longFieldType); + } else { + rareTerms = search(indexSearcher, query, aggregationBuilder, longFieldType); + } + verify.accept(rareTerms); + } + } + } + +} diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregatorTests.java index 4bfd9ac36be8e..58b4aa8a6e419 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregatorTests.java @@ -52,7 +52,6 @@ import org.elasticsearch.index.mapper.Uid; import org.elasticsearch.search.SearchHit; import org.elasticsearch.search.aggregations.Aggregation; -import org.elasticsearch.search.aggregations.AggregationExecutionException; import org.elasticsearch.search.aggregations.Aggregations; import org.elasticsearch.search.aggregations.Aggregator; import org.elasticsearch.search.aggregations.AggregatorTestCase; @@ -273,7 +272,8 @@ public void testUnmapped() throws Exception { ValueType[] valueTypes = new ValueType[]{ValueType.STRING, ValueType.LONG}; String[] fieldNames = new String[]{"string", "long"}; for (int i = 0; i < fieldNames.length; i++) { - RareTermsAggregationBuilder aggregationBuilder = new RareTermsAggregationBuilder("_name", valueTypes[i]) + RareTermsAggregationBuilder aggregationBuilder = new RareTermsAggregationBuilder("_name") + .userValueTypeHint(valueTypes[i]) .field(fieldNames[i]); Aggregator aggregator = createAggregator(aggregationBuilder, indexSearcher, fieldType1, fieldType2); aggregator.preCollection(); @@ -308,9 +308,9 @@ public void testRangeField() throws Exception { try (IndexReader indexReader = maybeWrapReaderEs(indexWriter.getReader())) { IndexSearcher indexSearcher = newIndexSearcher(indexReader); - RareTermsAggregationBuilder aggregationBuilder = new RareTermsAggregationBuilder("_name", null) + RareTermsAggregationBuilder aggregationBuilder = new RareTermsAggregationBuilder("_name") .field("field"); - expectThrows(AggregationExecutionException.class, + expectThrows(IllegalArgumentException.class, () -> createAggregator(aggregationBuilder, indexSearcher, fieldType)); } } @@ -322,7 +322,8 @@ public void testNestedTerms() throws IOException { Query query = new MatchAllDocsQuery(); testBothCases(query, dataset, aggregation -> { - TermsAggregationBuilder terms = new TermsAggregationBuilder("the_terms", ValueType.STRING).field(KEYWORD_FIELD); + TermsAggregationBuilder terms = new TermsAggregationBuilder("the_terms").userValueTypeHint(ValueType.STRING) + .field(KEYWORD_FIELD); aggregation.field(LONG_FIELD).maxDocCount(1).subAggregation(terms); }, agg -> { @@ -340,7 +341,8 @@ public void testNestedTerms() throws IOException { ); testBothCases(query, dataset, aggregation -> { - TermsAggregationBuilder terms = new TermsAggregationBuilder("the_terms", ValueType.STRING).field(KEYWORD_FIELD); + TermsAggregationBuilder terms = new TermsAggregationBuilder("the_terms").userValueTypeHint(ValueType.STRING) + .field(KEYWORD_FIELD); aggregation.field(KEYWORD_FIELD).maxDocCount(1).subAggregation(terms); }, agg -> { @@ -375,10 +377,12 @@ public void testGlobalAggregationWithScore() throws IOException { Aggregator.SubAggCollectionMode collectionMode = randomFrom(Aggregator.SubAggCollectionMode.values()); GlobalAggregationBuilder globalBuilder = new GlobalAggregationBuilder("global") .subAggregation( - new RareTermsAggregationBuilder("terms", ValueType.STRING) + new RareTermsAggregationBuilder("terms") + .userValueTypeHint(ValueType.STRING) .field("keyword") .subAggregation( - new RareTermsAggregationBuilder("sub_terms", ValueType.STRING) + new RareTermsAggregationBuilder("sub_terms") + .userValueTypeHint(ValueType.STRING) .field("keyword") .subAggregation( new TopHitsAggregationBuilder("top_hits") @@ -422,7 +426,8 @@ public void testWithNestedAggregations() throws IOException { indexWriter.commit(); NestedAggregationBuilder nested = new NestedAggregationBuilder("nested", "nested_object") - .subAggregation(new RareTermsAggregationBuilder("terms", ValueType.LONG) + .subAggregation(new RareTermsAggregationBuilder("terms") + .userValueTypeHint(ValueType.LONG) .field("nested_value") .maxDocCount(1) ); @@ -455,7 +460,8 @@ public void testWithNestedScoringAggregations() throws IOException { indexWriter.commit(); for (boolean withScore : new boolean[]{true, false}) { NestedAggregationBuilder nested = new NestedAggregationBuilder("nested", "nested_object") - .subAggregation(new RareTermsAggregationBuilder("terms", ValueType.LONG) + .subAggregation(new RareTermsAggregationBuilder("terms") + .userValueTypeHint(ValueType.LONG) .field("nested_value") .maxDocCount(2) .subAggregation( @@ -580,7 +586,8 @@ private void executeTestCase(boolean reduced, Query query, List dataset, try (IndexReader indexReader = DirectoryReader.open(directory)) { IndexSearcher indexSearcher = newIndexSearcher(indexReader); - RareTermsAggregationBuilder aggregationBuilder = new RareTermsAggregationBuilder("_name", valueType); + RareTermsAggregationBuilder aggregationBuilder = new RareTermsAggregationBuilder("_name"); + aggregationBuilder.userValueTypeHint(valueType); if (configure != null) { configure.accept(aggregationBuilder); } diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorTests.java index 57b255a157382..d1cd50ba2f2ba 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorTests.java @@ -128,13 +128,16 @@ protected A createAggregator(AggregationBuilder aggregati @Override protected AggregationBuilder createAggBuilderForTypeTest(MappedFieldType fieldType, String fieldName) { - return new TermsAggregationBuilder("foo", ValueType.STRING).field(fieldName); + return new TermsAggregationBuilder("foo").field(fieldName); } @Override protected List getSupportedValuesSourceTypes() { return Collections.unmodifiableList(asList(CoreValuesSourceType.NUMERIC, - CoreValuesSourceType.BYTES)); + CoreValuesSourceType.BYTES, + CoreValuesSourceType.IP, + CoreValuesSourceType.DATE, + CoreValuesSourceType.BOOLEAN)); } public void testGlobalOrdinalsExecutionHint() throws Exception { @@ -147,7 +150,7 @@ public void testGlobalOrdinalsExecutionHint() throws Exception { // We do not use LuceneTestCase.newSearcher because we need a DirectoryReader IndexSearcher indexSearcher = new IndexSearcher(indexReader); - TermsAggregationBuilder aggregationBuilder = new TermsAggregationBuilder("_name", ValueType.STRING) + TermsAggregationBuilder aggregationBuilder = new TermsAggregationBuilder("_name").userValueTypeHint(ValueType.STRING) .field("string") .collectMode(Aggregator.SubAggCollectionMode.BREADTH_FIRST); MappedFieldType fieldType = new KeywordFieldMapper.KeywordFieldType(); @@ -199,7 +202,8 @@ public void testSimple() throws Exception { try (IndexReader indexReader = maybeWrapReaderEs(indexWriter.getReader())) { IndexSearcher indexSearcher = newIndexSearcher(indexReader); for (TermsAggregatorFactory.ExecutionMode executionMode : TermsAggregatorFactory.ExecutionMode.values()) { - TermsAggregationBuilder aggregationBuilder = new TermsAggregationBuilder("_name", ValueType.STRING) + TermsAggregationBuilder aggregationBuilder = new TermsAggregationBuilder("_name") + .userValueTypeHint(ValueType.STRING) .executionHint(executionMode.toString()) .field("string") .order(BucketOrder.key(true)); @@ -270,7 +274,8 @@ public void testStringIncludeExclude() throws Exception { fieldType.setHasDocValues(true); String executionHint = randomFrom(TermsAggregatorFactory.ExecutionMode.values()).toString(); - TermsAggregationBuilder aggregationBuilder = new TermsAggregationBuilder("_name", ValueType.STRING) + TermsAggregationBuilder aggregationBuilder = new TermsAggregationBuilder("_name") + .userValueTypeHint(ValueType.STRING) .executionHint(executionHint) .includeExclude(new IncludeExclude("val00.+", null)) .field("mv_field") @@ -308,7 +313,7 @@ public void testStringIncludeExclude() throws Exception { MappedFieldType fieldType2 = new KeywordFieldMapper.KeywordFieldType(); fieldType2.setName("sv_field"); fieldType2.setHasDocValues(true); - aggregationBuilder = new TermsAggregationBuilder("_name", ValueType.STRING) + aggregationBuilder = new TermsAggregationBuilder("_name").userValueTypeHint(ValueType.STRING) .executionHint(executionHint) .includeExclude(new IncludeExclude("val00.+", null)) .field("sv_field") @@ -332,7 +337,7 @@ public void testStringIncludeExclude() throws Exception { assertEquals(1L, result.getBuckets().get(4).getDocCount()); assertTrue(AggregationInspectionHelper.hasValue((InternalTerms)result)); - aggregationBuilder = new TermsAggregationBuilder("_name", ValueType.STRING) + aggregationBuilder = new TermsAggregationBuilder("_name").userValueTypeHint(ValueType.STRING) .executionHint(executionHint) .includeExclude(new IncludeExclude("val00.+", "(val000|val001)")) .field("mv_field") @@ -362,7 +367,7 @@ public void testStringIncludeExclude() throws Exception { assertEquals(1L, result.getBuckets().get(7).getDocCount()); assertTrue(AggregationInspectionHelper.hasValue((InternalTerms)result)); - aggregationBuilder = new TermsAggregationBuilder("_name", ValueType.STRING) + aggregationBuilder = new TermsAggregationBuilder("_name").userValueTypeHint(ValueType.STRING) .executionHint(executionHint) .includeExclude(new IncludeExclude(null, "val00.+")) .field("mv_field") @@ -379,7 +384,7 @@ public void testStringIncludeExclude() throws Exception { assertEquals(1L, result.getBuckets().get(1).getDocCount()); assertTrue(AggregationInspectionHelper.hasValue((InternalTerms)result)); - aggregationBuilder = new TermsAggregationBuilder("_name", ValueType.STRING) + aggregationBuilder = new TermsAggregationBuilder("_name").userValueTypeHint(ValueType.STRING) .executionHint(executionHint) .includeExclude(new IncludeExclude(new String[]{"val000", "val010"}, null)) .field("mv_field") @@ -396,7 +401,7 @@ public void testStringIncludeExclude() throws Exception { assertEquals(1L, result.getBuckets().get(1).getDocCount()); assertTrue(AggregationInspectionHelper.hasValue((InternalTerms)result)); - aggregationBuilder = new TermsAggregationBuilder("_name", ValueType.STRING) + aggregationBuilder = new TermsAggregationBuilder("_name").userValueTypeHint(ValueType.STRING) .executionHint(executionHint) .includeExclude(new IncludeExclude(null, new String[]{"val001", "val002", "val003", "val004", "val005", "val006", "val007", "val008", "val009", "val011"})) @@ -452,7 +457,8 @@ public void testNumericIncludeExclude() throws Exception { fieldType.setHasDocValues(true ); String executionHint = randomFrom(TermsAggregatorFactory.ExecutionMode.values()).toString(); - TermsAggregationBuilder aggregationBuilder = new TermsAggregationBuilder("_name", ValueType.LONG) + TermsAggregationBuilder aggregationBuilder = new TermsAggregationBuilder("_name") + .userValueTypeHint(ValueType.LONG) .executionHint(executionHint) .includeExclude(new IncludeExclude(new long[]{0, 5}, null)) .field("long_field") @@ -469,7 +475,7 @@ public void testNumericIncludeExclude() throws Exception { assertEquals(1L, result.getBuckets().get(1).getDocCount()); assertTrue(AggregationInspectionHelper.hasValue((InternalTerms)result)); - aggregationBuilder = new TermsAggregationBuilder("_name", ValueType.LONG) + aggregationBuilder = new TermsAggregationBuilder("_name").userValueTypeHint(ValueType.LONG) .executionHint(executionHint) .includeExclude(new IncludeExclude(null, new long[]{0, 5})) .field("long_field") @@ -493,7 +499,7 @@ public void testNumericIncludeExclude() throws Exception { fieldType = new NumberFieldMapper.NumberFieldType(NumberFieldMapper.NumberType.DOUBLE); fieldType.setName("double_field"); fieldType.setHasDocValues(true ); - aggregationBuilder = new TermsAggregationBuilder("_name", ValueType.DOUBLE) + aggregationBuilder = new TermsAggregationBuilder("_name").userValueTypeHint(ValueType.DOUBLE) .executionHint(executionHint) .includeExclude(new IncludeExclude(new double[]{0.0, 5.0}, null)) .field("double_field") @@ -510,7 +516,7 @@ public void testNumericIncludeExclude() throws Exception { assertEquals(1L, result.getBuckets().get(1).getDocCount()); assertTrue(AggregationInspectionHelper.hasValue((InternalTerms)result)); - aggregationBuilder = new TermsAggregationBuilder("_name", ValueType.DOUBLE) + aggregationBuilder = new TermsAggregationBuilder("_name").userValueTypeHint(ValueType.DOUBLE) .executionHint(executionHint) .includeExclude(new IncludeExclude(null, new double[]{0.0, 5.0})) .field("double_field") @@ -671,7 +677,8 @@ private void termsAggregator(ValueType valueType, MappedFieldType fieldType, String executionHint = randomFrom(TermsAggregatorFactory.ExecutionMode.values()).toString(); logger.info("bucket_order={} size={} execution_hint={}", bucketOrder, size, executionHint); IndexSearcher indexSearcher = newIndexSearcher(indexReader); - AggregationBuilder aggregationBuilder = new TermsAggregationBuilder("_name", valueType) + AggregationBuilder aggregationBuilder = new TermsAggregationBuilder("_name") + .userValueTypeHint(valueType) .executionHint(executionHint) .size(size) .shardSize(size) @@ -701,7 +708,8 @@ private void termsAggregator(ValueType valueType, MappedFieldType fieldType, MappedFieldType filterFieldType = new KeywordFieldMapper.KeywordFieldType(); filterFieldType.setName("include"); aggregationBuilder = new FilterAggregationBuilder("_name1", QueryBuilders.termQuery("include", "yes")); - aggregationBuilder.subAggregation(new TermsAggregationBuilder("_name2", valueType) + aggregationBuilder.subAggregation(new TermsAggregationBuilder("_name2") + .userValueTypeHint(valueType) .executionHint(executionHint) .size(numTerms) .collectMode(randomFrom(Aggregator.SubAggCollectionMode.values())) @@ -767,7 +775,8 @@ private void termsAggregatorWithNestedMaxAgg(ValueType valueType, MappedFiel logger.info("bucket_order={} size={} execution_hint={}, collect_mode={}", bucketOrder, size, executionHint, collectionMode); IndexSearcher indexSearcher = newIndexSearcher(indexReader); - AggregationBuilder aggregationBuilder = new TermsAggregationBuilder("_name", valueType) + AggregationBuilder aggregationBuilder = new TermsAggregationBuilder("_name") + .userValueTypeHint(valueType) .executionHint(executionHint) .collectMode(collectionMode) .size(size) @@ -813,7 +822,8 @@ public void testEmpty() throws Exception { fieldType1.setHasDocValues(true); try (IndexReader indexReader = maybeWrapReaderEs(indexWriter.getReader())) { IndexSearcher indexSearcher = newIndexSearcher(indexReader); - TermsAggregationBuilder aggregationBuilder = new TermsAggregationBuilder("_name", ValueType.STRING) + TermsAggregationBuilder aggregationBuilder = new TermsAggregationBuilder("_name") + .userValueTypeHint(ValueType.STRING) .field("string"); Aggregator aggregator = createAggregator(aggregationBuilder, indexSearcher, fieldType1); aggregator.preCollection(); @@ -823,7 +833,7 @@ public void testEmpty() throws Exception { assertEquals("_name", result.getName()); assertEquals(0, result.getBuckets().size()); - aggregationBuilder = new TermsAggregationBuilder("_name", ValueType.LONG) + aggregationBuilder = new TermsAggregationBuilder("_name").userValueTypeHint(ValueType.LONG) .field("long"); aggregator = createAggregator(aggregationBuilder, indexSearcher, fieldType2); aggregator.preCollection(); @@ -833,7 +843,7 @@ public void testEmpty() throws Exception { assertEquals("_name", result.getName()); assertEquals(0, result.getBuckets().size()); - aggregationBuilder = new TermsAggregationBuilder("_name", ValueType.DOUBLE) + aggregationBuilder = new TermsAggregationBuilder("_name").userValueTypeHint(ValueType.DOUBLE) .field("double"); aggregator = createAggregator(aggregationBuilder, indexSearcher, fieldType3); aggregator.preCollection(); @@ -855,7 +865,8 @@ public void testUnmapped() throws Exception { ValueType[] valueTypes = new ValueType[]{ValueType.STRING, ValueType.LONG, ValueType.DOUBLE}; String[] fieldNames = new String[]{"string", "long", "double"}; for (int i = 0; i < fieldNames.length; i++) { - TermsAggregationBuilder aggregationBuilder = new TermsAggregationBuilder("_name", valueTypes[i]) + TermsAggregationBuilder aggregationBuilder = new TermsAggregationBuilder("_name") + .userValueTypeHint(valueTypes[i]) .field(fieldNames[i]); Aggregator aggregator = createAggregator(aggregationBuilder, indexSearcher, (MappedFieldType) null); aggregator.preCollection(); @@ -892,7 +903,8 @@ public void testUnmappedWithMissing() throws Exception { for (int i = 0; i < fieldNames.length; i++) { - TermsAggregationBuilder aggregationBuilder = new TermsAggregationBuilder("_name", valueTypes[i]) + TermsAggregationBuilder aggregationBuilder = new TermsAggregationBuilder("_name") + .userValueTypeHint(valueTypes[i]) .field(fieldNames[i]).missing(missingValues[i]); Aggregator aggregator = createAggregator(aggregationBuilder, indexSearcher, fieldType1); aggregator.preCollection(); @@ -927,9 +939,8 @@ public void testRangeField() throws Exception { fieldType.setName(fieldName); IndexSearcher indexSearcher = newIndexSearcher(indexReader); - TermsAggregationBuilder aggregationBuilder = new TermsAggregationBuilder("_name", null) .field(fieldName); - // Note - other places we throw IllegalArgumentException - expectThrows(AggregationExecutionException.class, () -> { + TermsAggregationBuilder aggregationBuilder = new TermsAggregationBuilder("_name") .field(fieldName); + expectThrows(IllegalArgumentException.class, () -> { createAggregator(aggregationBuilder, indexSearcher, fieldType); }); } @@ -951,9 +962,8 @@ public void testGeoPointField() throws Exception { fieldType.setName("field"); IndexSearcher indexSearcher = newIndexSearcher(indexReader); - TermsAggregationBuilder aggregationBuilder = new TermsAggregationBuilder("_name", null) .field(field); - // Note - other places we throw IllegalArgumentException - expectThrows(AggregationExecutionException.class, () -> { + TermsAggregationBuilder aggregationBuilder = new TermsAggregationBuilder("_name") .field(field); + expectThrows(IllegalArgumentException.class, () -> { createAggregator(aggregationBuilder, indexSearcher, fieldType); }); } @@ -975,7 +985,7 @@ public void testIpField() throws Exception { fieldType.setName("field"); IndexSearcher indexSearcher = newIndexSearcher(indexReader); - TermsAggregationBuilder aggregationBuilder = new TermsAggregationBuilder("_name", null) .field(field); + TermsAggregationBuilder aggregationBuilder = new TermsAggregationBuilder("_name") .field(field); // Note - other places we throw IllegalArgumentException Aggregator aggregator = createAggregator(aggregationBuilder, indexSearcher, fieldType); aggregator.preCollection(); @@ -1010,12 +1020,13 @@ public void testNestedTermsAgg() throws Exception { IndexSearcher indexSearcher = newIndexSearcher(indexReader); String executionHint = randomFrom(TermsAggregatorFactory.ExecutionMode.values()).toString(); Aggregator.SubAggCollectionMode collectionMode = randomFrom(Aggregator.SubAggCollectionMode.values()); - TermsAggregationBuilder aggregationBuilder = new TermsAggregationBuilder("_name1", ValueType.STRING) + TermsAggregationBuilder aggregationBuilder = new TermsAggregationBuilder("_name1") + .userValueTypeHint(ValueType.STRING) .executionHint(executionHint) .collectMode(collectionMode) .field("field1") .order(BucketOrder.key(true)) - .subAggregation(new TermsAggregationBuilder("_name2", ValueType.STRING) + .subAggregation(new TermsAggregationBuilder("_name2").userValueTypeHint(ValueType.STRING) .executionHint(executionHint) .collectMode(collectionMode) .field("field2") @@ -1053,7 +1064,7 @@ public void testNestedTermsAgg() throws Exception { public void testMixLongAndDouble() throws Exception { for (TermsAggregatorFactory.ExecutionMode executionMode : TermsAggregatorFactory.ExecutionMode.values()) { - TermsAggregationBuilder aggregationBuilder = new TermsAggregationBuilder("_name", ValueType.LONG) + TermsAggregationBuilder aggregationBuilder = new TermsAggregationBuilder("_name").userValueTypeHint(ValueType.LONG) .executionHint(executionMode.toString()) .field("number") .order(BucketOrder.key(true)); @@ -1125,13 +1136,13 @@ public void testGlobalAggregationWithScore() throws IOException { Aggregator.SubAggCollectionMode collectionMode = randomFrom(Aggregator.SubAggCollectionMode.values()); GlobalAggregationBuilder globalBuilder = new GlobalAggregationBuilder("global") .subAggregation( - new TermsAggregationBuilder("terms", ValueType.STRING) + new TermsAggregationBuilder("terms").userValueTypeHint(ValueType.STRING) .executionHint(executionHint) .collectMode(collectionMode) .field("keyword") .order(BucketOrder.key(true)) .subAggregation( - new TermsAggregationBuilder("sub_terms", ValueType.STRING) + new TermsAggregationBuilder("sub_terms").userValueTypeHint(ValueType.STRING) .executionHint(executionHint) .collectMode(collectionMode) .field("keyword").order(BucketOrder.key(true)) @@ -1178,7 +1189,7 @@ public void testWithNestedAggregations() throws IOException { for (Aggregator.SubAggCollectionMode mode : Aggregator.SubAggCollectionMode.values()) { for (boolean withScore : new boolean[]{true, false}) { NestedAggregationBuilder nested = new NestedAggregationBuilder("nested", "nested_object") - .subAggregation(new TermsAggregationBuilder("terms", ValueType.LONG) + .subAggregation(new TermsAggregationBuilder("terms").userValueTypeHint(ValueType.LONG) .field("nested_value") // force the breadth_first mode .collectMode(mode) @@ -1244,7 +1255,7 @@ public void testOrderByPipelineAggregation() throws Exception { "script", new Script("2.718")); TermsAggregationBuilder termsAgg = terms("terms") .field("field") - .valueType(ValueType.STRING) + .userValueTypeHint(ValueType.STRING) .order(BucketOrder.aggregation("script", true)) .subAggregation(bucketScriptAgg); diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/AvgAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/AvgAggregatorTests.java index 46f08d786a856..c832e890179f5 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/AvgAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/AvgAggregatorTests.java @@ -522,7 +522,7 @@ public void testOrderByEmptyAggregation() throws IOException { fieldType.setName("value"); fieldType.setHasDocValues(true); - AggregationBuilder aggregationBuilder = new TermsAggregationBuilder("terms", ValueType.NUMERIC) + AggregationBuilder aggregationBuilder = new TermsAggregationBuilder("terms").userValueTypeHint(ValueType.NUMERIC) .field("value") .order(BucketOrder.compound(BucketOrder.aggregation("filter>avg", true))) .subAggregation(AggregationBuilders.filter("filter", termQuery("value", 100)) diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/CardinalityAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/CardinalityAggregatorTests.java index 87c3c12c6bb57..e9a203e671963 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/CardinalityAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/CardinalityAggregatorTests.java @@ -135,7 +135,7 @@ public void testQueryFiltersAll() throws IOException { } public void testUnmappedMissingString() throws IOException { - CardinalityAggregationBuilder aggregationBuilder = new CardinalityAggregationBuilder("name", null) + CardinalityAggregationBuilder aggregationBuilder = new CardinalityAggregationBuilder("name") .field("number").missing("🍌🍌🍌"); testCase(aggregationBuilder, new MatchAllDocsQuery(), iw -> { @@ -149,7 +149,7 @@ public void testUnmappedMissingString() throws IOException { } public void testUnmappedMissingNumber() throws IOException { - CardinalityAggregationBuilder aggregationBuilder = new CardinalityAggregationBuilder("name", null) + CardinalityAggregationBuilder aggregationBuilder = new CardinalityAggregationBuilder("name") .field("number").missing(1234); testCase(aggregationBuilder, new MatchAllDocsQuery(), iw -> { @@ -163,7 +163,7 @@ public void testUnmappedMissingNumber() throws IOException { } public void testUnmappedMissingGeoPoint() throws IOException { - CardinalityAggregationBuilder aggregationBuilder = new CardinalityAggregationBuilder("name", null) + CardinalityAggregationBuilder aggregationBuilder = new CardinalityAggregationBuilder("name") .field("number").missing(new GeoPoint(42.39561, -71.13051)); testCase(aggregationBuilder, new MatchAllDocsQuery(), iw -> { @@ -181,7 +181,7 @@ private void testCase(Query query, CheckedConsumer { @Override protected final CardinalityAggregationBuilder createTestAggregatorBuilder() { - CardinalityAggregationBuilder factory = new CardinalityAggregationBuilder(randomAlphaOfLengthBetween(3, 10), null); + CardinalityAggregationBuilder factory = new CardinalityAggregationBuilder(randomAlphaOfLengthBetween(3, 10)); String field = randomNumericField(); randomFieldOrScript(factory, field); if (randomBoolean()) { diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/ExtendedStatsAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/ExtendedStatsAggregatorTests.java index 83713ff52af84..001f18e8eea47 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/ExtendedStatsAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/ExtendedStatsAggregatorTests.java @@ -42,6 +42,8 @@ public class ExtendedStatsAggregatorTests extends AggregatorTestCase { private static final double TOLERANCE = 1e-5; + // TODO: Add script test cases. Should fail with defaultValuesSourceType() commented out. + public void testEmpty() throws IOException { MappedFieldType ft = new NumberFieldMapper.NumberFieldType(NumberFieldMapper.NumberType.LONG); diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregatorTests.java index 4f27cc1e3099e..c692d45371039 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregatorTests.java @@ -28,11 +28,15 @@ import org.elasticsearch.common.geo.GeoPoint; import org.elasticsearch.index.mapper.GeoPointFieldMapper; import org.elasticsearch.index.mapper.MappedFieldType; +import org.elasticsearch.search.aggregations.AggregationBuilder; import org.elasticsearch.search.aggregations.AggregatorTestCase; import org.elasticsearch.search.aggregations.support.AggregationInspectionHelper; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import org.elasticsearch.test.geo.RandomGeoGenerator; import java.io.IOException; +import java.util.List; public class GeoCentroidAggregatorTests extends AggregatorTestCase { @@ -178,4 +182,13 @@ private void assertCentroid(RandomIndexWriter w, GeoPoint expectedCentroid) thro } } + @Override + protected AggregationBuilder createAggBuilderForTypeTest(MappedFieldType fieldType, String fieldName) { + return new GeoCentroidAggregationBuilder("foo").field(fieldName); + } + + @Override + protected List getSupportedValuesSourceTypes() { + return List.of(CoreValuesSourceType.GEOPOINT); + } } diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/HDRPercentileRanksAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/HDRPercentileRanksAggregatorTests.java index 9d9c74f283b45..5891981121000 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/HDRPercentileRanksAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/HDRPercentileRanksAggregatorTests.java @@ -44,6 +44,20 @@ public class HDRPercentileRanksAggregatorTests extends AggregatorTestCase { + @Override + protected AggregationBuilder createAggBuilderForTypeTest(MappedFieldType fieldType, String fieldName) { + return new PercentileRanksAggregationBuilder("hdr_ranks", new double[]{0.1, 0.5, 12}) + .field(fieldName) + .percentilesConfig(new PercentilesConfig.Hdr()); + } + + @Override + protected List getSupportedValuesSourceTypes() { + return List.of(CoreValuesSourceType.NUMERIC, + CoreValuesSourceType.DATE, + CoreValuesSourceType.BOOLEAN); + } + public void testEmpty() throws IOException { PercentileRanksAggregationBuilder aggBuilder = new PercentileRanksAggregationBuilder("my_agg", new double[]{0.5}) .field("field") diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/HDRPercentilesAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/HDRPercentilesAggregatorTests.java index 40ba3ef617b8c..14f2981051122 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/HDRPercentilesAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/HDRPercentilesAggregatorTests.java @@ -19,9 +19,11 @@ package org.elasticsearch.search.aggregations.metrics; +import org.apache.lucene.document.BinaryDocValuesField; import org.apache.lucene.document.LongPoint; import org.apache.lucene.document.NumericDocValuesField; import org.apache.lucene.document.SortedNumericDocValuesField; +import org.apache.lucene.document.SortedSetDocValuesField; import org.apache.lucene.index.DirectoryReader; import org.apache.lucene.index.IndexReader; import org.apache.lucene.index.RandomIndexWriter; @@ -30,13 +32,20 @@ import org.apache.lucene.search.MatchAllDocsQuery; import org.apache.lucene.search.Query; import org.apache.lucene.store.Directory; +import org.apache.lucene.util.BytesRef; import org.elasticsearch.common.CheckedConsumer; +import org.elasticsearch.index.mapper.KeywordFieldMapper; import org.elasticsearch.index.mapper.MappedFieldType; import org.elasticsearch.index.mapper.NumberFieldMapper; +import org.elasticsearch.index.mapper.RangeFieldMapper; +import org.elasticsearch.index.mapper.RangeType; +import org.elasticsearch.search.aggregations.AggregationBuilder; import org.elasticsearch.search.aggregations.AggregatorTestCase; import org.elasticsearch.search.aggregations.support.AggregationInspectionHelper; import java.io.IOException; +import java.util.Collections; +import java.util.List; import java.util.function.Consumer; import static java.util.Arrays.asList; @@ -46,6 +55,20 @@ public class HDRPercentilesAggregatorTests extends AggregatorTestCase { + @Override + protected AggregationBuilder createAggBuilderForTypeTest(MappedFieldType fieldType, String fieldName) { + return new PercentilesAggregationBuilder("hdr_percentiles") + .field(fieldName) + .percentilesConfig(new PercentilesConfig.Hdr()); + } + + @Override + protected List getSupportedValuesSourceTypes() { + return List.of(CoreValuesSourceType.NUMERIC, + CoreValuesSourceType.DATE, + CoreValuesSourceType.BOOLEAN); + } + public void testNoDocs() throws IOException { testCase(new MatchAllDocsQuery(), iw -> { // Intentionally not writing any docs @@ -55,6 +78,39 @@ public void testNoDocs() throws IOException { }); } + /** + * Attempting to use HDRPercentileAggregation on a string field throws IllegalArgumentException + */ + public void testStringField() throws IOException { + final String fieldName = "string"; + MappedFieldType fieldType = new KeywordFieldMapper.KeywordFieldType(); + fieldType.setName(fieldName); + fieldType.setHasDocValues(true); + expectThrows(IllegalArgumentException.class, + () -> testCase(new DocValuesFieldExistsQuery(fieldName), iw -> { + iw.addDocument(singleton(new SortedSetDocValuesField("string", new BytesRef("bogus")))); + iw.addDocument(singleton(new SortedSetDocValuesField("string", new BytesRef("zwomp")))); + iw.addDocument(singleton(new SortedSetDocValuesField("string", new BytesRef("foobar")))); + }, hdr -> {}, fieldType, fieldName)); + } + + /** + * Attempting to use HDRPercentileAggregation on a range field throws IllegalArgumentException + */ + @AwaitsFix(bugUrl = "https://github.com/elastic/elasticsearch/issues/42949") + public void testRangeField() throws IOException { + // Currently fails (throws ClassCast exception), but should be fixed once HDRPercentileAggregation uses the ValuesSource registry + final String fieldName = "range"; + MappedFieldType fieldType = new RangeFieldMapper.Builder(fieldName, RangeType.DOUBLE).fieldType(); + fieldType.setName(fieldName); + RangeFieldMapper.Range range =new RangeFieldMapper.Range(RangeType.DOUBLE, 1.0D, 5.0D, true, true); + BytesRef encodedRange = RangeType.DOUBLE.encodeRanges(Collections.singleton(range)); + expectThrows(IllegalArgumentException.class, + () -> testCase(new DocValuesFieldExistsQuery(fieldName), iw -> { + iw.addDocument(singleton(new BinaryDocValuesField(fieldName, encodedRange))); + }, hdr -> {}, fieldType, fieldName)); + } + public void testNoMatchingField() throws IOException { testCase(new MatchAllDocsQuery(), iw -> { iw.addDocument(singleton(new SortedNumericDocValuesField("wrong_number", 7))); @@ -133,6 +189,13 @@ public void testHdrThenTdigestSettings() throws Exception { private void testCase(Query query, CheckedConsumer buildIndex, Consumer verify) throws IOException { + MappedFieldType fieldType = new NumberFieldMapper.NumberFieldType(NumberFieldMapper.NumberType.LONG); + fieldType.setName("number"); + testCase(query, buildIndex, verify, fieldType, "number"); + } + + private void testCase(Query query, CheckedConsumer buildIndex, + Consumer verify, MappedFieldType fieldType, String fieldName) throws IOException { try (Directory directory = newDirectory()) { try (RandomIndexWriter indexWriter = new RandomIndexWriter(random(), directory)) { buildIndex.accept(indexWriter); @@ -144,14 +207,12 @@ private void testCase(Query query, CheckedConsumermax", true))) .subAggregation(AggregationBuilders.filter("filter", termQuery("value", 100)) @@ -831,7 +831,7 @@ public void testEarlyTermination() throws Exception { IndexSearcher indexSearcher = newSearcher(indexReader, true, true); MaxAggregationBuilder maxAggregationBuilder = new MaxAggregationBuilder("max") .field("values"); - ValueCountAggregationBuilder countAggregationBuilder = new ValueCountAggregationBuilder("count", null) + ValueCountAggregationBuilder countAggregationBuilder = new ValueCountAggregationBuilder("count") .field("values"); MaxAggregator maxAggregator = createAggregator(maxAggregationBuilder, indexSearcher, fieldType); @@ -881,9 +881,10 @@ public void testNestedEarlyTermination() throws Exception { for (Aggregator.SubAggCollectionMode collectionMode : Aggregator.SubAggCollectionMode.values()) { MaxAggregationBuilder maxAggregationBuilder = new MaxAggregationBuilder("max") .field("values"); - ValueCountAggregationBuilder countAggregationBuilder = new ValueCountAggregationBuilder("count", null) + ValueCountAggregationBuilder countAggregationBuilder = new ValueCountAggregationBuilder("count") .field("values"); - TermsAggregationBuilder termsAggregationBuilder = new TermsAggregationBuilder("terms", ValueType.NUMERIC) + TermsAggregationBuilder termsAggregationBuilder = new TermsAggregationBuilder("terms") + .userValueTypeHint(ValueType.NUMERIC) .field("value").collectMode(collectionMode) .subAggregation(new MaxAggregationBuilder("sub_max").field("invalid")); diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/MinAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/MinAggregatorTests.java index a54eef6f88126..dfbbc5151ee2f 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/MinAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/MinAggregatorTests.java @@ -23,6 +23,7 @@ import org.apache.lucene.document.DoublePoint; import org.apache.lucene.document.Field; import org.apache.lucene.document.FloatPoint; +import org.apache.lucene.document.InetAddressPoint; import org.apache.lucene.document.IntPoint; import org.apache.lucene.document.LongPoint; import org.apache.lucene.document.NumericDocValuesField; @@ -55,6 +56,7 @@ import org.elasticsearch.index.IndexSettings; import org.elasticsearch.index.mapper.ContentPath; import org.elasticsearch.index.mapper.DateFieldMapper; +import org.elasticsearch.index.mapper.IpFieldMapper; import org.elasticsearch.index.mapper.KeywordFieldMapper; import org.elasticsearch.index.mapper.MappedFieldType; import org.elasticsearch.index.mapper.Mapper; @@ -85,8 +87,6 @@ import org.elasticsearch.search.aggregations.bucket.terms.TermsAggregationBuilder; import org.elasticsearch.search.aggregations.support.AggregationInspectionHelper; import org.elasticsearch.search.aggregations.support.FieldContext; -import org.elasticsearch.search.aggregations.support.ValueType; -import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; import org.elasticsearch.search.internal.SearchContext; import org.elasticsearch.search.lookup.LeafDocLookup; @@ -263,6 +263,20 @@ public void testQueryFiltersAll() throws IOException { }); } + public void testIpField() throws IOException { + final String fieldName = "IP_field"; + MinAggregationBuilder aggregationBuilder = new MinAggregationBuilder("min").field(fieldName); + + MappedFieldType fieldType = new IpFieldMapper.IpFieldType(); + fieldType.setName(fieldName); + + boolean v4 = randomBoolean(); + expectThrows(IllegalArgumentException.class, () -> testCase(aggregationBuilder, new MatchAllDocsQuery(), iw -> { + iw.addDocument(singleton(new SortedSetDocValuesField(fieldName, new BytesRef(InetAddressPoint.encode(randomIp(v4)))))); + iw.addDocument(singleton(new SortedSetDocValuesField(fieldName, new BytesRef(InetAddressPoint.encode(randomIp(v4)))))); + }, min -> fail("expected an exception"), fieldType)); + } + public void testUnmappedWithMissingField() throws IOException { MinAggregationBuilder aggregationBuilder = new MinAggregationBuilder("min").field("does_not_exist").missing(0L); @@ -291,7 +305,7 @@ public void testUnsupportedType() { }, (Consumer) min -> { fail("Should have thrown exception"); }, fieldType)); - assertEquals(e.getMessage(), "Expected numeric type on field [not_a_number], but got [keyword]"); + assertEquals("Field [not_a_number] of type [keyword(indexed,tokenized)] is not supported for aggregation [min]", e.getMessage()); } public void testBadMissingField() { @@ -600,7 +614,7 @@ public void testMultiValuedFieldWithScriptParams() throws IOException { } public void testOrderByEmptyAggregation() throws IOException { - AggregationBuilder termsBuilder = new TermsAggregationBuilder("terms", ValueType.NUMERIC) + AggregationBuilder termsBuilder = new TermsAggregationBuilder("terms") .field("number") .order(BucketOrder.compound(BucketOrder.aggregation("filter>min", true))) .subAggregation(new FilterAggregationBuilder("filter", termQuery("number", 100)) @@ -666,11 +680,11 @@ public void testScriptCaching() throws IOException { fieldType.setName("number"); MinAggregationBuilder aggregationBuilder = new MinAggregationBuilder("min") .field("number") - .script(new Script(ScriptType.INLINE, MockScriptEngine.NAME, INVERT_SCRIPT, Collections.emptyMap()));; + .script(new Script(ScriptType.INLINE, MockScriptEngine.NAME, INVERT_SCRIPT, Collections.emptyMap())); MinAggregationBuilder nonDeterministicAggregationBuilder = new MinAggregationBuilder("min") .field("number") - .script(new Script(ScriptType.INLINE, MockScriptEngine.NAME, RANDOM_SCRIPT, Collections.emptyMap()));; + .script(new Script(ScriptType.INLINE, MockScriptEngine.NAME, RANDOM_SCRIPT, Collections.emptyMap())); try (Directory directory = newDirectory()) { RandomIndexWriter indexWriter = new RandomIndexWriter(random(), directory); @@ -882,10 +896,10 @@ private Aggregator mockAggregator() { return mock(Aggregator.class); } - private ValuesSourceConfig mockNumericValuesSourceConfig(String fieldName, - NumberFieldMapper.NumberType numType, - boolean indexed) { - ValuesSourceConfig config = mock(ValuesSourceConfig.class); + private ValuesSourceConfig mockNumericValuesSourceConfig(String fieldName, + NumberFieldMapper.NumberType numType, + boolean indexed) { + ValuesSourceConfig config = mock(ValuesSourceConfig.class); MappedFieldType ft = new NumberFieldMapper.NumberFieldType(numType); ft.setName(fieldName); ft.setIndexOptions(indexed ? IndexOptions.DOCS : IndexOptions.NONE); @@ -894,9 +908,9 @@ private ValuesSourceConfig mockNumericValuesSourceConfig(S return config; } - private ValuesSourceConfig mockDateValuesSourceConfig(String fieldName, boolean indexed, + private ValuesSourceConfig mockDateValuesSourceConfig(String fieldName, boolean indexed, DateFieldMapper.Resolution resolution) { - ValuesSourceConfig config = mock(ValuesSourceConfig.class); + ValuesSourceConfig config = mock(ValuesSourceConfig.class); Mapper.BuilderContext builderContext = new Mapper.BuilderContext( Settings.builder().put(IndexMetadata.SETTING_VERSION_CREATED, Version.CURRENT).build(), new ContentPath()); diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/ScriptedMetricAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/ScriptedMetricAggregatorTests.java index 44ca8820fdfd8..56206f268e1c0 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/ScriptedMetricAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/ScriptedMetricAggregatorTests.java @@ -426,7 +426,8 @@ protected QueryShardContext queryShardContextMock(IndexSearcher searcher, MockScriptEngine scriptEngine = new MockScriptEngine(MockScriptEngine.NAME, SCRIPTS, Collections.emptyMap()); Map engines = Collections.singletonMap(scriptEngine.getType(), scriptEngine); ScriptService scriptService = new ScriptService(Settings.EMPTY, engines, ScriptModule.CORE_CONTEXTS); - return new QueryShardContext(0, indexSettings, bigArrays, null, null, mapperService, null, scriptService, - xContentRegistry(), writableRegistry(), null, null, System::currentTimeMillis, null, null, () -> true); + return new QueryShardContext(0, indexSettings, BigArrays.NON_RECYCLING_INSTANCE, null, + null, mapperService, null, scriptService, xContentRegistry(), writableRegistry(), + null, null, System::currentTimeMillis, null, null, () -> true, null); } } diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/StatsAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/StatsAggregatorTests.java index 42804780727ed..70f83472a7292 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/StatsAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/StatsAggregatorTests.java @@ -72,6 +72,8 @@ public class StatsAggregatorTests extends AggregatorTestCase { private static final String VALUE_SCRIPT_NAME = "value_script"; private static final String FIELD_SCRIPT_NAME = "field_script"; + // TODO: Script tests, should fail with defaultValuesSourceType disabled. + public void testEmpty() throws IOException { final MappedFieldType ft = new NumberFieldMapper.NumberFieldType(NumberType.LONG); ft.setName("field"); diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/TDigestPercentileRanksAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/TDigestPercentileRanksAggregatorTests.java index 2541583e94580..a3ba44a55479f 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/TDigestPercentileRanksAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/TDigestPercentileRanksAggregatorTests.java @@ -44,6 +44,20 @@ public class TDigestPercentileRanksAggregatorTests extends AggregatorTestCase { + @Override + protected AggregationBuilder createAggBuilderForTypeTest(MappedFieldType fieldType, String fieldName) { + return new PercentileRanksAggregationBuilder("tdigest_ranks", new double[]{0.1, 0.5, 12}) + .field(fieldName) + .percentilesConfig(new PercentilesConfig.TDigest()); + } + + @Override + protected List getSupportedValuesSourceTypes() { + return List.of(CoreValuesSourceType.NUMERIC, + CoreValuesSourceType.DATE, + CoreValuesSourceType.BOOLEAN); + } + public void testEmpty() throws IOException { PercentileRanksAggregationBuilder aggBuilder = new PercentileRanksAggregationBuilder("my_agg", new double[]{0.5}) .field("field") diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/TDigestPercentilesAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/TDigestPercentilesAggregatorTests.java index aa65ae6f28bdd..15300284d9f6b 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/TDigestPercentilesAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/TDigestPercentilesAggregatorTests.java @@ -46,6 +46,20 @@ public class TDigestPercentilesAggregatorTests extends AggregatorTestCase { + @Override + protected AggregationBuilder createAggBuilderForTypeTest(MappedFieldType fieldType, String fieldName) { + return new PercentilesAggregationBuilder("tdist_percentiles") + .field(fieldName) + .percentilesConfig(new PercentilesConfig.TDigest()); + } + + @Override + protected List getSupportedValuesSourceTypes() { + return List.of(CoreValuesSourceType.NUMERIC, + CoreValuesSourceType.DATE, + CoreValuesSourceType.BOOLEAN); + } + public void testNoDocs() throws IOException { testCase(new MatchAllDocsQuery(), iw -> { // Intentionally not writing any docs diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregatorTests.java index b60c6743ebcf9..9673fecc92719 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregatorTests.java @@ -60,6 +60,40 @@ public class ValueCountAggregatorTests extends AggregatorTestCase { private static final String FIELD_NAME = "field"; + /** Script to return the {@code _value} provided by aggs framework. */ + private static final String VALUE_SCRIPT = "_value"; + private static final String SINGLE_SCRIPT = "single"; + + @Override + protected AggregationBuilder createAggBuilderForTypeTest(MappedFieldType fieldType, String fieldName) { + return new ValueCountAggregationBuilder("foo").field(fieldName); + } + + @Override + protected List getSupportedValuesSourceTypes() { + return List.of( + CoreValuesSourceType.NUMERIC, + CoreValuesSourceType.BYTES, + CoreValuesSourceType.GEOPOINT, + CoreValuesSourceType.RANGE + ); + } + + @Override + protected ScriptService getMockScriptService() { + Map, Object>> scripts = new HashMap<>(); + + scripts.put(VALUE_SCRIPT, vars -> (Double.valueOf((String) vars.get("_value")) + 1)); + scripts.put(SINGLE_SCRIPT, vars -> 1); + + MockScriptEngine scriptEngine = new MockScriptEngine(MockScriptEngine.NAME, + scripts, + Collections.emptyMap()); + Map engines = Collections.singletonMap(scriptEngine.getType(), scriptEngine); + + return new ScriptService(Settings.EMPTY, engines, ScriptModule.CORE_CONTEXTS); + } + public void testNoDocs() throws IOException { for (ValueType valueType : ValueType.values()) { testCase(new MatchAllDocsQuery(), valueType, iw -> { @@ -126,7 +160,7 @@ public void testQueryFiltersAll() throws IOException { } public void testUnmappedMissingString() throws IOException { - ValueCountAggregationBuilder aggregationBuilder = new ValueCountAggregationBuilder("name", null) + ValueCountAggregationBuilder aggregationBuilder = new ValueCountAggregationBuilder("name") .field("number").missing("🍌🍌🍌"); testCase(aggregationBuilder, new MatchAllDocsQuery(), iw -> { @@ -140,7 +174,7 @@ public void testUnmappedMissingString() throws IOException { } public void testUnmappedMissingNumber() throws IOException { - ValueCountAggregationBuilder aggregationBuilder = new ValueCountAggregationBuilder("name", null) + ValueCountAggregationBuilder aggregationBuilder = new ValueCountAggregationBuilder("name") .field("number").missing(1234); testCase(aggregationBuilder, new MatchAllDocsQuery(), iw -> { @@ -154,7 +188,7 @@ public void testUnmappedMissingNumber() throws IOException { } public void testUnmappedMissingGeoPoint() throws IOException { - ValueCountAggregationBuilder aggregationBuilder = new ValueCountAggregationBuilder("name", null) + ValueCountAggregationBuilder aggregationBuilder = new ValueCountAggregationBuilder("name") .field("number").missing(new GeoPoint(42.39561, -71.13051)); testCase(aggregationBuilder, new MatchAllDocsQuery(), iw -> { @@ -174,7 +208,7 @@ public void testRangeFieldValues() throws IOException { final String fieldName = "rangeField"; MappedFieldType fieldType = new RangeFieldMapper.Builder(fieldName, rangeType).fieldType(); fieldType.setName(fieldName); - final ValueCountAggregationBuilder aggregationBuilder = new ValueCountAggregationBuilder("_name", null).field(fieldName); + final ValueCountAggregationBuilder aggregationBuilder = new ValueCountAggregationBuilder("_name").field(fieldName); Set multiRecord = new HashSet<>(2); multiRecord.add(range1); multiRecord.add(range2); @@ -189,15 +223,126 @@ public void testRangeFieldValues() throws IOException { }, fieldType); } + public void testValueScriptNumber() throws IOException { + ValueCountAggregationBuilder aggregationBuilder = new ValueCountAggregationBuilder("name") + .field(FIELD_NAME) + .script(new Script(ScriptType.INLINE, MockScriptEngine.NAME, VALUE_SCRIPT, Collections.emptyMap())); + + MappedFieldType fieldType = createMappedFieldType(ValueType.NUMERIC); + fieldType.setName(FIELD_NAME); + fieldType.setHasDocValues(true); + + testCase(aggregationBuilder, new MatchAllDocsQuery(), iw -> { + iw.addDocument(singleton(new NumericDocValuesField(FIELD_NAME, 7))); + iw.addDocument(singleton(new NumericDocValuesField(FIELD_NAME, 8))); + iw.addDocument(singleton(new NumericDocValuesField(FIELD_NAME, 9))); + }, card -> { + assertEquals(3, card.getValue(), 0); + assertTrue(AggregationInspectionHelper.hasValue(card)); + }, fieldType); + } + + public void testSingleScriptNumber() throws IOException { + ValueCountAggregationBuilder aggregationBuilder = new ValueCountAggregationBuilder("name") + .field(FIELD_NAME); + + MappedFieldType fieldType = createMappedFieldType(ValueType.NUMERIC); + fieldType.setName(FIELD_NAME); + fieldType.setHasDocValues(true); + + testCase(aggregationBuilder, new MatchAllDocsQuery(), iw -> { + Document doc = new Document(); + doc.add(new SortedNumericDocValuesField(FIELD_NAME, 7)); + doc.add(new SortedNumericDocValuesField(FIELD_NAME, 7)); + iw.addDocument(doc); + + doc = new Document(); + doc.add(new SortedNumericDocValuesField(FIELD_NAME, 8)); + doc.add(new SortedNumericDocValuesField(FIELD_NAME, 8)); + iw.addDocument(doc); + + doc = new Document(); + doc.add(new SortedNumericDocValuesField(FIELD_NAME, 1)); + doc.add(new SortedNumericDocValuesField(FIELD_NAME, 1)); + iw.addDocument(doc); + }, card -> { + // note: this is 6, even though the script returns a single value. ValueCount does not de-dedupe + assertEquals(6, card.getValue(), 0); + assertTrue(AggregationInspectionHelper.hasValue(card)); + }, fieldType); + } + + public void testValueScriptString() throws IOException { + ValueCountAggregationBuilder aggregationBuilder = new ValueCountAggregationBuilder("name") + .field(FIELD_NAME) + .script(new Script(ScriptType.INLINE, MockScriptEngine.NAME, VALUE_SCRIPT, Collections.emptyMap())); + + MappedFieldType fieldType = createMappedFieldType(ValueType.STRING); + fieldType.setName(FIELD_NAME); + fieldType.setHasDocValues(true); + + testCase(aggregationBuilder, new MatchAllDocsQuery(), iw -> { + iw.addDocument(singleton(new SortedDocValuesField(FIELD_NAME, new BytesRef("1")))); + iw.addDocument(singleton(new SortedDocValuesField(FIELD_NAME, new BytesRef("2")))); + iw.addDocument(singleton(new SortedDocValuesField(FIELD_NAME, new BytesRef("3")))); + }, card -> { + assertEquals(3, card.getValue(), 0); + assertTrue(AggregationInspectionHelper.hasValue(card)); + }, fieldType); + } + + public void testSingleScriptString() throws IOException { + ValueCountAggregationBuilder aggregationBuilder = new ValueCountAggregationBuilder("name") + .field(FIELD_NAME); + + MappedFieldType fieldType = createMappedFieldType(ValueType.STRING); + fieldType.setName(FIELD_NAME); + fieldType.setHasDocValues(true); + + testCase(aggregationBuilder, new MatchAllDocsQuery(), iw -> { + Document doc = new Document(); + // Note: unlike numerics, lucene de-dupes strings so we increment here + doc.add(new SortedSetDocValuesField(FIELD_NAME, new BytesRef("1"))); + doc.add(new SortedSetDocValuesField(FIELD_NAME, new BytesRef("2"))); + iw.addDocument(doc); + + doc = new Document(); + doc.add(new SortedSetDocValuesField(FIELD_NAME, new BytesRef("3"))); + doc.add(new SortedSetDocValuesField(FIELD_NAME, new BytesRef("4"))); + iw.addDocument(doc); + + doc = new Document(); + doc.add(new SortedSetDocValuesField(FIELD_NAME, new BytesRef("5"))); + doc.add(new SortedSetDocValuesField(FIELD_NAME, new BytesRef("6"))); + iw.addDocument(doc); + }, card -> { + // note: this is 6, even though the script returns a single value. ValueCount does not de-dedupe + assertEquals(6, card.getValue(), 0); + assertTrue(AggregationInspectionHelper.hasValue(card)); + }, fieldType); + } + private void testCase(Query query, ValueType valueType, CheckedConsumer indexer, Consumer verify) throws IOException { + // Test both with and without the userValueTypeHint + testCase(query, valueType, indexer, verify, true); + testCase(query, valueType, indexer, verify, false); + } + + private void testCase(Query query, + ValueType valueType, + CheckedConsumer indexer, + Consumer verify, boolean testWithHint) throws IOException { MappedFieldType fieldType = createMappedFieldType(valueType); fieldType.setName(FIELD_NAME); fieldType.setHasDocValues(true); - ValueCountAggregationBuilder aggregationBuilder = new ValueCountAggregationBuilder("_name", valueType); + ValueCountAggregationBuilder aggregationBuilder = new ValueCountAggregationBuilder("_name"); + if (valueType != null && testWithHint) { + aggregationBuilder.userValueTypeHint(valueType); + } aggregationBuilder.field(FIELD_NAME); testCase(aggregationBuilder, query, indexer, verify, fieldType); diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/ValueCountTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/ValueCountTests.java index 591c096eea147..846bf79afc8d2 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/ValueCountTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/ValueCountTests.java @@ -25,7 +25,7 @@ public class ValueCountTests extends BaseAggregationTestCase aggBuilders = new HashSet<>(); aggBuilders.add(singleBucketAgg); aggBuilders.add(multiBucketAgg); diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/BucketScriptAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/BucketScriptAggregatorTests.java index 97e5fac24dcda..8f2eaa83509ce 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/BucketScriptAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/BucketScriptAggregatorTests.java @@ -78,7 +78,7 @@ public void testScript() throws IOException { fieldType1.setHasDocValues(true); FiltersAggregationBuilder filters = new FiltersAggregationBuilder("placeholder", new MatchAllQueryBuilder()) - .subAggregation(new TermsAggregationBuilder("the_terms", ValueType.STRING).field("the_field") + .subAggregation(new TermsAggregationBuilder("the_terms").userValueTypeHint(ValueType.STRING).field("the_field") .subAggregation(new AvgAggregationBuilder("the_avg").field("number_field"))) .subAggregation(new BucketScriptPipelineAggregationBuilder("bucket_script", Collections.singletonMap("the_avg", "the_terms['test1']>the_avg.value"), diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/ExtendedStatsBucketTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/ExtendedStatsBucketTests.java index 90cc9be95e3be..a7e2db4bada33 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/ExtendedStatsBucketTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/ExtendedStatsBucketTests.java @@ -62,7 +62,7 @@ public void testSigmaFromInt() throws Exception { public void testValidate() { AggregationBuilder singleBucketAgg = new GlobalAggregationBuilder("global"); - AggregationBuilder multiBucketAgg = new TermsAggregationBuilder("terms", ValueType.STRING); + AggregationBuilder multiBucketAgg = new TermsAggregationBuilder("terms").userValueTypeHint(ValueType.STRING); final Set aggBuilders = new HashSet<>(); aggBuilders.add(singleBucketAgg); aggBuilders.add(multiBucketAgg); diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/MaxBucketTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/MaxBucketTests.java index edbc1cda3eae2..7713a9f8d095c 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/MaxBucketTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/MaxBucketTests.java @@ -39,7 +39,7 @@ protected MaxBucketPipelineAggregationBuilder doCreateTestAggregatorFactory(Stri public void testValidate() { AggregationBuilder singleBucketAgg = new GlobalAggregationBuilder("global"); - AggregationBuilder multiBucketAgg = new TermsAggregationBuilder("terms", ValueType.STRING); + AggregationBuilder multiBucketAgg = new TermsAggregationBuilder("terms").userValueTypeHint(ValueType.STRING); final Set aggBuilders = new HashSet<>(); aggBuilders.add(singleBucketAgg); aggBuilders.add(multiBucketAgg); diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/MinBucketTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/MinBucketTests.java index 057e074a90cfc..e9d3e15a3fa2b 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/MinBucketTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/MinBucketTests.java @@ -39,7 +39,7 @@ protected MinBucketPipelineAggregationBuilder doCreateTestAggregatorFactory(Stri public void testValidate() { AggregationBuilder singleBucketAgg = new GlobalAggregationBuilder("global"); - AggregationBuilder multiBucketAgg = new TermsAggregationBuilder("terms", ValueType.STRING); + AggregationBuilder multiBucketAgg = new TermsAggregationBuilder("terms").userValueTypeHint(ValueType.STRING); final Set aggBuilders = new HashSet<>(); aggBuilders.add(singleBucketAgg); aggBuilders.add(multiBucketAgg); diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/PercentilesBucketTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/PercentilesBucketTests.java index 1918d26e1e1af..0b8757441cdb4 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/PercentilesBucketTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/PercentilesBucketTests.java @@ -66,7 +66,7 @@ public void testPercentsFromMixedArray() throws Exception { public void testValidate() { AggregationBuilder singleBucketAgg = new GlobalAggregationBuilder("global"); - AggregationBuilder multiBucketAgg = new TermsAggregationBuilder("terms", ValueType.STRING); + AggregationBuilder multiBucketAgg = new TermsAggregationBuilder("terms").userValueTypeHint(ValueType.STRING); final Set aggBuilders = new HashSet<>(); aggBuilders.add(singleBucketAgg); aggBuilders.add(multiBucketAgg); diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/PipelineAggregationHelperTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/PipelineAggregationHelperTests.java index 2d67735468984..02e932d607d2a 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/PipelineAggregationHelperTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/PipelineAggregationHelperTests.java @@ -115,7 +115,7 @@ public static class MockBucket { * @param values Array of values to compute metric for * @param metric A metric builder which defines what kind of metric should be returned for the values */ - public static double calculateMetric(double[] values, ValuesSourceAggregationBuilder metric) { + public static double calculateMetric(double[] values, ValuesSourceAggregationBuilder metric) { if (metric instanceof MinAggregationBuilder) { double accumulator = Double.POSITIVE_INFINITY; diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/SerialDiffIT.java b/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/SerialDiffIT.java index 9d5c790628fc6..ab664b2e0c16e 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/SerialDiffIT.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/SerialDiffIT.java @@ -24,7 +24,6 @@ import org.elasticsearch.common.collect.EvictingQueue; import org.elasticsearch.search.aggregations.bucket.histogram.Histogram; import org.elasticsearch.search.aggregations.bucket.histogram.Histogram.Bucket; -import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.test.ESIntegTestCase; import org.hamcrest.Matchers; @@ -57,7 +56,7 @@ public class SerialDiffIT extends ESIntegTestCase { static int numBuckets; static int lag; static BucketHelpers.GapPolicy gapPolicy; - static ValuesSourceAggregationBuilder> metric; + static ValuesSourceAggregationBuilder> metric; static List mockHisto; static Map> testValues; @@ -77,8 +76,8 @@ public String toString(){ } } - private ValuesSourceAggregationBuilder> randomMetric(String name, String field) { + private ValuesSourceAggregationBuilder< + ? extends ValuesSourceAggregationBuilder> randomMetric(String name, String field) { int rand = randomIntBetween(0,3); switch (rand) { diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/StatsBucketTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/StatsBucketTests.java index aac81c19be6bd..dffef51f643dd 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/StatsBucketTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/StatsBucketTests.java @@ -40,7 +40,7 @@ protected StatsBucketPipelineAggregationBuilder doCreateTestAggregatorFactory(St public void testValidate() { AggregationBuilder singleBucketAgg = new GlobalAggregationBuilder("global"); - AggregationBuilder multiBucketAgg = new TermsAggregationBuilder("terms", ValueType.STRING); + AggregationBuilder multiBucketAgg = new TermsAggregationBuilder("terms").userValueTypeHint(ValueType.STRING); final Set aggBuilders = new HashSet<>(); aggBuilders.add(singleBucketAgg); aggBuilders.add(multiBucketAgg); diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/SumBucketTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/SumBucketTests.java index 0bcbf592458aa..fc8c1be801313 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/SumBucketTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/SumBucketTests.java @@ -39,7 +39,7 @@ protected SumBucketPipelineAggregationBuilder doCreateTestAggregatorFactory(Stri public void testValidate() { AggregationBuilder singleBucketAgg = new GlobalAggregationBuilder("global"); - AggregationBuilder multiBucketAgg = new TermsAggregationBuilder("terms", ValueType.STRING); + AggregationBuilder multiBucketAgg = new TermsAggregationBuilder("terms").userValueTypeHint(ValueType.STRING); final Set aggBuilders = new HashSet<>(); aggBuilders.add(singleBucketAgg); aggBuilders.add(multiBucketAgg); diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/support/CoreValuesSourceTypeTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/support/CoreValuesSourceTypeTests.java index 5f57a8ee9c12e..a9f500bd09154 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/support/CoreValuesSourceTypeTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/support/CoreValuesSourceTypeTests.java @@ -19,60 +19,21 @@ package org.elasticsearch.search.aggregations.support; -import org.elasticsearch.common.io.stream.AbstractWriteableEnumTestCase; - -import java.io.IOException; +import org.elasticsearch.test.ESTestCase; import static org.hamcrest.Matchers.equalTo; -public class CoreValuesSourceTypeTests extends AbstractWriteableEnumTestCase { - - public CoreValuesSourceTypeTests() { - super(CoreValuesSourceType::fromStream); - } - - @Override - public void testValidOrdinals() { - assertThat(CoreValuesSourceType.ANY.ordinal(), equalTo(0)); - assertThat(CoreValuesSourceType.NUMERIC.ordinal(), equalTo(1)); - assertThat(CoreValuesSourceType.BYTES.ordinal(), equalTo(2)); - assertThat(CoreValuesSourceType.GEOPOINT.ordinal(), equalTo(3)); - assertThat(CoreValuesSourceType.RANGE.ordinal(), equalTo(4)); - assertThat(CoreValuesSourceType.HISTOGRAM.ordinal(), equalTo(5)); - } +public class CoreValuesSourceTypeTests extends ESTestCase { - @Override public void testFromString() { - assertThat(CoreValuesSourceType.fromString("any"), equalTo(CoreValuesSourceType.ANY)); assertThat(CoreValuesSourceType.fromString("numeric"), equalTo(CoreValuesSourceType.NUMERIC)); assertThat(CoreValuesSourceType.fromString("bytes"), equalTo(CoreValuesSourceType.BYTES)); assertThat(CoreValuesSourceType.fromString("geopoint"), equalTo(CoreValuesSourceType.GEOPOINT)); assertThat(CoreValuesSourceType.fromString("range"), equalTo(CoreValuesSourceType.RANGE)); - assertThat(CoreValuesSourceType.fromString("histogram"), equalTo(CoreValuesSourceType.HISTOGRAM)); IllegalArgumentException e = expectThrows(IllegalArgumentException.class, () -> CoreValuesSourceType.fromString("does_not_exist")); assertThat(e.getMessage(), equalTo("No enum constant org.elasticsearch.search.aggregations.support.CoreValuesSourceType.DOES_NOT_EXIST")); expectThrows(NullPointerException.class, () -> CoreValuesSourceType.fromString(null)); } - - @Override - public void testReadFrom() throws IOException { - assertReadFromStream(0, CoreValuesSourceType.ANY); - assertReadFromStream(1, CoreValuesSourceType.NUMERIC); - assertReadFromStream(2, CoreValuesSourceType.BYTES); - assertReadFromStream(3, CoreValuesSourceType.GEOPOINT); - assertReadFromStream(4, CoreValuesSourceType.RANGE); - assertReadFromStream(5, CoreValuesSourceType.HISTOGRAM); - } - - @Override - public void testWriteTo() throws IOException { - assertWriteToStream(CoreValuesSourceType.ANY, 0); - assertWriteToStream(CoreValuesSourceType.NUMERIC, 1); - assertWriteToStream(CoreValuesSourceType.BYTES, 2); - assertWriteToStream(CoreValuesSourceType.GEOPOINT, 3); - assertWriteToStream(CoreValuesSourceType.RANGE, 4); - assertWriteToStream(CoreValuesSourceType.HISTOGRAM, 5); - } } diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/support/ValueTypeTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/support/ValueTypeTests.java index 84995277fb7da..9bc7eb9a22988 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/support/ValueTypeTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/support/ValueTypeTests.java @@ -24,26 +24,59 @@ public class ValueTypeTests extends ESTestCase { public void testResolve() { - assertEquals(ValueType.STRING, ValueType.resolveForScript("string")); - assertEquals(ValueType.DOUBLE, ValueType.resolveForScript("float")); - assertEquals(ValueType.DOUBLE, ValueType.resolveForScript("double")); - assertEquals(ValueType.LONG, ValueType.resolveForScript("byte")); - assertEquals(ValueType.LONG, ValueType.resolveForScript("short")); - assertEquals(ValueType.LONG, ValueType.resolveForScript("integer")); - assertEquals(ValueType.LONG, ValueType.resolveForScript("long")); - assertEquals(ValueType.DATE, ValueType.resolveForScript("date")); - assertEquals(ValueType.IP, ValueType.resolveForScript("ip")); - assertEquals(ValueType.BOOLEAN, ValueType.resolveForScript("boolean")); + assertEquals(ValueType.STRING, ValueType.lenientParse("string")); + assertEquals(ValueType.DOUBLE, ValueType.lenientParse("float")); + assertEquals(ValueType.DOUBLE, ValueType.lenientParse("double")); + assertEquals(ValueType.LONG, ValueType.lenientParse("byte")); + assertEquals(ValueType.LONG, ValueType.lenientParse("short")); + assertEquals(ValueType.LONG, ValueType.lenientParse("integer")); + assertEquals(ValueType.LONG, ValueType.lenientParse("long")); + assertEquals(ValueType.DATE, ValueType.lenientParse("date")); + assertEquals(ValueType.IP, ValueType.lenientParse("ip")); + assertEquals(ValueType.BOOLEAN, ValueType.lenientParse("boolean")); } public void testCompatibility() { assertTrue(ValueType.DOUBLE.isA(ValueType.NUMERIC)); + assertTrue(ValueType.DOUBLE.isA(ValueType.NUMBER)); + assertTrue(ValueType.DOUBLE.isA(ValueType.LONG)); + assertTrue(ValueType.DOUBLE.isA(ValueType.BOOLEAN)); + assertTrue(ValueType.DOUBLE.isA(ValueType.DATE)); + assertTrue(ValueType.DOUBLE.isA(ValueType.DOUBLE)); + + assertTrue(ValueType.LONG.isA(ValueType.NUMERIC)); + assertTrue(ValueType.LONG.isA(ValueType.NUMBER)); + assertTrue(ValueType.LONG.isA(ValueType.LONG)); + assertTrue(ValueType.LONG.isA(ValueType.BOOLEAN)); + assertTrue(ValueType.LONG.isA(ValueType.DATE)); + assertTrue(ValueType.LONG.isA(ValueType.DOUBLE)); + + assertTrue(ValueType.DATE.isA(ValueType.NUMERIC)); + assertTrue(ValueType.DATE.isA(ValueType.NUMBER)); assertTrue(ValueType.DATE.isA(ValueType.LONG)); + assertTrue(ValueType.DATE.isA(ValueType.BOOLEAN)); + assertTrue(ValueType.DATE.isA(ValueType.DATE)); + assertTrue(ValueType.DATE.isA(ValueType.DOUBLE)); + + assertTrue(ValueType.NUMERIC.isA(ValueType.NUMERIC)); assertTrue(ValueType.NUMERIC.isA(ValueType.NUMBER)); + assertTrue(ValueType.NUMERIC.isA(ValueType.LONG)); + assertTrue(ValueType.NUMERIC.isA(ValueType.BOOLEAN)); + assertTrue(ValueType.NUMERIC.isA(ValueType.DATE)); + assertTrue(ValueType.NUMERIC.isA(ValueType.DOUBLE)); + + assertTrue(ValueType.BOOLEAN.isA(ValueType.NUMERIC)); assertTrue(ValueType.BOOLEAN.isA(ValueType.NUMBER)); + assertTrue(ValueType.BOOLEAN.isA(ValueType.LONG)); + assertTrue(ValueType.BOOLEAN.isA(ValueType.BOOLEAN)); + assertTrue(ValueType.BOOLEAN.isA(ValueType.DATE)); + assertTrue(ValueType.BOOLEAN.isA(ValueType.DOUBLE)); + assertFalse(ValueType.STRING.isA(ValueType.NUMBER)); assertFalse(ValueType.DATE.isA(ValueType.IP)); - } + assertTrue(ValueType.IP.isA(ValueType.STRING)); + assertTrue(ValueType.STRING.isA(ValueType.IP)); + } } diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/support/ValuesSourceConfigTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/support/ValuesSourceConfigTests.java index 2831aad4da25f..44e097a324496 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/support/ValuesSourceConfigTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/support/ValuesSourceConfigTests.java @@ -31,6 +31,7 @@ import org.elasticsearch.index.query.QueryShardContext; import org.elasticsearch.test.ESSingleNodeTestCase; +// TODO: This whole set of tests needs to be rethought. public class ValuesSourceConfigTests extends ESSingleNodeTestCase { public void testKeyword() throws Exception { @@ -44,9 +45,9 @@ public void testKeyword() throws Exception { try (Engine.Searcher searcher = indexService.getShard(0).acquireSearcher("test")) { QueryShardContext context = indexService.newQueryShardContext(0, searcher, () -> 42L, null); - ValuesSourceConfig config = ValuesSourceConfig.resolve( - context, null, "bytes", null, null, null, null); - ValuesSource.Bytes valuesSource = config.toValuesSource(context); + ValuesSourceConfig config = ValuesSourceConfig.resolve( + context, null, "bytes", null, null, null, null, CoreValuesSourceType.BYTES, null); + ValuesSource.Bytes valuesSource = (ValuesSource.Bytes) config.toValuesSource(); LeafReaderContext ctx = searcher.getIndexReader().leaves().get(0); SortedBinaryDocValues values = valuesSource.bytesValues(ctx); assertTrue(values.advanceExact(0)); @@ -66,16 +67,16 @@ public void testEmptyKeyword() throws Exception { try (Engine.Searcher searcher = indexService.getShard(0).acquireSearcher("test")) { QueryShardContext context = indexService.newQueryShardContext(0, searcher, () -> 42L, null); - ValuesSourceConfig config = ValuesSourceConfig.resolve( - context, null, "bytes", null, null, null, null); - ValuesSource.Bytes valuesSource = config.toValuesSource(context); + ValuesSourceConfig config = ValuesSourceConfig.resolve( + context, null, "bytes", null, null, null, null, CoreValuesSourceType.BYTES, null); + ValuesSource.Bytes valuesSource = (ValuesSource.Bytes) config.toValuesSource(); LeafReaderContext ctx = searcher.getIndexReader().leaves().get(0); SortedBinaryDocValues values = valuesSource.bytesValues(ctx); assertFalse(values.advanceExact(0)); config = ValuesSourceConfig.resolve( - context, null, "bytes", null, "abc", null, null); - valuesSource = config.toValuesSource(context); + context, null, "bytes", null, "abc", null, null, CoreValuesSourceType.BYTES, null); + valuesSource = (ValuesSource.Bytes) config.toValuesSource(); values = valuesSource.bytesValues(ctx); assertTrue(values.advanceExact(0)); assertEquals(1, values.docValueCount()); @@ -92,14 +93,14 @@ public void testUnmappedKeyword() throws Exception { try (Engine.Searcher searcher = indexService.getShard(0).acquireSearcher("test")) { QueryShardContext context = indexService.newQueryShardContext(0, searcher, () -> 42L, null); - ValuesSourceConfig config = ValuesSourceConfig.resolve( - context, ValueType.STRING, "bytes", null, null, null, null); - ValuesSource.Bytes valuesSource = config.toValuesSource(context); + ValuesSourceConfig config = ValuesSourceConfig.resolve( + context, ValueType.STRING, "bytes", null, null, null, null, CoreValuesSourceType.BYTES, null); + ValuesSource.Bytes valuesSource = (ValuesSource.Bytes) config.toValuesSource(); assertNull(valuesSource); config = ValuesSourceConfig.resolve( - context, ValueType.STRING, "bytes", null, "abc", null, null); - valuesSource = config.toValuesSource(context); + context, ValueType.STRING, "bytes", null, "abc", null, null, CoreValuesSourceType.BYTES, null); + valuesSource = (ValuesSource.Bytes) config.toValuesSource(); LeafReaderContext ctx = searcher.getIndexReader().leaves().get(0); SortedBinaryDocValues values = valuesSource.bytesValues(ctx); assertTrue(values.advanceExact(0)); @@ -119,9 +120,9 @@ public void testLong() throws Exception { try (Engine.Searcher searcher = indexService.getShard(0).acquireSearcher("test")) { QueryShardContext context = indexService.newQueryShardContext(0, searcher, () -> 42L, null); - ValuesSourceConfig config = ValuesSourceConfig.resolve( - context, null, "long", null, null, null, null); - ValuesSource.Numeric valuesSource = config.toValuesSource(context); + ValuesSourceConfig config = ValuesSourceConfig.resolve( + context, null, "long", null, null, null, null, CoreValuesSourceType.BYTES, null); + ValuesSource.Numeric valuesSource = (ValuesSource.Numeric) config.toValuesSource(); LeafReaderContext ctx = searcher.getIndexReader().leaves().get(0); SortedNumericDocValues values = valuesSource.longValues(ctx); assertTrue(values.advanceExact(0)); @@ -141,16 +142,16 @@ public void testEmptyLong() throws Exception { try (Engine.Searcher searcher = indexService.getShard(0).acquireSearcher("test")) { QueryShardContext context = indexService.newQueryShardContext(0, searcher, () -> 42L, null); - ValuesSourceConfig config = ValuesSourceConfig.resolve( - context, null, "long", null, null, null, null); - ValuesSource.Numeric valuesSource = config.toValuesSource(context); + ValuesSourceConfig config = ValuesSourceConfig.resolve( + context, null, "long", null, null, null, null, CoreValuesSourceType.BYTES, null); + ValuesSource.Numeric valuesSource = (ValuesSource.Numeric) config.toValuesSource(); LeafReaderContext ctx = searcher.getIndexReader().leaves().get(0); SortedNumericDocValues values = valuesSource.longValues(ctx); assertFalse(values.advanceExact(0)); config = ValuesSourceConfig.resolve( - context, null, "long", null, 42, null, null); - valuesSource = config.toValuesSource(context); + context, null, "long", null, 42, null, null, CoreValuesSourceType.BYTES, null); + valuesSource = (ValuesSource.Numeric) config.toValuesSource(); values = valuesSource.longValues(ctx); assertTrue(values.advanceExact(0)); assertEquals(1, values.docValueCount()); @@ -168,14 +169,14 @@ public void testUnmappedLong() throws Exception { try (Engine.Searcher searcher = indexService.getShard(0).acquireSearcher("test")) { QueryShardContext context = indexService.newQueryShardContext(0, searcher, () -> 42L, null); - ValuesSourceConfig config = ValuesSourceConfig.resolve( - context, ValueType.NUMBER, "long", null, null, null, null); - ValuesSource.Numeric valuesSource = config.toValuesSource(context); + ValuesSourceConfig config = ValuesSourceConfig.resolve( + context, ValueType.NUMBER, "long", null, null, null, null, CoreValuesSourceType.BYTES, null); + ValuesSource.Numeric valuesSource = (ValuesSource.Numeric) config.toValuesSource(); assertNull(valuesSource); config = ValuesSourceConfig.resolve( - context, ValueType.NUMBER, "long", null, 42, null, null); - valuesSource = config.toValuesSource(context); + context, ValueType.NUMBER, "long", null, 42, null, null, CoreValuesSourceType.BYTES, null); + valuesSource = (ValuesSource.Numeric) config.toValuesSource(); LeafReaderContext ctx = searcher.getIndexReader().leaves().get(0); SortedNumericDocValues values = valuesSource.longValues(ctx); assertTrue(values.advanceExact(0)); @@ -195,9 +196,9 @@ public void testBoolean() throws Exception { try (Engine.Searcher searcher = indexService.getShard(0).acquireSearcher("test")) { QueryShardContext context = indexService.newQueryShardContext(0, searcher, () -> 42L, null); - ValuesSourceConfig config = ValuesSourceConfig.resolve( - context, null, "bool", null, null, null, null); - ValuesSource.Numeric valuesSource = config.toValuesSource(context); + ValuesSourceConfig config = ValuesSourceConfig.resolve( + context, null, "bool", null, null, null, null, CoreValuesSourceType.BYTES, null); + ValuesSource.Numeric valuesSource = (ValuesSource.Numeric) config.toValuesSource(); LeafReaderContext ctx = searcher.getIndexReader().leaves().get(0); SortedNumericDocValues values = valuesSource.longValues(ctx); assertTrue(values.advanceExact(0)); @@ -217,16 +218,16 @@ public void testEmptyBoolean() throws Exception { try (Engine.Searcher searcher = indexService.getShard(0).acquireSearcher("test")) { QueryShardContext context = indexService.newQueryShardContext(0, searcher, () -> 42L, null); - ValuesSourceConfig config = ValuesSourceConfig.resolve( - context, null, "bool", null, null, null, null); - ValuesSource.Numeric valuesSource = config.toValuesSource(context); + ValuesSourceConfig config = ValuesSourceConfig.resolve( + context, null, "bool", null, null, null, null, CoreValuesSourceType.BYTES, null); + ValuesSource.Numeric valuesSource = (ValuesSource.Numeric) config.toValuesSource(); LeafReaderContext ctx = searcher.getIndexReader().leaves().get(0); SortedNumericDocValues values = valuesSource.longValues(ctx); assertFalse(values.advanceExact(0)); config = ValuesSourceConfig.resolve( - context, null, "bool", null, true, null, null); - valuesSource = config.toValuesSource(context); + context, null, "bool", null, true, null, null, CoreValuesSourceType.BYTES, null); + valuesSource = (ValuesSource.Numeric) config.toValuesSource(); values = valuesSource.longValues(ctx); assertTrue(values.advanceExact(0)); assertEquals(1, values.docValueCount()); @@ -244,14 +245,14 @@ public void testUnmappedBoolean() throws Exception { try (Engine.Searcher searcher = indexService.getShard(0).acquireSearcher("test")) { QueryShardContext context = indexService.newQueryShardContext(0, searcher, () -> 42L, null); - ValuesSourceConfig config = ValuesSourceConfig.resolve( - context, ValueType.BOOLEAN, "bool", null, null, null, null); - ValuesSource.Numeric valuesSource = config.toValuesSource(context); + ValuesSourceConfig config = ValuesSourceConfig.resolve( + context, ValueType.BOOLEAN, "bool", null, null, null, null, CoreValuesSourceType.BYTES, null); + ValuesSource.Numeric valuesSource = (ValuesSource.Numeric) config.toValuesSource(); assertNull(valuesSource); config = ValuesSourceConfig.resolve( - context, ValueType.BOOLEAN, "bool", null, true, null, null); - valuesSource = config.toValuesSource(context); + context, ValueType.BOOLEAN, "bool", null, true, null, null, CoreValuesSourceType.BYTES, null); + valuesSource = (ValuesSource.Numeric) config.toValuesSource(); LeafReaderContext ctx = searcher.getIndexReader().leaves().get(0); SortedNumericDocValues values = valuesSource.longValues(ctx); assertTrue(values.advanceExact(0)); @@ -265,8 +266,8 @@ public void testTypeFieldDeprecation() { try (Engine.Searcher searcher = indexService.getShard(0).acquireSearcher("test")) { QueryShardContext context = indexService.newQueryShardContext(0, searcher, () -> 42L, null); - ValuesSourceConfig config = ValuesSourceConfig.resolve( - context, null, TypeFieldMapper.NAME, null, null, null, null); + ValuesSourceConfig config = ValuesSourceConfig.resolve( + context, null, TypeFieldMapper.NAME, null, null, null, null, CoreValuesSourceType.BYTES, null); assertWarnings(QueryShardContext.TYPES_DEPRECATION_MESSAGE); } } @@ -281,9 +282,9 @@ public void testFieldAlias() throws Exception { try (Engine.Searcher searcher = indexService.getShard(0).acquireSearcher("test")) { QueryShardContext context = indexService.newQueryShardContext(0, searcher, () -> 42L, null); - ValuesSourceConfig config = ValuesSourceConfig.resolve( - context, ValueType.STRING, "alias", null, null, null, null); - ValuesSource.Bytes valuesSource = config.toValuesSource(context); + ValuesSourceConfig config = ValuesSourceConfig.resolve( + context, ValueType.STRING, "alias", null, null, null, null, CoreValuesSourceType.BYTES, null); + ValuesSource.Bytes valuesSource = (ValuesSource.Bytes) config.toValuesSource(); LeafReaderContext ctx = searcher.getIndexReader().leaves().get(0); SortedBinaryDocValues values = valuesSource.bytesValues(ctx); diff --git a/server/src/test/java/org/elasticsearch/search/fetch/subphase/highlight/HighlightBuilderTests.java b/server/src/test/java/org/elasticsearch/search/fetch/subphase/highlight/HighlightBuilderTests.java index a5f67ab9347e3..9a7ff85a14884 100644 --- a/server/src/test/java/org/elasticsearch/search/fetch/subphase/highlight/HighlightBuilderTests.java +++ b/server/src/test/java/org/elasticsearch/search/fetch/subphase/highlight/HighlightBuilderTests.java @@ -280,7 +280,7 @@ public void testBuildSearchContextHighlight() throws IOException { // shard context will only need indicesQueriesRegistry for building Query objects nested in highlighter QueryShardContext mockShardContext = new QueryShardContext(0, idxSettings, BigArrays.NON_RECYCLING_INSTANCE, null, null, null, null, null, xContentRegistry(), namedWriteableRegistry, - null, null, System::currentTimeMillis, null, null, () -> true) { + null, null, System::currentTimeMillis, null, null, () -> true, null) { @Override public MappedFieldType fieldMapper(String name) { TextFieldMapper.Builder builder = new TextFieldMapper.Builder(name); diff --git a/server/src/test/java/org/elasticsearch/search/rescore/QueryRescorerBuilderTests.java b/server/src/test/java/org/elasticsearch/search/rescore/QueryRescorerBuilderTests.java index 2be9c0d57a639..e0ab8d0945e9e 100644 --- a/server/src/test/java/org/elasticsearch/search/rescore/QueryRescorerBuilderTests.java +++ b/server/src/test/java/org/elasticsearch/search/rescore/QueryRescorerBuilderTests.java @@ -144,7 +144,7 @@ public void testBuildRescoreSearchContext() throws ElasticsearchParseException, // shard context will only need indicesQueriesRegistry for building Query objects nested in query rescorer QueryShardContext mockShardContext = new QueryShardContext(0, idxSettings, BigArrays.NON_RECYCLING_INSTANCE, null, null, null, null, null, - xContentRegistry(), namedWriteableRegistry, null, null, () -> nowInMillis, null, null, () -> true) { + xContentRegistry(), namedWriteableRegistry, null, null, () -> nowInMillis, null, null, () -> true, null) { @Override public MappedFieldType fieldMapper(String name) { TextFieldMapper.Builder builder = new TextFieldMapper.Builder(name); @@ -188,7 +188,7 @@ public void testRewritingKeepsSettings() throws IOException { // shard context will only need indicesQueriesRegistry for building Query objects nested in query rescorer QueryShardContext mockShardContext = new QueryShardContext(0, idxSettings, BigArrays.NON_RECYCLING_INSTANCE, null, null, null, null, null, - xContentRegistry(), namedWriteableRegistry, null, null, () -> nowInMillis, null, null, () -> true) { + xContentRegistry(), namedWriteableRegistry, null, null, () -> nowInMillis, null, null, () -> true, null) { @Override public MappedFieldType fieldMapper(String name) { TextFieldMapper.Builder builder = new TextFieldMapper.Builder(name); diff --git a/server/src/test/java/org/elasticsearch/search/sort/AbstractSortTestCase.java b/server/src/test/java/org/elasticsearch/search/sort/AbstractSortTestCase.java index 7a830ab6f2fa2..a5d3628327a76 100644 --- a/server/src/test/java/org/elasticsearch/search/sort/AbstractSortTestCase.java +++ b/server/src/test/java/org/elasticsearch/search/sort/AbstractSortTestCase.java @@ -200,7 +200,7 @@ protected final QueryShardContext createMockShardContext(IndexSearcher searcher) }; return new QueryShardContext(0, idxSettings, BigArrays.NON_RECYCLING_INSTANCE, bitsetFilterCache, indexFieldDataLookup, null, null, scriptService, xContentRegistry(), namedWriteableRegistry, null, searcher, - () -> randomNonNegativeLong(), null, null, () -> true) { + () -> randomNonNegativeLong(), null, null, () -> true, null) { @Override public MappedFieldType fieldMapper(String name) { diff --git a/server/src/test/java/org/elasticsearch/search/suggest/AbstractSuggestionBuilderTestCase.java b/server/src/test/java/org/elasticsearch/search/suggest/AbstractSuggestionBuilderTestCase.java index 1b3dc5ccbb9c8..e6d94f9009b85 100644 --- a/server/src/test/java/org/elasticsearch/search/suggest/AbstractSuggestionBuilderTestCase.java +++ b/server/src/test/java/org/elasticsearch/search/suggest/AbstractSuggestionBuilderTestCase.java @@ -181,7 +181,7 @@ public void testBuild() throws IOException { ((Script) invocation.getArguments()[0]).getIdOrCode())); QueryShardContext mockShardContext = new QueryShardContext(0, idxSettings, BigArrays.NON_RECYCLING_INSTANCE, null, null, mapperService, null, scriptService, xContentRegistry(), namedWriteableRegistry, null, null, - System::currentTimeMillis, null, null, () -> true); + System::currentTimeMillis, null, null, () -> true, null); SuggestionContext suggestionContext = suggestionBuilder.build(mockShardContext); assertEquals(toBytesRef(suggestionBuilder.text()), suggestionContext.getText()); diff --git a/server/src/test/java/org/elasticsearch/snapshots/SnapshotResiliencyTests.java b/server/src/test/java/org/elasticsearch/snapshots/SnapshotResiliencyTests.java index cc89b7a6f88b4..ab75cc09d177d 100644 --- a/server/src/test/java/org/elasticsearch/snapshots/SnapshotResiliencyTests.java +++ b/server/src/test/java/org/elasticsearch/snapshots/SnapshotResiliencyTests.java @@ -1276,7 +1276,8 @@ public void onFailure(final Exception e) { client, new MetaStateService(nodeEnv, namedXContentRegistry), Collections.emptyList(), - emptyMap() + emptyMap(), + null ); final RecoverySettings recoverySettings = new RecoverySettings(settings, clusterSettings); final ActionFilters actionFilters = new ActionFilters(emptySet()); diff --git a/test/framework/src/main/java/org/elasticsearch/search/aggregations/AggregatorTestCase.java b/test/framework/src/main/java/org/elasticsearch/search/aggregations/AggregatorTestCase.java index 8a5dc765cb99a..4fb332fbdfbf3 100644 --- a/test/framework/src/main/java/org/elasticsearch/search/aggregations/AggregatorTestCase.java +++ b/test/framework/src/main/java/org/elasticsearch/search/aggregations/AggregatorTestCase.java @@ -61,15 +61,12 @@ import org.elasticsearch.index.fielddata.IndexFieldDataCache; import org.elasticsearch.index.fielddata.IndexFieldDataService; import org.elasticsearch.index.mapper.BinaryFieldMapper; -import org.elasticsearch.index.mapper.BooleanFieldMapper; import org.elasticsearch.index.mapper.CompletionFieldMapper; import org.elasticsearch.index.mapper.ContentPath; -import org.elasticsearch.index.mapper.DateFieldMapper; import org.elasticsearch.index.mapper.DocumentMapper; import org.elasticsearch.index.mapper.FieldAliasMapper; import org.elasticsearch.index.mapper.FieldMapper; import org.elasticsearch.index.mapper.GeoShapeFieldMapper; -import org.elasticsearch.index.mapper.IpFieldMapper; import org.elasticsearch.index.mapper.MappedFieldType; import org.elasticsearch.index.mapper.Mapper; import org.elasticsearch.index.mapper.Mapper.BuilderContext; @@ -89,10 +86,12 @@ import org.elasticsearch.indices.mapper.MapperRegistry; import org.elasticsearch.mock.orig.Mockito; import org.elasticsearch.script.ScriptService; +import org.elasticsearch.search.SearchModule; import org.elasticsearch.search.aggregations.MultiBucketConsumerService.MultiBucketConsumer; import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator.PipelineTree; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.search.aggregations.support.ValuesSourceType; import org.elasticsearch.search.fetch.FetchPhase; import org.elasticsearch.search.fetch.subphase.FetchDocValuesPhase; @@ -103,6 +102,7 @@ import org.elasticsearch.test.ESTestCase; import org.elasticsearch.test.InternalAggregationTestCase; import org.junit.After; +import org.junit.BeforeClass; import java.io.IOException; import java.util.ArrayList; @@ -133,6 +133,7 @@ public abstract class AggregatorTestCase extends ESTestCase { private static final String NESTEDFIELD_PREFIX = "nested_"; private List releasables = new ArrayList<>(); private static final String TYPE_NAME = "type"; + protected static ValuesSourceRegistry valuesSourceRegistry; // A list of field types that should not be tested, or are not currently supported private static List TYPE_TEST_BLACKLIST; @@ -168,6 +169,12 @@ private static void registerFieldTypes(SearchContext searchContext, MapperServic } } + @BeforeClass + public static void initValuesSourceRegistry() { + SearchModule searchModule = new SearchModule(Settings.EMPTY, List.of()); + valuesSourceRegistry = searchModule.getValuesSourceRegistry(); + } + protected A createAggregator(AggregationBuilder aggregationBuilder, IndexSearcher indexSearcher, MappedFieldType... fieldTypes) throws IOException { @@ -340,7 +347,8 @@ protected QueryShardContext queryShardContextMock(IndexSearcher searcher, return new QueryShardContext(0, indexSettings, bigArrays, null, getIndexFieldDataLookup(mapperService, circuitBreakerService), mapperService, null, getMockScriptService(), xContentRegistry(), - writableRegistry(), null, searcher, System::currentTimeMillis, null, null, () -> true); + writableRegistry(), null, searcher, System::currentTimeMillis, null, null, () -> true, + valuesSourceRegistry); } /** @@ -613,9 +621,6 @@ public void testSupportedFieldTypes() throws IOException { if (supportedVSTypes.isEmpty()) { // If the test says it doesn't support any VStypes, it has not been converted yet so skip return; - } else if (supportedVSTypes.contains(CoreValuesSourceType.ANY)) { - throw new IllegalArgumentException("Tests should not specify CoreValuesSourceType.ANY as a supported ValuesSourceType, " + - "but should instead list the concrete ValuesSourceTypes that are supported"); } for (Map.Entry mappedType : mapperRegistry.getMapperParsers().entrySet()) { @@ -674,27 +679,55 @@ public void testSupportedFieldTypes() throws IOException { */ private void writeTestDoc(MappedFieldType fieldType, String fieldName, RandomIndexWriter iw) throws IOException { - if (fieldType.getValuesSourceType().equals(CoreValuesSourceType.NUMERIC)) { - // TODO note: once VS refactor adds DATE/BOOLEAN, this conditional will go away - if (fieldType.typeName().equals(DateFieldMapper.CONTENT_TYPE) - || fieldType.typeName().equals(DateFieldMapper.DATE_NANOS_CONTENT_TYPE)) { - iw.addDocument(singleton(new SortedNumericDocValuesField(fieldName, randomNonNegativeLong()))); - } else if (fieldType.typeName().equals(BooleanFieldMapper.CONTENT_TYPE)) { - iw.addDocument(singleton(new SortedNumericDocValuesField(fieldName, randomBoolean() ? 0 : 1))); + String typeName = fieldType.typeName(); + ValuesSourceType vst = fieldType.getValuesSourceType(); + Document doc = new Document(); + String json; + + + if (vst.equals(CoreValuesSourceType.NUMERIC)) { + long v; + if (typeName.equals(NumberFieldMapper.NumberType.DOUBLE.typeName())) { + double d = Math.abs(randomDouble()); + v = NumericUtils.doubleToSortableLong(d); + json = "{ \"" + fieldName + "\" : \"" + d + "\" }"; + } else if (typeName.equals(NumberFieldMapper.NumberType.FLOAT.typeName())) { + float f = Math.abs(randomFloat()); + v = NumericUtils.floatToSortableInt(f); + json = "{ \"" + fieldName + "\" : \"" + f + "\" }"; + } else if (typeName.equals(NumberFieldMapper.NumberType.HALF_FLOAT.typeName())) { + float f = Math.abs(randomFloat()); + v = HalfFloatPoint.halfFloatToSortableShort(f); + json = "{ \"" + fieldName + "\" : \"" + f + "\" }"; } else { - iw.addDocument(singleton(new SortedNumericDocValuesField(fieldName, randomLong()))); + v = randomLong(); + json = "{ \"" + fieldName + "\" : \"" + v + "\" }"; } - } else if (fieldType.getValuesSourceType().equals(CoreValuesSourceType.BYTES)) { - if (fieldType.typeName().equals(BinaryFieldMapper.CONTENT_TYPE)) { - iw.addDocument(singleton(new BinaryFieldMapper.CustomBinaryDocValuesField(fieldName, new BytesRef("a").bytes))); - } else if (fieldType.typeName().equals(IpFieldMapper.CONTENT_TYPE)) { - // TODO note: once VS refactor adds IP, this conditional will go away - boolean v4 = randomBoolean(); - iw.addDocument(singleton(new SortedSetDocValuesField(fieldName, new BytesRef(InetAddressPoint.encode(randomIp(v4)))))); + doc.add(new SortedNumericDocValuesField(fieldName, v)); + + } else if (vst.equals(CoreValuesSourceType.BYTES)) { + if (typeName.equals(BinaryFieldMapper.CONTENT_TYPE)) { + doc.add(new BinaryFieldMapper.CustomBinaryDocValuesField(fieldName, new BytesRef("a").bytes)); + json = "{ \"" + fieldName + "\" : \"a\" }"; } else { iw.addDocument(singleton(new SortedSetDocValuesField(fieldName, new BytesRef("a")))); } - } else if (fieldType.getValuesSourceType().equals(CoreValuesSourceType.RANGE)) { + } else if (vst.equals(CoreValuesSourceType.DATE)) { + // positive integer because date_nanos gets unhappy with large longs + long v; + v = Math.abs(randomInt()); + doc.add(new SortedNumericDocValuesField(fieldName, v)); + json = "{ \"" + fieldName + "\" : \"" + v + "\" }"; + } else if (vst.equals(CoreValuesSourceType.BOOLEAN)) { + long v; + v = randomBoolean() ? 0 : 1; + doc.add(new SortedNumericDocValuesField(fieldName, v)); + json = "{ \"" + fieldName + "\" : \"" + (v == 0 ? "false" : "true") + "\" }"; + } else if (vst.equals(CoreValuesSourceType.IP)) { + InetAddress ip = randomIp(randomBoolean()); + json = "{ \"" + fieldName + "\" : \"" + NetworkAddress.format(ip) + "\" }"; + doc.add(new SortedSetDocValuesField(fieldName, new BytesRef(InetAddressPoint.encode(ip)))); + } else if (vst.equals(CoreValuesSourceType.RANGE)) { Object start; Object end; RangeType rangeType; @@ -729,13 +762,22 @@ private void writeTestDoc(MappedFieldType fieldType, String fieldName, RandomInd } final RangeFieldMapper.Range range = new RangeFieldMapper.Range(rangeType, start, end, true, true); - iw.addDocument(singleton(new BinaryDocValuesField(fieldName, rangeType.encodeRanges(Collections.singleton(range))))); + doc.add(new BinaryDocValuesField(fieldName, rangeType.encodeRanges(Collections.singleton(range)))); + json = "{ \"" + fieldName + "\" : { \n" + + " \"gte\" : \"" + start + "\",\n" + + " \"lte\" : \"" + end + "\"\n" + + " }}"; } else if (fieldType.getValuesSourceType().equals(CoreValuesSourceType.GEOPOINT)) { - iw.addDocument(singleton(new LatLonDocValuesField(fieldName, randomDouble(), randomDouble()))); + double lat = randomDouble(); + double lon = randomDouble(); + doc.add(new LatLonDocValuesField(fieldName, lat, lon)); + json = "{ \"" + fieldName + "\" : \"[" + lon + "," + lat + "]\" }"; } else { throw new IllegalStateException("Unknown field type [" + fieldType.typeName() + "]"); } + doc.add(new StoredField("_source", new BytesRef(json))); + iw.addDocument(doc); } private class MockParserContext extends Mapper.TypeParser.ParserContext { diff --git a/test/framework/src/main/java/org/elasticsearch/search/aggregations/BaseAggregationTestCase.java b/test/framework/src/main/java/org/elasticsearch/search/aggregations/BaseAggregationTestCase.java index f9b938c1c358f..82b08a3994a18 100644 --- a/test/framework/src/main/java/org/elasticsearch/search/aggregations/BaseAggregationTestCase.java +++ b/test/framework/src/main/java/org/elasticsearch/search/aggregations/BaseAggregationTestCase.java @@ -199,7 +199,7 @@ public String randomNumericField() { } } - protected void randomFieldOrScript(ValuesSourceAggregationBuilder factory, String field) { + protected void randomFieldOrScript(ValuesSourceAggregationBuilder factory, String field) { int choice = randomInt(2); switch (choice) { case 0: diff --git a/test/framework/src/main/java/org/elasticsearch/test/AbstractBuilderTestCase.java b/test/framework/src/main/java/org/elasticsearch/test/AbstractBuilderTestCase.java index 32d932cd5608a..3c7fd7a84b5ba 100644 --- a/test/framework/src/main/java/org/elasticsearch/test/AbstractBuilderTestCase.java +++ b/test/framework/src/main/java/org/elasticsearch/test/AbstractBuilderTestCase.java @@ -416,7 +416,7 @@ public void close() throws IOException { QueryShardContext createShardContext(IndexSearcher searcher) { return new QueryShardContext(0, idxSettings, BigArrays.NON_RECYCLING_INSTANCE, bitsetFilterCache, indexFieldDataService::getForField, mapperService, similarityService, scriptService, xContentRegistry, - namedWriteableRegistry, this.client, searcher, () -> nowInMillis, null, indexNameMatcher(), () -> true); + namedWriteableRegistry, this.client, searcher, () -> nowInMillis, null, indexNameMatcher(), () -> true, null); } ScriptModule createScriptModule(List scriptPlugins) { diff --git a/test/framework/src/test/java/org/elasticsearch/search/MockSearchServiceTests.java b/test/framework/src/test/java/org/elasticsearch/search/MockSearchServiceTests.java index d0f41a3e0078f..94bd637781c83 100644 --- a/test/framework/src/test/java/org/elasticsearch/search/MockSearchServiceTests.java +++ b/test/framework/src/test/java/org/elasticsearch/search/MockSearchServiceTests.java @@ -43,7 +43,7 @@ public void testAssertNoInFlightContext() { final long nowInMillis = randomNonNegativeLong(); SearchContext s = new TestSearchContext(new QueryShardContext(0, new IndexSettings(EMPTY_INDEX_METADATA, Settings.EMPTY), BigArrays.NON_RECYCLING_INSTANCE, null, null, null, null, null, - xContentRegistry(), writableRegistry(), null, null, () -> nowInMillis, null, null, () -> true)) { + xContentRegistry(), writableRegistry(), null, null, () -> nowInMillis, null, null, () -> true, null)) { @Override public SearchShardTarget shardTarget() { diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/AnalyticsPlugin.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/AnalyticsPlugin.java index 476b4056336ed..cceef8a052e4c 100644 --- a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/AnalyticsPlugin.java +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/AnalyticsPlugin.java @@ -26,9 +26,11 @@ import org.elasticsearch.plugins.Plugin; import org.elasticsearch.plugins.SearchPlugin; import org.elasticsearch.script.ScriptService; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.threadpool.ThreadPool; import org.elasticsearch.watcher.ResourceWatcherService; import org.elasticsearch.xpack.analytics.action.TransportAnalyticsStatsAction; +import org.elasticsearch.xpack.analytics.aggregations.metrics.AnalyticsPercentilesAggregatorFactory; import org.elasticsearch.xpack.analytics.boxplot.BoxplotAggregationBuilder; import org.elasticsearch.xpack.analytics.boxplot.InternalBoxplot; import org.elasticsearch.xpack.analytics.cumulativecardinality.CumulativeCardinalityPipelineAggregationBuilder; @@ -54,6 +56,7 @@ import java.util.Collections; import java.util.List; import java.util.Map; +import java.util.function.Consumer; import static java.util.Collections.singletonList; @@ -86,12 +89,14 @@ public List getAggregations() { StringStatsAggregationBuilder.NAME, StringStatsAggregationBuilder::new, usage.track(AnalyticsUsage.Item.STRING_STATS, checkLicense(StringStatsAggregationBuilder.PARSER))) - .addResultReader(InternalStringStats::new), + .addResultReader(InternalStringStats::new) + .setAggregatorRegistrar(StringStatsAggregationBuilder::registerAggregators), new AggregationSpec( BoxplotAggregationBuilder.NAME, BoxplotAggregationBuilder::new, usage.track(AnalyticsUsage.Item.BOXPLOT, checkLicense(BoxplotAggregationBuilder.PARSER))) - .addResultReader(InternalBoxplot::new), + .addResultReader(InternalBoxplot::new) + .setAggregatorRegistrar(BoxplotAggregationBuilder::registerAggregators), new AggregationSpec( TopMetricsAggregationBuilder.NAME, TopMetricsAggregationBuilder::new, @@ -133,6 +138,12 @@ public Map getMappers() { return Collections.singletonMap(HistogramFieldMapper.CONTENT_TYPE, new HistogramFieldMapper.TypeParser()); } + @Override + public List> getBareAggregatorRegistrar() { + return List.of(AnalyticsPercentilesAggregatorFactory::registerPercentilesAggregator, + AnalyticsPercentilesAggregatorFactory::registerPercentileRanksAggregator); + } + @Override public Collection createComponents(Client client, ClusterService clusterService, ThreadPool threadPool, ResourceWatcherService resourceWatcherService, ScriptService scriptService, NamedXContentRegistry xContentRegistry, diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/AbstractHistoBackedHDRPercentilesAggregator.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/AbstractHistoBackedHDRPercentilesAggregator.java new file mode 100644 index 0000000000000..86918db07656c --- /dev/null +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/AbstractHistoBackedHDRPercentilesAggregator.java @@ -0,0 +1,121 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +package org.elasticsearch.xpack.analytics.aggregations.metrics; + +import org.HdrHistogram.DoubleHistogram; +import org.apache.lucene.index.LeafReaderContext; +import org.apache.lucene.search.ScoreMode; +import org.elasticsearch.common.lease.Releasables; +import org.elasticsearch.common.util.ArrayUtils; +import org.elasticsearch.common.util.BigArrays; +import org.elasticsearch.common.util.ObjectArray; +import org.elasticsearch.index.fielddata.HistogramValue; +import org.elasticsearch.index.fielddata.HistogramValues; +import org.elasticsearch.search.DocValueFormat; +import org.elasticsearch.search.aggregations.Aggregator; +import org.elasticsearch.search.aggregations.LeafBucketCollector; +import org.elasticsearch.search.aggregations.LeafBucketCollectorBase; +import org.elasticsearch.search.aggregations.metrics.NumericMetricsAggregator; +import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.ValuesSource; +import org.elasticsearch.search.internal.SearchContext; +import org.elasticsearch.xpack.analytics.aggregations.support.HistogramValuesSource; + +import java.io.IOException; +import java.util.List; +import java.util.Map; + +abstract class AbstractHistoBackedHDRPercentilesAggregator extends NumericMetricsAggregator.MultiValue { + + private static int indexOfKey(double[] keys, double key) { + return ArrayUtils.binarySearch(keys, key, 0.001); + } + + protected final double[] keys; + protected final ValuesSource valuesSource; + protected final DocValueFormat format; + protected ObjectArray states; + protected final int numberOfSignificantValueDigits; + protected final boolean keyed; + + AbstractHistoBackedHDRPercentilesAggregator(String name, ValuesSource valuesSource, SearchContext context, Aggregator parent, + double[] keys, int numberOfSignificantValueDigits, boolean keyed, DocValueFormat formatter, + List pipelineAggregators, Map metaData) throws IOException { + super(name, context, parent, pipelineAggregators, metaData); + this.valuesSource = valuesSource; + this.keyed = keyed; + this.format = formatter; + this.states = context.bigArrays().newObjectArray(1); + this.keys = keys; + this.numberOfSignificantValueDigits = numberOfSignificantValueDigits; + } + + @Override + public ScoreMode scoreMode() { + return valuesSource != null && valuesSource.needsScores() ? ScoreMode.COMPLETE : ScoreMode.COMPLETE_NO_SCORES; + } + + @Override + public LeafBucketCollector getLeafCollector(LeafReaderContext ctx, + final LeafBucketCollector sub) throws IOException { + if (valuesSource == null) { + return LeafBucketCollector.NO_OP_COLLECTOR; + } + final BigArrays bigArrays = context.bigArrays(); + final HistogramValues values = ((HistogramValuesSource.Histogram)valuesSource).getHistogramValues(ctx); + + return new LeafBucketCollectorBase(sub, values) { + @Override + public void collect(int doc, long bucket) throws IOException { + DoubleHistogram state = getExistingOrNewHistogram(bigArrays, bucket); + if (values.advanceExact(doc)) { + final HistogramValue sketch = values.histogram(); + while (sketch.next()) { + state.recordValueWithCount(sketch.value(), sketch.count()); + } + } + } + }; + } + + private DoubleHistogram getExistingOrNewHistogram(final BigArrays bigArrays, long bucket) { + states = bigArrays.grow(states, bucket + 1); + DoubleHistogram state = states.get(bucket); + if (state == null) { + state = new DoubleHistogram(numberOfSignificantValueDigits); + /* Set the histogram to autosize so it can resize itself as + the data range increases. Resize operations should be + rare as the histogram buckets are exponential (on the top + level). In the future we could expose the range as an + option on the request so the histogram can be fixed at + initialisation and doesn't need resizing. + */ + state.setAutoResize(true); + states.set(bucket, state); + } + return state; + } + + @Override + public boolean hasMetric(String name) { + return indexOfKey(keys, Double.parseDouble(name)) >= 0; + } + + protected DoubleHistogram getState(long bucketOrd) { + if (bucketOrd >= states.size()) { + return null; + } + final DoubleHistogram state = states.get(bucketOrd); + return state; + } + + @Override + protected void doClose() { + Releasables.close(states); + } + +} diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/AbstractHistoBackedTDigestPercentilesAggregator.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/AbstractHistoBackedTDigestPercentilesAggregator.java new file mode 100644 index 0000000000000..f4ae77c95b536 --- /dev/null +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/AbstractHistoBackedTDigestPercentilesAggregator.java @@ -0,0 +1,113 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +package org.elasticsearch.xpack.analytics.aggregations.metrics; + +import org.apache.lucene.index.LeafReaderContext; +import org.apache.lucene.search.ScoreMode; +import org.elasticsearch.common.lease.Releasables; +import org.elasticsearch.common.util.ArrayUtils; +import org.elasticsearch.common.util.BigArrays; +import org.elasticsearch.common.util.ObjectArray; +import org.elasticsearch.index.fielddata.HistogramValue; +import org.elasticsearch.index.fielddata.HistogramValues; +import org.elasticsearch.search.DocValueFormat; +import org.elasticsearch.search.aggregations.Aggregator; +import org.elasticsearch.search.aggregations.LeafBucketCollector; +import org.elasticsearch.search.aggregations.LeafBucketCollectorBase; +import org.elasticsearch.search.aggregations.metrics.NumericMetricsAggregator; +import org.elasticsearch.search.aggregations.metrics.TDigestState; +import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.ValuesSource; +import org.elasticsearch.search.internal.SearchContext; +import org.elasticsearch.xpack.analytics.aggregations.support.HistogramValuesSource; + +import java.io.IOException; +import java.util.List; +import java.util.Map; + +abstract class AbstractHistoBackedTDigestPercentilesAggregator extends NumericMetricsAggregator.MultiValue { + + private static int indexOfKey(double[] keys, double key) { + return ArrayUtils.binarySearch(keys, key, 0.001); + } + + protected final double[] keys; + protected final ValuesSource valuesSource; + protected final DocValueFormat formatter; + protected ObjectArray states; + protected final double compression; + protected final boolean keyed; + + AbstractHistoBackedTDigestPercentilesAggregator(String name, ValuesSource valuesSource, SearchContext context, Aggregator parent, + double[] keys, double compression, boolean keyed, DocValueFormat formatter, + List pipelineAggregators, Map metaData) throws IOException { + super(name, context, parent, pipelineAggregators, metaData); + this.valuesSource = valuesSource; + this.keyed = keyed; + this.formatter = formatter; + this.states = context.bigArrays().newObjectArray(1); + this.keys = keys; + this.compression = compression; + } + + @Override + public ScoreMode scoreMode() { + return valuesSource != null && valuesSource.needsScores() ? ScoreMode.COMPLETE : ScoreMode.COMPLETE_NO_SCORES; + } + + @Override + public LeafBucketCollector getLeafCollector(LeafReaderContext ctx, + final LeafBucketCollector sub) throws IOException { + if (valuesSource == null) { + return LeafBucketCollector.NO_OP_COLLECTOR; + } + final BigArrays bigArrays = context.bigArrays(); + final HistogramValues values = ((HistogramValuesSource.Histogram)valuesSource).getHistogramValues(ctx); + + return new LeafBucketCollectorBase(sub, values) { + @Override + public void collect(int doc, long bucket) throws IOException { + TDigestState state = getExistingOrNewHistogram(bigArrays, bucket); + if (values.advanceExact(doc)) { + final HistogramValue sketch = values.histogram(); + while(sketch.next()) { + state.add(sketch.value(), sketch.count()); + } + } + } + }; + } + + private TDigestState getExistingOrNewHistogram(final BigArrays bigArrays, long bucket) { + states = bigArrays.grow(states, bucket + 1); + TDigestState state = states.get(bucket); + if (state == null) { + state = new TDigestState(compression); + states.set(bucket, state); + } + return state; + } + + @Override + public boolean hasMetric(String name) { + return indexOfKey(keys, Double.parseDouble(name)) >= 0; + } + + protected TDigestState getState(long bucketOrd) { + if (bucketOrd >= states.size()) { + return null; + } + final TDigestState state = states.get(bucketOrd); + return state; + } + + @Override + protected void doClose() { + Releasables.close(states); + } + +} diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/AnalyticsPercentilesAggregatorFactory.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/AnalyticsPercentilesAggregatorFactory.java new file mode 100644 index 0000000000000..2a50482d5f59f --- /dev/null +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/AnalyticsPercentilesAggregatorFactory.java @@ -0,0 +1,61 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +package org.elasticsearch.xpack.analytics.aggregations.metrics; + +import org.elasticsearch.search.aggregations.metrics.PercentileRanksAggregationBuilder; +import org.elasticsearch.search.aggregations.metrics.PercentilesAggregationBuilder; +import org.elasticsearch.search.aggregations.metrics.PercentilesAggregatorSupplier; +import org.elasticsearch.search.aggregations.metrics.PercentilesConfig; +import org.elasticsearch.search.aggregations.metrics.PercentilesMethod; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; +import org.elasticsearch.xpack.analytics.aggregations.support.AnalyticsValuesSourceType; + +public class AnalyticsPercentilesAggregatorFactory { + public static void registerPercentilesAggregator(ValuesSourceRegistry valuesSourceRegistry) { + valuesSourceRegistry.register(PercentilesAggregationBuilder.NAME, + AnalyticsValuesSourceType.HISTOGRAM, + (PercentilesAggregatorSupplier) (name, valuesSource, context, parent, percents, percentilesConfig, keyed, + formatter, pipelineAggregators, metaData) -> { + + if (percentilesConfig.getMethod().equals(PercentilesMethod.TDIGEST)) { + double compression = ((PercentilesConfig.TDigest)percentilesConfig).getCompression(); + return new HistoBackedTDigestPercentilesAggregator(name, valuesSource, context, parent, + percents, compression, keyed, formatter, pipelineAggregators, metaData); + + } else if (percentilesConfig.getMethod().equals(PercentilesMethod.HDR)) { + int numSigFig = ((PercentilesConfig.Hdr)percentilesConfig).getNumberOfSignificantValueDigits(); + return new HistoBackedHDRPercentilesAggregator(name, valuesSource, context, parent, + percents, numSigFig, keyed, formatter, pipelineAggregators, metaData); + } + + throw new IllegalArgumentException("Percentiles algorithm: [" + percentilesConfig.getMethod().toString() + "] " + + "is not compatible with Histogram field"); + }); + } + + public static void registerPercentileRanksAggregator(ValuesSourceRegistry valuesSourceRegistry) { + valuesSourceRegistry.register(PercentileRanksAggregationBuilder.NAME, + AnalyticsValuesSourceType.HISTOGRAM, + (PercentilesAggregatorSupplier) (name, valuesSource, context, parent, percents, percentilesConfig, keyed, + formatter, pipelineAggregators, metaData) -> { + + if (percentilesConfig.getMethod().equals(PercentilesMethod.TDIGEST)) { + double compression = ((PercentilesConfig.TDigest)percentilesConfig).getCompression(); + return new HistoBackedTDigestPercentileRanksAggregator(name, valuesSource, context, parent, + percents, compression, keyed, formatter, pipelineAggregators, metaData); + + } else if (percentilesConfig.getMethod().equals(PercentilesMethod.HDR)) { + int numSigFig = ((PercentilesConfig.Hdr)percentilesConfig).getNumberOfSignificantValueDigits(); + return new HistoBackedHDRPercentileRanksAggregator(name, valuesSource, context, parent, + percents, numSigFig, keyed, formatter, pipelineAggregators, metaData); + } + + throw new IllegalArgumentException("Percentiles algorithm: [" + percentilesConfig.getMethod().toString() + "] " + + "is not compatible with Histogram field"); + }); + } +} diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedHDRPercentileRanksAggregator.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedHDRPercentileRanksAggregator.java new file mode 100644 index 0000000000000..dc9bca57ec3cd --- /dev/null +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedHDRPercentileRanksAggregator.java @@ -0,0 +1,59 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +package org.elasticsearch.xpack.analytics.aggregations.metrics; + +import org.HdrHistogram.DoubleHistogram; +import org.elasticsearch.search.DocValueFormat; +import org.elasticsearch.search.aggregations.Aggregator; +import org.elasticsearch.search.aggregations.InternalAggregation; +import org.elasticsearch.search.aggregations.metrics.InternalHDRPercentileRanks; +import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.ValuesSource; +import org.elasticsearch.search.internal.SearchContext; + +import java.io.IOException; +import java.util.List; +import java.util.Map; + +class HistoBackedHDRPercentileRanksAggregator extends AbstractHistoBackedHDRPercentilesAggregator { + + HistoBackedHDRPercentileRanksAggregator(String name, ValuesSource valuesSource, SearchContext context, Aggregator parent, + double[] percents, int numberOfSignificantValueDigits, boolean keyed, DocValueFormat format, + List pipelineAggregators, Map metaData) throws IOException { + super(name, valuesSource, context, parent, percents, numberOfSignificantValueDigits, keyed, format, pipelineAggregators, + metaData); + } + + @Override + public InternalAggregation buildAggregation(long owningBucketOrdinal) { + DoubleHistogram state = getState(owningBucketOrdinal); + if (state == null) { + return buildEmptyAggregation(); + } else { + return new InternalHDRPercentileRanks(name, keys, state, keyed, format, pipelineAggregators(), metaData()); + } + } + + @Override + public InternalAggregation buildEmptyAggregation() { + DoubleHistogram state; + state = new DoubleHistogram(numberOfSignificantValueDigits); + state.setAutoResize(true); + return new InternalHDRPercentileRanks(name, keys, state, + keyed, format, pipelineAggregators(), metaData()); + } + + @Override + public double metric(String name, long bucketOrd) { + DoubleHistogram state = getState(bucketOrd); + if (state == null) { + return Double.NaN; + } else { + return InternalHDRPercentileRanks.percentileRank(state, Double.valueOf(name)); + } + } +} diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedHDRPercentilesAggregator.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedHDRPercentilesAggregator.java new file mode 100644 index 0000000000000..283f07fcc71f4 --- /dev/null +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedHDRPercentilesAggregator.java @@ -0,0 +1,60 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +package org.elasticsearch.xpack.analytics.aggregations.metrics; + +import org.HdrHistogram.DoubleHistogram; +import org.elasticsearch.search.DocValueFormat; +import org.elasticsearch.search.aggregations.Aggregator; +import org.elasticsearch.search.aggregations.InternalAggregation; +import org.elasticsearch.search.aggregations.metrics.InternalHDRPercentiles; +import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.ValuesSource; +import org.elasticsearch.search.internal.SearchContext; + +import java.io.IOException; +import java.util.List; +import java.util.Map; + +public class HistoBackedHDRPercentilesAggregator extends AbstractHistoBackedHDRPercentilesAggregator { + + HistoBackedHDRPercentilesAggregator(String name, ValuesSource valuesSource, SearchContext context, Aggregator parent, double[] percents, + int numberOfSignificantValueDigits, boolean keyed, DocValueFormat formatter, + List pipelineAggregators, Map metaData) throws IOException { + super(name, valuesSource, context, parent, percents, numberOfSignificantValueDigits, keyed, formatter, + pipelineAggregators, metaData); + } + + @Override + public InternalAggregation buildAggregation(long owningBucketOrdinal) { + DoubleHistogram state = getState(owningBucketOrdinal); + if (state == null) { + return buildEmptyAggregation(); + } else { + return new InternalHDRPercentiles(name, keys, state, keyed, format, pipelineAggregators(), metaData()); + } + } + + @Override + public double metric(String name, long bucketOrd) { + DoubleHistogram state = getState(bucketOrd); + if (state == null) { + return Double.NaN; + } else { + return state.getValueAtPercentile(Double.parseDouble(name)); + } + } + + @Override + public InternalAggregation buildEmptyAggregation() { + DoubleHistogram state; + state = new DoubleHistogram(numberOfSignificantValueDigits); + state.setAutoResize(true); + return new InternalHDRPercentiles(name, keys, state, + keyed, + format, pipelineAggregators(), metaData()); + } +} diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedTDigestPercentileRanksAggregator.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedTDigestPercentileRanksAggregator.java new file mode 100644 index 0000000000000..7bd1f9d35e995 --- /dev/null +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedTDigestPercentileRanksAggregator.java @@ -0,0 +1,62 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +package org.elasticsearch.xpack.analytics.aggregations.metrics; + +import org.elasticsearch.search.DocValueFormat; +import org.elasticsearch.search.aggregations.Aggregator; +import org.elasticsearch.search.aggregations.InternalAggregation; +import org.elasticsearch.search.aggregations.metrics.InternalTDigestPercentileRanks; +import org.elasticsearch.search.aggregations.metrics.TDigestState; +import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.ValuesSource; +import org.elasticsearch.search.internal.SearchContext; + +import java.io.IOException; +import java.util.List; +import java.util.Map; + +class HistoBackedTDigestPercentileRanksAggregator extends AbstractHistoBackedTDigestPercentilesAggregator { + + HistoBackedTDigestPercentileRanksAggregator(String name, + ValuesSource valuesSource, + SearchContext context, + Aggregator parent, + double[] percents, + double compression, + boolean keyed, + DocValueFormat formatter, + List pipelineAggregators, + Map metaData) throws IOException { + super(name, valuesSource, context, parent, percents, compression, keyed, formatter, pipelineAggregators, metaData); + } + + @Override + public InternalAggregation buildAggregation(long owningBucketOrdinal) { + TDigestState state = getState(owningBucketOrdinal); + if (state == null) { + return buildEmptyAggregation(); + } else { + return new InternalTDigestPercentileRanks(name, keys, state, keyed, formatter, pipelineAggregators(), metaData()); + } + } + + @Override + public InternalAggregation buildEmptyAggregation() { + return new InternalTDigestPercentileRanks(name, keys, new TDigestState(compression), keyed, + formatter, pipelineAggregators(), metaData()); + } + + @Override + public double metric(String name, long bucketOrd) { + TDigestState state = getState(bucketOrd); + if (state == null) { + return Double.NaN; + } else { + return InternalTDigestPercentileRanks.percentileRank(state, Double.valueOf(name)); + } + } +} diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedTDigestPercentilesAggregator.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedTDigestPercentilesAggregator.java new file mode 100644 index 0000000000000..6b7f33178ebf1 --- /dev/null +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedTDigestPercentilesAggregator.java @@ -0,0 +1,62 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +package org.elasticsearch.xpack.analytics.aggregations.metrics; + +import org.elasticsearch.search.DocValueFormat; +import org.elasticsearch.search.aggregations.Aggregator; +import org.elasticsearch.search.aggregations.InternalAggregation; +import org.elasticsearch.search.aggregations.metrics.InternalTDigestPercentiles; +import org.elasticsearch.search.aggregations.metrics.TDigestState; +import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.ValuesSource; +import org.elasticsearch.search.internal.SearchContext; + +import java.io.IOException; +import java.util.List; +import java.util.Map; + +class HistoBackedTDigestPercentilesAggregator extends AbstractHistoBackedTDigestPercentilesAggregator { + + HistoBackedTDigestPercentilesAggregator(String name, + ValuesSource valuesSource, + SearchContext context, + Aggregator parent, + double[] percents, + double compression, + boolean keyed, + DocValueFormat formatter, + List pipelineAggregators, + Map metaData) throws IOException { + super(name, valuesSource, context, parent, percents, compression, keyed, formatter, pipelineAggregators, metaData); + } + + @Override + public InternalAggregation buildAggregation(long owningBucketOrdinal) { + TDigestState state = getState(owningBucketOrdinal); + if (state == null) { + return buildEmptyAggregation(); + } else { + return new InternalTDigestPercentiles(name, keys, state, keyed, formatter, pipelineAggregators(), metaData()); + } + } + + @Override + public double metric(String name, long bucketOrd) { + TDigestState state = getState(bucketOrd); + if (state == null) { + return Double.NaN; + } else { + return state.quantile(Double.parseDouble(name) / 100); + } + } + + @Override + public InternalAggregation buildEmptyAggregation() { + return new InternalTDigestPercentiles(name, keys, new TDigestState(compression), keyed, + formatter, pipelineAggregators(), metaData()); + } +} diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/support/AnalyticsValuesSourceType.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/support/AnalyticsValuesSourceType.java new file mode 100644 index 0000000000000..8ff07fda257b5 --- /dev/null +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/support/AnalyticsValuesSourceType.java @@ -0,0 +1,59 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ +package org.elasticsearch.xpack.analytics.aggregations.support; + +import org.elasticsearch.index.fielddata.IndexFieldData; +import org.elasticsearch.index.fielddata.IndexHistogramFieldData; +import org.elasticsearch.script.AggregationScript; +import org.elasticsearch.search.DocValueFormat; +import org.elasticsearch.search.aggregations.AggregationExecutionException; +import org.elasticsearch.search.aggregations.support.FieldContext; +import org.elasticsearch.search.aggregations.support.ValueType; +import org.elasticsearch.search.aggregations.support.ValuesSource; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; + +import java.util.Locale; +import java.util.function.LongSupplier; + +public enum AnalyticsValuesSourceType implements ValuesSourceType { + HISTOGRAM() { + @Override + public ValuesSource getEmpty() { + // TODO: Is this the correct exception type here? + throw new IllegalArgumentException("Can't deal with unmapped HistogramValuesSource type " + this.value()); + } + + @Override + public ValuesSource getScript(AggregationScript.LeafFactory script, ValueType scriptValueType) { + throw new AggregationExecutionException("value source of type [" + this.value() + "] is not supported by scripts"); + } + + @Override + public ValuesSource getField(FieldContext fieldContext, AggregationScript.LeafFactory script) { + final IndexFieldData indexFieldData = fieldContext.indexFieldData(); + + if (!(indexFieldData instanceof IndexHistogramFieldData)) { + throw new IllegalArgumentException("Expected histogram type on field [" + fieldContext.field() + + "], but got [" + fieldContext.fieldType().typeName() + "]"); + } + return new HistogramValuesSource.Histogram.Fielddata((IndexHistogramFieldData) indexFieldData); + } + + @Override + public ValuesSource replaceMissing(ValuesSource valuesSource, Object rawMissing, DocValueFormat docValueFormat, LongSupplier now) { + throw new IllegalArgumentException("Can't apply missing values on a " + valuesSource.getClass()); + } + }; + + + public static ValuesSourceType fromString(String name) { + return valueOf(name.trim().toUpperCase(Locale.ROOT)); + } + + public String value() { + return name().toLowerCase(Locale.ROOT); + } +} diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/support/HistogramValuesSource.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/support/HistogramValuesSource.java new file mode 100644 index 0000000000000..54a9582b4f34b --- /dev/null +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/support/HistogramValuesSource.java @@ -0,0 +1,51 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +package org.elasticsearch.xpack.analytics.aggregations.support; + +import org.apache.lucene.index.LeafReaderContext; +import org.elasticsearch.index.fielddata.DocValueBits; +import org.elasticsearch.index.fielddata.HistogramValues; +import org.elasticsearch.index.fielddata.IndexHistogramFieldData; +import org.elasticsearch.index.fielddata.SortedBinaryDocValues; + +import java.io.IOException; + +public class HistogramValuesSource { + public abstract static class Histogram extends org.elasticsearch.search.aggregations.support.ValuesSource { + + public abstract HistogramValues getHistogramValues(LeafReaderContext context) throws IOException; + + public static class Fielddata extends Histogram { + + protected final IndexHistogramFieldData indexFieldData; + + public Fielddata(IndexHistogramFieldData indexFieldData) { + this.indexFieldData = indexFieldData; + } + + @Override + public SortedBinaryDocValues bytesValues(LeafReaderContext context) { + return indexFieldData.load(context).getBytesValues(); + } + + @Override + public DocValueBits docsWithValue(LeafReaderContext context) throws IOException { + HistogramValues values = getHistogramValues(context); + return new DocValueBits() { + @Override + public boolean advanceExact(int doc) throws IOException { + return values.advanceExact(doc); + } + }; + } + + public HistogramValues getHistogramValues(LeafReaderContext context) throws IOException { + return indexFieldData.load(context).getHistogramValues(); + } + } + } +} diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregationBuilder.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregationBuilder.java index daba298b19fa9..9d118d1ba2816 100644 --- a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregationBuilder.java +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregationBuilder.java @@ -16,11 +16,11 @@ import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.metrics.PercentilesMethod; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; -import org.elasticsearch.search.aggregations.support.ValueType; import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; -import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.util.Map; @@ -35,14 +35,14 @@ public class BoxplotAggregationBuilder extends ValuesSourceAggregationBuilder.Le public static final ObjectParser PARSER = ObjectParser.fromBuilder(NAME, BoxplotAggregationBuilder::new); static { - ValuesSourceParserHelper.declareAnyFields(PARSER, true, true); + ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, false); PARSER.declareDouble(BoxplotAggregationBuilder::compression, COMPRESSION_FIELD); } private double compression = 100.0; public BoxplotAggregationBuilder(String name) { - super(name, CoreValuesSourceType.NUMERIC, ValueType.NUMERIC); + super(name); } protected BoxplotAggregationBuilder(BoxplotAggregationBuilder clone, @@ -51,6 +51,10 @@ protected BoxplotAggregationBuilder(BoxplotAggregationBuilder clone, this.compression = clone.compression; } + public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + BoxplotAggregatorFactory.registerAggregators(valuesSourceRegistry); + } + @Override protected AggregationBuilder shallowCopy(AggregatorFactories.Builder factoriesBuilder, Map metadata) { return new BoxplotAggregationBuilder(this, factoriesBuilder, metadata); @@ -60,7 +64,7 @@ protected AggregationBuilder shallowCopy(AggregatorFactories.Builder factoriesBu * Read from a stream. */ public BoxplotAggregationBuilder(StreamInput in) throws IOException { - super(in, CoreValuesSourceType.NUMERIC, ValueType.NUMERIC); + super(in); compression = in.readDouble(); } @@ -69,6 +73,11 @@ protected void innerWriteTo(StreamOutput out) throws IOException { out.writeDouble(compression); } + @Override + protected ValuesSourceType defaultValueSourceType() { + return CoreValuesSourceType.NUMERIC; + } + /** * Expert: set the compression. Higher values improve accuracy but also * memory usage. Only relevant when using {@link PercentilesMethod#TDIGEST}. @@ -92,7 +101,7 @@ public double compression() { @Override protected BoxplotAggregatorFactory innerBuild(QueryShardContext queryShardContext, - ValuesSourceConfig config, + ValuesSourceConfig config, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder) throws IOException { return new BoxplotAggregatorFactory(name, config, compression, queryShardContext, parent, subFactoriesBuilder, metadata); diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregator.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregator.java index f917ad87507df..6938728592d9d 100644 --- a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregator.java +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregator.java @@ -24,6 +24,7 @@ import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.internal.SearchContext; +import org.elasticsearch.xpack.analytics.aggregations.support.HistogramValuesSource; import java.io.IOException; import java.util.List; @@ -60,8 +61,8 @@ public LeafBucketCollector getLeafCollector(LeafReaderContext ctx, return LeafBucketCollector.NO_OP_COLLECTOR; } final BigArrays bigArrays = context.bigArrays(); - if (valuesSource instanceof ValuesSource.Histogram) { - final HistogramValues values = ((ValuesSource.Histogram)valuesSource).getHistogramValues(ctx); + if (valuesSource instanceof HistogramValuesSource.Histogram) { + final HistogramValues values = ((HistogramValuesSource.Histogram)valuesSource).getHistogramValues(ctx); return new LeafBucketCollectorBase(sub, values) { @Override public void collect(int doc, long bucket) throws IOException { diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregatorFactory.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregatorFactory.java index ff3ce32aa063e..aca50bdc7b2cb 100644 --- a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregatorFactory.java +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregatorFactory.java @@ -7,25 +7,36 @@ package org.elasticsearch.xpack.analytics.boxplot; import org.elasticsearch.index.query.QueryShardContext; +import org.elasticsearch.search.aggregations.AggregationExecutionException; import org.elasticsearch.search.aggregations.Aggregator; import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.search.internal.SearchContext; +import org.elasticsearch.xpack.analytics.aggregations.support.AnalyticsValuesSourceType; import java.io.IOException; import java.util.List; import java.util.Map; -public class BoxplotAggregatorFactory extends ValuesSourceAggregatorFactory { +public class BoxplotAggregatorFactory extends ValuesSourceAggregatorFactory { private final double compression; + static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + valuesSourceRegistry.register(BoxplotAggregationBuilder.NAME, + List.of(CoreValuesSourceType.NUMERIC, AnalyticsValuesSourceType.HISTOGRAM), + (BoxplotAggregatorSupplier) BoxplotAggregator::new); + } + BoxplotAggregatorFactory(String name, - ValuesSourceConfig config, + ValuesSourceConfig config, double compression, QueryShardContext queryShardContext, AggregatorFactory parent, @@ -51,9 +62,15 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, Aggregator parent, boolean collectsFromSingleBucket, List pipelineAggregators, - Map metadata) throws IOException { - return new BoxplotAggregator(name, valuesSource, config.format(), compression, searchContext, parent, - pipelineAggregators, metadata); - } + Map metaData) throws IOException { + AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), + BoxplotAggregationBuilder.NAME); + if (aggregatorSupplier instanceof BoxplotAggregatorSupplier == false) { + throw new AggregationExecutionException("Registry miss-match - expected BoxplotAggregatorSupplier, found [" + + aggregatorSupplier.getClass().toString() + "]"); + } + return ((BoxplotAggregatorSupplier) aggregatorSupplier).build(name, valuesSource, config.format(), compression, + searchContext, parent, pipelineAggregators, metaData); + } } diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregatorSupplier.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregatorSupplier.java new file mode 100644 index 0000000000000..3555014b8e9ca --- /dev/null +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregatorSupplier.java @@ -0,0 +1,31 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +package org.elasticsearch.xpack.analytics.boxplot; + +import org.elasticsearch.search.DocValueFormat; +import org.elasticsearch.search.aggregations.Aggregator; +import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.ValuesSource; +import org.elasticsearch.search.internal.SearchContext; + +import java.io.IOException; +import java.util.List; +import java.util.Map; + +@FunctionalInterface +public interface BoxplotAggregatorSupplier extends AggregatorSupplier { + Aggregator build(String name, + ValuesSource valuesSource, + DocValueFormat formatter, + double compression, + SearchContext context, + Aggregator parent, + List pipelineAggregators, + Map metaData) throws IOException; + +} diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/mapper/HistogramFieldMapper.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/mapper/HistogramFieldMapper.java index 6b77018c11638..ceab5dc8dfb1e 100644 --- a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/mapper/HistogramFieldMapper.java +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/mapper/HistogramFieldMapper.java @@ -34,11 +34,11 @@ import org.elasticsearch.index.fielddata.HistogramValue; import org.elasticsearch.index.fielddata.HistogramValues; import org.elasticsearch.index.fielddata.IndexFieldData; +import org.elasticsearch.index.fielddata.IndexFieldData.XFieldComparatorSource.Nested; import org.elasticsearch.index.fielddata.IndexFieldDataCache; import org.elasticsearch.index.fielddata.IndexHistogramFieldData; import org.elasticsearch.index.fielddata.ScriptDocValues; import org.elasticsearch.index.fielddata.SortedBinaryDocValues; -import org.elasticsearch.index.fielddata.IndexFieldData.XFieldComparatorSource.Nested; import org.elasticsearch.index.mapper.FieldMapper; import org.elasticsearch.index.mapper.MappedFieldType; import org.elasticsearch.index.mapper.Mapper; @@ -51,10 +51,10 @@ import org.elasticsearch.indices.breaker.CircuitBreakerService; import org.elasticsearch.search.DocValueFormat; import org.elasticsearch.search.MultiValueMode; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import org.elasticsearch.search.sort.BucketedSort; import org.elasticsearch.search.sort.SortOrder; -import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; -import org.elasticsearch.search.aggregations.support.ValuesSourceType; +import org.elasticsearch.xpack.analytics.aggregations.support.AnalyticsValuesSourceType; import java.io.IOException; import java.util.Iterator; @@ -280,8 +280,7 @@ public BucketedSort newBucketedSort(BigArrays bigArrays, Object missingValue, Mu @Override public ValuesSourceType getValuesSourceType() { - // TODO: Histogram ValuesSourceType should move into this plugin. - return CoreValuesSourceType.HISTOGRAM; + return AnalyticsValuesSourceType.HISTOGRAM; } @Override diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregationBuilder.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregationBuilder.java index a41fbe079cee9..b33d87d3a634f 100644 --- a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregationBuilder.java +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregationBuilder.java @@ -15,31 +15,31 @@ import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; -import org.elasticsearch.search.aggregations.support.ValueType; -import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; -import org.elasticsearch.search.aggregations.support.ValuesSourceParserHelper; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.util.Map; import java.util.Objects; -public class StringStatsAggregationBuilder extends ValuesSourceAggregationBuilder { +public class StringStatsAggregationBuilder extends ValuesSourceAggregationBuilder { + public static final String NAME = "string_stats"; private static final ParseField SHOW_DISTRIBUTION_FIELD = new ParseField("show_distribution"); public static final ObjectParser PARSER = ObjectParser.fromBuilder(NAME, StringStatsAggregationBuilder::new); static { - ValuesSourceParserHelper.declareBytesFields(PARSER, true, true); + ValuesSourceAggregationBuilder.declareFields(PARSER, true, true, false); PARSER.declareBoolean(StringStatsAggregationBuilder::showDistribution, StringStatsAggregationBuilder.SHOW_DISTRIBUTION_FIELD); } private boolean showDistribution = false; public StringStatsAggregationBuilder(String name) { - super(name, CoreValuesSourceType.BYTES, ValueType.STRING); + super(name); } public StringStatsAggregationBuilder(StringStatsAggregationBuilder clone, @@ -51,13 +51,18 @@ public StringStatsAggregationBuilder(StringStatsAggregationBuilder clone, /** Read from a stream. */ public StringStatsAggregationBuilder(StreamInput in) throws IOException { - super(in, CoreValuesSourceType.BYTES, ValueType.STRING); + super(in); this.showDistribution = in.readBoolean(); } @Override - protected AggregationBuilder shallowCopy(AggregatorFactories.Builder factoriesBuilder, Map metadata) { - return new StringStatsAggregationBuilder(this, factoriesBuilder, metadata); + protected ValuesSourceType defaultValueSourceType() { + return CoreValuesSourceType.BYTES; + } + + @Override + protected AggregationBuilder shallowCopy(AggregatorFactories.Builder factoriesBuilder, Map metaData) { + return new StringStatsAggregationBuilder(this, factoriesBuilder, metaData); } @Override @@ -72,7 +77,7 @@ public BucketCardinality bucketCardinality() { @Override protected StringStatsAggregatorFactory innerBuild(QueryShardContext queryShardContext, - ValuesSourceConfig config, + ValuesSourceConfig config, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder) throws IOException { return new StringStatsAggregatorFactory(name, config, showDistribution, queryShardContext, parent, subFactoriesBuilder, metadata); @@ -108,6 +113,10 @@ public StringStatsAggregationBuilder showDistribution(boolean showDistribution) return this; } + public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { + StringStatsAggregatorFactory.registerAggregators(valuesSourceRegistry); + } + @Override public int hashCode() { return Objects.hash(super.hashCode(), showDistribution); diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregatorFactory.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregatorFactory.java index 8944f4f3f5350..e74c49e5825d4 100644 --- a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregatorFactory.java +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregatorFactory.java @@ -7,24 +7,29 @@ package org.elasticsearch.xpack.analytics.stringstats; import org.elasticsearch.index.query.QueryShardContext; +import org.elasticsearch.search.DocValueFormat; +import org.elasticsearch.search.aggregations.AggregationExecutionException; import org.elasticsearch.search.aggregations.Aggregator; import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; import java.util.List; import java.util.Map; -class StringStatsAggregatorFactory extends ValuesSourceAggregatorFactory { +class StringStatsAggregatorFactory extends ValuesSourceAggregatorFactory { private final boolean showDistribution; - StringStatsAggregatorFactory(String name, ValuesSourceConfig config, + StringStatsAggregatorFactory(String name, ValuesSourceConfig config, Boolean showDistribution, QueryShardContext queryShardContext, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, Map metadata) @@ -33,6 +38,24 @@ class StringStatsAggregatorFactory extends ValuesSourceAggregatorFactory pipelineAggregators, + Map metaData) throws IOException { + return new StringStatsAggregator(name, showDistribution, (ValuesSource.Bytes) valuesSource, + format, context, parent, pipelineAggregators, metaData); + } + }); + } + @Override protected Aggregator createUnmapped(SearchContext searchContext, Aggregator parent, @@ -43,14 +66,21 @@ protected Aggregator createUnmapped(SearchContext searchContext, } @Override - protected Aggregator doCreateInternal(ValuesSource.Bytes valuesSource, + protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, List pipelineAggregators, - Map metadata) throws IOException { - return new StringStatsAggregator(name, showDistribution, valuesSource, config.format(), searchContext, parent, - pipelineAggregators, metadata); + Map metaData) throws IOException { + AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), + StringStatsAggregationBuilder.NAME); + + if (aggregatorSupplier instanceof StringStatsAggregatorSupplier == false) { + throw new AggregationExecutionException("Registry miss-match - expected StringStatsAggregatorSupplier, found [" + + aggregatorSupplier.getClass().toString() + "]"); + } + return ((StringStatsAggregatorSupplier) aggregatorSupplier).build(name, valuesSource, showDistribution, config.format(), + searchContext, parent, pipelineAggregators, metaData); } } diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregatorSupplier.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregatorSupplier.java new file mode 100644 index 0000000000000..d996ec0b309a1 --- /dev/null +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregatorSupplier.java @@ -0,0 +1,29 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ +package org.elasticsearch.xpack.analytics.stringstats; + +import org.elasticsearch.search.DocValueFormat; +import org.elasticsearch.search.aggregations.Aggregator; +import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.AggregatorSupplier; +import org.elasticsearch.search.aggregations.support.ValuesSource; +import org.elasticsearch.search.internal.SearchContext; + +import java.io.IOException; +import java.util.List; +import java.util.Map; + +public interface StringStatsAggregatorSupplier extends AggregatorSupplier { + + Aggregator build(String name, + ValuesSource valuesSource, + boolean showDistribution, + DocValueFormat format, + SearchContext context, + Aggregator parent, + List pipelineAggregators, + Map metaData) throws IOException; +} diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/topmetrics/TopMetricsAggregatorFactory.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/topmetrics/TopMetricsAggregatorFactory.java index 61f4ab968fe1c..86779aa232b0e 100644 --- a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/topmetrics/TopMetricsAggregatorFactory.java +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/topmetrics/TopMetricsAggregatorFactory.java @@ -13,6 +13,7 @@ import org.elasticsearch.search.aggregations.AggregatorFactories.Builder; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; import org.elasticsearch.search.aggregations.support.MultiValuesSourceFieldConfig; import org.elasticsearch.search.aggregations.support.ValueType; import org.elasticsearch.search.aggregations.support.ValuesSource; @@ -58,11 +59,12 @@ protected TopMetricsAggregator createInternal(SearchContext searchContext, Aggre + "] index level setting."); } List metricSources = metricFields.stream().map(config -> { - ValuesSourceConfig resolved = ValuesSourceConfig.resolve( + ValuesSourceConfig resolved = ValuesSourceConfig.resolve( searchContext.getQueryShardContext(), ValueType.NUMERIC, - config.getFieldName(), config.getScript(), config.getMissing(), config.getTimeZone(), null); + config.getFieldName(), config.getScript(), config.getMissing(), config.getTimeZone(), null, + CoreValuesSourceType.NUMERIC, TopMetricsAggregationBuilder.NAME); return new TopMetricsAggregator.MetricSource(config.getFieldName(), resolved.format(), - resolved.toValuesSource(searchContext.getQueryShardContext())); + (ValuesSource.Numeric) resolved.toValuesSource()); }).collect(toList()); return new TopMetricsAggregator(name, searchContext, parent, pipelineAggregators, metadata, size, sortBuilders.get(0), metricSources); diff --git a/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregatorTests.java b/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregatorTests.java index 31a532d5314be..d5ca763d66a85 100644 --- a/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregatorTests.java +++ b/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregatorTests.java @@ -30,6 +30,10 @@ import org.elasticsearch.search.aggregations.bucket.histogram.HistogramAggregationBuilder; import org.elasticsearch.search.aggregations.bucket.histogram.InternalHistogram; import org.elasticsearch.search.aggregations.support.AggregationInspectionHelper; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; +import org.elasticsearch.xpack.analytics.aggregations.support.AnalyticsValuesSourceType; +import org.junit.BeforeClass; import java.io.IOException; import java.util.function.Consumer; @@ -39,6 +43,39 @@ public class BoxplotAggregatorTests extends AggregatorTestCase { + /** Script to return the {@code _value} provided by aggs framework. */ + public static final String VALUE_SCRIPT = "_value"; + + + @BeforeClass() + public static void registerBuilder() { + BoxplotAggregationBuilder.registerAggregators(valuesSourceRegistry); + } + + @Override + protected AggregationBuilder createAggBuilderForTypeTest(MappedFieldType fieldType, String fieldName) { + return new BoxplotAggregationBuilder("foo").field(fieldName); + } + + @Override + protected List getSupportedValuesSourceTypes() { + return List.of(CoreValuesSourceType.NUMERIC, AnalyticsValuesSourceType.HISTOGRAM); + } + + @Override + protected ScriptService getMockScriptService() { + Map, Object>> scripts = new HashMap<>(); + + scripts.put(VALUE_SCRIPT, vars -> ((Number) vars.get("_value")).doubleValue() + 1); + + MockScriptEngine scriptEngine = new MockScriptEngine(MockScriptEngine.NAME, + scripts, + Collections.emptyMap()); + Map engines = Collections.singletonMap(scriptEngine.getType(), scriptEngine); + + return new ScriptService(Settings.EMPTY, engines, ScriptModule.CORE_CONTEXTS); + } + public void testNoMatchingField() throws IOException { testCase(new MatchAllDocsQuery(), iw -> { iw.addDocument(singleton(new SortedNumericDocValuesField("wrong_number", 7))); @@ -154,7 +191,8 @@ public void testUnsupportedType() { }, (Consumer) boxplot -> { fail("Should have thrown exception"); }, fieldType)); - assertEquals(e.getMessage(), "Expected numeric type on field [not_a_number], but got [keyword]"); + assertEquals(e.getMessage(), "Field [not_a_number] of type [keyword(indexed,tokenized)] " + + "is not supported for aggregation [boxplot]"); } public void testBadMissingField() { diff --git a/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/cumulativecardinality/CumulativeCardinalityAggregatorTests.java b/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/cumulativecardinality/CumulativeCardinalityAggregatorTests.java index fd7adddd4234c..a8894e3c13a92 100644 --- a/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/cumulativecardinality/CumulativeCardinalityAggregatorTests.java +++ b/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/cumulativecardinality/CumulativeCardinalityAggregatorTests.java @@ -29,7 +29,6 @@ import org.elasticsearch.search.aggregations.bucket.histogram.Histogram; import org.elasticsearch.search.aggregations.metrics.CardinalityAggregationBuilder; import org.elasticsearch.search.aggregations.metrics.SumAggregationBuilder; -import org.elasticsearch.search.aggregations.support.ValueType; import java.io.IOException; import java.util.Arrays; @@ -64,7 +63,7 @@ public void testSimple() throws IOException { DateHistogramAggregationBuilder aggBuilder = new DateHistogramAggregationBuilder("histo"); aggBuilder.calendarInterval(DateHistogramInterval.DAY).field(HISTO_FIELD); - aggBuilder.subAggregation(new CardinalityAggregationBuilder("the_cardinality", ValueType.NUMERIC).field(VALUE_FIELD)); + aggBuilder.subAggregation(new CardinalityAggregationBuilder("the_cardinality").field(VALUE_FIELD)); aggBuilder.subAggregation(new CumulativeCardinalityPipelineAggregationBuilder("cumulative_card", "the_cardinality")); executeTestCase(query, aggBuilder, histogram -> { @@ -84,7 +83,7 @@ public void testAllNull() throws IOException { DateHistogramAggregationBuilder aggBuilder = new DateHistogramAggregationBuilder("histo"); aggBuilder.calendarInterval(DateHistogramInterval.DAY).field(HISTO_FIELD); - aggBuilder.subAggregation(new CardinalityAggregationBuilder("the_cardinality", ValueType.NUMERIC).field("foo")); + aggBuilder.subAggregation(new CardinalityAggregationBuilder("the_cardinality").field("foo")); aggBuilder.subAggregation(new CumulativeCardinalityPipelineAggregationBuilder("cumulative_card", "the_cardinality")); executeTestCase(query, aggBuilder, histogram -> { diff --git a/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/HDRPreAggregatedPercentileRanksAggregatorTests.java b/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/HDRPreAggregatedPercentileRanksAggregatorTests.java index 843ff9447fde5..fab7bd91e0ea3 100644 --- a/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/HDRPreAggregatedPercentileRanksAggregatorTests.java +++ b/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/HDRPreAggregatedPercentileRanksAggregatorTests.java @@ -16,20 +16,49 @@ import org.apache.lucene.store.Directory; import org.elasticsearch.common.io.stream.BytesStreamOutput; import org.elasticsearch.index.mapper.MappedFieldType; +import org.elasticsearch.search.aggregations.AggregationBuilder; import org.elasticsearch.search.aggregations.AggregatorTestCase; import org.elasticsearch.search.aggregations.metrics.InternalHDRPercentileRanks; import org.elasticsearch.search.aggregations.metrics.Percentile; import org.elasticsearch.search.aggregations.metrics.PercentileRanks; import org.elasticsearch.search.aggregations.metrics.PercentileRanksAggregationBuilder; +import org.elasticsearch.search.aggregations.metrics.PercentilesConfig; import org.elasticsearch.search.aggregations.metrics.PercentilesMethod; import org.elasticsearch.search.aggregations.support.AggregationInspectionHelper; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; +import org.elasticsearch.xpack.analytics.aggregations.metrics.AnalyticsPercentilesAggregatorFactory; +import org.elasticsearch.xpack.analytics.aggregations.support.AnalyticsValuesSourceType; import org.hamcrest.Matchers; +import org.junit.BeforeClass; import java.io.IOException; import java.util.Iterator; +import java.util.List; public class HDRPreAggregatedPercentileRanksAggregatorTests extends AggregatorTestCase { + @BeforeClass + public static void registerBuilder() { + AnalyticsPercentilesAggregatorFactory.registerPercentileRanksAggregator(valuesSourceRegistry); + } + + @Override + protected AggregationBuilder createAggBuilderForTypeTest(MappedFieldType fieldType, String fieldName) { + return new PercentileRanksAggregationBuilder("hdr_percentiles", new double[]{1.0}) + .field(fieldName) + .percentilesConfig(new PercentilesConfig.Hdr()); + } + + @Override + protected List getSupportedValuesSourceTypes() { + // Note: this is the same list as Core, plus Analytics + return List.of(CoreValuesSourceType.NUMERIC, + CoreValuesSourceType.DATE, + CoreValuesSourceType.BOOLEAN, + AnalyticsValuesSourceType.HISTOGRAM); + } + private BinaryDocValuesField getDocValue(String fieldName, double[] values) throws IOException { DoubleHistogram histogram = new DoubleHistogram(3);//default for (double value : values) { @@ -62,7 +91,7 @@ public void testSimple() throws IOException { fieldType.setName("field"); try (IndexReader reader = w.getReader()) { IndexSearcher searcher = new IndexSearcher(reader); - PercentileRanks ranks = search(searcher, new MatchAllDocsQuery(), aggBuilder, fieldType); + PercentileRanks ranks = searchAndReduce(searcher, new MatchAllDocsQuery(), aggBuilder, fieldType); Iterator rankIterator = ranks.iterator(); Percentile rank = rankIterator.next(); assertEquals(0.1, rank.getValue(), 0d); diff --git a/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/HDRPreAggregatedPercentilesAggregatorTests.java b/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/HDRPreAggregatedPercentilesAggregatorTests.java index 5d4e5c05b2e9d..d71797039d78d 100644 --- a/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/HDRPreAggregatedPercentilesAggregatorTests.java +++ b/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/HDRPreAggregatedPercentilesAggregatorTests.java @@ -19,22 +19,51 @@ import org.elasticsearch.common.CheckedConsumer; import org.elasticsearch.common.io.stream.BytesStreamOutput; import org.elasticsearch.index.mapper.MappedFieldType; +import org.elasticsearch.search.aggregations.AggregationBuilder; import org.elasticsearch.search.aggregations.Aggregator; import org.elasticsearch.search.aggregations.AggregatorTestCase; import org.elasticsearch.search.aggregations.metrics.InternalHDRPercentiles; import org.elasticsearch.search.aggregations.metrics.PercentilesAggregationBuilder; +import org.elasticsearch.search.aggregations.metrics.PercentilesConfig; import org.elasticsearch.search.aggregations.metrics.PercentilesMethod; import org.elasticsearch.search.aggregations.support.AggregationInspectionHelper; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; +import org.elasticsearch.xpack.analytics.aggregations.metrics.AnalyticsPercentilesAggregatorFactory; +import org.elasticsearch.xpack.analytics.aggregations.support.AnalyticsValuesSourceType; +import org.junit.BeforeClass; import java.io.IOException; import java.util.Iterator; +import java.util.List; import java.util.function.Consumer; import static java.util.Collections.singleton; public class HDRPreAggregatedPercentilesAggregatorTests extends AggregatorTestCase { - private BinaryDocValuesField getDocValue(String fieldName, double[] values) throws IOException { + @BeforeClass + public static void registerBuilder() { + AnalyticsPercentilesAggregatorFactory.registerPercentilesAggregator(valuesSourceRegistry); + } + + @Override + protected AggregationBuilder createAggBuilderForTypeTest(MappedFieldType fieldType, String fieldName) { + return new PercentilesAggregationBuilder("hdr_percentiles") + .field(fieldName) + .percentilesConfig(new PercentilesConfig.Hdr()); + } + + @Override + protected List getSupportedValuesSourceTypes() { + // Note: this is the same list as Core, plus Analytics + return List.of(CoreValuesSourceType.NUMERIC, + CoreValuesSourceType.DATE, + CoreValuesSourceType.BOOLEAN, + AnalyticsValuesSourceType.HISTOGRAM); + } + + private BinaryDocValuesField getDocValue(String fieldName, double[] values) throws IOException { DoubleHistogram histogram = new DoubleHistogram(3);//default for (double value : values) { histogram.recordValue(value); @@ -54,7 +83,7 @@ private BinaryDocValuesField getDocValue(String fieldName, double[] values) thro } return new BinaryDocValuesField(fieldName, streamOutput.bytes().toBytesRef()); - } + } public void testNoMatchingField() throws IOException { testCase(new MatchAllDocsQuery(), iw -> { diff --git a/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/TDigestPreAggregatedPercentileRanksAggregatorTests.java b/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/TDigestPreAggregatedPercentileRanksAggregatorTests.java index 879173a5bc26e..ad80192a3c67b 100644 --- a/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/TDigestPreAggregatedPercentileRanksAggregatorTests.java +++ b/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/TDigestPreAggregatedPercentileRanksAggregatorTests.java @@ -16,23 +16,52 @@ import org.apache.lucene.store.Directory; import org.elasticsearch.common.io.stream.BytesStreamOutput; import org.elasticsearch.index.mapper.MappedFieldType; +import org.elasticsearch.search.aggregations.AggregationBuilder; import org.elasticsearch.search.aggregations.AggregatorTestCase; import org.elasticsearch.search.aggregations.metrics.InternalTDigestPercentileRanks; import org.elasticsearch.search.aggregations.metrics.Percentile; import org.elasticsearch.search.aggregations.metrics.PercentileRanks; import org.elasticsearch.search.aggregations.metrics.PercentileRanksAggregationBuilder; +import org.elasticsearch.search.aggregations.metrics.PercentilesConfig; import org.elasticsearch.search.aggregations.metrics.PercentilesMethod; import org.elasticsearch.search.aggregations.metrics.TDigestState; import org.elasticsearch.search.aggregations.support.AggregationInspectionHelper; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; +import org.elasticsearch.xpack.analytics.aggregations.metrics.AnalyticsPercentilesAggregatorFactory; +import org.elasticsearch.xpack.analytics.aggregations.support.AnalyticsValuesSourceType; import org.hamcrest.Matchers; +import org.junit.BeforeClass; import java.io.IOException; import java.util.Collection; import java.util.Iterator; +import java.util.List; public class TDigestPreAggregatedPercentileRanksAggregatorTests extends AggregatorTestCase { + @BeforeClass + public static void registerBuilder() { + AnalyticsPercentilesAggregatorFactory.registerPercentileRanksAggregator(valuesSourceRegistry); + } + + @Override + protected AggregationBuilder createAggBuilderForTypeTest(MappedFieldType fieldType, String fieldName) { + return new PercentileRanksAggregationBuilder("tdigest_percentiles", new double[]{1.0}) + .field(fieldName) + .percentilesConfig(new PercentilesConfig.TDigest()); + } + + @Override + protected List getSupportedValuesSourceTypes() { + // Note: this is the same list as Core, plus Analytics + return List.of(CoreValuesSourceType.NUMERIC, + CoreValuesSourceType.DATE, + CoreValuesSourceType.BOOLEAN, + AnalyticsValuesSourceType.HISTOGRAM); + } + private BinaryDocValuesField getDocValue(String fieldName, double[] values) throws IOException { TDigest histogram = new TDigestState(100.0); //default for (double value : values) { diff --git a/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/TDigestPreAggregatedPercentilesAggregatorTests.java b/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/TDigestPreAggregatedPercentilesAggregatorTests.java index e1340619256cf..6b084535addb8 100644 --- a/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/TDigestPreAggregatedPercentilesAggregatorTests.java +++ b/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/TDigestPreAggregatedPercentilesAggregatorTests.java @@ -19,24 +19,53 @@ import org.elasticsearch.common.CheckedConsumer; import org.elasticsearch.common.io.stream.BytesStreamOutput; import org.elasticsearch.index.mapper.MappedFieldType; +import org.elasticsearch.search.aggregations.AggregationBuilder; import org.elasticsearch.search.aggregations.Aggregator; import org.elasticsearch.search.aggregations.AggregatorTestCase; import org.elasticsearch.search.aggregations.metrics.InternalTDigestPercentiles; import org.elasticsearch.search.aggregations.metrics.PercentilesAggregationBuilder; +import org.elasticsearch.search.aggregations.metrics.PercentilesConfig; import org.elasticsearch.search.aggregations.metrics.PercentilesMethod; import org.elasticsearch.search.aggregations.metrics.TDigestState; import org.elasticsearch.search.aggregations.support.AggregationInspectionHelper; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; +import org.elasticsearch.xpack.analytics.aggregations.metrics.AnalyticsPercentilesAggregatorFactory; +import org.elasticsearch.xpack.analytics.aggregations.support.AnalyticsValuesSourceType; +import org.junit.BeforeClass; import java.io.IOException; import java.util.Collection; import java.util.Iterator; +import java.util.List; import java.util.function.Consumer; import static java.util.Collections.singleton; public class TDigestPreAggregatedPercentilesAggregatorTests extends AggregatorTestCase { - private BinaryDocValuesField getDocValue(String fieldName, double[] values) throws IOException { + @BeforeClass + public static void registerBuilder() { + AnalyticsPercentilesAggregatorFactory.registerPercentilesAggregator(valuesSourceRegistry); + } + + @Override + protected AggregationBuilder createAggBuilderForTypeTest(MappedFieldType fieldType, String fieldName) { + return new PercentilesAggregationBuilder("tdigest_percentiles") + .field(fieldName) + .percentilesConfig(new PercentilesConfig.TDigest()); + } + + @Override + protected List getSupportedValuesSourceTypes() { + // Note: this is the same list as Core, plus Analytics + return List.of(CoreValuesSourceType.NUMERIC, + CoreValuesSourceType.DATE, + CoreValuesSourceType.BOOLEAN, + AnalyticsValuesSourceType.HISTOGRAM); + } + + private BinaryDocValuesField getDocValue(String fieldName, double[] values) throws IOException { TDigest histogram = new TDigestState(100.0); //default for (double value : values) { histogram.add(value); @@ -51,7 +80,7 @@ private BinaryDocValuesField getDocValue(String fieldName, double[] values) thro streamOutput.writeDouble(centroid.mean()); } return new BinaryDocValuesField(fieldName, streamOutput.bytes().toBytesRef()); - } + } public void testNoMatchingField() throws IOException { testCase(new MatchAllDocsQuery(), iw -> { diff --git a/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregatorTests.java b/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregatorTests.java index 5e565cc833c50..8d09f336d0bf7 100644 --- a/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregatorTests.java +++ b/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregatorTests.java @@ -27,6 +27,9 @@ import org.elasticsearch.search.aggregations.bucket.terms.TermsAggregationBuilder; import org.elasticsearch.search.aggregations.bucket.terms.TermsAggregator; import org.elasticsearch.search.aggregations.support.ValueType; +import org.junit.BeforeClass; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; +import org.elasticsearch.search.lookup.LeafDocLookup; import java.io.IOException; import java.util.Arrays; @@ -37,6 +40,13 @@ public class StringStatsAggregatorTests extends AggregatorTestCase { + @BeforeClass() + public static void registerBuilder() { + StringStatsAggregationBuilder.registerAggregators(valuesSourceRegistry); + } + private static final String VALUE_SCRIPT_NAME = "value_script"; + private static final String FIELD_SCRIPT_NAME = "field_script"; + private void testCase(Query query, CheckedConsumer buildIndex, Consumer verify) throws IOException { @@ -207,9 +217,10 @@ public void testNestedAggregation() throws IOException { textFieldType.setName("text"); textFieldType.setFielddata(true); - TermsAggregationBuilder aggregationBuilder = new TermsAggregationBuilder("terms", ValueType.NUMERIC) + TermsAggregationBuilder aggregationBuilder = new TermsAggregationBuilder("terms") + .userValueTypeHint(ValueType.NUMERIC) .field("value") - .subAggregation(new StringStatsAggregationBuilder("text_stats").field("text").valueType(ValueType.STRING)); + .subAggregation(new StringStatsAggregationBuilder("text_stats").field("text").userValueTypeHint(ValueType.STRING)); Directory directory = newDirectory(); RandomIndexWriter indexWriter = new RandomIndexWriter(random(), directory); diff --git a/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/topmetrics/TopMetricsAggregatorTests.java b/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/topmetrics/TopMetricsAggregatorTests.java index d1d624ab7e68c..aa77d5c6418e2 100644 --- a/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/topmetrics/TopMetricsAggregatorTests.java +++ b/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/topmetrics/TopMetricsAggregatorTests.java @@ -55,7 +55,6 @@ import org.elasticsearch.search.aggregations.bucket.terms.Terms; import org.elasticsearch.search.aggregations.bucket.terms.TermsAggregationBuilder; import org.elasticsearch.search.aggregations.support.MultiValuesSourceFieldConfig; -import org.elasticsearch.search.aggregations.support.ValueType; import org.elasticsearch.search.internal.SearchContext; import org.elasticsearch.search.sort.FieldSortBuilder; import org.elasticsearch.search.sort.GeoDistanceSortBuilder; @@ -283,14 +282,14 @@ private InternalTopMetrics collectFromNewYorkAndLA(TopMetricsAggregationBuilder public void testSortByGeoDistancDescending() throws IOException { TopMetricsAggregationBuilder builder = simpleBuilder(new GeoDistanceSortBuilder("s", 35.7796, 78.6382).order(SortOrder.DESC)); - InternalTopMetrics result = collectFromNewYorkAndLA(builder); + InternalTopMetrics result = collectFromNewYorkAndLA(builder); assertThat(result.getSortOrder(), equalTo(SortOrder.DESC)); assertThat(result.getTopMetrics(), equalTo(singletonList(top(1.2054632268631617E7, 3.0)))); } public void testSortByGeoDistanceAscending() throws IOException { TopMetricsAggregationBuilder builder = simpleBuilder(new GeoDistanceSortBuilder("s", 35.7796, 78.6382).order(SortOrder.ASC)); - InternalTopMetrics result = collectFromNewYorkAndLA(builder); + InternalTopMetrics result = collectFromNewYorkAndLA(builder); assertThat(result.getSortOrder(), equalTo(SortOrder.ASC)); assertThat(result.getTopMetrics(), equalTo(singletonList(top(1.1062351376961706E7, 2.0)))); } @@ -305,7 +304,7 @@ public void testSortByGeoDistanceTwoHits() throws IOException { public void testInsideTerms() throws IOException { TopMetricsAggregationBuilder builder = simpleBuilder(new FieldSortBuilder("s").order(SortOrder.ASC)); - TermsAggregationBuilder terms = new TermsAggregationBuilder("terms", ValueType.DOUBLE).field("c").subAggregation(builder); + TermsAggregationBuilder terms = new TermsAggregationBuilder("terms").field("c").subAggregation(builder); Terms result = (Terms) collect(terms, new MatchAllDocsQuery(), writer -> { writer.addDocument(Arrays.asList(doubleField("c", 1.0), doubleField("s", 1.0), doubleField("m", 2.0))); writer.addDocument(Arrays.asList(doubleField("c", 1.0), doubleField("s", 2.0), doubleField("m", 3.0))); diff --git a/x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/rollup/RollupField.java b/x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/rollup/RollupField.java index a6a1feaf133f5..f22349008e0f2 100644 --- a/x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/rollup/RollupField.java +++ b/x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/rollup/RollupField.java @@ -70,7 +70,7 @@ public class RollupField { * @param extra The type of value this field is (VALUE, INTERVAL, etc) * @return formatted field name */ - public static String formatFieldName(ValuesSourceAggregationBuilder source, String extra) { + public static String formatFieldName(ValuesSourceAggregationBuilder source, String extra) { return source.field() + "." + source.getType() + "." + extra; } diff --git a/x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/security/authz/accesscontrol/DocumentSubsetBitsetCacheTests.java b/x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/security/authz/accesscontrol/DocumentSubsetBitsetCacheTests.java index 48ac7bea6a910..adcdb6f323aa1 100644 --- a/x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/security/authz/accesscontrol/DocumentSubsetBitsetCacheTests.java +++ b/x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/security/authz/accesscontrol/DocumentSubsetBitsetCacheTests.java @@ -524,7 +524,7 @@ private TestIndexContext testIndex(MapperService mapperService, Client client) t final QueryShardContext shardContext = new QueryShardContext(shardId.id(), indexSettings, BigArrays.NON_RECYCLING_INSTANCE, null, null, mapperService, null, null, xContentRegistry(), writableRegistry(), - client, new IndexSearcher(directoryReader), () -> nowInMillis, null, null, () -> true); + client, new IndexSearcher(directoryReader), () -> nowInMillis, null, null, () -> true, null); context = new TestIndexContext(directory, iw, directoryReader, shardContext, leaf); return context; diff --git a/x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/security/authz/accesscontrol/SecurityIndexReaderWrapperIntegrationTests.java b/x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/security/authz/accesscontrol/SecurityIndexReaderWrapperIntegrationTests.java index 422e5dd0128ba..e7a29c7083824 100644 --- a/x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/security/authz/accesscontrol/SecurityIndexReaderWrapperIntegrationTests.java +++ b/x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/security/authz/accesscontrol/SecurityIndexReaderWrapperIntegrationTests.java @@ -93,7 +93,7 @@ public void testDLS() throws Exception { final long nowInMillis = randomNonNegativeLong(); QueryShardContext realQueryShardContext = new QueryShardContext(shardId.id(), indexSettings, BigArrays.NON_RECYCLING_INSTANCE, null, null, mapperService, null, null, xContentRegistry(), writableRegistry(), - client, null, () -> nowInMillis, null, null, () -> true); + client, null, () -> nowInMillis, null, null, () -> true, null); QueryShardContext queryShardContext = spy(realQueryShardContext); DocumentSubsetBitsetCache bitsetCache = new DocumentSubsetBitsetCache(Settings.EMPTY, Executors.newSingleThreadExecutor()); XPackLicenseState licenseState = mock(XPackLicenseState.class); @@ -227,7 +227,7 @@ public void testDLSWithLimitedPermissions() throws Exception { final long nowInMillis = randomNonNegativeLong(); QueryShardContext realQueryShardContext = new QueryShardContext(shardId.id(), indexSettings, BigArrays.NON_RECYCLING_INSTANCE, null, null, mapperService, null, null, xContentRegistry(), writableRegistry(), - client, null, () -> nowInMillis, null, null, () -> true); + client, null, () -> nowInMillis, null, null, () -> true, null); QueryShardContext queryShardContext = spy(realQueryShardContext); DocumentSubsetBitsetCache bitsetCache = new DocumentSubsetBitsetCache(Settings.EMPTY, Executors.newSingleThreadExecutor()); diff --git a/x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/transform/transforms/MockDeprecatedAggregationBuilder.java b/x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/transform/transforms/MockDeprecatedAggregationBuilder.java index d02c0e994c969..fe224728209a7 100644 --- a/x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/transform/transforms/MockDeprecatedAggregationBuilder.java +++ b/x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/transform/transforms/MockDeprecatedAggregationBuilder.java @@ -17,16 +17,15 @@ import org.elasticsearch.search.aggregations.AggregatorFactories.Builder; import org.elasticsearch.search.aggregations.AggregatorFactory; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; -import org.elasticsearch.search.aggregations.support.ValueType; -import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.util.Map; -public class MockDeprecatedAggregationBuilder extends ValuesSourceAggregationBuilder { +public class MockDeprecatedAggregationBuilder extends ValuesSourceAggregationBuilder { public static final String NAME = "deprecated_agg"; public static final String DEPRECATION_MESSAGE = "expected deprecation message from MockDeprecatedAggregationBuilder"; @@ -40,19 +39,24 @@ protected MockDeprecatedAggregationBuilder(MockDeprecatedAggregationBuilder clon } @Override - protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metadata) { - return new MockDeprecatedAggregationBuilder(this, factoriesBuilder, metadata); + protected ValuesSourceType defaultValueSourceType() { + return CoreValuesSourceType.BYTES; + } + + @Override + protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metaData) { + return new MockDeprecatedAggregationBuilder(this, factoriesBuilder, metaData); } public MockDeprecatedAggregationBuilder() { - super(NAME, CoreValuesSourceType.NUMERIC, ValueType.NUMERIC); + super(NAME); } /** * Read from a stream. */ protected MockDeprecatedAggregationBuilder(StreamInput in) throws IOException { - super(in, null, null); + super(in); } @Override @@ -70,10 +74,10 @@ public BucketCardinality bucketCardinality() { } @Override - protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, - ValuesSourceConfig config, - AggregatorFactory parent, - Builder subFactoriesBuilder) throws IOException { + protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardContext, + ValuesSourceConfig config, + AggregatorFactory parent, + Builder subFactoriesBuilder) throws IOException { return null; } diff --git a/x-pack/plugin/mapper-constant-keyword/src/main/java/org/elasticsearch/xpack/constantkeyword/mapper/ConstantKeywordFieldMapper.java b/x-pack/plugin/mapper-constant-keyword/src/main/java/org/elasticsearch/xpack/constantkeyword/mapper/ConstantKeywordFieldMapper.java index 115968b69ee65..9ffcdbcfdc21f 100644 --- a/x-pack/plugin/mapper-constant-keyword/src/main/java/org/elasticsearch/xpack/constantkeyword/mapper/ConstantKeywordFieldMapper.java +++ b/x-pack/plugin/mapper-constant-keyword/src/main/java/org/elasticsearch/xpack/constantkeyword/mapper/ConstantKeywordFieldMapper.java @@ -43,6 +43,8 @@ import org.elasticsearch.index.mapper.ParseContext; import org.elasticsearch.index.mapper.TypeParsers; import org.elasticsearch.index.query.QueryShardContext; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; /** * A {@link FieldMapper} that assigns every document the same value. @@ -248,6 +250,10 @@ public Query regexpQuery(String value, int flags, int maxDeterminizedStates, } } + @Override + public ValuesSourceType getValuesSourceType() { + return CoreValuesSourceType.BYTES; + } } ConstantKeywordFieldMapper(String simpleName, MappedFieldType fieldType, MappedFieldType defaultFieldType, diff --git a/x-pack/plugin/mapper-flattened/src/main/java/org/elasticsearch/xpack/flattened/mapper/FlatObjectFieldMapper.java b/x-pack/plugin/mapper-flattened/src/main/java/org/elasticsearch/xpack/flattened/mapper/FlatObjectFieldMapper.java index 9bbb2a5d6d6c1..5d5ed7bd48e8e 100644 --- a/x-pack/plugin/mapper-flattened/src/main/java/org/elasticsearch/xpack/flattened/mapper/FlatObjectFieldMapper.java +++ b/x-pack/plugin/mapper-flattened/src/main/java/org/elasticsearch/xpack/flattened/mapper/FlatObjectFieldMapper.java @@ -53,10 +53,10 @@ import org.elasticsearch.indices.breaker.CircuitBreakerService; import org.elasticsearch.search.DocValueFormat; import org.elasticsearch.search.MultiValueMode; -import org.elasticsearch.search.sort.BucketedSort; -import org.elasticsearch.search.sort.SortOrder; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; import org.elasticsearch.search.aggregations.support.ValuesSourceType; +import org.elasticsearch.search.sort.BucketedSort; +import org.elasticsearch.search.sort.SortOrder; import java.io.IOException; import java.util.Iterator; @@ -541,6 +541,11 @@ public IndexFieldData.Builder fielddataBuilder(String fullyQualifiedIndexName) { failIfNoDocValues(); return new DocValuesIndexFieldData.Builder(); } + + @Override + public ValuesSourceType getValuesSourceType() { + return CoreValuesSourceType.BYTES; + } } private FlatObjectFieldParser fieldParser; diff --git a/x-pack/plugin/ml/src/main/java/org/elasticsearch/xpack/ml/datafeed/extractor/aggregation/RollupDataExtractorFactory.java b/x-pack/plugin/ml/src/main/java/org/elasticsearch/xpack/ml/datafeed/extractor/aggregation/RollupDataExtractorFactory.java index d66936957e071..a8e2b70863ebe 100644 --- a/x-pack/plugin/ml/src/main/java/org/elasticsearch/xpack/ml/datafeed/extractor/aggregation/RollupDataExtractorFactory.java +++ b/x-pack/plugin/ml/src/main/java/org/elasticsearch/xpack/ml/datafeed/extractor/aggregation/RollupDataExtractorFactory.java @@ -118,7 +118,7 @@ public static void create(Client client, ); return; } - final List> flattenedAggs = new ArrayList<>(); + final List> flattenedAggs = new ArrayList<>(); flattenAggregations(datafeed.getParsedAggregations(xContentRegistry) .getAggregatorFactories(), datafeedHistogramAggregation, flattenedAggs); @@ -149,7 +149,7 @@ private static boolean validInterval(long datafeedInterval, ParsedRollupCaps rol private static void flattenAggregations(final Collection datafeedAggregations, final AggregationBuilder datafeedHistogramAggregation, - final List> flattenedAggregations) { + final List> flattenedAggregations) { for (AggregationBuilder aggregationBuilder : datafeedAggregations) { if (aggregationBuilder.equals(datafeedHistogramAggregation) == false) { flattenedAggregations.add((ValuesSourceAggregationBuilder)aggregationBuilder); @@ -158,8 +158,8 @@ private static void flattenAggregations(final Collection dat } } - private static boolean hasAggregations(ParsedRollupCaps rollupCaps, List> datafeedAggregations) { - for (ValuesSourceAggregationBuilder aggregationBuilder : datafeedAggregations) { + private static boolean hasAggregations(ParsedRollupCaps rollupCaps, List> datafeedAggregations) { + for (ValuesSourceAggregationBuilder aggregationBuilder : datafeedAggregations) { String type = aggregationBuilder.getType(); String field = aggregationBuilder.field(); if (aggregationBuilder instanceof TermsAggregationBuilder) { diff --git a/x-pack/plugin/rollup/src/main/java/org/elasticsearch/xpack/rollup/RollupRequestTranslator.java b/x-pack/plugin/rollup/src/main/java/org/elasticsearch/xpack/rollup/RollupRequestTranslator.java index ba74d16397f1a..aa39b0a44f6cc 100644 --- a/x-pack/plugin/rollup/src/main/java/org/elasticsearch/xpack/rollup/RollupRequestTranslator.java +++ b/x-pack/plugin/rollup/src/main/java/org/elasticsearch/xpack/rollup/RollupRequestTranslator.java @@ -319,7 +319,10 @@ private static List translateTerms(TermsAggregationBuilder s return translateVSAggBuilder(source, registry, () -> { TermsAggregationBuilder rolledTerms - = new TermsAggregationBuilder(source.getName(), source.valueType()); + = new TermsAggregationBuilder(source.getName()); + if (source.userValueTypeHint() != null) { + rolledTerms.userValueTypeHint(source.userValueTypeHint()); + } rolledTerms.field(RollupField.formatFieldName(source, RollupField.VALUE)); rolledTerms.includeExclude(source.includeExclude()); if (source.collectMode() != null) { diff --git a/x-pack/plugin/rollup/src/main/java/org/elasticsearch/xpack/rollup/job/RollupIndexer.java b/x-pack/plugin/rollup/src/main/java/org/elasticsearch/xpack/rollup/job/RollupIndexer.java index cc3f0afc5e8dd..448e495746bd6 100644 --- a/x-pack/plugin/rollup/src/main/java/org/elasticsearch/xpack/rollup/job/RollupIndexer.java +++ b/x-pack/plugin/rollup/src/main/java/org/elasticsearch/xpack/rollup/job/RollupIndexer.java @@ -298,16 +298,17 @@ static List createAggregationBuilders(final List countBuilder = new ValueCountAggregationBuilder( - formatFieldName(field, AvgAggregationBuilder.NAME, RollupField.COUNT_FIELD), ValueType.NUMERIC); + formatFieldName(field, AvgAggregationBuilder.NAME, RollupField.COUNT_FIELD)); countBuilder.field(field); builders.add(countBuilder); } else if (metric.equals(MetricConfig.SUM.getPreferredName())) { newBuilder = new SumAggregationBuilder(formatFieldName(field, SumAggregationBuilder.NAME, RollupField.VALUE)); } else if (metric.equals(MetricConfig.VALUE_COUNT.getPreferredName())) { // TODO allow non-numeric value_counts. - // Hardcoding this is fine for now since the job validation guarantees that all metric fields are numerics + // I removed the hard coding of NUMERIC as part of cleaning up targetValueType, but I don't think that resolves + // the above to do note -- Tozzi 2019-12-06 newBuilder = new ValueCountAggregationBuilder( - formatFieldName(field, ValueCountAggregationBuilder.NAME, RollupField.VALUE), ValueType.NUMERIC); + formatFieldName(field, ValueCountAggregationBuilder.NAME, RollupField.VALUE)); } else { throw new IllegalArgumentException("Unsupported metric type [" + metric + "]"); } diff --git a/x-pack/plugin/rollup/src/test/java/org/elasticsearch/xpack/rollup/RollupJobIdentifierUtilTests.java b/x-pack/plugin/rollup/src/test/java/org/elasticsearch/xpack/rollup/RollupJobIdentifierUtilTests.java index 555b07a0b3987..205c25ed60ce6 100644 --- a/x-pack/plugin/rollup/src/test/java/org/elasticsearch/xpack/rollup/RollupJobIdentifierUtilTests.java +++ b/x-pack/plugin/rollup/src/test/java/org/elasticsearch/xpack/rollup/RollupJobIdentifierUtilTests.java @@ -308,7 +308,7 @@ public void testComparableNoTermsVsTerms() { DateHistogramAggregationBuilder builder = new DateHistogramAggregationBuilder("foo").field("foo") .calendarInterval(new DateHistogramInterval("1h")) - .subAggregation(new TermsAggregationBuilder("histo", ValueType.STRING).field("bar")); + .subAggregation(new TermsAggregationBuilder("histo").userValueTypeHint(ValueType.STRING).field("bar")); Set caps = new HashSet<>(2); caps.add(cap); diff --git a/x-pack/plugin/rollup/src/test/java/org/elasticsearch/xpack/rollup/RollupRequestTranslationTests.java b/x-pack/plugin/rollup/src/test/java/org/elasticsearch/xpack/rollup/RollupRequestTranslationTests.java index ea5a81cf7178b..2d6e0f97ca8c0 100644 --- a/x-pack/plugin/rollup/src/test/java/org/elasticsearch/xpack/rollup/RollupRequestTranslationTests.java +++ b/x-pack/plugin/rollup/src/test/java/org/elasticsearch/xpack/rollup/RollupRequestTranslationTests.java @@ -62,7 +62,7 @@ public void testBasicDateHisto() { .extendedBounds(new ExtendedBounds(0L, 1000L)) .subAggregation(new MaxAggregationBuilder("the_max").field("max_field")) .subAggregation(new AvgAggregationBuilder("the_avg").field("avg_field")); - + List translated = translateAggregation(histo, namedWriteableRegistry); assertThat(translated.size(), equalTo(1)); assertThat(translated.get(0), Matchers.instanceOf(DateHistogramAggregationBuilder.class)); @@ -302,7 +302,7 @@ public void testAvgMetric() { public void testStringTerms() throws IOException { - TermsAggregationBuilder terms = new TermsAggregationBuilder("test_string_terms", ValueType.STRING); + TermsAggregationBuilder terms = new TermsAggregationBuilder("test_string_terms").userValueTypeHint(ValueType.STRING); terms.field("foo") .subAggregation(new MaxAggregationBuilder("the_max").field("max_field")) .subAggregation(new AvgAggregationBuilder("the_avg").field("avg_field")); diff --git a/x-pack/plugin/rollup/src/test/java/org/elasticsearch/xpack/rollup/RollupResponseTranslationTests.java b/x-pack/plugin/rollup/src/test/java/org/elasticsearch/xpack/rollup/RollupResponseTranslationTests.java index 95386e18cf480..32aa68e6216eb 100644 --- a/x-pack/plugin/rollup/src/test/java/org/elasticsearch/xpack/rollup/RollupResponseTranslationTests.java +++ b/x-pack/plugin/rollup/src/test/java/org/elasticsearch/xpack/rollup/RollupResponseTranslationTests.java @@ -448,7 +448,7 @@ public void testUnsupportedMultiBucket() throws IOException { .must(QueryBuilders.termQuery("field", "foo")) .should(QueryBuilders.termQuery("field", "bar")); SignificantTermsAggregationBuilder builder = new SignificantTermsAggregationBuilder( - "test", ValueType.STRING) + "test") .field("field") .backgroundFilter(filter); @@ -1015,9 +1015,11 @@ public void testMetric() throws IOException { public void testUnsupportedMetric() throws IOException { - AggregationBuilder nonRollup = new CardinalityAggregationBuilder("test_metric", ValueType.LONG).field("foo"); + AggregationBuilder nonRollup = new CardinalityAggregationBuilder("test_metric").userValueTypeHint(ValueType.LONG) + .field("foo"); String fieldName = "foo.max." + RollupField.VALUE; - AggregationBuilder rollup = new CardinalityAggregationBuilder("test_metric", ValueType.LONG).field(fieldName); + AggregationBuilder rollup = new CardinalityAggregationBuilder("test_metric").userValueTypeHint(ValueType.LONG) + .field(fieldName); NumberFieldMapper.Builder nrValueMapper = new NumberFieldMapper.Builder("foo", NumberFieldMapper.NumberType.LONG); @@ -1049,10 +1051,10 @@ public void testUnsupportedMetric() throws IOException { } public void testStringTerms() throws IOException { - TermsAggregationBuilder nonRollupTerms = new TermsAggregationBuilder("terms", ValueType.STRING) + TermsAggregationBuilder nonRollupTerms = new TermsAggregationBuilder("terms").userValueTypeHint(ValueType.STRING) .field("stringField"); - TermsAggregationBuilder rollupTerms = new TermsAggregationBuilder("terms", ValueType.STRING) + TermsAggregationBuilder rollupTerms = new TermsAggregationBuilder("terms").userValueTypeHint(ValueType.STRING) .field("stringfield.terms." + RollupField.VALUE) .subAggregation(new SumAggregationBuilder("terms." + RollupField.COUNT_FIELD) .field("stringfield.terms." + RollupField.COUNT_FIELD)); @@ -1090,10 +1092,10 @@ public void testStringTerms() throws IOException { } public void testStringTermsNullValue() throws IOException { - TermsAggregationBuilder nonRollupTerms = new TermsAggregationBuilder("terms", ValueType.STRING) + TermsAggregationBuilder nonRollupTerms = new TermsAggregationBuilder("terms").userValueTypeHint(ValueType.STRING) .field("stringField"); - TermsAggregationBuilder rollupTerms = new TermsAggregationBuilder("terms", ValueType.STRING) + TermsAggregationBuilder rollupTerms = new TermsAggregationBuilder("terms").userValueTypeHint(ValueType.STRING) .field("stringfield.terms." + RollupField.VALUE) .subAggregation(new SumAggregationBuilder("terms." + RollupField.COUNT_FIELD) .field("stringfield.terms." + RollupField.COUNT_FIELD)); @@ -1138,10 +1140,10 @@ public void testStringTermsNullValue() throws IOException { } public void testLongTerms() throws IOException { - TermsAggregationBuilder nonRollupTerms = new TermsAggregationBuilder("terms", ValueType.LONG) + TermsAggregationBuilder nonRollupTerms = new TermsAggregationBuilder("terms").userValueTypeHint(ValueType.LONG) .field("longField"); - TermsAggregationBuilder rollupTerms = new TermsAggregationBuilder("terms", ValueType.LONG) + TermsAggregationBuilder rollupTerms = new TermsAggregationBuilder("terms").userValueTypeHint(ValueType.LONG) .field("longfield.terms." + RollupField.VALUE) .subAggregation(new SumAggregationBuilder("terms." + RollupField.COUNT_FIELD) .field("longfield.terms." + RollupField.COUNT_FIELD)); diff --git a/x-pack/plugin/rollup/src/test/java/org/elasticsearch/xpack/rollup/job/RollupIndexerIndexingTests.java b/x-pack/plugin/rollup/src/test/java/org/elasticsearch/xpack/rollup/job/RollupIndexerIndexingTests.java index 341160917a19e..16e5a3eeffad1 100644 --- a/x-pack/plugin/rollup/src/test/java/org/elasticsearch/xpack/rollup/job/RollupIndexerIndexingTests.java +++ b/x-pack/plugin/rollup/src/test/java/org/elasticsearch/xpack/rollup/job/RollupIndexerIndexingTests.java @@ -93,7 +93,7 @@ private void setup() { settings = createIndexSettings(); queryShardContext = new QueryShardContext(0, settings, BigArrays.NON_RECYCLING_INSTANCE, null, null, null, null, null, - null, null, null, null, () -> 0L, null, null, () -> true); + null, null, null, null, () -> 0L, null, null, () -> true, null); } public void testSimpleDateHisto() throws Exception { diff --git a/x-pack/plugin/transform/src/main/java/org/elasticsearch/xpack/transform/transforms/pivot/Aggregations.java b/x-pack/plugin/transform/src/main/java/org/elasticsearch/xpack/transform/transforms/pivot/Aggregations.java index 81a78cd465ac3..7a6261505009e 100644 --- a/x-pack/plugin/transform/src/main/java/org/elasticsearch/xpack/transform/transforms/pivot/Aggregations.java +++ b/x-pack/plugin/transform/src/main/java/org/elasticsearch/xpack/transform/transforms/pivot/Aggregations.java @@ -184,7 +184,7 @@ public static Tuple, Map> getAggregationInpu } if (agg instanceof ValuesSourceAggregationBuilder) { - ValuesSourceAggregationBuilder valueSourceAggregation = (ValuesSourceAggregationBuilder) agg; + ValuesSourceAggregationBuilder valueSourceAggregation = (ValuesSourceAggregationBuilder) agg; return new Tuple<>( Collections.singletonMap(valueSourceAggregation.getName(), valueSourceAggregation.field()), Collections.singletonMap(agg.getName(), agg.getType()) diff --git a/x-pack/plugin/wildcard/src/main/java/org/elasticsearch/xpack/wildcard/mapper/WildcardFieldMapper.java b/x-pack/plugin/wildcard/src/main/java/org/elasticsearch/xpack/wildcard/mapper/WildcardFieldMapper.java index 568e4f502dbe0..ffb3a81aeaf67 100644 --- a/x-pack/plugin/wildcard/src/main/java/org/elasticsearch/xpack/wildcard/mapper/WildcardFieldMapper.java +++ b/x-pack/plugin/wildcard/src/main/java/org/elasticsearch/xpack/wildcard/mapper/WildcardFieldMapper.java @@ -59,6 +59,8 @@ import org.elasticsearch.index.similarity.SimilarityProvider; import org.elasticsearch.indices.breaker.CircuitBreakerService; import org.elasticsearch.search.MultiValueMode; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.nio.charset.StandardCharsets; @@ -76,15 +78,15 @@ public class WildcardFieldMapper extends FieldMapper { public static final String CONTENT_TYPE = "wildcard"; - public static short MAX_CLAUSES_IN_APPROXIMATION_QUERY = 10; - public static final int NGRAM_SIZE = 3; + public static short MAX_CLAUSES_IN_APPROXIMATION_QUERY = 10; + public static final int NGRAM_SIZE = 3; static final NamedAnalyzer WILDCARD_ANALYZER = new NamedAnalyzer("_wildcard", AnalyzerScope.GLOBAL, new Analyzer() { @Override public TokenStreamComponents createComponents(String fieldName) { Tokenizer tokenizer = new NGramTokenizer(NGRAM_SIZE, NGRAM_SIZE); return new TokenStreamComponents(tokenizer); } - }); + }); public static class Defaults { public static final MappedFieldType FIELD_TYPE = new WildcardFieldType(); @@ -98,14 +100,14 @@ public static class Defaults { FIELD_TYPE.setOmitNorms(true); FIELD_TYPE.freeze(); } - public static final int IGNORE_ABOVE = Integer.MAX_VALUE; + public static final int IGNORE_ABOVE = Integer.MAX_VALUE; } public static class Builder extends FieldMapper.Builder { protected int ignoreAbove = Defaults.IGNORE_ABOVE; private IndexAnalyzers indexAnalyzers; private String normalizerName; - + public Builder(String name) { super(name, Defaults.FIELD_TYPE, Defaults.FIELD_TYPE); @@ -115,7 +117,7 @@ public Builder(String name) { @Override public Builder docValues(boolean docValues) { if (docValues == false) { - throw new MapperParsingException("The field [" + name + "] cannot have doc values = false"); + throw new MapperParsingException("The field [" + name + "] cannot have doc values = false"); } return this; } @@ -131,64 +133,64 @@ public Builder indexOptions(IndexOptions indexOptions) { @Override public Builder store(boolean store) { if (store) { - throw new MapperParsingException("The field [" + name + "] cannot have store = true"); + throw new MapperParsingException("The field [" + name + "] cannot have store = true"); } return this; } @Override public Builder similarity(SimilarityProvider similarity) { - throw new MapperParsingException("The field [" + name + "] cannot have custom similarities"); + throw new MapperParsingException("The field [" + name + "] cannot have custom similarities"); } @Override public Builder index(boolean index) { if (index == false) { - throw new MapperParsingException("The field [" + name + "] cannot have index = false"); + throw new MapperParsingException("The field [" + name + "] cannot have index = false"); } return this; } - + public Builder ignoreAbove(int ignoreAbove) { if (ignoreAbove < 0) { throw new IllegalArgumentException("[ignore_above] must be positive, got " + ignoreAbove); } this.ignoreAbove = ignoreAbove; return this; - } - - + } + + @Override protected void setupFieldType(BuilderContext context) { super.setupFieldType(context); fieldType().setHasDocValues(true); fieldType().setTokenized(false); - fieldType().setIndexOptions(IndexOptions.DOCS); + fieldType().setIndexOptions(IndexOptions.DOCS); } @Override public WildcardFieldType fieldType() { return (WildcardFieldType) super.fieldType(); } - + public Builder normalizer(IndexAnalyzers indexAnalyzers, String name) { this.indexAnalyzers = indexAnalyzers; this.normalizerName = name; return builder; - } + } @Override public WildcardFieldMapper build(BuilderContext context) { - setupFieldType(context); + setupFieldType(context); if (normalizerName != null) { NamedAnalyzer normalizer = indexAnalyzers.getNormalizer(normalizerName); if (normalizer == null) { throw new MapperParsingException("normalizer [" + normalizerName + "] not found for field [" + name + "]"); } fieldType().setNormalizer(normalizer); - } + } return new WildcardFieldMapper( - name, fieldType, defaultFieldType, ignoreAbove, + name, fieldType, defaultFieldType, ignoreAbove, context.indexSettings(), multiFieldsBuilder.build(this, context), copyTo); } } @@ -199,7 +201,7 @@ public static class TypeParser implements Mapper.TypeParser { throws MapperParsingException { WildcardFieldMapper.Builder builder = new WildcardFieldMapper.Builder(name); parseField(builder, name, node, parserContext); - + for (Iterator> iterator = node.entrySet().iterator(); iterator.hasNext();) { Map.Entry entry = iterator.next(); String propName = entry.getKey(); @@ -213,21 +215,21 @@ public static class TypeParser implements Mapper.TypeParser { } iterator.remove(); } - } - + } + return builder; } } - - public static final char TOKEN_START_OR_END_CHAR = 0; - + + public static final char TOKEN_START_OR_END_CHAR = 0; + public static final class WildcardFieldType extends MappedFieldType { - + private NamedAnalyzer normalizer = null; - public WildcardFieldType() { + public WildcardFieldType() { setIndexAnalyzer(Lucene.KEYWORD_ANALYZER); - setSearchAnalyzer(Lucene.KEYWORD_ANALYZER); + setSearchAnalyzer(Lucene.KEYWORD_ANALYZER); } protected WildcardFieldType(WildcardFieldType ref) { @@ -239,7 +241,7 @@ public WildcardFieldType clone() { WildcardFieldType result = new WildcardFieldType(this); return result; } - + @Override public boolean equals(Object o) { @@ -253,8 +255,8 @@ public boolean equals(Object o) { @Override public int hashCode() { return 31 * super.hashCode() + Objects.hash(normalizer); - } - + } + private NamedAnalyzer normalizer() { return normalizer; } @@ -262,7 +264,7 @@ private NamedAnalyzer normalizer() { public void setNormalizer(NamedAnalyzer normalizer) { checkIfFrozen(); this.normalizer = normalizer; - } + } @Override public void checkCompatibility(MappedFieldType otherFT, List conflicts) { @@ -272,22 +274,22 @@ public void checkCompatibility(MappedFieldType otherFT, List conflicts) conflicts.add("mapper [" + name() + "] has different [normalizer]"); } } - + // Holds parsed information about the wildcard pattern static class PatternStructure { - boolean openStart, openEnd, hasSymbols; + boolean openStart, openEnd, hasSymbols; int lastGap =0; int wildcardCharCount, wildcardStringCount; String[] fragments; Integer [] precedingGapSizes; final String pattern; - + @SuppressWarnings("fallthrough") // Intentionally uses fallthrough mirroring implementation in Lucene's WildcardQuery PatternStructure (String wildcardText) { this.pattern = wildcardText; ArrayList fragmentList = new ArrayList<>(); ArrayList precedingGapSizeList = new ArrayList<>(); - StringBuilder sb = new StringBuilder(); + StringBuilder sb = new StringBuilder(); for (int i = 0; i < wildcardText.length();) { final int c = wildcardText.codePointAt(i); int length = Character.charCount(c); @@ -296,10 +298,10 @@ static class PatternStructure { if (i == 0) { openStart = true; } - openEnd = true; + openEnd = true; hasSymbols = true; wildcardStringCount++; - + if (sb.length() > 0) { precedingGapSizeList.add(lastGap); fragmentList.add(sb.toString()); @@ -317,10 +319,10 @@ static class PatternStructure { if (sb.length() > 0) { precedingGapSizeList.add(lastGap); fragmentList.add(sb.toString()); - sb = new StringBuilder(); + sb = new StringBuilder(); lastGap = 0; } - + if (lastGap != Integer.MAX_VALUE) { lastGap++; } @@ -347,19 +349,19 @@ static class PatternStructure { } fragments = fragmentList.toArray(new String[0]); precedingGapSizes = precedingGapSizeList.toArray(new Integer[0]); - + } - + public boolean needsVerification() { // Return true if term queries are not enough evidence if (fragments.length == 1 && wildcardCharCount == 0) { - // The one case where we don't need verification is when + // The one case where we don't need verification is when // we have a single fragment and no ? characters return false; } return true; } - + // Returns number of positions for last gap (Integer.MAX means unlimited gap) public int getPrecedingGapSize(int fragmentNum) { return precedingGapSizes[fragmentNum]; @@ -379,17 +381,17 @@ public boolean equals(Object obj) { PatternStructure other = (PatternStructure) obj; return pattern.equals(other.pattern); } - - + + } - + @Override public Query wildcardQuery(String wildcardPattern, RewriteMethod method, QueryShardContext context) { if (normalizer != null) { wildcardPattern = StringFieldType.normalizeWildcardPattern(name(), wildcardPattern, normalizer); } - PatternStructure patternStructure = new PatternStructure(wildcardPattern); + PatternStructure patternStructure = new PatternStructure(wildcardPattern); ArrayList tokens = new ArrayList<>(); for (int i = 0; i < patternStructure.fragments.length; i++) { @@ -398,7 +400,7 @@ public Query wildcardQuery(String wildcardPattern, RewriteMethod method, QuerySh if (fLength == 0) { continue; } - + // Add any start/end of string character if (i == 0 && patternStructure.openStart == false) { // Start-of-string anchored (is not a leading wildcard) @@ -411,7 +413,7 @@ public Query wildcardQuery(String wildcardPattern, RewriteMethod method, QuerySh if (fragment.codePointCount(0, fragment.length()) <= NGRAM_SIZE) { tokens.add(fragment); } else { - // Break fragment into multiple Ngrams + // Break fragment into multiple Ngrams TokenStream tokenizer = WILDCARD_ANALYZER.tokenStream(name(), fragment); CharTermAttribute termAtt = tokenizer.addAttribute(CharTermAttribute.class); String lastUnusedToken = null; @@ -425,7 +427,7 @@ public Query wildcardQuery(String wildcardPattern, RewriteMethod method, QuerySh if (takeThis) { tokens.add(tokenValue); } else { - lastUnusedToken = tokenValue; + lastUnusedToken = tokenValue; } // alternate takeThis = !takeThis; @@ -445,7 +447,7 @@ public Query wildcardQuery(String wildcardPattern, RewriteMethod method, QuerySh if (patternStructure.isMatchAll()) { return new MatchAllDocsQuery(); - } + } BooleanQuery approximation = createApproximationQuery(tokens); if (approximation.clauses().size() > 1 || patternStructure.needsVerification()) { BooleanQuery.Builder verifyingBuilder = new BooleanQuery.Builder(); @@ -455,7 +457,7 @@ public Query wildcardQuery(String wildcardPattern, RewriteMethod method, QuerySh return verifyingBuilder.build(); } return approximation; - } + } private BooleanQuery createApproximationQuery(ArrayList tokens) { BooleanQuery.Builder bqBuilder = new BooleanQuery.Builder(); @@ -473,7 +475,7 @@ private BooleanQuery createApproximationQuery(ArrayList tokens) { // TODO we can be smarter about pruning here. e.g. // * Avoid wildcard queries if there are sufficient numbers of other terms that are full 3grams that are cheaper term queries // * We can select terms on their scarcity rather than even spreads across the search string. - + return bqBuilder.build(); } @@ -504,11 +506,11 @@ public Query existsQuery(QueryShardContext context) { public Query termQuery(Object value, QueryShardContext context) { return wildcardQuery(BytesRefs.toString(value), MultiTermQuery.CONSTANT_SCORE_REWRITE, context); } - + @Override public Query prefixQuery(String value, MultiTermQuery.RewriteMethod method, QueryShardContext context) { return wildcardQuery(value + "*", method, context); - } + } @Override public Query termsQuery(List values, QueryShardContext context) { @@ -517,8 +519,8 @@ public Query termsQuery(List values, QueryShardContext context) { bq.add(termQuery(value, context), Occur.SHOULD); } return new ConstantScoreQuery(bq.build()); - } - + } + @Override public IndexFieldData.Builder fielddataBuilder(String fullyQualifiedIndexName) { failIfNoDocValues(); @@ -531,8 +533,12 @@ public IndexFieldData build(IndexSettings indexSettings, MappedFieldType fiel }}; } + @Override + public ValuesSourceType getValuesSourceType() { + return CoreValuesSourceType.BYTES; + } - String normalize(String value) throws IOException { + String normalize(String value) throws IOException { if (normalizer != null) { try (TokenStream ts = normalizer.tokenStream(name(), value)) { final CharTermAttribute termAtt = ts.addAttribute(CharTermAttribute.class); @@ -553,10 +559,10 @@ String normalize(String value) throws IOException { } } return value; - } - + } + } - + static class WildcardBytesBinaryDVIndexFieldData extends BytesBinaryDVIndexFieldData{ WildcardBytesBinaryDVIndexFieldData(Index index, String fieldName) { @@ -569,7 +575,7 @@ public SortField sortField(Object missingValue, MultiValueMode sortMode, Nested sortMode, nested); return new SortField(getFieldName(), source, reverse); } - + } private int ignoreAbove; @@ -579,9 +585,9 @@ private WildcardFieldMapper(String simpleName, MappedFieldType fieldType, Mapped super(simpleName, fieldType, defaultFieldType, indexSettings, multiFields, copyTo); this.ignoreAbove = ignoreAbove; assert fieldType.indexOptions() == IndexOptions.DOCS; - + ngramFieldType = fieldType.clone(); - ngramFieldType.setTokenized(true); + ngramFieldType.setTokenized(true); ngramFieldType.freeze(); } @@ -590,8 +596,8 @@ private WildcardFieldMapper(String simpleName, MappedFieldType fieldType, Mapped // pkg-private for testing int ignoreAbove() { return ignoreAbove; - } - + } + @Override protected WildcardFieldMapper clone() { return (WildcardFieldMapper) super.clone(); @@ -612,9 +618,9 @@ protected void doXContentBody(XContentBuilder builder, boolean includeDefaults, builder.field("normalizer", fieldType().normalizer().name()); } else if (includeDefaults) { builder.nullField("normalizer"); - } + } } - + @Override protected void parseCreateField(ParseContext context, List fields) throws IOException { final String value; @@ -629,13 +635,13 @@ protected void parseCreateField(ParseContext context, List field } } ParseContext.Document parseDoc = context.doc(); - - createFields(value, parseDoc, fields); - } - + + createFields(value, parseDoc, fields); + } + // For internal use by Lucene only - used to define ngram index final MappedFieldType ngramFieldType; - + void createFields(String value, Document parseDoc, Listfields) throws IOException { if (value == null || value.length() > ignoreAbove) { return; @@ -644,25 +650,25 @@ void createFields(String value, Document parseDoc, Listfields) t String ngramValue = TOKEN_START_OR_END_CHAR + value + TOKEN_START_OR_END_CHAR + TOKEN_START_OR_END_CHAR; Field ngramField = new Field(fieldType().name(), ngramValue, ngramFieldType); fields.add(ngramField); - + CustomBinaryDocValuesField dvField = (CustomBinaryDocValuesField) parseDoc.getByKey(fieldType().name()); if (dvField == null) { dvField = new CustomBinaryDocValuesField(fieldType().name(), value.getBytes(StandardCharsets.UTF_8)); parseDoc.addWithKey(fieldType().name(), dvField); } else { dvField.add(value.getBytes(StandardCharsets.UTF_8)); - } + } } @Override protected String contentType() { return CONTENT_TYPE; } - + @Override protected void doMerge(Mapper mergeWith) { super.doMerge(mergeWith); this.ignoreAbove = ((WildcardFieldMapper) mergeWith).ignoreAbove; - } + } } diff --git a/x-pack/plugin/wildcard/src/test/java/org/elasticsearch/xpack/wildcard/mapper/WildcardFieldMapperTests.java b/x-pack/plugin/wildcard/src/test/java/org/elasticsearch/xpack/wildcard/mapper/WildcardFieldMapperTests.java index 11116ecee0f73..c86d62740e67d 100644 --- a/x-pack/plugin/wildcard/src/test/java/org/elasticsearch/xpack/wildcard/mapper/WildcardFieldMapperTests.java +++ b/x-pack/plugin/wildcard/src/test/java/org/elasticsearch/xpack/wildcard/mapper/WildcardFieldMapperTests.java @@ -243,7 +243,7 @@ protected final QueryShardContext createMockShardContext() { }; return new QueryShardContext(0, idxSettings, BigArrays.NON_RECYCLING_INSTANCE, bitsetFilterCache, indexFieldDataLookup, null, null, null, xContentRegistry(), null, null, null, - () -> randomNonNegativeLong(), null, null, () -> true) { + () -> randomNonNegativeLong(), null, null, () -> true, null) { @Override public MappedFieldType fieldMapper(String name) { From 3a621f8621efdcb1d11d604c7515638fd97e1e60 Mon Sep 17 00:00:00 2001 From: Mark Tozzi Date: Tue, 7 Apr 2020 12:05:31 -0400 Subject: [PATCH 02/15] First batch of easy fixes --- .../bucket/geogrid/GeoGridAggregationBuilder.java | 3 +-- .../histogram/AutoDateHistogramAggregationBuilder.java | 8 -------- .../histogram/DateHistogramAggregationBuilder.java | 4 ++-- .../bucket/histogram/HistogramAggregationBuilder.java | 5 ++--- .../bucket/histogram/HistogramAggregatorFactory.java | 9 +++++++-- .../aggregations/bucket/range/AbstractRangeBuilder.java | 2 +- .../significant/SignificantTermsAggregationBuilder.java | 3 +-- .../bucket/terms/TermsAggregationBuilder.java | 3 +-- .../aggregations/metrics/InternalHDRPercentileRanks.java | 4 ++-- .../aggregations/metrics/InternalHDRPercentiles.java | 4 ++-- 10 files changed, 19 insertions(+), 26 deletions(-) diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoGridAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoGridAggregationBuilder.java index 7d872316caba6..87a34ebe979b1 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoGridAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoGridAggregationBuilder.java @@ -44,8 +44,7 @@ import java.util.Objects; import java.util.function.Function; -public abstract class GeoGridAggregationBuilder extends ValuesSourceAggregationBuilder - implements MultiBucketAggregationBuilder { +public abstract class GeoGridAggregationBuilder extends ValuesSourceAggregationBuilder { /* recognized field names in JSON */ static final ParseField FIELD_PRECISION = new ParseField("precision"); static final ParseField FIELD_SIZE = new ParseField("size"); diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/AutoDateHistogramAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/AutoDateHistogramAggregationBuilder.java index be4f4c84d4c34..fab115e9dd063 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/AutoDateHistogramAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/AutoDateHistogramAggregationBuilder.java @@ -150,14 +150,6 @@ protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map - implements MultiBucketAggregationBuilder, DateIntervalConsumer { + implements DateIntervalConsumer { public static final String NAME = "date_histogram"; @@ -152,7 +152,7 @@ protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map - implements MultiBucketAggregationBuilder { +public class HistogramAggregationBuilder extends ValuesSourceAggregationBuilder { public static final String NAME = "histogram"; private static final ObjectParser EXTENDED_BOUNDS_PARSER = new ObjectParser<>( @@ -121,7 +120,7 @@ protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map, R extends Range> - extends ValuesSourceAggregationBuilder implements MultiBucketAggregationBuilder { + extends ValuesSourceAggregationBuilder { protected final InternalRange.Factory rangeFactory; protected List ranges = new ArrayList<>(); diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregationBuilder.java index f8ebc4686773a..f0f8116911383 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregationBuilder.java @@ -50,8 +50,7 @@ import static org.elasticsearch.index.query.AbstractQueryBuilder.parseInnerQueryBuilder; -public class SignificantTermsAggregationBuilder extends ValuesSourceAggregationBuilder - implements MultiBucketAggregationBuilder { +public class SignificantTermsAggregationBuilder extends ValuesSourceAggregationBuilder { public static final String NAME = "significant_terms"; static final ParseField BACKGROUND_FILTER = new ParseField("background_filter"); diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregationBuilder.java index d2ad7e0359003..900d607542c19 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregationBuilder.java @@ -47,8 +47,7 @@ import java.util.Map; import java.util.Objects; -public class TermsAggregationBuilder extends ValuesSourceAggregationBuilder - implements MultiBucketAggregationBuilder { +public class TermsAggregationBuilder extends ValuesSourceAggregationBuilder { public static final String NAME = "terms"; public static final ParseField EXECUTION_HINT_FIELD_NAME = new ParseField("execution_hint"); diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalHDRPercentileRanks.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalHDRPercentileRanks.java index 891a10c847e23..1bd0f46b3206a 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalHDRPercentileRanks.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalHDRPercentileRanks.java @@ -30,8 +30,8 @@ public class InternalHDRPercentileRanks extends AbstractInternalHDRPercentiles i public static final String NAME = "hdr_percentile_ranks"; public InternalHDRPercentileRanks(String name, double[] cdfValues, DoubleHistogram state, boolean keyed, DocValueFormat formatter, - List pipelineAggregators, Map metaData) { - super(name, cdfValues, state, keyed, formatter, pipelineAggregators, metaData); + Map metaData) { + super(name, cdfValues, state, keyed, formatter, metaData); } /** diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalHDRPercentiles.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalHDRPercentiles.java index 9b34c5c71e159..c647bfbc5be13 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalHDRPercentiles.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalHDRPercentiles.java @@ -30,8 +30,8 @@ public class InternalHDRPercentiles extends AbstractInternalHDRPercentiles imple public static final String NAME = "hdr_percentiles"; public InternalHDRPercentiles(String name, double[] percents, DoubleHistogram state, boolean keyed, DocValueFormat formatter, - List pipelineAggregators, Map metaData) { - super(name, percents, state, keyed, formatter, pipelineAggregators, metaData); + Map metaData) { + super(name, percents, state, keyed, formatter, metaData); } /** From 229a65f090c0e418e848ead2e282913b7eaea53b Mon Sep 17 00:00:00 2001 From: Mark Tozzi Date: Tue, 7 Apr 2020 12:33:39 -0400 Subject: [PATCH 03/15] Remove List.of from ValuesSourceRegistry Note that we intend to have a follow up PR dealing with the mutability of the registry, so I didn't even try to address that here. --- .../support/ValuesSourceRegistry.java | 39 +++++-------------- 1 file changed, 10 insertions(+), 29 deletions(-) diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceRegistry.java b/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceRegistry.java index 5356821c5f58f..176155f7beb5e 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceRegistry.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceRegistry.java @@ -27,7 +27,11 @@ import org.elasticsearch.search.SearchModule; import org.elasticsearch.search.aggregations.AggregationExecutionException; +import java.lang.reflect.Array; import java.util.AbstractMap; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.HashMap; import java.util.List; import java.util.Map; import java.util.function.Predicate; @@ -40,7 +44,7 @@ */ public class ValuesSourceRegistry { // Maps Aggregation names to (ValuesSourceType, Supplier) pairs, keyed by ValuesSourceType - private Map, AggregatorSupplier>>> aggregatorRegistry = Map.of(); + private Map, AggregatorSupplier>>> aggregatorRegistry = new HashMap<>(); /** * Register a ValuesSource to Aggregator mapping. @@ -59,15 +63,14 @@ public class ValuesSourceRegistry { */ public synchronized void register(String aggregationName, Predicate appliesTo, AggregatorSupplier aggregatorSupplier) { - AbstractMap.SimpleEntry[] mappings; + AbstractMap.SimpleEntry newSupplier = new AbstractMap.SimpleEntry<>(appliesTo, aggregatorSupplier); if (aggregatorRegistry.containsKey(aggregationName)) { - List currentMappings = aggregatorRegistry.get(aggregationName); - mappings = (AbstractMap.SimpleEntry[]) currentMappings.toArray(new AbstractMap.SimpleEntry[currentMappings.size() + 1]); + aggregatorRegistry.get(aggregationName).add(newSupplier); } else { - mappings = new AbstractMap.SimpleEntry[1]; + List, AggregatorSupplier>> supplierList = new ArrayList<>(); + supplierList.add(newSupplier); + aggregatorRegistry.put(aggregationName, supplierList); } - mappings[mappings.length - 1] = new AbstractMap.SimpleEntry<>(appliesTo, aggregatorSupplier); - aggregatorRegistry = copyAndAdd(aggregatorRegistry,new AbstractMap.SimpleEntry<>(aggregationName, List.of(mappings))); } /** @@ -168,26 +171,4 @@ && findMatchingSuppier(valuesSourceType, aggregatorRegistry.get(aggregationName) } } } - - private static Map copyAndAdd(Map source, Map.Entry newValue) { - Map.Entry[] entries; - if (source.containsKey(newValue.getKey())) { - // Replace with new value - entries = new Map.Entry[source.size()]; - int i = 0; - for (Map.Entry entry : source.entrySet()) { - if (entry.getKey() == newValue.getKey()) { - entries[i] = newValue; - } else { - entries[i] = entry; - } - i++; - } - } else { - entries = source.entrySet().toArray(new Map.Entry[source.size() + 1]); - entries[entries.length - 1] = newValue; - } - return Map.ofEntries(entries); - } - } From bd90d431d3b7a5dca7512def7017c9c909591228 Mon Sep 17 00:00:00 2001 From: Mark Tozzi Date: Tue, 7 Apr 2020 13:06:12 -0400 Subject: [PATCH 04/15] More compiler fixes --- .../bucket/histogram/DateHistogramAggregatorFactory.java | 5 +++-- .../bucket/missing/MissingAggregationBuilder.java | 2 +- .../bucket/range/AbstractRangeAggregatorFactory.java | 3 ++- .../significant/SignificantTermsAggregatorFactory.java | 7 ++++--- .../bucket/terms/RareTermsAggregatorFactory.java | 7 ++++--- .../aggregations/bucket/terms/TermsAggregatorFactory.java | 5 +++-- .../search/aggregations/metrics/AvgAggregatorFactory.java | 3 ++- .../search/aggregations/metrics/MaxAggregatorFactory.java | 3 ++- .../metrics/MedianAbsoluteDeviationAggregatorFactory.java | 2 +- .../search/aggregations/metrics/MinAggregatorFactory.java | 3 ++- .../metrics/PercentileRanksAggregatorFactory.java | 3 ++- .../aggregations/metrics/PercentilesAggregatorFactory.java | 5 +++-- .../search/aggregations/metrics/PercentilesConfig.java | 4 ++-- .../aggregations/metrics/StatsAggregatorFactory.java | 3 ++- .../search/aggregations/metrics/SumAggregatorFactory.java | 3 ++- .../metrics/WeightedAvgAggregationBuilder.java | 2 +- .../search/aggregations/support/ValueType.java | 6 ++++-- 17 files changed, 40 insertions(+), 26 deletions(-) diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/DateHistogramAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/DateHistogramAggregatorFactory.java index 870313e7cf7b9..e8a3bd8699a60 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/DateHistogramAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/DateHistogramAggregatorFactory.java @@ -39,6 +39,7 @@ import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; +import java.util.Arrays; import java.util.List; import java.util.Map; @@ -46,7 +47,7 @@ public final class DateHistogramAggregatorFactory extends ValuesSourceAggregator public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { valuesSourceRegistry.register(DateHistogramAggregationBuilder.NAME, - List.of(CoreValuesSourceType.DATE, CoreValuesSourceType.NUMERIC, CoreValuesSourceType.BOOLEAN), + Arrays.asList(CoreValuesSourceType.DATE, CoreValuesSourceType.NUMERIC, CoreValuesSourceType.BOOLEAN), (DateHistogramAggregationSupplier) (String name, AggregatorFactories factories, Rounding rounding, @@ -133,7 +134,7 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, aggregatorSupplier.getClass().toString() + "]"); } return ((DateHistogramAggregationSupplier) aggregatorSupplier).build(name, factories, rounding, shardRounding, order, keyed, - minDocCount, extendedBounds, valuesSource, config.format(), searchContext, parent, pipelineAggregators, metaData); + minDocCount, extendedBounds, valuesSource, config.format(), searchContext, parent, pipelineAggregators, metadata); } @Override diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/missing/MissingAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/missing/MissingAggregationBuilder.java index 62d76914fef29..440ac9aeccc49 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/missing/MissingAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/missing/MissingAggregationBuilder.java @@ -91,7 +91,7 @@ protected ValuesSourceAggregatorFactory innerBuild(QueryShardContext queryShardC ValuesSourceConfig config, AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException { - return new MissingAggregatorFactory(name, config, queryShardContext, parent, subFactoriesBuilder, metaData); + return new MissingAggregatorFactory(name, config, queryShardContext, parent, subFactoriesBuilder, metadata); } @Override diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/AbstractRangeAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/AbstractRangeAggregatorFactory.java index 93f9a670de9f0..28f021bb695d4 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/AbstractRangeAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/AbstractRangeAggregatorFactory.java @@ -38,6 +38,7 @@ import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; +import java.util.Arrays; import java.util.List; import java.util.Map; @@ -50,7 +51,7 @@ public class AbstractRangeAggregatorFactory extends ValuesSourc public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry, String aggregationName) { valuesSourceRegistry.register(aggregationName, - List.of(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN), + Arrays.asList(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN), new RangeAggregatorSupplier() { @Override public Aggregator build(String name, diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorFactory.java index 943fe5b065887..d1b0e8636d2e3 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorFactory.java @@ -59,6 +59,7 @@ import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; +import java.util.Arrays; import java.util.List; import java.util.Map; @@ -80,11 +81,11 @@ public class SignificantTermsAggregatorFactory extends ValuesSourceAggregatorFac static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { valuesSourceRegistry.register(SignificantTermsAggregationBuilder.NAME, - List.of(CoreValuesSourceType.BYTES, CoreValuesSourceType.IP), + Arrays.asList(CoreValuesSourceType.BYTES, CoreValuesSourceType.IP), SignificantTermsAggregatorFactory.bytesSupplier()); valuesSourceRegistry.register(SignificantTermsAggregationBuilder.NAME, - List.of(CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN, CoreValuesSourceType.NUMERIC), + Arrays.asList(CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN, CoreValuesSourceType.NUMERIC), SignificantTermsAggregatorFactory.numericSupplier()); } @@ -326,7 +327,7 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, // TODO we should refactor so that we don't need to use this Factory as a singleton (e.g. stop passing `this` to the aggregators) return sigTermsAggregatorSupplier.build(name, factories, valuesSource, config.format(), bucketCountThresholds, includeExclude, executionHint, searchContext, parent, - significanceHeuristic, this, pipelineAggregators, metaData); + significanceHeuristic, this, pipelineAggregators, metadata); } public enum ExecutionMode { diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregatorFactory.java index 84466b3e72d81..e58d7e54a7227 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregatorFactory.java @@ -39,6 +39,7 @@ import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; +import java.util.Arrays; import java.util.List; import java.util.Map; @@ -49,11 +50,11 @@ public class RareTermsAggregatorFactory extends ValuesSourceAggregatorFactory { static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { valuesSourceRegistry.register(RareTermsAggregationBuilder.NAME, - List.of(CoreValuesSourceType.BYTES, CoreValuesSourceType.IP), + Arrays.asList(CoreValuesSourceType.BYTES, CoreValuesSourceType.IP), RareTermsAggregatorFactory.bytesSupplier()); valuesSourceRegistry.register(RareTermsAggregationBuilder.NAME, - List.of(CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN, CoreValuesSourceType.NUMERIC), + Arrays.asList(CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN, CoreValuesSourceType.NUMERIC), RareTermsAggregatorFactory.numericSupplier()); } @@ -173,7 +174,7 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, } return ((RareTermsAggregatorSupplier) aggregatorSupplier).build(name, factories, valuesSource, config.format(), - maxDocCount, precision, includeExclude, searchContext, parent, pipelineAggregators, metaData); + maxDocCount, precision, includeExclude, searchContext, parent, pipelineAggregators, metadata); } public enum ExecutionMode { diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorFactory.java index 4eae7ad1fe249..3d811796f6d8b 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorFactory.java @@ -47,6 +47,7 @@ import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; +import java.util.Arrays; import java.util.List; import java.util.Map; @@ -65,11 +66,11 @@ public class TermsAggregatorFactory extends ValuesSourceAggregatorFactory { // TODO: Registration should happen on the actual aggregator classes static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { valuesSourceRegistry.register(TermsAggregationBuilder.NAME, - List.of(CoreValuesSourceType.BYTES, CoreValuesSourceType.IP), + Arrays.asList(CoreValuesSourceType.BYTES, CoreValuesSourceType.IP), TermsAggregatorFactory.bytesSupplier()); valuesSourceRegistry.register(TermsAggregationBuilder.NAME, - List.of(CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN, CoreValuesSourceType.NUMERIC), + Arrays.asList(CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN, CoreValuesSourceType.NUMERIC), TermsAggregatorFactory.numericSupplier()); } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/AvgAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/AvgAggregatorFactory.java index ca987fede2068..71c0c83ebd1ea 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/AvgAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/AvgAggregatorFactory.java @@ -36,6 +36,7 @@ import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; +import java.util.Arrays; import java.util.List; import java.util.Map; @@ -49,7 +50,7 @@ class AvgAggregatorFactory extends ValuesSourceAggregatorFactory { static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { valuesSourceRegistry.register(AvgAggregationBuilder.NAME, - List.of(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN), + Arrays.asList(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN), new MetricAggregatorSupplier() { @Override public Aggregator build(String name, diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MaxAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MaxAggregatorFactory.java index 24073faf025fe..ef7d45c68f799 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MaxAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MaxAggregatorFactory.java @@ -35,6 +35,7 @@ import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; +import java.util.Arrays; import java.util.List; import java.util.Map; @@ -42,7 +43,7 @@ class MaxAggregatorFactory extends ValuesSourceAggregatorFactory { static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { valuesSourceRegistry.register(MaxAggregationBuilder.NAME, - List.of(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN), + Arrays.asList(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN), new MinMaxAggregatorSupplier() { @Override public Aggregator build(String name, diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregatorFactory.java index 4725a8db78b1c..8143a5a401d87 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregatorFactory.java @@ -47,7 +47,7 @@ public class MedianAbsoluteDeviationAggregatorFactory extends ValuesSourceAggreg QueryShardContext queryShardContext, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, - Map metaData, + Map metadata, double compression) throws IOException { super(name, config, queryShardContext, parent, subFactoriesBuilder, metadata); diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MinAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MinAggregatorFactory.java index 3b48768ad156a..47ca00eae4350 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MinAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MinAggregatorFactory.java @@ -35,6 +35,7 @@ import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; +import java.util.Arrays; import java.util.List; import java.util.Map; @@ -42,7 +43,7 @@ class MinAggregatorFactory extends ValuesSourceAggregatorFactory { static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { valuesSourceRegistry.register(MinAggregationBuilder.NAME, - List.of(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN), + Arrays.asList(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN), new MinMaxAggregatorSupplier() { @Override public Aggregator build(String name, diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentileRanksAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentileRanksAggregatorFactory.java index 748743093c969..29e56b7417b6c 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentileRanksAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentileRanksAggregatorFactory.java @@ -35,6 +35,7 @@ import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; +import java.util.Arrays; import java.util.List; import java.util.Map; @@ -46,7 +47,7 @@ class PercentileRanksAggregatorFactory extends ValuesSourceAggregatorFactory { static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { valuesSourceRegistry.register(PercentileRanksAggregationBuilder.NAME, - List.of(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN), + Arrays.asList(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN), new PercentilesAggregatorSupplier() { @Override public Aggregator build(String name, ValuesSource valuesSource, SearchContext context, Aggregator parent, diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesAggregatorFactory.java index 44aa22a325248..32bec871a9f44 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesAggregatorFactory.java @@ -35,6 +35,7 @@ import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; +import java.util.Arrays; import java.util.List; import java.util.Map; @@ -50,7 +51,7 @@ class PercentilesAggregatorFactory extends ValuesSourceAggregatorFactory { static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { valuesSourceRegistry.register(PercentilesAggregationBuilder.NAME, - List.of(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN), + Arrays.asList(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN), new PercentilesAggregatorSupplier() { @Override public Aggregator build(String name, ValuesSource valuesSource, SearchContext context, Aggregator parent, @@ -101,6 +102,6 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, } PercentilesAggregatorSupplier percentilesAggregatorSupplier = (PercentilesAggregatorSupplier) aggregatorSupplier; return percentilesAggregatorSupplier.build(name, valuesSource, searchContext, parent, percents, percentilesConfig, keyed, - config.format(), pipelineAggregators, metaData); + config.format(), pipelineAggregators, metadata); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesConfig.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesConfig.java index b1da934210135..9a32c63eb1a9c 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesConfig.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesConfig.java @@ -131,7 +131,7 @@ public double getCompression() { public Aggregator createPercentilesAggregator(String name, ValuesSource valuesSource, SearchContext context, Aggregator parent, double[] values, boolean keyed, DocValueFormat formatter, List pipelineAggregators, - Map metaData) throws IOException { + Map metadata) throws IOException { return new TDigestPercentilesAggregator(name, valuesSource, context, parent, values, compression, keyed, formatter, pipelineAggregators, metadata); } @@ -207,7 +207,7 @@ public int getNumberOfSignificantValueDigits() { public Aggregator createPercentilesAggregator(String name, ValuesSource valuesSource, SearchContext context, Aggregator parent, double[] values, boolean keyed, DocValueFormat formatter, List pipelineAggregators, - Map metaData) throws IOException { + Map metadata) throws IOException { return new HDRPercentilesAggregator(name, valuesSource, context, parent, values, numberOfSignificantValueDigits, keyed, formatter, pipelineAggregators, metadata); } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/StatsAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/StatsAggregatorFactory.java index 19cb223a31f8b..7c9941f84e41a 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/StatsAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/StatsAggregatorFactory.java @@ -36,6 +36,7 @@ import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; +import java.util.Arrays; import java.util.List; import java.util.Map; @@ -52,7 +53,7 @@ class StatsAggregatorFactory extends ValuesSourceAggregatorFactory { static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { valuesSourceRegistry.register(StatsAggregationBuilder.NAME, - List.of(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN), + Arrays.asList(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN), new MetricAggregatorSupplier() { @Override public Aggregator build(String name, diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/SumAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/SumAggregatorFactory.java index 38c8111a48801..b0a83dd124e7a 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/SumAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/SumAggregatorFactory.java @@ -36,6 +36,7 @@ import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; +import java.util.Arrays; import java.util.List; import java.util.Map; @@ -52,7 +53,7 @@ class SumAggregatorFactory extends ValuesSourceAggregatorFactory { static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { valuesSourceRegistry.register(SumAggregationBuilder.NAME, - List.of(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN), + Arrays.asList(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN), new MetricAggregatorSupplier() { @Override public Aggregator build(String name, diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/WeightedAvgAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/WeightedAvgAggregationBuilder.java index b6ec1c843376b..770a9acc7c3b0 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/WeightedAvgAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/WeightedAvgAggregationBuilder.java @@ -109,7 +109,7 @@ protected MultiValuesSourceAggregatorFactory innerBuild(QueryShardContext queryS DocValueFormat format, AggregatorFactory parent, Builder subFactoriesBuilder) throws IOException { - return new WeightedAvgAggregatorFactory(name, configs, format, queryShardContext, parent, subFactoriesBuilder, metaData); + return new WeightedAvgAggregatorFactory(name, configs, format, queryShardContext, parent, subFactoriesBuilder, metadata); } @Override diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/support/ValueType.java b/server/src/main/java/org/elasticsearch/search/aggregations/support/ValueType.java index 068790030bd49..026289af284bd 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/support/ValueType.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/support/ValueType.java @@ -28,6 +28,8 @@ import java.io.IOException; import java.time.ZoneOffset; +import java.util.Arrays; +import java.util.List; import java.util.Set; public enum ValueType implements Writeable { @@ -73,9 +75,9 @@ public ValuesSourceType getValuesSourceType() { return valuesSourceType; } - private static Set numericValueTypes = Set.of(ValueType.DOUBLE, ValueType.DATE, ValueType.LONG, ValueType.NUMBER, + private static List numericValueTypes = Arrays.asList(ValueType.DOUBLE, ValueType.DATE, ValueType.LONG, ValueType.NUMBER, ValueType.NUMERIC, ValueType.BOOLEAN); - private static Set stringValueTypes = Set.of(ValueType.STRING, ValueType.IP); + private static List stringValueTypes = Arrays.asList(ValueType.STRING, ValueType.IP); /** * This is a bit of a hack to mirror the old {@link ValueType} behavior, which would allow a rough compatibility between types. This From 51911409bb7416e3dcec9ae67bb77b9a04c200a7 Mon Sep 17 00:00:00 2001 From: Mark Tozzi Date: Tue, 7 Apr 2020 13:14:27 -0400 Subject: [PATCH 05/15] More compiler fixes --- .../search/aggregations/AggregatorTestCase.java | 13 ++++++++++--- .../xpack/rollup/RollupRequestTranslator.java | 6 +++--- 2 files changed, 13 insertions(+), 6 deletions(-) diff --git a/test/framework/src/main/java/org/elasticsearch/search/aggregations/AggregatorTestCase.java b/test/framework/src/main/java/org/elasticsearch/search/aggregations/AggregatorTestCase.java index 4fb332fbdfbf3..019272fefdbcf 100644 --- a/test/framework/src/main/java/org/elasticsearch/search/aggregations/AggregatorTestCase.java +++ b/test/framework/src/main/java/org/elasticsearch/search/aggregations/AggregatorTestCase.java @@ -19,10 +19,13 @@ package org.elasticsearch.search.aggregations; import org.apache.lucene.document.BinaryDocValuesField; +import org.apache.lucene.document.Document; +import org.apache.lucene.document.HalfFloatPoint; import org.apache.lucene.document.InetAddressPoint; import org.apache.lucene.document.LatLonDocValuesField; import org.apache.lucene.document.SortedNumericDocValuesField; import org.apache.lucene.document.SortedSetDocValuesField; +import org.apache.lucene.document.StoredField; import org.apache.lucene.index.AssertingDirectoryReader; import org.apache.lucene.index.CompositeReaderContext; import org.apache.lucene.index.DirectoryReader; @@ -41,12 +44,14 @@ import org.apache.lucene.search.Weight; import org.apache.lucene.store.Directory; import org.apache.lucene.util.BytesRef; +import org.apache.lucene.util.NumericUtils; import org.elasticsearch.Version; import org.elasticsearch.cluster.metadata.IndexMetadata; import org.elasticsearch.common.breaker.CircuitBreaker; import org.elasticsearch.common.lease.Releasable; import org.elasticsearch.common.lease.Releasables; import org.elasticsearch.common.lucene.index.ElasticsearchDirectoryReader; +import org.elasticsearch.common.network.NetworkAddress; import org.elasticsearch.common.settings.Settings; import org.elasticsearch.common.text.Text; import org.elasticsearch.common.util.BigArrays; @@ -71,6 +76,7 @@ import org.elasticsearch.index.mapper.Mapper; import org.elasticsearch.index.mapper.Mapper.BuilderContext; import org.elasticsearch.index.mapper.MapperService; +import org.elasticsearch.index.mapper.NumberFieldMapper; import org.elasticsearch.index.mapper.ObjectMapper; import org.elasticsearch.index.mapper.ObjectMapper.Nested; import org.elasticsearch.index.mapper.RangeFieldMapper; @@ -105,6 +111,7 @@ import org.junit.BeforeClass; import java.io.IOException; +import java.net.InetAddress; import java.util.ArrayList; import java.util.Arrays; import java.util.Collections; @@ -171,7 +178,7 @@ private static void registerFieldTypes(SearchContext searchContext, MapperServic @BeforeClass public static void initValuesSourceRegistry() { - SearchModule searchModule = new SearchModule(Settings.EMPTY, List.of()); + SearchModule searchModule = new SearchModule(Settings.EMPTY, false, Collections.EMPTY_LIST); valuesSourceRegistry = searchModule.getValuesSourceRegistry(); } @@ -684,7 +691,6 @@ private void writeTestDoc(MappedFieldType fieldType, String fieldName, RandomInd Document doc = new Document(); String json; - if (vst.equals(CoreValuesSourceType.NUMERIC)) { long v; if (typeName.equals(NumberFieldMapper.NumberType.DOUBLE.typeName())) { @@ -710,7 +716,8 @@ private void writeTestDoc(MappedFieldType fieldType, String fieldName, RandomInd doc.add(new BinaryFieldMapper.CustomBinaryDocValuesField(fieldName, new BytesRef("a").bytes)); json = "{ \"" + fieldName + "\" : \"a\" }"; } else { - iw.addDocument(singleton(new SortedSetDocValuesField(fieldName, new BytesRef("a")))); + doc.add(new SortedSetDocValuesField(fieldName, new BytesRef("a"))); + json = "{ \"" + fieldName + "\" : \"a\" }"; } } else if (vst.equals(CoreValuesSourceType.DATE)) { // positive integer because date_nanos gets unhappy with large longs diff --git a/x-pack/plugin/rollup/src/main/java/org/elasticsearch/xpack/rollup/RollupRequestTranslator.java b/x-pack/plugin/rollup/src/main/java/org/elasticsearch/xpack/rollup/RollupRequestTranslator.java index aa39b0a44f6cc..bc8f4065131d1 100644 --- a/x-pack/plugin/rollup/src/main/java/org/elasticsearch/xpack/rollup/RollupRequestTranslator.java +++ b/x-pack/plugin/rollup/src/main/java/org/elasticsearch/xpack/rollup/RollupRequestTranslator.java @@ -357,7 +357,7 @@ private static List translateTerms(TermsAggregationBuilder s * @param The type of ValueSourceAggBuilder that we are working with * @return the translated multi-bucket ValueSourceAggBuilder */ - private static > List + private static > List translateVSAggBuilder(T source, NamedWriteableRegistry registry, Supplier factory) { T rolled = factory.get(); @@ -483,8 +483,8 @@ private static List translateVSLeaf(ValuesSourceAggregationB NamedWriteableAwareStreamInput in = new NamedWriteableAwareStreamInput(stream, registry)) { - ValuesSourceAggregationBuilder serialized - = ((ValuesSourceAggregationBuilder)in.readNamedWriteable(AggregationBuilder.class)) + ValuesSourceAggregationBuilder serialized + = ((ValuesSourceAggregationBuilder)in.readNamedWriteable(AggregationBuilder.class)) .field(RollupField.formatFieldName(metric, RollupField.VALUE)); return Collections.singletonList(serialized); From 892fc4b675ebb15cb205fb22a214fac331cf82f1 Mon Sep 17 00:00:00 2001 From: Mark Tozzi Date: Tue, 7 Apr 2020 14:23:02 -0400 Subject: [PATCH 06/15] More compiler fixes --- .../missing/MissingAggregatorTests.java | 3 ++- .../SignificantTermsAggregatorTests.java | 5 ++++- .../metrics/CardinalityAggregatorTests.java | 2 +- .../metrics/GeoCentroidAggregatorTests.java | 3 ++- .../HDRPercentileRanksAggregatorTests.java | 7 ++++++- .../HDRPercentilesAggregatorTests.java | 5 ++++- ...TDigestPercentileRanksAggregatorTests.java | 7 ++++++- .../TDigestPercentilesAggregatorTests.java | 7 ++++++- .../metrics/ValueCountAggregatorTests.java | 19 ++++++++++++++++++- .../aggregations/pipeline/MovAvgIT.java | 11 +++++------ ...stoBackedHDRPercentileRanksAggregator.java | 5 ++--- .../HistoBackedHDRPercentilesAggregator.java | 6 ++---- ...ackedTDigestPercentileRanksAggregator.java | 4 ++-- ...stoBackedTDigestPercentilesAggregator.java | 4 ++-- .../boxplot/BoxplotAggregatorFactory.java | 3 ++- .../ttest/TTestAggregationBuilder.java | 17 ++++++++++++----- .../ttest/TTestAggregatorFactory.java | 7 +++---- 17 files changed, 79 insertions(+), 36 deletions(-) diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/missing/MissingAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/missing/MissingAggregatorTests.java index aefe998660a97..1950efe14b313 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/missing/MissingAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/missing/MissingAggregatorTests.java @@ -51,6 +51,7 @@ import java.io.IOException; import java.util.ArrayList; +import java.util.Arrays; import java.util.Collection; import java.util.HashMap; import java.util.List; @@ -195,7 +196,7 @@ public void testMatchSparseRangeField() throws IOException { assertEquals(finalDocsMissingAggField, internalMissing.getDocCount()); assertTrue(AggregationInspectionHelper.hasValue(internalMissing)); }, - List.of(aggFieldType, anotherFieldType) + Arrays.asList(aggFieldType, anotherFieldType) ); } diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorTests.java index e94ca73aa5797..c43e3574f7488 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorTests.java @@ -88,10 +88,11 @@ protected AggregationBuilder createAggBuilderForTypeTest(MappedFieldType fieldTy @Override protected List getSupportedValuesSourceTypes() { - return List.of(CoreValuesSourceType.NUMERIC, + return Arrays.asList(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.BYTES); } + /* NOTE - commented out instead of deleted, because we probably need to backport this function in AggregatorTestCase @Override protected List unsupportedMappedFieldTypes() { return List.of( @@ -102,6 +103,8 @@ protected List unsupportedMappedFieldTypes() { ); } + */ + /** * For each provided field type, we also register an alias with name {@code -alias}. */ diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/CardinalityAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/CardinalityAggregatorTests.java index e9a203e671963..7c2f3106283d8 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/CardinalityAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/CardinalityAggregatorTests.java @@ -65,7 +65,7 @@ public void testRangeFieldValues() throws IOException { final String fieldName = "rangeField"; MappedFieldType fieldType = new RangeFieldMapper.Builder(fieldName, rangeType).fieldType(); fieldType.setName(fieldName); - final CardinalityAggregationBuilder aggregationBuilder = new CardinalityAggregationBuilder("_name", null).field(fieldName); + final CardinalityAggregationBuilder aggregationBuilder = new CardinalityAggregationBuilder("_name").field(fieldName); Set multiRecord = new HashSet<>(2); multiRecord.add(range1); multiRecord.add(range2); diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregatorTests.java index c692d45371039..bcb16dabcca85 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregatorTests.java @@ -36,6 +36,7 @@ import org.elasticsearch.test.geo.RandomGeoGenerator; import java.io.IOException; +import java.util.Arrays; import java.util.List; public class GeoCentroidAggregatorTests extends AggregatorTestCase { @@ -189,6 +190,6 @@ protected AggregationBuilder createAggBuilderForTypeTest(MappedFieldType fieldTy @Override protected List getSupportedValuesSourceTypes() { - return List.of(CoreValuesSourceType.GEOPOINT); + return Arrays.asList(CoreValuesSourceType.GEOPOINT); } } diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/HDRPercentileRanksAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/HDRPercentileRanksAggregatorTests.java index 5891981121000..77a4ab3283bb9 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/HDRPercentileRanksAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/HDRPercentileRanksAggregatorTests.java @@ -30,16 +30,21 @@ import org.apache.lucene.util.NumericUtils; import org.elasticsearch.index.mapper.MappedFieldType; import org.elasticsearch.index.mapper.NumberFieldMapper; +import org.elasticsearch.search.aggregations.AggregationBuilder; import org.elasticsearch.search.aggregations.AggregatorTestCase; import org.elasticsearch.search.aggregations.metrics.Percentile; import org.elasticsearch.search.aggregations.metrics.PercentileRanks; import org.elasticsearch.search.aggregations.metrics.PercentileRanksAggregationBuilder; import org.elasticsearch.search.aggregations.metrics.PercentilesMethod; import org.elasticsearch.search.aggregations.support.AggregationInspectionHelper; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import org.hamcrest.Matchers; import java.io.IOException; +import java.util.Arrays; import java.util.Iterator; +import java.util.List; public class HDRPercentileRanksAggregatorTests extends AggregatorTestCase { @@ -53,7 +58,7 @@ protected AggregationBuilder createAggBuilderForTypeTest(MappedFieldType fieldTy @Override protected List getSupportedValuesSourceTypes() { - return List.of(CoreValuesSourceType.NUMERIC, + return Arrays.asList(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN); } diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/HDRPercentilesAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/HDRPercentilesAggregatorTests.java index 14f2981051122..076f4cf3bd14c 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/HDRPercentilesAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/HDRPercentilesAggregatorTests.java @@ -42,8 +42,11 @@ import org.elasticsearch.search.aggregations.AggregationBuilder; import org.elasticsearch.search.aggregations.AggregatorTestCase; import org.elasticsearch.search.aggregations.support.AggregationInspectionHelper; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; +import java.util.Arrays; import java.util.Collections; import java.util.List; import java.util.function.Consumer; @@ -64,7 +67,7 @@ protected AggregationBuilder createAggBuilderForTypeTest(MappedFieldType fieldTy @Override protected List getSupportedValuesSourceTypes() { - return List.of(CoreValuesSourceType.NUMERIC, + return Arrays.asList(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN); } diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/TDigestPercentileRanksAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/TDigestPercentileRanksAggregatorTests.java index a3ba44a55479f..f175f46006b87 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/TDigestPercentileRanksAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/TDigestPercentileRanksAggregatorTests.java @@ -30,16 +30,21 @@ import org.apache.lucene.util.NumericUtils; import org.elasticsearch.index.mapper.MappedFieldType; import org.elasticsearch.index.mapper.NumberFieldMapper; +import org.elasticsearch.search.aggregations.AggregationBuilder; import org.elasticsearch.search.aggregations.AggregatorTestCase; import org.elasticsearch.search.aggregations.metrics.Percentile; import org.elasticsearch.search.aggregations.metrics.PercentileRanks; import org.elasticsearch.search.aggregations.metrics.PercentileRanksAggregationBuilder; import org.elasticsearch.search.aggregations.metrics.PercentilesMethod; import org.elasticsearch.search.aggregations.support.AggregationInspectionHelper; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import org.hamcrest.Matchers; import java.io.IOException; +import java.util.Arrays; import java.util.Iterator; +import java.util.List; public class TDigestPercentileRanksAggregatorTests extends AggregatorTestCase { @@ -53,7 +58,7 @@ protected AggregationBuilder createAggBuilderForTypeTest(MappedFieldType fieldTy @Override protected List getSupportedValuesSourceTypes() { - return List.of(CoreValuesSourceType.NUMERIC, + return Arrays.asList(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN); } diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/TDigestPercentilesAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/TDigestPercentilesAggregatorTests.java index 15300284d9f6b..85867dc834da2 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/TDigestPercentilesAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/TDigestPercentilesAggregatorTests.java @@ -33,10 +33,15 @@ import org.elasticsearch.common.CheckedConsumer; import org.elasticsearch.index.mapper.MappedFieldType; import org.elasticsearch.index.mapper.NumberFieldMapper; +import org.elasticsearch.search.aggregations.AggregationBuilder; import org.elasticsearch.search.aggregations.AggregatorTestCase; import org.elasticsearch.search.aggregations.support.AggregationInspectionHelper; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; +import java.util.Arrays; +import java.util.List; import java.util.function.Consumer; import static java.util.Arrays.asList; @@ -55,7 +60,7 @@ protected AggregationBuilder createAggBuilderForTypeTest(MappedFieldType fieldTy @Override protected List getSupportedValuesSourceTypes() { - return List.of(CoreValuesSourceType.NUMERIC, + return Arrays.asList(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN); } diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregatorTests.java index 9673fecc92719..e7b98e2740f31 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregatorTests.java @@ -20,10 +20,12 @@ package org.elasticsearch.search.aggregations.metrics; import org.apache.lucene.document.BinaryDocValuesField; +import org.apache.lucene.document.Document; import org.apache.lucene.document.IntPoint; import org.apache.lucene.document.NumericDocValuesField; import org.apache.lucene.document.SortedDocValuesField; import org.apache.lucene.document.SortedNumericDocValuesField; +import org.apache.lucene.document.SortedSetDocValuesField; import org.apache.lucene.index.DirectoryReader; import org.apache.lucene.index.IndexReader; import org.apache.lucene.index.RandomIndexWriter; @@ -35,6 +37,7 @@ import org.apache.lucene.util.BytesRef; import org.elasticsearch.common.CheckedConsumer; import org.elasticsearch.common.geo.GeoPoint; +import org.elasticsearch.common.settings.Settings; import org.elasticsearch.index.mapper.BooleanFieldMapper; import org.elasticsearch.index.mapper.DateFieldMapper; import org.elasticsearch.index.mapper.GeoPointFieldMapper; @@ -44,15 +47,29 @@ import org.elasticsearch.index.mapper.NumberFieldMapper; import org.elasticsearch.index.mapper.RangeFieldMapper; import org.elasticsearch.index.mapper.RangeType; +import org.elasticsearch.script.MockScriptEngine; +import org.elasticsearch.script.Script; +import org.elasticsearch.script.ScriptEngine; +import org.elasticsearch.script.ScriptModule; +import org.elasticsearch.script.ScriptService; +import org.elasticsearch.script.ScriptType; +import org.elasticsearch.search.aggregations.AggregationBuilder; import org.elasticsearch.search.aggregations.AggregatorTestCase; import org.elasticsearch.search.aggregations.support.AggregationInspectionHelper; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; import org.elasticsearch.search.aggregations.support.ValueType; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.util.Arrays; +import java.util.Collections; +import java.util.HashMap; import java.util.HashSet; +import java.util.List; +import java.util.Map; import java.util.Set; import java.util.function.Consumer; +import java.util.function.Function; import static java.util.Collections.singleton; @@ -71,7 +88,7 @@ protected AggregationBuilder createAggBuilderForTypeTest(MappedFieldType fieldTy @Override protected List getSupportedValuesSourceTypes() { - return List.of( + return Arrays.asList( CoreValuesSourceType.NUMERIC, CoreValuesSourceType.BYTES, CoreValuesSourceType.GEOPOINT, diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/MovAvgIT.java b/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/MovAvgIT.java index 254e324ab9f72..a478c78e6a279 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/MovAvgIT.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/MovAvgIT.java @@ -73,7 +73,7 @@ public class MovAvgIT extends ESIntegTestCase { static int period; static HoltWintersModel.SeasonalityType seasonalityType; static BucketHelpers.GapPolicy gapPolicy; - static ValuesSourceAggregationBuilder> metric; + static ValuesSourceAggregationBuilder> metric; static List mockHisto; static Map> testValues; @@ -655,7 +655,7 @@ public void testPredictNegativeKeysAtStart() { SearchResponse response = client() .prepareSearch("neg_idx") - + .addAggregation( histogram("histo") .field(INTERVAL_FIELD) @@ -877,7 +877,7 @@ public void testHoltWintersNotEnoughData() { public void testTwoMovAvgsWithPredictions() { SearchResponse response = client() .prepareSearch("double_predict") - + .addAggregation( histogram("histo") .field(INTERVAL_FIELD) @@ -1221,7 +1221,7 @@ public void testPredictWithNonEmptyBuckets() throws Exception { SearchResponse response = client() .prepareSearch("predict_non_empty") - + .addAggregation( histogram("histo") .field(INTERVAL_FIELD) @@ -1324,8 +1324,7 @@ private MovAvgModelBuilder randomModelBuilder(double padding) { } } - private ValuesSourceAggregationBuilder> randomMetric(String name, String field) { + private ValuesSourceAggregationBuilder> randomMetric(String name, String field) { int rand = randomIntBetween(0,3); switch (rand) { diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedHDRPercentileRanksAggregator.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedHDRPercentileRanksAggregator.java index dc9bca57ec3cd..a737797e7c320 100644 --- a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedHDRPercentileRanksAggregator.java +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedHDRPercentileRanksAggregator.java @@ -34,7 +34,7 @@ public InternalAggregation buildAggregation(long owningBucketOrdinal) { if (state == null) { return buildEmptyAggregation(); } else { - return new InternalHDRPercentileRanks(name, keys, state, keyed, format, pipelineAggregators(), metaData()); + return new InternalHDRPercentileRanks(name, keys, state, keyed, format, metadata()); } } @@ -43,8 +43,7 @@ public InternalAggregation buildEmptyAggregation() { DoubleHistogram state; state = new DoubleHistogram(numberOfSignificantValueDigits); state.setAutoResize(true); - return new InternalHDRPercentileRanks(name, keys, state, - keyed, format, pipelineAggregators(), metaData()); + return new InternalHDRPercentileRanks(name, keys, state, keyed, format, metadata()); } @Override diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedHDRPercentilesAggregator.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedHDRPercentilesAggregator.java index 283f07fcc71f4..6abb4ba05c279 100644 --- a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedHDRPercentilesAggregator.java +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedHDRPercentilesAggregator.java @@ -34,7 +34,7 @@ public InternalAggregation buildAggregation(long owningBucketOrdinal) { if (state == null) { return buildEmptyAggregation(); } else { - return new InternalHDRPercentiles(name, keys, state, keyed, format, pipelineAggregators(), metaData()); + return new InternalHDRPercentiles(name, keys, state, keyed, format, metadata()); } } @@ -53,8 +53,6 @@ public InternalAggregation buildEmptyAggregation() { DoubleHistogram state; state = new DoubleHistogram(numberOfSignificantValueDigits); state.setAutoResize(true); - return new InternalHDRPercentiles(name, keys, state, - keyed, - format, pipelineAggregators(), metaData()); + return new InternalHDRPercentiles(name, keys, state, keyed, format, metadata()); } } diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedTDigestPercentileRanksAggregator.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedTDigestPercentileRanksAggregator.java index 7bd1f9d35e995..8ac2d81fd6cb2 100644 --- a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedTDigestPercentileRanksAggregator.java +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedTDigestPercentileRanksAggregator.java @@ -40,14 +40,14 @@ public InternalAggregation buildAggregation(long owningBucketOrdinal) { if (state == null) { return buildEmptyAggregation(); } else { - return new InternalTDigestPercentileRanks(name, keys, state, keyed, formatter, pipelineAggregators(), metaData()); + return new InternalTDigestPercentileRanks(name, keys, state, keyed, formatter, pipelineAggregators(), metadata()); } } @Override public InternalAggregation buildEmptyAggregation() { return new InternalTDigestPercentileRanks(name, keys, new TDigestState(compression), keyed, - formatter, pipelineAggregators(), metaData()); + formatter, pipelineAggregators(), metadata()); } @Override diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedTDigestPercentilesAggregator.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedTDigestPercentilesAggregator.java index 6b7f33178ebf1..382a88a71d9db 100644 --- a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedTDigestPercentilesAggregator.java +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedTDigestPercentilesAggregator.java @@ -40,7 +40,7 @@ public InternalAggregation buildAggregation(long owningBucketOrdinal) { if (state == null) { return buildEmptyAggregation(); } else { - return new InternalTDigestPercentiles(name, keys, state, keyed, formatter, pipelineAggregators(), metaData()); + return new InternalTDigestPercentiles(name, keys, state, keyed, formatter, pipelineAggregators(), metadata()); } } @@ -57,6 +57,6 @@ public double metric(String name, long bucketOrd) { @Override public InternalAggregation buildEmptyAggregation() { return new InternalTDigestPercentiles(name, keys, new TDigestState(compression), keyed, - formatter, pipelineAggregators(), metaData()); + formatter, pipelineAggregators(), metadata()); } } diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregatorFactory.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregatorFactory.java index aca50bdc7b2cb..73d539372d688 100644 --- a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregatorFactory.java +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregatorFactory.java @@ -22,6 +22,7 @@ import org.elasticsearch.xpack.analytics.aggregations.support.AnalyticsValuesSourceType; import java.io.IOException; +import java.util.Arrays; import java.util.List; import java.util.Map; @@ -31,7 +32,7 @@ public class BoxplotAggregatorFactory extends ValuesSourceAggregatorFactory { static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { valuesSourceRegistry.register(BoxplotAggregationBuilder.NAME, - List.of(CoreValuesSourceType.NUMERIC, AnalyticsValuesSourceType.HISTOGRAM), + Arrays.asList(CoreValuesSourceType.NUMERIC, AnalyticsValuesSourceType.HISTOGRAM), (BoxplotAggregatorSupplier) BoxplotAggregator::new); } diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/ttest/TTestAggregationBuilder.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/ttest/TTestAggregationBuilder.java index a03a34d6e3f1b..9b802e4df37f3 100644 --- a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/ttest/TTestAggregationBuilder.java +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/ttest/TTestAggregationBuilder.java @@ -17,6 +17,7 @@ import org.elasticsearch.search.aggregations.AggregationBuilder; import org.elasticsearch.search.aggregations.AggregatorFactories; import org.elasticsearch.search.aggregations.AggregatorFactory; +import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; import org.elasticsearch.search.aggregations.support.MultiValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.MultiValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.MultiValuesSourceFieldConfig; @@ -24,12 +25,13 @@ import org.elasticsearch.search.aggregations.support.ValueType; import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; +import org.elasticsearch.search.aggregations.support.ValuesSourceType; import java.io.IOException; import java.util.Map; import java.util.Objects; -public class TTestAggregationBuilder extends MultiValuesSourceAggregationBuilder.LeafOnly { +public class TTestAggregationBuilder extends MultiValuesSourceAggregationBuilder.LeafOnly { public static final String NAME = "t_test"; public static final ParseField A_FIELD = new ParseField("a"); public static final ParseField B_FIELD = new ParseField("b"); @@ -53,7 +55,7 @@ public class TTestAggregationBuilder extends MultiValuesSourceAggregationBuilder private int tails = 2; public TTestAggregationBuilder(String name) { - super(name, ValueType.NUMERIC); + super(name); } public TTestAggregationBuilder(TTestAggregationBuilder clone, @@ -91,7 +93,7 @@ public TTestAggregationBuilder tails(int tails) { } public TTestAggregationBuilder(StreamInput in) throws IOException { - super(in, ValueType.NUMERIC); + super(in); testType = in.readEnum(TTestType.class); tails = in.readVInt(); } @@ -112,11 +114,16 @@ protected void innerWriteTo(StreamOutput out) throws IOException { out.writeVInt(tails); } + @Override + protected ValuesSourceType defaultValueSourceType() { + return CoreValuesSourceType.NUMERIC; + } + @Override - protected MultiValuesSourceAggregatorFactory innerBuild( + protected MultiValuesSourceAggregatorFactory innerBuild( QueryShardContext queryShardContext, - Map> configs, + Map configs, DocValueFormat format, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder) throws IOException { diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/ttest/TTestAggregatorFactory.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/ttest/TTestAggregatorFactory.java index 49d26a616105d..e7e9cad8c616f 100644 --- a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/ttest/TTestAggregatorFactory.java +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/ttest/TTestAggregatorFactory.java @@ -14,7 +14,6 @@ import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; import org.elasticsearch.search.aggregations.support.MultiValuesSource; import org.elasticsearch.search.aggregations.support.MultiValuesSourceAggregatorFactory; -import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; import org.elasticsearch.search.internal.SearchContext; @@ -22,12 +21,12 @@ import java.util.List; import java.util.Map; -class TTestAggregatorFactory extends MultiValuesSourceAggregatorFactory{ +class TTestAggregatorFactory extends MultiValuesSourceAggregatorFactory { private final TTestType testType; private final int tails; - TTestAggregatorFactory(String name, Map> configs, TTestType testType, int tails, + TTestAggregatorFactory(String name, Map configs, TTestType testType, int tails, DocValueFormat format, QueryShardContext queryShardContext, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, Map metadata) throws IOException { @@ -55,7 +54,7 @@ protected Aggregator createUnmapped(SearchContext searchContext, @Override protected Aggregator doCreateInternal(SearchContext searchContext, - Map> configs, + Map configs, DocValueFormat format, Aggregator parent, boolean collectsFromSingleBucket, From 85a8e4294344028d3c27a5898c9abaa5744f202a Mon Sep 17 00:00:00 2001 From: Mark Tozzi Date: Tue, 7 Apr 2020 14:57:02 -0400 Subject: [PATCH 07/15] Precommit is happy and so am I --- .../histogram/HistogramAggregatorFactory.java | 1 - .../aggregations/support/ValueType.java | 1 - .../support/ValuesSourceRegistry.java | 2 - .../search/SearchModuleTests.java | 1 - .../SignificantTermsAggregatorTests.java | 4 -- .../aggregations/pipeline/MovAvgIT.java | 45 +++++++++---------- .../aggregations/AggregatorTestCase.java | 3 +- .../xpack/analytics/AnalyticsPlugin.java | 2 +- .../ttest/TTestAggregationBuilder.java | 1 - .../boxplot/BoxplotAggregatorTests.java | 13 +++++- ...regatedPercentileRanksAggregatorTests.java | 3 +- ...eAggregatedPercentilesAggregatorTests.java | 3 +- ...regatedPercentileRanksAggregatorTests.java | 3 +- ...eAggregatedPercentilesAggregatorTests.java | 3 +- .../StringStatsAggregatorTests.java | 2 - .../xpack/rollup/job/RollupIndexer.java | 1 - 16 files changed, 44 insertions(+), 44 deletions(-) diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/HistogramAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/HistogramAggregatorFactory.java index 60c6bdbc1cab0..9fde2e1cbaee5 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/HistogramAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/HistogramAggregatorFactory.java @@ -34,7 +34,6 @@ import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; import org.elasticsearch.search.aggregations.support.ValuesSourceRegistry; -import org.elasticsearch.search.aggregations.support.ValuesSourceType; import org.elasticsearch.search.internal.SearchContext; import java.io.IOException; diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/support/ValueType.java b/server/src/main/java/org/elasticsearch/search/aggregations/support/ValueType.java index 026289af284bd..f643960ba4e8d 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/support/ValueType.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/support/ValueType.java @@ -30,7 +30,6 @@ import java.time.ZoneOffset; import java.util.Arrays; import java.util.List; -import java.util.Set; public enum ValueType implements Writeable { diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceRegistry.java b/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceRegistry.java index 176155f7beb5e..c7c90dee04248 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceRegistry.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceRegistry.java @@ -27,10 +27,8 @@ import org.elasticsearch.search.SearchModule; import org.elasticsearch.search.aggregations.AggregationExecutionException; -import java.lang.reflect.Array; import java.util.AbstractMap; import java.util.ArrayList; -import java.util.Arrays; import java.util.HashMap; import java.util.List; import java.util.Map; diff --git a/server/src/test/java/org/elasticsearch/search/SearchModuleTests.java b/server/src/test/java/org/elasticsearch/search/SearchModuleTests.java index ab99e796ccc00..1f316e2dc0280 100644 --- a/server/src/test/java/org/elasticsearch/search/SearchModuleTests.java +++ b/server/src/test/java/org/elasticsearch/search/SearchModuleTests.java @@ -48,7 +48,6 @@ import org.elasticsearch.search.aggregations.pipeline.MovAvgModel; import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator; import org.elasticsearch.search.aggregations.pipeline.SimpleModel; -import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory; diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorTests.java index c43e3574f7488..b1ba159ef0efd 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorTests.java @@ -46,10 +46,6 @@ import org.elasticsearch.index.mapper.TextFieldMapper.TextFieldType; import org.elasticsearch.index.query.QueryBuilder; import org.elasticsearch.index.query.QueryBuilders; -import org.elasticsearch.search.aggregations.AggregationExecutionException; -import org.elasticsearch.search.aggregations.AggregatorTestCase; -import org.elasticsearch.search.aggregations.bucket.significant.SignificantTermsAggregatorFactory.ExecutionMode; -import org.elasticsearch.search.aggregations.bucket.terms.IncludeExclude; import org.elasticsearch.search.aggregations.AggregationBuilder; import org.elasticsearch.search.aggregations.AggregatorTestCase; import org.elasticsearch.search.aggregations.bucket.significant.SignificantTermsAggregatorFactory.ExecutionMode; diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/MovAvgIT.java b/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/MovAvgIT.java index a478c78e6a279..8055cf09c6ef4 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/MovAvgIT.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/pipeline/MovAvgIT.java @@ -19,28 +19,6 @@ package org.elasticsearch.search.aggregations.pipeline; -import static org.elasticsearch.common.xcontent.XContentFactory.jsonBuilder; -import static org.elasticsearch.search.aggregations.AggregationBuilders.avg; -import static org.elasticsearch.search.aggregations.AggregationBuilders.histogram; -import static org.elasticsearch.search.aggregations.AggregationBuilders.max; -import static org.elasticsearch.search.aggregations.AggregationBuilders.min; -import static org.elasticsearch.search.aggregations.AggregationBuilders.range; -import static org.elasticsearch.search.aggregations.PipelineAggregatorBuilders.derivative; -import static org.elasticsearch.search.aggregations.PipelineAggregatorBuilders.movingAvg; -import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertSearchResponse; -import static org.hamcrest.Matchers.equalTo; -import static org.hamcrest.Matchers.is; -import static org.hamcrest.core.IsNull.notNullValue; -import static org.hamcrest.core.IsNull.nullValue; - -import java.util.ArrayList; -import java.util.Arrays; -import java.util.Collection; -import java.util.HashMap; -import java.util.Iterator; -import java.util.List; -import java.util.Map; - import org.elasticsearch.action.ActionRequestValidationException; import org.elasticsearch.action.bulk.BulkRequestBuilder; import org.elasticsearch.action.index.IndexRequestBuilder; @@ -53,11 +31,32 @@ import org.elasticsearch.search.aggregations.bucket.histogram.Histogram; import org.elasticsearch.search.aggregations.bucket.histogram.Histogram.Bucket; import org.elasticsearch.search.aggregations.metrics.Avg; -import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.test.ESIntegTestCase; import org.hamcrest.Matchers; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Collection; +import java.util.HashMap; +import java.util.Iterator; +import java.util.List; +import java.util.Map; + +import static org.elasticsearch.common.xcontent.XContentFactory.jsonBuilder; +import static org.elasticsearch.search.aggregations.AggregationBuilders.avg; +import static org.elasticsearch.search.aggregations.AggregationBuilders.histogram; +import static org.elasticsearch.search.aggregations.AggregationBuilders.max; +import static org.elasticsearch.search.aggregations.AggregationBuilders.min; +import static org.elasticsearch.search.aggregations.AggregationBuilders.range; +import static org.elasticsearch.search.aggregations.PipelineAggregatorBuilders.derivative; +import static org.elasticsearch.search.aggregations.PipelineAggregatorBuilders.movingAvg; +import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertSearchResponse; +import static org.hamcrest.Matchers.equalTo; +import static org.hamcrest.Matchers.is; +import static org.hamcrest.core.IsNull.notNullValue; +import static org.hamcrest.core.IsNull.nullValue; + @ESIntegTestCase.SuiteScopeTestCase public class MovAvgIT extends ESIntegTestCase { private static final String INTERVAL_FIELD = "l_value"; diff --git a/test/framework/src/main/java/org/elasticsearch/search/aggregations/AggregatorTestCase.java b/test/framework/src/main/java/org/elasticsearch/search/aggregations/AggregatorTestCase.java index 019272fefdbcf..bc46c253ecc42 100644 --- a/test/framework/src/main/java/org/elasticsearch/search/aggregations/AggregatorTestCase.java +++ b/test/framework/src/main/java/org/elasticsearch/search/aggregations/AggregatorTestCase.java @@ -123,7 +123,6 @@ import java.util.function.Function; import java.util.stream.Collectors; -import static java.util.Collections.singleton; import static org.elasticsearch.test.InternalAggregationTestCase.DEFAULT_MAX_BUCKETS; import static org.mockito.Matchers.anyObject; import static org.mockito.Matchers.anyString; @@ -178,7 +177,7 @@ private static void registerFieldTypes(SearchContext searchContext, MapperServic @BeforeClass public static void initValuesSourceRegistry() { - SearchModule searchModule = new SearchModule(Settings.EMPTY, false, Collections.EMPTY_LIST); + SearchModule searchModule = new SearchModule(Settings.EMPTY, false, Collections.emptyList()); valuesSourceRegistry = searchModule.getValuesSourceRegistry(); } diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/AnalyticsPlugin.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/AnalyticsPlugin.java index cceef8a052e4c..a03c9c2acca07 100644 --- a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/AnalyticsPlugin.java +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/AnalyticsPlugin.java @@ -140,7 +140,7 @@ public Map getMappers() { @Override public List> getBareAggregatorRegistrar() { - return List.of(AnalyticsPercentilesAggregatorFactory::registerPercentilesAggregator, + return Arrays.asList(AnalyticsPercentilesAggregatorFactory::registerPercentilesAggregator, AnalyticsPercentilesAggregatorFactory::registerPercentileRanksAggregator); } diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/ttest/TTestAggregationBuilder.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/ttest/TTestAggregationBuilder.java index 9b802e4df37f3..fa0028b3b4b04 100644 --- a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/ttest/TTestAggregationBuilder.java +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/ttest/TTestAggregationBuilder.java @@ -23,7 +23,6 @@ import org.elasticsearch.search.aggregations.support.MultiValuesSourceFieldConfig; import org.elasticsearch.search.aggregations.support.MultiValuesSourceParseHelper; import org.elasticsearch.search.aggregations.support.ValueType; -import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceConfig; import org.elasticsearch.search.aggregations.support.ValuesSourceType; diff --git a/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregatorTests.java b/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregatorTests.java index d5ca763d66a85..5e090e42aedd2 100644 --- a/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregatorTests.java +++ b/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregatorTests.java @@ -19,9 +19,14 @@ import org.apache.lucene.store.Directory; import org.apache.lucene.util.BytesRef; import org.elasticsearch.common.CheckedConsumer; +import org.elasticsearch.common.settings.Settings; import org.elasticsearch.index.mapper.KeywordFieldMapper; import org.elasticsearch.index.mapper.MappedFieldType; import org.elasticsearch.index.mapper.NumberFieldMapper; +import org.elasticsearch.script.MockScriptEngine; +import org.elasticsearch.script.ScriptEngine; +import org.elasticsearch.script.ScriptModule; +import org.elasticsearch.script.ScriptService; import org.elasticsearch.search.aggregations.AggregationBuilder; import org.elasticsearch.search.aggregations.AggregatorTestCase; import org.elasticsearch.search.aggregations.InternalAggregation; @@ -36,7 +41,13 @@ import org.junit.BeforeClass; import java.io.IOException; +import java.util.Arrays; +import java.util.Collections; +import java.util.HashMap; +import java.util.List; +import java.util.Map; import java.util.function.Consumer; +import java.util.function.Function; import static java.util.Collections.singleton; import static org.hamcrest.Matchers.equalTo; @@ -59,7 +70,7 @@ protected AggregationBuilder createAggBuilderForTypeTest(MappedFieldType fieldTy @Override protected List getSupportedValuesSourceTypes() { - return List.of(CoreValuesSourceType.NUMERIC, AnalyticsValuesSourceType.HISTOGRAM); + return Arrays.asList(CoreValuesSourceType.NUMERIC, AnalyticsValuesSourceType.HISTOGRAM); } @Override diff --git a/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/HDRPreAggregatedPercentileRanksAggregatorTests.java b/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/HDRPreAggregatedPercentileRanksAggregatorTests.java index fab7bd91e0ea3..54e15365e27f9 100644 --- a/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/HDRPreAggregatedPercentileRanksAggregatorTests.java +++ b/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/HDRPreAggregatedPercentileRanksAggregatorTests.java @@ -33,6 +33,7 @@ import org.junit.BeforeClass; import java.io.IOException; +import java.util.Arrays; import java.util.Iterator; import java.util.List; @@ -53,7 +54,7 @@ protected AggregationBuilder createAggBuilderForTypeTest(MappedFieldType fieldTy @Override protected List getSupportedValuesSourceTypes() { // Note: this is the same list as Core, plus Analytics - return List.of(CoreValuesSourceType.NUMERIC, + return Arrays.asList(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN, AnalyticsValuesSourceType.HISTOGRAM); diff --git a/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/HDRPreAggregatedPercentilesAggregatorTests.java b/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/HDRPreAggregatedPercentilesAggregatorTests.java index d71797039d78d..d79433cda5648 100644 --- a/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/HDRPreAggregatedPercentilesAggregatorTests.java +++ b/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/HDRPreAggregatedPercentilesAggregatorTests.java @@ -34,6 +34,7 @@ import org.junit.BeforeClass; import java.io.IOException; +import java.util.Arrays; import java.util.Iterator; import java.util.List; import java.util.function.Consumer; @@ -57,7 +58,7 @@ protected AggregationBuilder createAggBuilderForTypeTest(MappedFieldType fieldTy @Override protected List getSupportedValuesSourceTypes() { // Note: this is the same list as Core, plus Analytics - return List.of(CoreValuesSourceType.NUMERIC, + return Arrays.asList(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN, AnalyticsValuesSourceType.HISTOGRAM); diff --git a/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/TDigestPreAggregatedPercentileRanksAggregatorTests.java b/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/TDigestPreAggregatedPercentileRanksAggregatorTests.java index ad80192a3c67b..dcca43eb8dec1 100644 --- a/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/TDigestPreAggregatedPercentileRanksAggregatorTests.java +++ b/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/TDigestPreAggregatedPercentileRanksAggregatorTests.java @@ -34,6 +34,7 @@ import org.junit.BeforeClass; import java.io.IOException; +import java.util.Arrays; import java.util.Collection; import java.util.Iterator; import java.util.List; @@ -56,7 +57,7 @@ protected AggregationBuilder createAggBuilderForTypeTest(MappedFieldType fieldTy @Override protected List getSupportedValuesSourceTypes() { // Note: this is the same list as Core, plus Analytics - return List.of(CoreValuesSourceType.NUMERIC, + return Arrays.asList(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN, AnalyticsValuesSourceType.HISTOGRAM); diff --git a/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/TDigestPreAggregatedPercentilesAggregatorTests.java b/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/TDigestPreAggregatedPercentilesAggregatorTests.java index 6b084535addb8..97e29384445d1 100644 --- a/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/TDigestPreAggregatedPercentilesAggregatorTests.java +++ b/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/mapper/TDigestPreAggregatedPercentilesAggregatorTests.java @@ -35,6 +35,7 @@ import org.junit.BeforeClass; import java.io.IOException; +import java.util.Arrays; import java.util.Collection; import java.util.Iterator; import java.util.List; @@ -59,7 +60,7 @@ protected AggregationBuilder createAggBuilderForTypeTest(MappedFieldType fieldTy @Override protected List getSupportedValuesSourceTypes() { // Note: this is the same list as Core, plus Analytics - return List.of(CoreValuesSourceType.NUMERIC, + return Arrays.asList(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.DATE, CoreValuesSourceType.BOOLEAN, AnalyticsValuesSourceType.HISTOGRAM); diff --git a/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregatorTests.java b/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregatorTests.java index 8d09f336d0bf7..b742e553916cd 100644 --- a/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregatorTests.java +++ b/x-pack/plugin/analytics/src/test/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregatorTests.java @@ -28,8 +28,6 @@ import org.elasticsearch.search.aggregations.bucket.terms.TermsAggregator; import org.elasticsearch.search.aggregations.support.ValueType; import org.junit.BeforeClass; -import org.elasticsearch.search.aggregations.support.ValuesSourceType; -import org.elasticsearch.search.lookup.LeafDocLookup; import java.io.IOException; import java.util.Arrays; diff --git a/x-pack/plugin/rollup/src/main/java/org/elasticsearch/xpack/rollup/job/RollupIndexer.java b/x-pack/plugin/rollup/src/main/java/org/elasticsearch/xpack/rollup/job/RollupIndexer.java index 448e495746bd6..f9c063f32664f 100644 --- a/x-pack/plugin/rollup/src/main/java/org/elasticsearch/xpack/rollup/job/RollupIndexer.java +++ b/x-pack/plugin/rollup/src/main/java/org/elasticsearch/xpack/rollup/job/RollupIndexer.java @@ -26,7 +26,6 @@ import org.elasticsearch.search.aggregations.metrics.MinAggregationBuilder; import org.elasticsearch.search.aggregations.metrics.SumAggregationBuilder; import org.elasticsearch.search.aggregations.metrics.ValueCountAggregationBuilder; -import org.elasticsearch.search.aggregations.support.ValueType; import org.elasticsearch.search.aggregations.support.ValuesSource; import org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder; import org.elasticsearch.search.builder.SearchSourceBuilder; From 547947ca591ba11ff7a360311942c67f589f5e8d Mon Sep 17 00:00:00 2001 From: Mark Tozzi Date: Tue, 7 Apr 2020 16:33:34 -0400 Subject: [PATCH 08/15] Add new Core VSTs to tests --- .../search/aggregations/metrics/MaxAggregatorTests.java | 7 +++++-- .../search/aggregations/metrics/StatsAggregatorTests.java | 7 +++++-- .../search/aggregations/metrics/SumAggregatorTests.java | 6 ++++-- .../aggregations/metrics/ValueCountAggregatorTests.java | 3 +++ .../search/aggregations/AggregatorTestCase.java | 3 ++- 5 files changed, 19 insertions(+), 7 deletions(-) diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/MaxAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/MaxAggregatorTests.java index 1e90a93180b51..84bb65037993f 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/MaxAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/MaxAggregatorTests.java @@ -91,7 +91,6 @@ import static java.util.Collections.emptyList; import static java.util.Collections.singleton; -import static java.util.Collections.singletonList; import static org.elasticsearch.index.query.QueryBuilders.termQuery; import static org.hamcrest.Matchers.equalTo; @@ -163,7 +162,11 @@ protected ScriptService getMockScriptService() { @Override protected List getSupportedValuesSourceTypes() { - return singletonList(CoreValuesSourceType.NUMERIC); + return Arrays.asList( + CoreValuesSourceType.NUMERIC, + CoreValuesSourceType.DATE, + CoreValuesSourceType.BOOLEAN + ); } @Override diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/StatsAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/StatsAggregatorTests.java index 70f83472a7292..97d548ebda945 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/StatsAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/StatsAggregatorTests.java @@ -49,6 +49,7 @@ import java.io.IOException; import java.util.ArrayList; +import java.util.Arrays; import java.util.Collection; import java.util.List; import java.util.Map; @@ -60,7 +61,6 @@ import static java.util.Collections.emptyMap; import static java.util.Collections.emptySet; import static java.util.Collections.singleton; -import static java.util.Collections.singletonList; import static java.util.Collections.singletonMap; import static java.util.stream.Collectors.toList; import static java.util.stream.Collectors.toSet; @@ -475,7 +475,10 @@ void add(double value) { @Override protected List getSupportedValuesSourceTypes() { - return singletonList(CoreValuesSourceType.NUMERIC); + return Arrays.asList( + CoreValuesSourceType.NUMERIC, + CoreValuesSourceType.DATE, + CoreValuesSourceType.BOOLEAN); } @Override diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/SumAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/SumAggregatorTests.java index dfc96ee38f465..c6957ce5825fa 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/SumAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/SumAggregatorTests.java @@ -69,7 +69,6 @@ import static java.util.Collections.emptyMap; import static java.util.Collections.singleton; -import static java.util.Collections.singletonList; import static java.util.Collections.singletonMap; import static java.util.stream.Collectors.toList; import static org.elasticsearch.search.aggregations.AggregationBuilders.sum; @@ -404,7 +403,10 @@ private void testCase(Query query, @Override protected List getSupportedValuesSourceTypes() { - return singletonList(CoreValuesSourceType.NUMERIC); + return Arrays.asList( + CoreValuesSourceType.NUMERIC, + CoreValuesSourceType.DATE, + CoreValuesSourceType.BOOLEAN); } @Override diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregatorTests.java index e7b98e2740f31..54cbe37cd222d 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregatorTests.java @@ -90,7 +90,10 @@ protected AggregationBuilder createAggBuilderForTypeTest(MappedFieldType fieldTy protected List getSupportedValuesSourceTypes() { return Arrays.asList( CoreValuesSourceType.NUMERIC, + CoreValuesSourceType.DATE, + CoreValuesSourceType.BOOLEAN, CoreValuesSourceType.BYTES, + CoreValuesSourceType.IP, CoreValuesSourceType.GEOPOINT, CoreValuesSourceType.RANGE ); diff --git a/test/framework/src/main/java/org/elasticsearch/search/aggregations/AggregatorTestCase.java b/test/framework/src/main/java/org/elasticsearch/search/aggregations/AggregatorTestCase.java index bc46c253ecc42..f6d44df6a6564 100644 --- a/test/framework/src/main/java/org/elasticsearch/search/aggregations/AggregatorTestCase.java +++ b/test/framework/src/main/java/org/elasticsearch/search/aggregations/AggregatorTestCase.java @@ -705,7 +705,8 @@ private void writeTestDoc(MappedFieldType fieldType, String fieldName, RandomInd v = HalfFloatPoint.halfFloatToSortableShort(f); json = "{ \"" + fieldName + "\" : \"" + f + "\" }"; } else { - v = randomLong(); + // smallest numeric is a byte so we select the smallest + v = Math.abs(randomByte()); json = "{ \"" + fieldName + "\" : \"" + v + "\" }"; } doc.add(new SortedNumericDocValuesField(fieldName, v)); From ec935eafd72f4769a90c2331cb02ba78324fe31e Mon Sep 17 00:00:00 2001 From: Mark Tozzi Date: Tue, 7 Apr 2020 16:45:09 -0400 Subject: [PATCH 09/15] Disabled supported type test on SigTerms until we can backport it's fix --- .../bucket/significant/SignificantTermsAggregatorTests.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorTests.java index b1ba159ef0efd..f0fc8ffcf79b4 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorTests.java @@ -82,13 +82,13 @@ protected AggregationBuilder createAggBuilderForTypeTest(MappedFieldType fieldTy return new SignificantTermsAggregationBuilder("foo").field(fieldName); } + /* NOTE - commented out instead of deleted, we need to backport https://github.com/elastic/elasticsearch/pull/52851 to support this. @Override protected List getSupportedValuesSourceTypes() { return Arrays.asList(CoreValuesSourceType.NUMERIC, CoreValuesSourceType.BYTES); } - /* NOTE - commented out instead of deleted, because we probably need to backport this function in AggregatorTestCase @Override protected List unsupportedMappedFieldTypes() { return List.of( From b5e4d98f006409b9d4681f01527ccef5a83aaabf Mon Sep 17 00:00:00 2001 From: Mark Tozzi Date: Tue, 7 Apr 2020 17:10:32 -0400 Subject: [PATCH 10/15] fix checkstyle --- .../bucket/significant/SignificantTermsAggregatorTests.java | 2 -- 1 file changed, 2 deletions(-) diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorTests.java index f0fc8ffcf79b4..da9803a1b6211 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorTests.java @@ -50,8 +50,6 @@ import org.elasticsearch.search.aggregations.AggregatorTestCase; import org.elasticsearch.search.aggregations.bucket.significant.SignificantTermsAggregatorFactory.ExecutionMode; import org.elasticsearch.search.aggregations.bucket.terms.IncludeExclude; -import org.elasticsearch.search.aggregations.support.CoreValuesSourceType; -import org.elasticsearch.search.aggregations.support.ValuesSourceType; import org.junit.Before; import java.io.IOException; From 75c5b15c33c65514d6ef77332b5f24374560995b Mon Sep 17 00:00:00 2001 From: Mark Tozzi Date: Thu, 9 Apr 2020 10:38:18 -0400 Subject: [PATCH 11/15] Fix test failure from semantic merge issue --- .../aggregations/bucket/terms/TermsAggregatorFactory.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorFactory.java index 4d5d89a935320..cff3fcfc7eedb 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorFactory.java @@ -101,10 +101,10 @@ public Aggregator build(String name, if (valuesSource instanceof ValuesSource.Bytes.WithOrdinals == false) { execution = ExecutionMode.MAP; } - final long maxOrd = execution == ExecutionMode.GLOBAL_ORDINALS ? getMaxOrd(valuesSource, context.searcher()) : -1; if (execution == null) { execution = ExecutionMode.GLOBAL_ORDINALS; } + final long maxOrd = execution == ExecutionMode.GLOBAL_ORDINALS ? getMaxOrd(valuesSource, context.searcher()) : -1; if (subAggCollectMode == null) { subAggCollectMode = SubAggCollectionMode.DEPTH_FIRST; // TODO can we remove concept of AggregatorFactories.EMPTY? From 525598ccd26ece0c393c332251686807bb3c89dc Mon Sep 17 00:00:00 2001 From: Mark Tozzi Date: Thu, 9 Apr 2020 13:21:57 -0400 Subject: [PATCH 12/15] Fix some metaData->metadata replacements that got lost --- .../stats/MatrixStatsAggregatorFactory.java | 4 +-- .../ArrayValuesSourceAggregationBuilder.java | 8 ++--- .../ArrayValuesSourceAggregatorFactory.java | 2 +- .../ChildrenAggregationBuilder.java | 4 +-- .../ParentAggregationBuilder.java | 4 +-- .../geogrid/GeoHashGridAggregatorFactory.java | 8 ++--- .../GeoTileGridAggregationBuilder.java | 2 +- .../geogrid/GeoTileGridAggregatorFactory.java | 8 ++--- .../AutoDateHistogramAggregationBuilder.java | 4 +-- .../AutoDateHistogramAggregatorFactory.java | 4 +-- .../DateHistogramAggregationSupplier.java | 2 +- .../DateHistogramAggregatorFactory.java | 12 +++---- .../histogram/HistogramAggregatorFactory.java | 8 ++--- .../missing/MissingAggregationBuilder.java | 4 +-- .../missing/MissingAggregatorFactory.java | 4 +-- .../range/AbstractRangeAggregatorFactory.java | 12 +++---- .../range/BinaryRangeAggregatorFactory.java | 4 +-- .../range/DateRangeAggregatorFactory.java | 4 +-- .../range/GeoDistanceAggregationBuilder.java | 4 +-- .../GeoDistanceRangeAggregatorFactory.java | 6 ++-- .../bucket/range/RangeAggregatorFactory.java | 4 +-- .../bucket/range/RangeAggregatorSupplier.java | 2 +- .../DiversifiedAggregationBuilder.java | 4 +-- .../SignificantTermsAggregationBuilder.java | 4 +-- .../SignificantTermsAggregatorFactory.java | 12 +++---- .../SignificantTermsAggregatorSupplier.java | 2 +- .../terms/RareTermsAggregationBuilder.java | 4 +-- .../terms/RareTermsAggregatorFactory.java | 8 ++--- .../terms/RareTermsAggregatorSupplier.java | 2 +- .../bucket/terms/TermsAggregationBuilder.java | 4 +-- .../bucket/terms/TermsAggregatorFactory.java | 18 +++++----- .../bucket/terms/TermsAggregatorSupplier.java | 2 +- .../metrics/AvgAggregatorFactory.java | 12 +++---- .../metrics/CardinalityAggregatorFactory.java | 8 ++--- .../CardinalityAggregatorSupplier.java | 2 +- .../ExtendedStatsAggregatorFactory.java | 4 +-- .../metrics/GeoBoundsAggregatorFactory.java | 8 ++--- .../metrics/GeoBoundsAggregatorSupplier.java | 2 +- .../GeoCentroidAggregationBuilder.java | 4 +-- .../metrics/GeoCentroidAggregatorFactory.java | 8 ++--- .../GeoCentroidAggregatorSupplier.java | 2 +- .../metrics/GeoGridAggregatorSupplier.java | 2 +- .../metrics/InternalHDRPercentileRanks.java | 4 +-- .../metrics/InternalHDRPercentiles.java | 4 +-- .../InternalTDigestPercentileRanks.java | 4 +-- .../metrics/InternalTDigestPercentiles.java | 4 +-- .../metrics/MaxAggregationBuilder.java | 4 +-- .../metrics/MaxAggregatorFactory.java | 12 +++---- ...ianAbsoluteDeviationAggregatorFactory.java | 10 +++--- ...anAbsoluteDeviationAggregatorSupplier.java | 2 +- .../metrics/MetricAggregatorSupplier.java | 2 +- .../metrics/MinAggregatorFactory.java | 12 +++---- .../metrics/MinMaxAggregatorSupplier.java | 2 +- .../PercentileRanksAggregatorFactory.java | 8 ++--- .../metrics/PercentilesAggregatorFactory.java | 4 +-- .../PercentilesAggregatorSupplier.java | 2 +- .../metrics/PercentilesConfig.java | 2 +- .../metrics/StatsAggregatorFactory.java | 12 +++---- .../metrics/SumAggregatorFactory.java | 8 ++--- .../metrics/ValueCountAggregationBuilder.java | 4 +-- .../metrics/ValueCountAggregatorFactory.java | 12 +++---- .../metrics/ValueCountAggregatorSupplier.java | 2 +- .../support/HistogramAggregatorSupplier.java | 2 +- .../MultiValuesSourceAggregationBuilder.java | 8 ++--- .../ValuesSourceAggregationBuilder.java | 4 +-- .../ValuesSourceAggregatorFactory.java | 6 ++-- .../allocation/AllocationServiceTests.java | 12 +++---- .../search/SearchModuleTests.java | 4 +-- ...ctHistoBackedHDRPercentilesAggregator.java | 4 +-- ...stoBackedTDigestPercentilesAggregator.java | 4 +-- ...AnalyticsPercentilesAggregatorFactory.java | 12 +++---- ...stoBackedHDRPercentileRanksAggregator.java | 4 +-- .../HistoBackedHDRPercentilesAggregator.java | 4 +-- ...ackedTDigestPercentileRanksAggregator.java | 4 +-- ...stoBackedTDigestPercentilesAggregator.java | 4 +-- .../boxplot/BoxplotAggregatorFactory.java | 4 +-- .../boxplot/BoxplotAggregatorSupplier.java | 2 +- .../StringStatsAggregationBuilder.java | 4 +-- .../StringStatsAggregatorFactory.java | 8 ++--- .../StringStatsAggregatorSupplier.java | 2 +- .../MockDeprecatedAggregationBuilder.java | 4 +-- .../CachedBlobContainerIndexInputTests.java | 4 +-- .../sql/jdbc/JdbcResultSetMetaDataTests.java | 36 +++++++++---------- .../xpack/sql/qa/jdbc/JdbcAssert.java | 10 +++--- .../xpack/sql/qa/jdbc/JdbcTestUtils.java | 24 ++++++------- .../qa/jdbc/ResultSetMetaDataTestCase.java | 6 ++-- 86 files changed, 258 insertions(+), 258 deletions(-) diff --git a/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/matrix/stats/MatrixStatsAggregatorFactory.java b/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/matrix/stats/MatrixStatsAggregatorFactory.java index 89c1efbc33175..89cb82c8640c4 100644 --- a/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/matrix/stats/MatrixStatsAggregatorFactory.java +++ b/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/matrix/stats/MatrixStatsAggregatorFactory.java @@ -61,7 +61,7 @@ protected Aggregator doCreateInternal(Map valuesSources, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, - Map metaData) throws IOException { + Map metadata) throws IOException { Map typedValuesSources = new HashMap<>(valuesSources.size()); for (Map.Entry entry : valuesSources.entrySet()) { if (entry.getValue() instanceof ValuesSource.Numeric == false) { @@ -71,6 +71,6 @@ protected Aggregator doCreateInternal(Map valuesSources, // TODO: There must be a better option than this. typedValuesSources.put(entry.getKey(), (ValuesSource.Numeric) entry.getValue()); } - return new MatrixStatsAggregator(name, typedValuesSources, searchContext, parent, multiValueMode, metaData); + return new MatrixStatsAggregator(name, typedValuesSources, searchContext, parent, multiValueMode, metadata); } } diff --git a/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/support/ArrayValuesSourceAggregationBuilder.java b/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/support/ArrayValuesSourceAggregationBuilder.java index ba23e689ecc67..2d644bf629c7f 100644 --- a/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/support/ArrayValuesSourceAggregationBuilder.java +++ b/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/support/ArrayValuesSourceAggregationBuilder.java @@ -49,8 +49,8 @@ protected LeafOnly(String name) { super(name); } - protected LeafOnly(LeafOnly clone, Builder factoriesBuilder, Map metaData) { - super(clone, factoriesBuilder, metaData); + protected LeafOnly(LeafOnly clone, Builder factoriesBuilder, Map metadata) { + super(clone, factoriesBuilder, metadata); if (factoriesBuilder.count() > 0) { throw new AggregationInitializationException("Aggregator [" + name + "] of type [" + getType() + "] cannot accept sub-aggregations"); @@ -90,8 +90,8 @@ protected ArrayValuesSourceAggregationBuilder(String name) { } protected ArrayValuesSourceAggregationBuilder(ArrayValuesSourceAggregationBuilder clone, - Builder factoriesBuilder, Map metaData) { - super(clone, factoriesBuilder, metaData); + Builder factoriesBuilder, Map metadata) { + super(clone, factoriesBuilder, metadata); this.fields = new ArrayList<>(clone.fields); this.userValueTypeHint = clone.userValueTypeHint; this.format = clone.format; diff --git a/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/support/ArrayValuesSourceAggregatorFactory.java b/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/support/ArrayValuesSourceAggregatorFactory.java index c9951bb777e6c..147f2157b0a0b 100644 --- a/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/support/ArrayValuesSourceAggregatorFactory.java +++ b/modules/aggs-matrix-stats/src/main/java/org/elasticsearch/search/aggregations/support/ArrayValuesSourceAggregatorFactory.java @@ -46,7 +46,7 @@ public ArrayValuesSourceAggregatorFactory(String name, Map metaData) throws IOException { + Map metadata) throws IOException { HashMap valuesSources = new HashMap<>(); for (Map.Entry config : configs.entrySet()) { diff --git a/modules/parent-join/src/main/java/org/elasticsearch/join/aggregations/ChildrenAggregationBuilder.java b/modules/parent-join/src/main/java/org/elasticsearch/join/aggregations/ChildrenAggregationBuilder.java index f965457209eaa..1e821f0a939f8 100644 --- a/modules/parent-join/src/main/java/org/elasticsearch/join/aggregations/ChildrenAggregationBuilder.java +++ b/modules/parent-join/src/main/java/org/elasticsearch/join/aggregations/ChildrenAggregationBuilder.java @@ -79,8 +79,8 @@ protected ValuesSourceType defaultValueSourceType() { } @Override - protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metaData) { - return new ChildrenAggregationBuilder(this, factoriesBuilder, metaData); + protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metadata) { + return new ChildrenAggregationBuilder(this, factoriesBuilder, metadata); } /** diff --git a/modules/parent-join/src/main/java/org/elasticsearch/join/aggregations/ParentAggregationBuilder.java b/modules/parent-join/src/main/java/org/elasticsearch/join/aggregations/ParentAggregationBuilder.java index 8c76abf7588ff..3914179ef264f 100644 --- a/modules/parent-join/src/main/java/org/elasticsearch/join/aggregations/ParentAggregationBuilder.java +++ b/modules/parent-join/src/main/java/org/elasticsearch/join/aggregations/ParentAggregationBuilder.java @@ -79,8 +79,8 @@ protected ValuesSourceType defaultValueSourceType() { } @Override - protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metaData) { - return new ParentAggregationBuilder(this, factoriesBuilder, metaData); + protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metadata) { + return new ParentAggregationBuilder(this, factoriesBuilder, metadata); } /** diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoHashGridAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoHashGridAggregatorFactory.java index 2a43ad10d33b0..3471b32b7c189 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoHashGridAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoHashGridAggregatorFactory.java @@ -78,7 +78,7 @@ protected Aggregator doCreateInternal(final ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, - Map metaData) throws IOException { + Map metadata) throws IOException { AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry() .getAggregator(config.valueSourceType(), GeoHashGridAggregationBuilder.NAME); if (aggregatorSupplier instanceof GeoGridAggregatorSupplier == false) { @@ -89,17 +89,17 @@ protected Aggregator doCreateInternal(final ValuesSource valuesSource, return asMultiBucketAggregator(this, searchContext, parent); } return ((GeoGridAggregatorSupplier) aggregatorSupplier).build(name, factories, valuesSource, precision, geoBoundingBox, - requiredSize, shardSize, searchContext, parent, metaData); + requiredSize, shardSize, searchContext, parent, metadata); } static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { valuesSourceRegistry.register(GeoHashGridAggregationBuilder.NAME, CoreValuesSourceType.GEOPOINT, (GeoGridAggregatorSupplier) (name, factories, valuesSource, precision, geoBoundingBox, requiredSize, shardSize, - aggregationContext, parent, metaData) -> { + aggregationContext, parent, metadata) -> { CellIdSource cellIdSource = new CellIdSource((ValuesSource.GeoPoint) valuesSource, precision, geoBoundingBox, Geohash::longEncode); return new GeoHashGridAggregator(name, factories, cellIdSource, requiredSize, shardSize, aggregationContext, - parent, metaData); + parent, metadata); }); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoTileGridAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoTileGridAggregationBuilder.java index 7026ab643cbfc..0c06a6f5bac61 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoTileGridAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoTileGridAggregationBuilder.java @@ -66,7 +66,7 @@ public GeoGridAggregationBuilder precision(int precision) { protected ValuesSourceAggregatorFactory createFactory( String name, ValuesSourceConfig config, int precision, int requiredSize, int shardSize, GeoBoundingBox geoBoundingBox, QueryShardContext queryShardContext, AggregatorFactory parent, - AggregatorFactories.Builder subFactoriesBuilder, Map metaData) throws IOException { + AggregatorFactories.Builder subFactoriesBuilder, Map metadata) throws IOException { return new GeoTileGridAggregatorFactory(name, config, precision, requiredSize, shardSize, geoBoundingBox, queryShardContext, parent, subFactoriesBuilder, metadata); } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoTileGridAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoTileGridAggregatorFactory.java index 7b3d341349ceb..406f46b3cafb1 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoTileGridAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/geogrid/GeoTileGridAggregatorFactory.java @@ -76,7 +76,7 @@ protected Aggregator doCreateInternal(final ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, - Map metaData) throws IOException { + Map metadata) throws IOException { AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry() .getAggregator(config.valueSourceType(), GeoTileGridAggregationBuilder.NAME); if (aggregatorSupplier instanceof GeoGridAggregatorSupplier == false) { @@ -87,17 +87,17 @@ protected Aggregator doCreateInternal(final ValuesSource valuesSource, return asMultiBucketAggregator(this, searchContext, parent); } return ((GeoGridAggregatorSupplier) aggregatorSupplier).build(name, factories, valuesSource, precision, geoBoundingBox, - requiredSize, shardSize, searchContext, parent, metaData); + requiredSize, shardSize, searchContext, parent, metadata); } static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { valuesSourceRegistry.register(GeoTileGridAggregationBuilder.NAME, CoreValuesSourceType.GEOPOINT, (GeoGridAggregatorSupplier) (name, factories, valuesSource, precision, geoBoundingBox, requiredSize, shardSize, - aggregationContext, parent, metaData) -> { + aggregationContext, parent, metadata) -> { CellIdSource cellIdSource = new CellIdSource((ValuesSource.GeoPoint) valuesSource, precision, geoBoundingBox, GeoTileUtils::longEncode); return new GeoTileGridAggregator(name, factories, cellIdSource, requiredSize, shardSize, aggregationContext, - parent, metaData); + parent, metadata); }); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/AutoDateHistogramAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/AutoDateHistogramAggregationBuilder.java index fab115e9dd063..643af94d16b5c 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/AutoDateHistogramAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/AutoDateHistogramAggregationBuilder.java @@ -146,8 +146,8 @@ protected ValuesSourceType defaultValueSourceType() { } @Override - protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metaData) { - return new AutoDateHistogramAggregationBuilder(this, factoriesBuilder, metaData); + protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metadata) { + return new AutoDateHistogramAggregationBuilder(this, factoriesBuilder, metadata); } @Override diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/AutoDateHistogramAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/AutoDateHistogramAggregatorFactory.java index 14cfac9158606..758bec06c44d9 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/AutoDateHistogramAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/AutoDateHistogramAggregatorFactory.java @@ -58,7 +58,7 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, - Map metaData) throws IOException { + Map metadata) throws IOException { if (valuesSource instanceof Numeric == false) { throw new AggregationExecutionException("ValuesSource type " + valuesSource.toString() + "is not supported for aggregation " + this.name()); @@ -66,7 +66,7 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, if (collectsFromSingleBucket == false) { return asMultiBucketAggregator(this, searchContext, parent); } - return createAggregator((Numeric) valuesSource, searchContext, parent, metaData); + return createAggregator((Numeric) valuesSource, searchContext, parent, metadata); } private Aggregator createAggregator(ValuesSource.Numeric valuesSource, diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/DateHistogramAggregationSupplier.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/DateHistogramAggregationSupplier.java index a2aebe6d19bad..623f75204e696 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/DateHistogramAggregationSupplier.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/DateHistogramAggregationSupplier.java @@ -46,5 +46,5 @@ Aggregator build(String name, DocValueFormat formatter, SearchContext aggregationContext, Aggregator parent, - Map metaData) throws IOException; + Map metadata) throws IOException; } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/DateHistogramAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/DateHistogramAggregatorFactory.java index c345600ee2cd7..9b407fd24a786 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/DateHistogramAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/DateHistogramAggregatorFactory.java @@ -58,9 +58,9 @@ public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry DocValueFormat formatter, SearchContext aggregationContext, Aggregator parent, - Map metaData) -> new DateHistogramAggregator(name, + Map metadata) -> new DateHistogramAggregator(name, factories, rounding, shardRounding, order, keyed, minDocCount, extendedBounds, (ValuesSource.Numeric) valuesSource, - formatter, aggregationContext, parent, metaData)); + formatter, aggregationContext, parent, metadata)); valuesSourceRegistry.register(DateHistogramAggregationBuilder.NAME, CoreValuesSourceType.RANGE, @@ -76,7 +76,7 @@ public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry DocValueFormat formatter, SearchContext aggregationContext, Aggregator parent, - Map metaData) -> { + Map metadata) -> { ValuesSource.Range rangeValueSource = (ValuesSource.Range) valuesSource; if (rangeValueSource.rangeType() != RangeType.DATE) { @@ -85,7 +85,7 @@ public static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry } return new DateRangeHistogramAggregator(name, factories, rounding, shardRounding, order, keyed, minDocCount, extendedBounds, rangeValueSource, formatter, - aggregationContext, parent, metaData); }); + aggregationContext, parent, metadata); }); } private final BucketOrder order; @@ -135,8 +135,8 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, @Override protected Aggregator createUnmapped(SearchContext searchContext, Aggregator parent, - Map metaData) throws IOException { + Map metadata) throws IOException { return new DateHistogramAggregator(name, factories, rounding, shardRounding, order, keyed, minDocCount, extendedBounds, - null, config.format(), searchContext, parent, metaData); + null, config.format(), searchContext, parent, metadata); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/HistogramAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/HistogramAggregatorFactory.java index 1c8520a52c276..dd850161d8b49 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/HistogramAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/histogram/HistogramAggregatorFactory.java @@ -61,14 +61,14 @@ public Aggregator build(String name, AggregatorFactories factories, double inter BucketOrder order, boolean keyed, long minDocCount, double minBound, double maxBound, ValuesSource valuesSource, DocValueFormat formatter, SearchContext context, Aggregator parent, - Map metaData) throws IOException { + Map metadata) throws IOException { ValuesSource.Range rangeValueSource = (ValuesSource.Range) valuesSource; if (rangeValueSource.rangeType().isNumeric() == false) { throw new IllegalArgumentException("Expected numeric range type but found non-numeric range [" + rangeValueSource.rangeType().name + "]"); } return new RangeHistogramAggregator(name, factories, interval, offset, order, keyed, minDocCount, minBound, - maxBound, rangeValueSource, formatter, context, parent, metaData); + maxBound, rangeValueSource, formatter, context, parent, metadata); } } ); @@ -83,9 +83,9 @@ public Aggregator build(String name, AggregatorFactories factories, double inter BucketOrder order, boolean keyed, long minDocCount, double minBound, double maxBound, ValuesSource valuesSource, DocValueFormat formatter, SearchContext context, Aggregator parent, - Map metaData) throws IOException { + Map metadata) throws IOException { return new NumericHistogramAggregator(name, factories, interval, offset, order, keyed, minDocCount, minBound, - maxBound, (ValuesSource.Numeric) valuesSource, formatter, context, parent, metaData); + maxBound, (ValuesSource.Numeric) valuesSource, formatter, context, parent, metadata); } } ); diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/missing/MissingAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/missing/MissingAggregationBuilder.java index 440ac9aeccc49..24a08f705ceae 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/missing/MissingAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/missing/MissingAggregationBuilder.java @@ -60,8 +60,8 @@ protected ValuesSourceType defaultValueSourceType() { } @Override - protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metaData) { - return new MissingAggregationBuilder(this, factoriesBuilder, metaData); + protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metadata) { + return new MissingAggregationBuilder(this, factoriesBuilder, metadata); } /** diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/missing/MissingAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/missing/MissingAggregatorFactory.java index f06fab847debd..08239212e139d 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/missing/MissingAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/missing/MissingAggregatorFactory.java @@ -35,8 +35,8 @@ public class MissingAggregatorFactory extends ValuesSourceAggregatorFactory { public MissingAggregatorFactory(String name, ValuesSourceConfig config, QueryShardContext queryShardContext, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, - Map metaData) throws IOException { - super(name, config, queryShardContext, parent, subFactoriesBuilder, metaData); + Map metadata) throws IOException { + super(name, config, queryShardContext, parent, subFactoriesBuilder, metadata); } @Override diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/AbstractRangeAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/AbstractRangeAggregatorFactory.java index d7a276fba1f99..e212d1d541126 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/AbstractRangeAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/AbstractRangeAggregatorFactory.java @@ -61,9 +61,9 @@ public Aggregator build(String name, boolean keyed, SearchContext context, Aggregator parent, - Map metaData) throws IOException { + Map metadata) throws IOException { return new RangeAggregator(name, factories, valuesSource, format, rangeFactory, ranges, keyed, context, parent, - metaData); + metadata); } }); } @@ -76,8 +76,8 @@ public AbstractRangeAggregatorFactory(String name, QueryShardContext queryShardContext, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, - Map metaData) throws IOException { - super(name, config, queryShardContext, parent, subFactoriesBuilder, metaData); + Map metadata) throws IOException { + super(name, config, queryShardContext, parent, subFactoriesBuilder, metadata); this.ranges = ranges; this.keyed = keyed; this.rangeFactory = rangeFactory; @@ -96,7 +96,7 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, - Map metaData) throws IOException { + Map metadata) throws IOException { AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), aggregationTypeName); if (aggregatorSupplier instanceof RangeAggregatorSupplier == false) { @@ -104,6 +104,6 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, aggregatorSupplier.getClass().toString() + "]"); } return ((RangeAggregatorSupplier)aggregatorSupplier).build(name, factories, (Numeric) valuesSource, config.format(), rangeFactory, - ranges, keyed, searchContext, parent, metaData); + ranges, keyed, searchContext, parent, metadata); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/BinaryRangeAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/BinaryRangeAggregatorFactory.java index 29d70e3c19799..06585389446fa 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/BinaryRangeAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/BinaryRangeAggregatorFactory.java @@ -59,13 +59,13 @@ protected Aggregator createUnmapped(SearchContext searchContext, Aggregator pare protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, - Map metaData) throws IOException { + Map metadata) throws IOException { if (valuesSource instanceof ValuesSource.Bytes == false) { throw new AggregationExecutionException("ValuesSource type " + valuesSource.toString() + "is not supported for aggregation " + this.name()); } return new BinaryRangeAggregator(name, factories, (ValuesSource.Bytes) valuesSource, config.format(), - ranges, keyed, searchContext, parent, metaData); + ranges, keyed, searchContext, parent, metadata); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/DateRangeAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/DateRangeAggregatorFactory.java index c981c13c839a9..4c3883d4819fd 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/DateRangeAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/DateRangeAggregatorFactory.java @@ -37,9 +37,9 @@ public DateRangeAggregatorFactory(String name, QueryShardContext queryShardContext, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, - Map metaData) throws IOException { + Map metadata) throws IOException { super(name, DateRangeAggregationBuilder.NAME, config, ranges, keyed, rangeFactory, queryShardContext, parent, subFactoriesBuilder, - metaData); + metadata); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/GeoDistanceAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/GeoDistanceAggregationBuilder.java index 569eb10be0991..f22d0e0d5f5ea 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/GeoDistanceAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/GeoDistanceAggregationBuilder.java @@ -273,8 +273,8 @@ protected ValuesSourceType defaultValueSourceType() { } @Override - protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metaData) { - return new GeoDistanceAggregationBuilder(this, factoriesBuilder, metaData); + protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metadata) { + return new GeoDistanceAggregationBuilder(this, factoriesBuilder, metadata); } GeoDistanceAggregationBuilder origin(GeoPoint origin) { diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/GeoDistanceRangeAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/GeoDistanceRangeAggregatorFactory.java index f5e43cd97828c..329d47d1d834b 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/GeoDistanceRangeAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/GeoDistanceRangeAggregatorFactory.java @@ -56,8 +56,8 @@ public GeoDistanceRangeAggregatorFactory(String name, ValuesSourceConfig config, Range[] ranges, DistanceUnit unit, GeoDistance distanceType, boolean keyed, QueryShardContext queryShardContext, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, - Map metaData) throws IOException { - super(name, config, queryShardContext, parent, subFactoriesBuilder, metaData); + Map metadata) throws IOException { + super(name, config, queryShardContext, parent, subFactoriesBuilder, metadata); this.origin = origin; this.ranges = ranges; this.unit = unit; @@ -78,7 +78,7 @@ protected Aggregator doCreateInternal(final ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, - Map metaData) throws IOException { + Map metadata) throws IOException { if (valuesSource instanceof ValuesSource.GeoPoint == false) { throw new AggregationExecutionException("ValuesSource type " + valuesSource.toString() + "is not supported for aggregation " + this.name()); diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/RangeAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/RangeAggregatorFactory.java index 7d30605d19646..f544f6a31b2c7 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/RangeAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/RangeAggregatorFactory.java @@ -39,9 +39,9 @@ public RangeAggregatorFactory(String name, QueryShardContext queryShardContext, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, - Map metaData) throws IOException { + Map metadata) throws IOException { super(name, RangeAggregationBuilder.NAME, config, ranges, keyed, rangeFactory, queryShardContext, parent, subFactoriesBuilder, - metaData); + metadata); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/RangeAggregatorSupplier.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/RangeAggregatorSupplier.java index a38d434af1735..e8d01fca01a63 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/RangeAggregatorSupplier.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/range/RangeAggregatorSupplier.java @@ -38,5 +38,5 @@ Aggregator build(String name, boolean keyed, SearchContext context, Aggregator parent, - Map metaData) throws IOException; + Map metadata) throws IOException; } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/sampler/DiversifiedAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/sampler/DiversifiedAggregationBuilder.java index 6c5e1d5cd4e47..37716bd9991d5 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/sampler/DiversifiedAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/sampler/DiversifiedAggregationBuilder.java @@ -72,8 +72,8 @@ protected ValuesSourceType defaultValueSourceType() { } @Override - protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metaData) { - return new DiversifiedAggregationBuilder(this, factoriesBuilder, metaData); + protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metadata) { + return new DiversifiedAggregationBuilder(this, factoriesBuilder, metadata); } /** diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregationBuilder.java index f0f8116911383..5fad5882fbce3 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregationBuilder.java @@ -134,8 +134,8 @@ protected ValuesSourceType defaultValueSourceType() { } @Override - protected SignificantTermsAggregationBuilder shallowCopy(Builder factoriesBuilder, Map metaData) { - return new SignificantTermsAggregationBuilder(this, factoriesBuilder, metaData); + protected SignificantTermsAggregationBuilder shallowCopy(Builder factoriesBuilder, Map metadata) { + return new SignificantTermsAggregationBuilder(this, factoriesBuilder, metadata); } protected AggregationBuilder doRewrite(QueryRewriteContext queryShardContext) throws IOException { diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorFactory.java index d85e96b6a2f76..c74c3dc074231 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorFactory.java @@ -105,7 +105,7 @@ public Aggregator build(String name, Aggregator parent, SignificanceHeuristic significanceHeuristic, SignificantTermsAggregatorFactory sigTermsFactory, - Map metaData) throws IOException { + Map metadata) throws IOException { ExecutionMode execution = null; if (executionHint != null) { @@ -125,7 +125,7 @@ public Aggregator build(String name, } return execution.create(name, factories, valuesSource, format, bucketCountThresholds, includeExclude, context, parent, - significanceHeuristic, sigTermsFactory, metaData); + significanceHeuristic, sigTermsFactory, metadata); } }; @@ -149,7 +149,7 @@ public Aggregator build(String name, Aggregator parent, SignificanceHeuristic significanceHeuristic, SignificantTermsAggregatorFactory sigTermsFactory, - Map metaData) throws IOException { + Map metadata) throws IOException { if ((includeExclude != null) && (includeExclude.isRegexBased())) { throw new IllegalArgumentException("Aggregation [" + name + "] cannot support regular expression style include/exclude " @@ -168,7 +168,7 @@ public Aggregator build(String name, return new SignificantLongTermsAggregator(name, factories, (ValuesSource.Numeric) valuesSource, format, bucketCountThresholds, context, parent, significanceHeuristic, sigTermsFactory, longFilter, - metaData); + metadata); } }; @@ -184,8 +184,8 @@ public Aggregator build(String name, QueryShardContext queryShardContext, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, - Map metaData) throws IOException { - super(name, config, queryShardContext, parent, subFactoriesBuilder, metaData); + Map metadata) throws IOException { + super(name, config, queryShardContext, parent, subFactoriesBuilder, metadata); if (config.unmapped() == false) { if (config.fieldContext().fieldType().isSearchable() == false) { diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorSupplier.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorSupplier.java index 10853370bcfdd..5f8ddf51387df 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorSupplier.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/SignificantTermsAggregatorSupplier.java @@ -43,5 +43,5 @@ Aggregator build(String name, Aggregator parent, SignificanceHeuristic significanceHeuristic, SignificantTermsAggregatorFactory sigTermsFactory, - Map metaData) throws IOException; + Map metadata) throws IOException; } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregationBuilder.java index e11edec07845e..a6d74e52e26bd 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregationBuilder.java @@ -84,8 +84,8 @@ protected ValuesSourceType defaultValueSourceType() { } @Override - protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metaData) { - return new RareTermsAggregationBuilder(this, factoriesBuilder, metaData); + protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metadata) { + return new RareTermsAggregationBuilder(this, factoriesBuilder, metadata); } /** diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregatorFactory.java index 12a00e70fae46..a9e5981112088 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregatorFactory.java @@ -72,7 +72,7 @@ public Aggregator build(String name, IncludeExclude includeExclude, SearchContext context, Aggregator parent, - Map metaData) throws IOException { + Map metadata) throws IOException { ExecutionMode execution = ExecutionMode.MAP; //TODO global ords not implemented yet, only supports "map" @@ -83,7 +83,7 @@ public Aggregator build(String name, } return execution.create(name, factories, valuesSource, format, - includeExclude, context, parent, metaData, maxDocCount, precision); + includeExclude, context, parent, metadata, maxDocCount, precision); } }; @@ -105,7 +105,7 @@ public Aggregator build(String name, IncludeExclude includeExclude, SearchContext context, Aggregator parent, - Map metaData) throws IOException { + Map metadata) throws IOException { if ((includeExclude != null) && (includeExclude.isRegexBased())) { throw new IllegalArgumentException("Aggregation [" + name + "] cannot support regular expression " + @@ -121,7 +121,7 @@ public Aggregator build(String name, longFilter = includeExclude.convertToLongFilter(format); } return new LongRareTermsAggregator(name, factories, (ValuesSource.Numeric) valuesSource, format, - context, parent, longFilter, maxDocCount, precision, metaData); + context, parent, longFilter, maxDocCount, precision, metadata); } }; } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregatorSupplier.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregatorSupplier.java index 05f09f90a55a3..1a8b20a8a8727 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregatorSupplier.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/RareTermsAggregatorSupplier.java @@ -38,5 +38,5 @@ Aggregator build(String name, IncludeExclude includeExclude, SearchContext context, Aggregator parent, - Map metaData) throws IOException; + Map metadata) throws IOException; } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregationBuilder.java index 900d607542c19..455215e55dbba 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregationBuilder.java @@ -125,8 +125,8 @@ protected ValuesSourceType defaultValueSourceType() { } @Override - protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metaData) { - return new TermsAggregationBuilder(this, factoriesBuilder, metaData); + protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metadata) { + return new TermsAggregationBuilder(this, factoriesBuilder, metadata); } /** diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorFactory.java index cff3fcfc7eedb..c1f85d3cbc392 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorFactory.java @@ -91,7 +91,7 @@ public Aggregator build(String name, Aggregator parent, SubAggCollectionMode subAggCollectMode, boolean showTermDocCountError, - Map metaData) throws IOException { + Map metadata) throws IOException { ExecutionMode execution = null; if (executionHint != null) { @@ -122,7 +122,7 @@ public Aggregator build(String name, // TODO: [Zach] we might want refactor and remove ExecutionMode#create(), moving that logic outside the enum return execution.create(name, factories, valuesSource, order, format, bucketCountThresholds, includeExclude, - context, parent, subAggCollectMode, showTermDocCountError, metaData); + context, parent, subAggCollectMode, showTermDocCountError, metadata); } }; @@ -147,7 +147,7 @@ public Aggregator build(String name, Aggregator parent, SubAggCollectionMode subAggCollectMode, boolean showTermDocCountError, - Map metaData) throws IOException { + Map metadata) throws IOException { if ((includeExclude != null) && (includeExclude.isRegexBased())) { throw new AggregationExecutionException("Aggregation [" + name + "] cannot support regular expression style " @@ -170,14 +170,14 @@ public Aggregator build(String name, } return new DoubleTermsAggregator(name, factories, (ValuesSource.Numeric) valuesSource, format, order, bucketCountThresholds, context, parent, subAggCollectMode, showTermDocCountError, longFilter, - metaData); + metadata); } if (includeExclude != null) { longFilter = includeExclude.convertToLongFilter(format); } return new LongTermsAggregator(name, factories, (ValuesSource.Numeric) valuesSource, format, order, bucketCountThresholds, context, parent, subAggCollectMode, showTermDocCountError, longFilter, - metaData); + metadata); } }; } @@ -193,8 +193,8 @@ public Aggregator build(String name, QueryShardContext queryShardContext, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, - Map metaData) throws IOException { - super(name, config, queryShardContext, parent, subFactoriesBuilder, metaData); + Map metadata) throws IOException { + super(name, config, queryShardContext, parent, subFactoriesBuilder, metadata); this.order = order; this.includeExclude = includeExclude; this.executionHint = executionHint; @@ -236,7 +236,7 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, - Map metaData) throws IOException { + Map metadata) throws IOException { if (collectsFromSingleBucket == false) { return asMultiBucketAggregator(this, searchContext, parent); } @@ -261,7 +261,7 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, return termsAggregatorSupplier.build(name, factories, valuesSource, order, config.format(), bucketCountThresholds, includeExclude, executionHint, searchContext, parent, collectMode, - showTermDocCountError, metaData); + showTermDocCountError, metadata); } // return the SubAggCollectionMode that this aggregation should use based on the expected size diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorSupplier.java b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorSupplier.java index 769ffbbef8489..240b451454ca9 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorSupplier.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/bucket/terms/TermsAggregatorSupplier.java @@ -42,5 +42,5 @@ Aggregator build(String name, Aggregator parent, Aggregator.SubAggCollectionMode subAggCollectMode, boolean showTermDocCountError, - Map metaData) throws IOException; + Map metadata) throws IOException; } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/AvgAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/AvgAggregatorFactory.java index 2352ca8092ecd..e8db1298f7cb0 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/AvgAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/AvgAggregatorFactory.java @@ -42,8 +42,8 @@ class AvgAggregatorFactory extends ValuesSourceAggregatorFactory { AvgAggregatorFactory(String name, ValuesSourceConfig config, QueryShardContext queryShardContext, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, - Map metaData) throws IOException { - super(name, config, queryShardContext, parent, subFactoriesBuilder, metaData); + Map metadata) throws IOException { + super(name, config, queryShardContext, parent, subFactoriesBuilder, metadata); } static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { @@ -56,8 +56,8 @@ public Aggregator build(String name, DocValueFormat formatter, SearchContext context, Aggregator parent, - Map metaData) throws IOException { - return new AvgAggregator(name, (Numeric) valuesSource, formatter, context, parent, metaData); + Map metadata) throws IOException { + return new AvgAggregator(name, (Numeric) valuesSource, formatter, context, parent, metadata); } }); } @@ -74,7 +74,7 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, - Map metaData) throws IOException { + Map metadata) throws IOException { AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), AvgAggregationBuilder.NAME); @@ -82,6 +82,6 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, throw new AggregationExecutionException("Registry miss-match - expected MetricAggregatorSupplier, found [" + aggregatorSupplier.getClass().toString() + "]"); } - return ((MetricAggregatorSupplier) aggregatorSupplier).build(name, valuesSource, config.format(), searchContext, parent, metaData); + return ((MetricAggregatorSupplier) aggregatorSupplier).build(name, valuesSource, config.format(), searchContext, parent, metadata); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/CardinalityAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/CardinalityAggregatorFactory.java index 63acc493cfc9b..9ff49225bd242 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/CardinalityAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/CardinalityAggregatorFactory.java @@ -60,8 +60,8 @@ public Aggregator build(String name, int precision, SearchContext context, Aggregator parent, - Map metaData) throws IOException { - return new CardinalityAggregator(name, valuesSource, precision, context, parent, metaData); + Map metadata) throws IOException { + return new CardinalityAggregator(name, valuesSource, precision, context, parent, metadata); } }; } @@ -78,7 +78,7 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, - Map metaData) throws IOException { + Map metadata) throws IOException { AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), CardinalityAggregationBuilder.NAME); if (aggregatorSupplier instanceof CardinalityAggregatorSupplier == false) { @@ -86,7 +86,7 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, aggregatorSupplier.getClass().toString() + "]"); } CardinalityAggregatorSupplier cardinalityAggregatorSupplier = (CardinalityAggregatorSupplier) aggregatorSupplier; - return cardinalityAggregatorSupplier.build(name, valuesSource, precision(), searchContext, parent, metaData); + return cardinalityAggregatorSupplier.build(name, valuesSource, precision(), searchContext, parent, metadata); } private int precision() { diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/CardinalityAggregatorSupplier.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/CardinalityAggregatorSupplier.java index b5d30e80a5947..e169a68614d48 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/CardinalityAggregatorSupplier.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/CardinalityAggregatorSupplier.java @@ -33,5 +33,5 @@ Aggregator build(String name, int precision, SearchContext context, Aggregator parent, - Map metaData) throws IOException; + Map metadata) throws IOException; } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ExtendedStatsAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ExtendedStatsAggregatorFactory.java index 78b41cd8460b2..289836526a972 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ExtendedStatsAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ExtendedStatsAggregatorFactory.java @@ -60,12 +60,12 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, - Map metaData) throws IOException { + Map metadata) throws IOException { if (valuesSource instanceof Numeric == false) { throw new AggregationExecutionException("ValuesSource type " + valuesSource.toString() + "is not supported for aggregation " + this.name()); } return new ExtendedStatsAggregator(name, (Numeric) valuesSource, config.format(), searchContext, - parent, sigma, metaData); + parent, sigma, metadata); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoBoundsAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoBoundsAggregatorFactory.java index 0dec0db35fb36..f4e2dea3bb70c 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoBoundsAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoBoundsAggregatorFactory.java @@ -62,7 +62,7 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, - Map metaData) throws IOException { + Map metadata) throws IOException { AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry() .getAggregator(config.valueSourceType(), GeoBoundsAggregationBuilder.NAME); @@ -71,13 +71,13 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, + GeoBoundsAggregatorSupplier.class.getName() + ", found [" + aggregatorSupplier.getClass().toString() + "]"); } - return ((GeoBoundsAggregatorSupplier) aggregatorSupplier).build(name, searchContext, parent, valuesSource, wrapLongitude, metaData); + return ((GeoBoundsAggregatorSupplier) aggregatorSupplier).build(name, searchContext, parent, valuesSource, wrapLongitude, metadata); } static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { valuesSourceRegistry.register(GeoBoundsAggregationBuilder.NAME, CoreValuesSourceType.GEOPOINT, - (GeoBoundsAggregatorSupplier) (name, aggregationContext, parent, valuesSource, wrapLongitude, metaData) + (GeoBoundsAggregatorSupplier) (name, aggregationContext, parent, valuesSource, wrapLongitude, metadata) -> new GeoBoundsAggregator(name, aggregationContext, parent, (ValuesSource.GeoPoint) valuesSource, - wrapLongitude, metaData)); + wrapLongitude, metadata)); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoBoundsAggregatorSupplier.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoBoundsAggregatorSupplier.java index 20d006734aa1e..0e2ac1aba6484 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoBoundsAggregatorSupplier.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoBoundsAggregatorSupplier.java @@ -32,5 +32,5 @@ public interface GeoBoundsAggregatorSupplier extends AggregatorSupplier { GeoBoundsAggregator build(String name, SearchContext aggregationContext, Aggregator parent, ValuesSource valuesSource, boolean wrapLongitude, - Map metaData) throws IOException; + Map metadata) throws IOException; } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregationBuilder.java index 98722f96552e3..57120898996fe 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregationBuilder.java @@ -65,8 +65,8 @@ protected ValuesSourceType defaultValueSourceType() { } @Override - protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metaData) { - return new GeoCentroidAggregationBuilder(this, factoriesBuilder, metaData); + protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metadata) { + return new GeoCentroidAggregationBuilder(this, factoriesBuilder, metadata); } /** diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregatorFactory.java index a7b96fd264b31..71b97cacc382c 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregatorFactory.java @@ -58,7 +58,7 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, - Map metaData) throws IOException { + Map metadata) throws IOException { AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), GeoCentroidAggregationBuilder.NAME); @@ -66,12 +66,12 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, throw new AggregationExecutionException("Registry miss-match - expected " + GeoCentroidAggregatorSupplier.class.getName() + ", found [" + aggregatorSupplier.getClass().toString() + "]"); } - return ((GeoCentroidAggregatorSupplier) aggregatorSupplier).build(name, searchContext, parent, valuesSource, metaData); + return ((GeoCentroidAggregatorSupplier) aggregatorSupplier).build(name, searchContext, parent, valuesSource, metadata); } static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { valuesSourceRegistry.register(GeoCentroidAggregationBuilder.NAME, CoreValuesSourceType.GEOPOINT, - (GeoCentroidAggregatorSupplier) (name, context, parent, valuesSource, metaData) -> - new GeoCentroidAggregator(name, context, parent, (ValuesSource.GeoPoint) valuesSource, metaData)); + (GeoCentroidAggregatorSupplier) (name, context, parent, valuesSource, metadata) -> + new GeoCentroidAggregator(name, context, parent, (ValuesSource.GeoPoint) valuesSource, metadata)); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregatorSupplier.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregatorSupplier.java index b4a4bbf98b23e..736c7df53c162 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregatorSupplier.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoCentroidAggregatorSupplier.java @@ -32,5 +32,5 @@ public interface GeoCentroidAggregatorSupplier extends AggregatorSupplier { GeoCentroidAggregator build(String name, SearchContext context, Aggregator parent, ValuesSource valuesSource, - Map metaData) throws IOException; + Map metadata) throws IOException; } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoGridAggregatorSupplier.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoGridAggregatorSupplier.java index f8cfae6d5bcea..aefd942c43912 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoGridAggregatorSupplier.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/GeoGridAggregatorSupplier.java @@ -36,5 +36,5 @@ public interface GeoGridAggregatorSupplier extends AggregatorSupplier { GeoGridAggregator build(String name, AggregatorFactories factories, ValuesSource valuesSource, int precision, GeoBoundingBox geoBoundingBox, int requiredSize, int shardSize, SearchContext aggregationContext, Aggregator parent, - Map metaData) throws IOException; + Map metadata) throws IOException; } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalHDRPercentileRanks.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalHDRPercentileRanks.java index 1bd0f46b3206a..8b5f3a8e257a5 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalHDRPercentileRanks.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalHDRPercentileRanks.java @@ -30,8 +30,8 @@ public class InternalHDRPercentileRanks extends AbstractInternalHDRPercentiles i public static final String NAME = "hdr_percentile_ranks"; public InternalHDRPercentileRanks(String name, double[] cdfValues, DoubleHistogram state, boolean keyed, DocValueFormat formatter, - Map metaData) { - super(name, cdfValues, state, keyed, formatter, metaData); + Map metadata) { + super(name, cdfValues, state, keyed, formatter, metadata); } /** diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalHDRPercentiles.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalHDRPercentiles.java index c647bfbc5be13..2883ea9db6081 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalHDRPercentiles.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalHDRPercentiles.java @@ -30,8 +30,8 @@ public class InternalHDRPercentiles extends AbstractInternalHDRPercentiles imple public static final String NAME = "hdr_percentiles"; public InternalHDRPercentiles(String name, double[] percents, DoubleHistogram state, boolean keyed, DocValueFormat formatter, - Map metaData) { - super(name, percents, state, keyed, formatter, metaData); + Map metadata) { + super(name, percents, state, keyed, formatter, metadata); } /** diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalTDigestPercentileRanks.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalTDigestPercentileRanks.java index 72cd06d3c67a9..3c4b5206fcf0b 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalTDigestPercentileRanks.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalTDigestPercentileRanks.java @@ -29,8 +29,8 @@ public class InternalTDigestPercentileRanks extends AbstractInternalTDigestPerce public static final String NAME = "tdigest_percentile_ranks"; public InternalTDigestPercentileRanks(String name, double[] cdfValues, TDigestState state, boolean keyed, DocValueFormat formatter, - Map metaData) { - super(name, cdfValues, state, keyed, formatter, metaData); + Map metadata) { + super(name, cdfValues, state, keyed, formatter, metadata); } /** diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalTDigestPercentiles.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalTDigestPercentiles.java index 780e508bbe6b4..f2e1d12285c2e 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalTDigestPercentiles.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/InternalTDigestPercentiles.java @@ -29,8 +29,8 @@ public class InternalTDigestPercentiles extends AbstractInternalTDigestPercentil public static final String NAME = "tdigest_percentiles"; public InternalTDigestPercentiles(String name, double[] percents, TDigestState state, boolean keyed, DocValueFormat formatter, - Map metaData) { - super(name, percents, state, keyed, formatter, metaData); + Map metadata) { + super(name, percents, state, keyed, formatter, metadata); } /** diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MaxAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MaxAggregationBuilder.java index b8d8cb2799a7c..9459c491d5fa1 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MaxAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MaxAggregationBuilder.java @@ -63,8 +63,8 @@ protected ValuesSourceType defaultValueSourceType() { } @Override - protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metaData) { - return new MaxAggregationBuilder(this, factoriesBuilder, metaData); + protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metadata) { + return new MaxAggregationBuilder(this, factoriesBuilder, metadata); } /** diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MaxAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MaxAggregatorFactory.java index 387c2b1a3d572..754877c27a879 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MaxAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MaxAggregatorFactory.java @@ -49,16 +49,16 @@ public Aggregator build(String name, ValuesSource valuesSource, SearchContext context, Aggregator parent, - Map metaData) throws IOException { - return new MaxAggregator(name, config, (Numeric) valuesSource, context, parent, metaData); + Map metadata) throws IOException { + return new MaxAggregator(name, config, (Numeric) valuesSource, context, parent, metadata); } }); } MaxAggregatorFactory(String name, ValuesSourceConfig config, QueryShardContext queryShardContext, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, - Map metaData) throws IOException { - super(name, config, queryShardContext, parent, subFactoriesBuilder, metaData); + Map metadata) throws IOException { + super(name, config, queryShardContext, parent, subFactoriesBuilder, metadata); } @Override @@ -73,7 +73,7 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, - Map metaData) throws IOException { + Map metadata) throws IOException { AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), MaxAggregationBuilder.NAME); @@ -82,6 +82,6 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, aggregatorSupplier.getClass().toString() + "]"); } return ((MinMaxAggregatorSupplier) aggregatorSupplier).build(name, config, valuesSource, searchContext, parent, - metaData); + metadata); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregatorFactory.java index edc403adf8975..abba4a6e09a07 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregatorFactory.java @@ -62,13 +62,13 @@ public Aggregator build(String name, DocValueFormat format, SearchContext context, Aggregator parent, - Map metaData, + Map metadata, double compression) throws IOException { return new MedianAbsoluteDeviationAggregator( name, context, parent, - metaData, + metadata, (ValuesSource.Numeric) valuesSource, format, compression @@ -80,7 +80,7 @@ public Aggregator build(String name, @Override protected Aggregator createUnmapped(SearchContext searchContext, Aggregator parent, - Map metaData) throws IOException { + Map metadata) throws IOException { return new MedianAbsoluteDeviationAggregator( name, @@ -98,7 +98,7 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, - Map metaData) throws IOException { + Map metadata) throws IOException { AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), MedianAbsoluteDeviationAggregationBuilder.NAME); @@ -107,6 +107,6 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, aggregatorSupplier.getClass().toString() + "]"); } return ((MedianAbsoluteDeviationAggregatorSupplier) aggregatorSupplier).build(name, valuesSource, config.format(), - searchContext, parent, metaData, compression); + searchContext, parent, metadata, compression); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregatorSupplier.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregatorSupplier.java index 034f3a27d44c5..f99092329a07d 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregatorSupplier.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MedianAbsoluteDeviationAggregatorSupplier.java @@ -33,6 +33,6 @@ Aggregator build(String name, DocValueFormat format, SearchContext context, Aggregator parent, - Map metaData, + Map metadata, double compression) throws IOException; } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MetricAggregatorSupplier.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MetricAggregatorSupplier.java index ebecf2b49c32e..9885f1f0dbab2 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MetricAggregatorSupplier.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MetricAggregatorSupplier.java @@ -33,5 +33,5 @@ Aggregator build(String name, DocValueFormat format, SearchContext context, Aggregator parent, - Map metaData) throws IOException; + Map metadata) throws IOException; } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MinAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MinAggregatorFactory.java index 4036c8a12b62e..7e39930892824 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MinAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MinAggregatorFactory.java @@ -49,16 +49,16 @@ public Aggregator build(String name, ValuesSource valuesSource, SearchContext context, Aggregator parent, - Map metaData) throws IOException { - return new MinAggregator(name, config, (Numeric) valuesSource, context, parent, metaData); + Map metadata) throws IOException { + return new MinAggregator(name, config, (Numeric) valuesSource, context, parent, metadata); } }); } MinAggregatorFactory(String name, ValuesSourceConfig config, QueryShardContext queryShardContext, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, - Map metaData) throws IOException { - super(name, config, queryShardContext, parent, subFactoriesBuilder, metaData); + Map metadata) throws IOException { + super(name, config, queryShardContext, parent, subFactoriesBuilder, metadata); } @Override @@ -73,7 +73,7 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, - Map metaData) throws IOException { + Map metadata) throws IOException { AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), MinAggregationBuilder.NAME); @@ -81,6 +81,6 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, throw new AggregationExecutionException("Registry miss-match - expected MinMaxAggregatorSupplier, found [" + aggregatorSupplier.getClass().toString() + "]"); } - return ((MinMaxAggregatorSupplier) aggregatorSupplier).build(name, config, valuesSource, searchContext, parent, metaData); + return ((MinMaxAggregatorSupplier) aggregatorSupplier).build(name, config, valuesSource, searchContext, parent, metadata); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MinMaxAggregatorSupplier.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MinMaxAggregatorSupplier.java index 6054559831269..69e697d694f06 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MinMaxAggregatorSupplier.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/MinMaxAggregatorSupplier.java @@ -33,5 +33,5 @@ Aggregator build(String name, ValuesSource valuesSource, SearchContext context, Aggregator parent, - Map metaData) throws IOException; + Map metadata) throws IOException; } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentileRanksAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentileRanksAggregatorFactory.java index 75342322fa4c0..26bf4160de730 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentileRanksAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentileRanksAggregatorFactory.java @@ -50,10 +50,10 @@ static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { @Override public Aggregator build(String name, ValuesSource valuesSource, SearchContext context, Aggregator parent, double[] percents, PercentilesConfig percentilesConfig, boolean keyed, DocValueFormat formatter, - Map metaData) throws IOException { + Map metadata) throws IOException { return percentilesConfig.createPercentileRanksAggregator(name, valuesSource, context, parent, percents, keyed, - formatter, metaData); + formatter, metadata); } } ); @@ -88,7 +88,7 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, - Map metaData) throws IOException { + Map metadata) throws IOException { AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), PercentileRanksAggregationBuilder.NAME); @@ -98,6 +98,6 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, } PercentilesAggregatorSupplier percentilesAggregatorSupplier = (PercentilesAggregatorSupplier) aggregatorSupplier; return percentilesAggregatorSupplier.build(name, valuesSource, searchContext, parent, percents, percentilesConfig, keyed, - config.format(), metaData); + config.format(), metadata); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesAggregatorFactory.java index b8aa428626ee1..4f39a5584eba1 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesAggregatorFactory.java @@ -54,10 +54,10 @@ static void registerAggregators(ValuesSourceRegistry valuesSourceRegistry) { @Override public Aggregator build(String name, ValuesSource valuesSource, SearchContext context, Aggregator parent, double[] percents, PercentilesConfig percentilesConfig, boolean keyed, DocValueFormat formatter, - Map metaData) throws IOException { + Map metadata) throws IOException { return percentilesConfig.createPercentilesAggregator(name, valuesSource, context, parent, percents, keyed, - formatter, metaData); + formatter, metadata); } } ); diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesAggregatorSupplier.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesAggregatorSupplier.java index 3a9d8b3d14f02..e7bda50e5f404 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesAggregatorSupplier.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesAggregatorSupplier.java @@ -36,5 +36,5 @@ Aggregator build(String name, PercentilesConfig percentilesConfig, boolean keyed, DocValueFormat formatter, - Map metaData) throws IOException; + Map metadata) throws IOException; } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesConfig.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesConfig.java index ec1ed9dcaff26..b59ee9933d011 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesConfig.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/PercentilesConfig.java @@ -69,7 +69,7 @@ public PercentilesMethod getMethod() { public abstract Aggregator createPercentilesAggregator(String name, ValuesSource valuesSource, SearchContext context, Aggregator parent, double[] values, boolean keyed, DocValueFormat formatter, - Map metaData) throws IOException; + Map metadata) throws IOException; abstract Aggregator createPercentileRanksAggregator(String name, ValuesSource valuesSource, SearchContext context, Aggregator parent, double[] values, boolean keyed, diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/StatsAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/StatsAggregatorFactory.java index 961c848b6a4c7..22809babf798f 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/StatsAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/StatsAggregatorFactory.java @@ -59,8 +59,8 @@ public Aggregator build(String name, DocValueFormat formatter, SearchContext context, Aggregator parent, - Map metaData) throws IOException { - return new StatsAggregator(name, (Numeric) valuesSource, formatter, context, parent, metaData); + Map metadata) throws IOException { + return new StatsAggregator(name, (Numeric) valuesSource, formatter, context, parent, metadata); } }); } @@ -68,8 +68,8 @@ public Aggregator build(String name, @Override protected Aggregator createUnmapped(SearchContext searchContext, Aggregator parent, - Map metaData) throws IOException { - return new StatsAggregator(name, null, config.format(), searchContext, parent, metaData); + Map metadata) throws IOException { + return new StatsAggregator(name, null, config.format(), searchContext, parent, metadata); } @Override @@ -77,7 +77,7 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, - Map metaData) throws IOException { + Map metadata) throws IOException { AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), StatsAggregationBuilder.NAME); @@ -85,6 +85,6 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, throw new AggregationExecutionException("Registry miss-match - expected MetricAggregatorSupplier, found [" + aggregatorSupplier.getClass().toString() + "]"); } - return ((MetricAggregatorSupplier) aggregatorSupplier).build(name, valuesSource, config.format(), searchContext, parent, metaData); + return ((MetricAggregatorSupplier) aggregatorSupplier).build(name, valuesSource, config.format(), searchContext, parent, metadata); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/SumAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/SumAggregatorFactory.java index 8e69ca31153d2..1cfb132c339f8 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/SumAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/SumAggregatorFactory.java @@ -59,8 +59,8 @@ public Aggregator build(String name, DocValueFormat formatter, SearchContext context, Aggregator parent, - Map metaData) throws IOException { - return new SumAggregator(name, (Numeric) valuesSource, formatter, context, parent, metaData); + Map metadata) throws IOException { + return new SumAggregator(name, (Numeric) valuesSource, formatter, context, parent, metadata); } }); } @@ -78,7 +78,7 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, - Map metaData) throws IOException { + Map metadata) throws IOException { AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), SumAggregationBuilder.NAME); @@ -86,6 +86,6 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, throw new AggregationExecutionException("Registry miss-match - expected MetricAggregatorSupplier, found [" + aggregatorSupplier.getClass().toString() + "]"); } - return ((MetricAggregatorSupplier) aggregatorSupplier).build(name, valuesSource, config.format(), searchContext, parent, metaData); + return ((MetricAggregatorSupplier) aggregatorSupplier).build(name, valuesSource, config.format(), searchContext, parent, metadata); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregationBuilder.java index 8ed6e3f02698a..bacc0dd4d8559 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregationBuilder.java @@ -66,8 +66,8 @@ protected ValuesSourceType defaultValueSourceType() { } @Override - protected AggregationBuilder shallowCopy(AggregatorFactories.Builder factoriesBuilder, Map metaData) { - return new ValueCountAggregationBuilder(this, factoriesBuilder, metaData); + protected AggregationBuilder shallowCopy(AggregatorFactories.Builder factoriesBuilder, Map metadata) { + return new ValueCountAggregationBuilder(this, factoriesBuilder, metadata); } /** diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregatorFactory.java index 3212beedffac5..ad3a41ba79db6 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregatorFactory.java @@ -44,16 +44,16 @@ public Aggregator build(String name, ValuesSource valuesSource, SearchContext aggregationContext, Aggregator parent, - Map metaData) throws IOException { - return new ValueCountAggregator(name, valuesSource, aggregationContext, parent, metaData); + Map metadata) throws IOException { + return new ValueCountAggregator(name, valuesSource, aggregationContext, parent, metadata); } }); } ValueCountAggregatorFactory(String name, ValuesSourceConfig config, QueryShardContext queryShardContext, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, - Map metaData) throws IOException { - super(name, config, queryShardContext, parent, subFactoriesBuilder, metaData); + Map metadata) throws IOException { + super(name, config, queryShardContext, parent, subFactoriesBuilder, metadata); } @Override @@ -68,7 +68,7 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, - Map metaData) throws IOException { + Map metadata) throws IOException { AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), ValueCountAggregationBuilder.NAME); if (aggregatorSupplier instanceof ValueCountAggregatorSupplier == false) { @@ -76,6 +76,6 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, aggregatorSupplier.getClass().toString() + "]"); } return ((ValueCountAggregatorSupplier) aggregatorSupplier) - .build(name, valuesSource, searchContext, parent, metaData); + .build(name, valuesSource, searchContext, parent, metadata); } } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregatorSupplier.java b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregatorSupplier.java index eed3900692333..f882afaf1d076 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregatorSupplier.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/metrics/ValueCountAggregatorSupplier.java @@ -31,5 +31,5 @@ Aggregator build(String name, ValuesSource valuesSource, SearchContext aggregationContext, Aggregator parent, - Map metaData) throws IOException; + Map metadata) throws IOException; } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/support/HistogramAggregatorSupplier.java b/server/src/main/java/org/elasticsearch/search/aggregations/support/HistogramAggregatorSupplier.java index 5ffcb6c3d3cd7..83bfa8abf70ec 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/support/HistogramAggregatorSupplier.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/support/HistogramAggregatorSupplier.java @@ -33,5 +33,5 @@ Aggregator build(String name, AggregatorFactories factories, double interval, do BucketOrder order, boolean keyed, long minDocCount, double minBound, double maxBound, @Nullable ValuesSource valuesSource, DocValueFormat formatter, SearchContext context, Aggregator parent, - Map metaData) throws IOException; + Map metadata) throws IOException; } diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/support/MultiValuesSourceAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/support/MultiValuesSourceAggregationBuilder.java index 1c36db7296472..9a5dcfcf3a703 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/support/MultiValuesSourceAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/support/MultiValuesSourceAggregationBuilder.java @@ -51,8 +51,8 @@ protected LeafOnly(String name) { super(name); } - protected LeafOnly(LeafOnly clone, Builder factoriesBuilder, Map metaData) { - super(clone, factoriesBuilder, metaData); + protected LeafOnly(LeafOnly clone, Builder factoriesBuilder, Map metadata) { + super(clone, factoriesBuilder, metadata); if (factoriesBuilder.count() > 0) { throw new AggregationInitializationException("Aggregator [" + name + "] of type [" + getType() + "] cannot accept sub-aggregations"); @@ -84,8 +84,8 @@ protected MultiValuesSourceAggregationBuilder(String name) { } protected MultiValuesSourceAggregationBuilder(MultiValuesSourceAggregationBuilder clone, - Builder factoriesBuilder, Map metaData) { - super(clone, factoriesBuilder, metaData); + Builder factoriesBuilder, Map metadata) { + super(clone, factoriesBuilder, metadata); this.fields = new HashMap<>(clone.fields); this.userValueTypeHint = clone.userValueTypeHint; diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceAggregationBuilder.java b/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceAggregationBuilder.java index a574b07ef8f2f..3f8aa17c2e52c 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceAggregationBuilder.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceAggregationBuilder.java @@ -126,8 +126,8 @@ protected ValuesSourceAggregationBuilder(String name) { } protected ValuesSourceAggregationBuilder(ValuesSourceAggregationBuilder clone, - Builder factoriesBuilder, Map metaData) { - super(clone, factoriesBuilder, metaData); + Builder factoriesBuilder, Map metadata) { + super(clone, factoriesBuilder, metadata); this.field = clone.field; this.userValueTypeHint = clone.userValueTypeHint; this.format = clone.format; diff --git a/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceAggregatorFactory.java b/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceAggregatorFactory.java index 9cf6759bcecbe..6d03d6f056b69 100644 --- a/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceAggregatorFactory.java +++ b/server/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceAggregatorFactory.java @@ -34,14 +34,14 @@ public abstract class ValuesSourceAggregatorFactory extends AggregatorFactory { public ValuesSourceAggregatorFactory(String name, ValuesSourceConfig config, QueryShardContext queryShardContext, AggregatorFactory parent, AggregatorFactories.Builder subFactoriesBuilder, - Map metaData) throws IOException { - super(name, queryShardContext, parent, subFactoriesBuilder, metaData); + Map metadata) throws IOException { + super(name, queryShardContext, parent, subFactoriesBuilder, metadata); this.config = config; } @Override public Aggregator createInternal(SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, - Map metaData) throws IOException { + Map metadata) throws IOException { ValuesSource vs = config.toValuesSource(); if (vs == null) { return createUnmapped(searchContext, parent, metadata); diff --git a/server/src/test/java/org/elasticsearch/cluster/routing/allocation/AllocationServiceTests.java b/server/src/test/java/org/elasticsearch/cluster/routing/allocation/AllocationServiceTests.java index 56e7d27c5344e..dbca012733754 100644 --- a/server/src/test/java/org/elasticsearch/cluster/routing/allocation/AllocationServiceTests.java +++ b/server/src/test/java/org/elasticsearch/cluster/routing/allocation/AllocationServiceTests.java @@ -149,7 +149,7 @@ public ShardAllocationDecision decideShardAllocation(ShardRouting shard, Routing nodesBuilder.add(new DiscoveryNode("node2", buildNewFakeTransportAddress(), Version.CURRENT)); nodesBuilder.add(new DiscoveryNode("node3", buildNewFakeTransportAddress(), Version.CURRENT)); - final Metadata.Builder metaData = Metadata.builder() + final Metadata.Builder metadata = Metadata.builder() // create 3 indices with different priorities. The high and low priority indices use the default allocator which (in this test) // does not allocate any replicas, whereas the medium priority one uses the unrealistic allocator which does allocate replicas .put(indexMetadata("highPriority", Settings.builder() @@ -166,14 +166,14 @@ public ShardAllocationDecision decideShardAllocation(ShardRouting shard, Routing .put(ExistingShardsAllocator.EXISTING_SHARDS_ALLOCATOR_SETTING.getKey(), "unknown"))); final RoutingTable.Builder routingTableBuilder = RoutingTable.builder() - .addAsRecovery(metaData.get("highPriority")) - .addAsRecovery(metaData.get("mediumPriority")) - .addAsRecovery(metaData.get("lowPriority")) - .addAsRecovery(metaData.get("invalid")); + .addAsRecovery(metadata.get("highPriority")) + .addAsRecovery(metadata.get("mediumPriority")) + .addAsRecovery(metadata.get("lowPriority")) + .addAsRecovery(metadata.get("invalid")); final ClusterState clusterState = ClusterState.builder(ClusterName.DEFAULT) .nodes(nodesBuilder) - .metadata(metaData) + .metadata(metadata) .routingTable(routingTableBuilder.build()) .build(); diff --git a/server/src/test/java/org/elasticsearch/search/SearchModuleTests.java b/server/src/test/java/org/elasticsearch/search/SearchModuleTests.java index 1f316e2dc0280..6799fc67b215d 100644 --- a/server/src/test/java/org/elasticsearch/search/SearchModuleTests.java +++ b/server/src/test/java/org/elasticsearch/search/SearchModuleTests.java @@ -383,8 +383,8 @@ protected ValuesSourceType defaultValueSourceType() { } @Override - protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metaData) { - return new TestAggregationBuilder(this, factoriesBuilder, metaData); + protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metadata) { + return new TestAggregationBuilder(this, factoriesBuilder, metadata); } /** * Read from a stream. diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/AbstractHistoBackedHDRPercentilesAggregator.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/AbstractHistoBackedHDRPercentilesAggregator.java index 130af5e17b683..d0c41e1f7b5a7 100644 --- a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/AbstractHistoBackedHDRPercentilesAggregator.java +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/AbstractHistoBackedHDRPercentilesAggregator.java @@ -42,8 +42,8 @@ private static int indexOfKey(double[] keys, double key) { AbstractHistoBackedHDRPercentilesAggregator(String name, ValuesSource valuesSource, SearchContext context, Aggregator parent, double[] keys, int numberOfSignificantValueDigits, boolean keyed, DocValueFormat formatter, - Map metaData) throws IOException { - super(name, context, parent, metaData); + Map metadata) throws IOException { + super(name, context, parent, metadata); this.valuesSource = valuesSource; this.keyed = keyed; this.format = formatter; diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/AbstractHistoBackedTDigestPercentilesAggregator.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/AbstractHistoBackedTDigestPercentilesAggregator.java index bce3bf40bb8a8..34c1d5ac84a9b 100644 --- a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/AbstractHistoBackedTDigestPercentilesAggregator.java +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/AbstractHistoBackedTDigestPercentilesAggregator.java @@ -42,8 +42,8 @@ private static int indexOfKey(double[] keys, double key) { AbstractHistoBackedTDigestPercentilesAggregator(String name, ValuesSource valuesSource, SearchContext context, Aggregator parent, double[] keys, double compression, boolean keyed, DocValueFormat formatter, - Map metaData) throws IOException { - super(name, context, parent, metaData); + Map metadata) throws IOException { + super(name, context, parent, metadata); this.valuesSource = valuesSource; this.keyed = keyed; this.formatter = formatter; diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/AnalyticsPercentilesAggregatorFactory.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/AnalyticsPercentilesAggregatorFactory.java index 0e40cb5876dda..f08610e965876 100644 --- a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/AnalyticsPercentilesAggregatorFactory.java +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/AnalyticsPercentilesAggregatorFactory.java @@ -19,17 +19,17 @@ public static void registerPercentilesAggregator(ValuesSourceRegistry valuesSour valuesSourceRegistry.register(PercentilesAggregationBuilder.NAME, AnalyticsValuesSourceType.HISTOGRAM, (PercentilesAggregatorSupplier) (name, valuesSource, context, parent, percents, percentilesConfig, keyed, - formatter, metaData) -> { + formatter, metadata) -> { if (percentilesConfig.getMethod().equals(PercentilesMethod.TDIGEST)) { double compression = ((PercentilesConfig.TDigest)percentilesConfig).getCompression(); return new HistoBackedTDigestPercentilesAggregator(name, valuesSource, context, parent, - percents, compression, keyed, formatter, metaData); + percents, compression, keyed, formatter, metadata); } else if (percentilesConfig.getMethod().equals(PercentilesMethod.HDR)) { int numSigFig = ((PercentilesConfig.Hdr)percentilesConfig).getNumberOfSignificantValueDigits(); return new HistoBackedHDRPercentilesAggregator(name, valuesSource, context, parent, - percents, numSigFig, keyed, formatter, metaData); + percents, numSigFig, keyed, formatter, metadata); } throw new IllegalArgumentException("Percentiles algorithm: [" + percentilesConfig.getMethod().toString() + "] " + @@ -41,17 +41,17 @@ public static void registerPercentileRanksAggregator(ValuesSourceRegistry values valuesSourceRegistry.register(PercentileRanksAggregationBuilder.NAME, AnalyticsValuesSourceType.HISTOGRAM, (PercentilesAggregatorSupplier) (name, valuesSource, context, parent, percents, percentilesConfig, keyed, - formatter, metaData) -> { + formatter, metadata) -> { if (percentilesConfig.getMethod().equals(PercentilesMethod.TDIGEST)) { double compression = ((PercentilesConfig.TDigest)percentilesConfig).getCompression(); return new HistoBackedTDigestPercentileRanksAggregator(name, valuesSource, context, parent, - percents, compression, keyed, formatter, metaData); + percents, compression, keyed, formatter, metadata); } else if (percentilesConfig.getMethod().equals(PercentilesMethod.HDR)) { int numSigFig = ((PercentilesConfig.Hdr)percentilesConfig).getNumberOfSignificantValueDigits(); return new HistoBackedHDRPercentileRanksAggregator(name, valuesSource, context, parent, - percents, numSigFig, keyed, formatter, metaData); + percents, numSigFig, keyed, formatter, metadata); } throw new IllegalArgumentException("Percentiles algorithm: [" + percentilesConfig.getMethod().toString() + "] " + diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedHDRPercentileRanksAggregator.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedHDRPercentileRanksAggregator.java index 65403d61f10f0..c88be4a3d27b2 100644 --- a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedHDRPercentileRanksAggregator.java +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedHDRPercentileRanksAggregator.java @@ -21,8 +21,8 @@ class HistoBackedHDRPercentileRanksAggregator extends AbstractHistoBackedHDRPerc HistoBackedHDRPercentileRanksAggregator(String name, ValuesSource valuesSource, SearchContext context, Aggregator parent, double[] percents, int numberOfSignificantValueDigits, boolean keyed, DocValueFormat format, - Map metaData) throws IOException { - super(name, valuesSource, context, parent, percents, numberOfSignificantValueDigits, keyed, format, metaData); + Map metadata) throws IOException { + super(name, valuesSource, context, parent, percents, numberOfSignificantValueDigits, keyed, format, metadata); } @Override diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedHDRPercentilesAggregator.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedHDRPercentilesAggregator.java index fcd6656f74a3b..736bb3169bab3 100644 --- a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedHDRPercentilesAggregator.java +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedHDRPercentilesAggregator.java @@ -21,9 +21,9 @@ public class HistoBackedHDRPercentilesAggregator extends AbstractHistoBackedHDRP HistoBackedHDRPercentilesAggregator(String name, ValuesSource valuesSource, SearchContext context, Aggregator parent, double[] percents, int numberOfSignificantValueDigits, boolean keyed, DocValueFormat formatter, - Map metaData) throws IOException { + Map metadata) throws IOException { super(name, valuesSource, context, parent, percents, numberOfSignificantValueDigits, keyed, formatter, - metaData); + metadata); } @Override diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedTDigestPercentileRanksAggregator.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedTDigestPercentileRanksAggregator.java index a434aa9ebc623..5acba9d1ac6d5 100644 --- a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedTDigestPercentileRanksAggregator.java +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedTDigestPercentileRanksAggregator.java @@ -27,8 +27,8 @@ class HistoBackedTDigestPercentileRanksAggregator extends AbstractHistoBackedTDi double compression, boolean keyed, DocValueFormat formatter, - Map metaData) throws IOException { - super(name, valuesSource, context, parent, percents, compression, keyed, formatter, metaData); + Map metadata) throws IOException { + super(name, valuesSource, context, parent, percents, compression, keyed, formatter, metadata); } @Override diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedTDigestPercentilesAggregator.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedTDigestPercentilesAggregator.java index 63cd4d6bd03e7..d8ce85f47763a 100644 --- a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedTDigestPercentilesAggregator.java +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/aggregations/metrics/HistoBackedTDigestPercentilesAggregator.java @@ -27,8 +27,8 @@ class HistoBackedTDigestPercentilesAggregator extends AbstractHistoBackedTDigest double compression, boolean keyed, DocValueFormat formatter, - Map metaData) throws IOException { - super(name, valuesSource, context, parent, percents, compression, keyed, formatter, metaData); + Map metadata) throws IOException { + super(name, valuesSource, context, parent, percents, compression, keyed, formatter, metadata); } @Override diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregatorFactory.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregatorFactory.java index 64f30011bd7f8..033a2bf1028b7 100644 --- a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregatorFactory.java +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregatorFactory.java @@ -58,7 +58,7 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, - Map metaData) throws IOException { + Map metadata) throws IOException { AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), BoxplotAggregationBuilder.NAME); @@ -67,6 +67,6 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, aggregatorSupplier.getClass().toString() + "]"); } return ((BoxplotAggregatorSupplier) aggregatorSupplier).build(name, valuesSource, config.format(), compression, - searchContext, parent, metaData); + searchContext, parent, metadata); } } diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregatorSupplier.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregatorSupplier.java index 3e3da6120a06e..a83d7ce2d0cf4 100644 --- a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregatorSupplier.java +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/boxplot/BoxplotAggregatorSupplier.java @@ -23,6 +23,6 @@ Aggregator build(String name, double compression, SearchContext context, Aggregator parent, - Map metaData) throws IOException; + Map metadata) throws IOException; } diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregationBuilder.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregationBuilder.java index b33d87d3a634f..1d33177474fee 100644 --- a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregationBuilder.java +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregationBuilder.java @@ -61,8 +61,8 @@ protected ValuesSourceType defaultValueSourceType() { } @Override - protected AggregationBuilder shallowCopy(AggregatorFactories.Builder factoriesBuilder, Map metaData) { - return new StringStatsAggregationBuilder(this, factoriesBuilder, metaData); + protected AggregationBuilder shallowCopy(AggregatorFactories.Builder factoriesBuilder, Map metadata) { + return new StringStatsAggregationBuilder(this, factoriesBuilder, metadata); } @Override diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregatorFactory.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregatorFactory.java index e2329c75b6b5c..af08820b52d51 100644 --- a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregatorFactory.java +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregatorFactory.java @@ -46,9 +46,9 @@ public Aggregator build(String name, DocValueFormat format, SearchContext context, Aggregator parent, - Map metaData) throws IOException { + Map metadata) throws IOException { return new StringStatsAggregator(name, showDistribution, (ValuesSource.Bytes) valuesSource, - format, context, parent, metaData); + format, context, parent, metadata); } }); } @@ -65,7 +65,7 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, SearchContext searchContext, Aggregator parent, boolean collectsFromSingleBucket, - Map metaData) throws IOException { + Map metadata) throws IOException { AggregatorSupplier aggregatorSupplier = queryShardContext.getValuesSourceRegistry().getAggregator(config.valueSourceType(), StringStatsAggregationBuilder.NAME); @@ -74,7 +74,7 @@ protected Aggregator doCreateInternal(ValuesSource valuesSource, aggregatorSupplier.getClass().toString() + "]"); } return ((StringStatsAggregatorSupplier) aggregatorSupplier).build(name, valuesSource, showDistribution, config.format(), - searchContext, parent, metaData); + searchContext, parent, metadata); } } diff --git a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregatorSupplier.java b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregatorSupplier.java index 3ff8d33eb9d69..9ff249679bfe5 100644 --- a/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregatorSupplier.java +++ b/x-pack/plugin/analytics/src/main/java/org/elasticsearch/xpack/analytics/stringstats/StringStatsAggregatorSupplier.java @@ -22,5 +22,5 @@ Aggregator build(String name, DocValueFormat format, SearchContext context, Aggregator parent, - Map metaData) throws IOException; + Map metadata) throws IOException; } diff --git a/x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/transform/transforms/MockDeprecatedAggregationBuilder.java b/x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/transform/transforms/MockDeprecatedAggregationBuilder.java index fe224728209a7..8ba1dc29ed7f9 100644 --- a/x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/transform/transforms/MockDeprecatedAggregationBuilder.java +++ b/x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/transform/transforms/MockDeprecatedAggregationBuilder.java @@ -44,8 +44,8 @@ protected ValuesSourceType defaultValueSourceType() { } @Override - protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metaData) { - return new MockDeprecatedAggregationBuilder(this, factoriesBuilder, metaData); + protected AggregationBuilder shallowCopy(Builder factoriesBuilder, Map metadata) { + return new MockDeprecatedAggregationBuilder(this, factoriesBuilder, metadata); } public MockDeprecatedAggregationBuilder() { diff --git a/x-pack/plugin/searchable-snapshots/src/test/java/org/elasticsearch/index/store/cache/CachedBlobContainerIndexInputTests.java b/x-pack/plugin/searchable-snapshots/src/test/java/org/elasticsearch/index/store/cache/CachedBlobContainerIndexInputTests.java index 149b208edcc99..e3a6acecb70ff 100644 --- a/x-pack/plugin/searchable-snapshots/src/test/java/org/elasticsearch/index/store/cache/CachedBlobContainerIndexInputTests.java +++ b/x-pack/plugin/searchable-snapshots/src/test/java/org/elasticsearch/index/store/cache/CachedBlobContainerIndexInputTests.java @@ -54,14 +54,14 @@ public void testRandomReads() throws IOException { final byte[] input = randomUnicodeOfLength(randomIntBetween(1, 100_000)).getBytes(StandardCharsets.UTF_8); final String blobName = randomUnicodeOfLength(10); - final StoreFileMetadata metaData = new StoreFileMetadata(fileName, input.length, "_na", Version.CURRENT.luceneVersion); + final StoreFileMetadata metadata = new StoreFileMetadata(fileName, input.length, "_na", Version.CURRENT.luceneVersion); final int partSize = randomBoolean() ? input.length : randomIntBetween(1, input.length); final BlobStoreIndexShardSnapshot snapshot = new BlobStoreIndexShardSnapshot( snapshotId.getName(), 0L, - singletonList(new BlobStoreIndexShardSnapshot.FileInfo(blobName, metaData, new ByteSizeValue(partSize))), + singletonList(new BlobStoreIndexShardSnapshot.FileInfo(blobName, metadata, new ByteSizeValue(partSize))), 0L, 0L, 0, diff --git a/x-pack/plugin/sql/jdbc/src/test/java/org/elasticsearch/xpack/sql/jdbc/JdbcResultSetMetaDataTests.java b/x-pack/plugin/sql/jdbc/src/test/java/org/elasticsearch/xpack/sql/jdbc/JdbcResultSetMetaDataTests.java index 6c3b270b73140..18b819813fab2 100644 --- a/x-pack/plugin/sql/jdbc/src/test/java/org/elasticsearch/xpack/sql/jdbc/JdbcResultSetMetaDataTests.java +++ b/x-pack/plugin/sql/jdbc/src/test/java/org/elasticsearch/xpack/sql/jdbc/JdbcResultSetMetaDataTests.java @@ -23,30 +23,30 @@ public class JdbcResultSetMetaDataTests extends ESTestCase { new JdbcColumnInfo("test_double", EsType.DOUBLE, EMPTY, EMPTY, EMPTY, EMPTY, 25), new JdbcColumnInfo("test_long", EsType.LONG, "test_table", "test", "schema", "custom_label", 20) ); - private final JdbcResultSetMetaData metaData = new JdbcResultSetMetaData(null, columns); + private final JdbcResultSetMetaData metadata = new JdbcResultSetMetaData(null, columns); public void testColumnsProperties() throws SQLException { int maxColumnIndex = columns.size(); - assertEquals(false, metaData.isAutoIncrement(randomIntBetween(1, maxColumnIndex))); - assertEquals(true, metaData.isCaseSensitive(randomIntBetween(1, maxColumnIndex))); - assertEquals(true, metaData.isSearchable(randomIntBetween(1, maxColumnIndex))); - assertEquals(false, metaData.isCurrency(randomIntBetween(1, maxColumnIndex))); - assertEquals(ResultSetMetaData.columnNullableUnknown, metaData.isNullable(randomIntBetween(1, maxColumnIndex))); - assertEquals(false, metaData.isSigned(1)); - assertEquals(true, metaData.isSigned(2)); - assertEquals(true, metaData.isSigned(3)); - assertEquals(true, metaData.isSigned(4)); + assertEquals(false, metadata.isAutoIncrement(randomIntBetween(1, maxColumnIndex))); + assertEquals(true, metadata.isCaseSensitive(randomIntBetween(1, maxColumnIndex))); + assertEquals(true, metadata.isSearchable(randomIntBetween(1, maxColumnIndex))); + assertEquals(false, metadata.isCurrency(randomIntBetween(1, maxColumnIndex))); + assertEquals(ResultSetMetaData.columnNullableUnknown, metadata.isNullable(randomIntBetween(1, maxColumnIndex))); + assertEquals(false, metadata.isSigned(1)); + assertEquals(true, metadata.isSigned(2)); + assertEquals(true, metadata.isSigned(3)); + assertEquals(true, metadata.isSigned(4)); } public void testColumnNamesAndLabels() throws SQLException { - assertEquals("test_keyword", metaData.getColumnName(1)); - assertEquals("test_integer", metaData.getColumnName(2)); - assertEquals("test_double", metaData.getColumnName(3)); - assertEquals("test_long", metaData.getColumnName(4)); + assertEquals("test_keyword", metadata.getColumnName(1)); + assertEquals("test_integer", metadata.getColumnName(2)); + assertEquals("test_double", metadata.getColumnName(3)); + assertEquals("test_long", metadata.getColumnName(4)); - assertEquals("test_keyword", metaData.getColumnLabel(1)); - assertEquals("test_integer", metaData.getColumnLabel(2)); - assertEquals("test_double", metaData.getColumnLabel(3)); - assertEquals("custom_label", metaData.getColumnLabel(4)); + assertEquals("test_keyword", metadata.getColumnLabel(1)); + assertEquals("test_integer", metadata.getColumnLabel(2)); + assertEquals("test_double", metadata.getColumnLabel(3)); + assertEquals("custom_label", metadata.getColumnLabel(4)); } } diff --git a/x-pack/plugin/sql/qa/src/main/java/org/elasticsearch/xpack/sql/qa/jdbc/JdbcAssert.java b/x-pack/plugin/sql/qa/src/main/java/org/elasticsearch/xpack/sql/qa/jdbc/JdbcAssert.java index fcdb3c65330c0..2bf4752f4a9e0 100644 --- a/x-pack/plugin/sql/qa/src/main/java/org/elasticsearch/xpack/sql/qa/jdbc/JdbcAssert.java +++ b/x-pack/plugin/sql/qa/src/main/java/org/elasticsearch/xpack/sql/qa/jdbc/JdbcAssert.java @@ -206,8 +206,8 @@ public static void assertResultSetData(ResultSet expected, ResultSet actual, Log private static void doAssertResultSetData(ResultSet expected, ResultSet actual, Logger logger, boolean lenientDataType, boolean lenientFloatingNumbers) throws SQLException { - ResultSetMetaData metaData = expected.getMetaData(); - int columns = metaData.getColumnCount(); + ResultSetMetaData metadata = expected.getMetaData(); + int columns = metadata.getColumnCount(); long count = 0; try { @@ -219,10 +219,10 @@ private static void doAssertResultSetData(ResultSet expected, ResultSet actual, } for (int column = 1; column <= columns; column++) { - int type = metaData.getColumnType(column); + int type = metadata.getColumnType(column); Class expectedColumnClass = null; try { - String columnClassName = metaData.getColumnClassName(column); + String columnClassName = metadata.getColumnClassName(column); // fix for CSV which returns the shortName not fully-qualified name if (columnClassName != null && !columnClassName.contains(".")) { @@ -258,7 +258,7 @@ private static void doAssertResultSetData(ResultSet expected, ResultSet actual, : actual.getObject(column); String msg = format(Locale.ROOT, "Different result for column [%s], entry [%d]", - metaData.getColumnName(column), count + 1); + metadata.getColumnName(column), count + 1); // handle nulls first if (expectedObject == null || actualObject == null) { diff --git a/x-pack/plugin/sql/qa/src/main/java/org/elasticsearch/xpack/sql/qa/jdbc/JdbcTestUtils.java b/x-pack/plugin/sql/qa/src/main/java/org/elasticsearch/xpack/sql/qa/jdbc/JdbcTestUtils.java index 8bfbcd370ec54..737bdaffc95cb 100644 --- a/x-pack/plugin/sql/qa/src/main/java/org/elasticsearch/xpack/sql/qa/jdbc/JdbcTestUtils.java +++ b/x-pack/plugin/sql/qa/src/main/java/org/elasticsearch/xpack/sql/qa/jdbc/JdbcTestUtils.java @@ -51,20 +51,20 @@ private JdbcTestUtils() {} static final LocalDate EPOCH = LocalDate.of(1970, 1, 1); static void logResultSetMetadata(ResultSet rs, Logger logger) throws SQLException { - ResultSetMetaData metaData = rs.getMetaData(); + ResultSetMetaData metadata = rs.getMetaData(); // header StringBuilder sb = new StringBuilder(); StringBuilder column = new StringBuilder(); - int columns = metaData.getColumnCount(); + int columns = metadata.getColumnCount(); for (int i = 1; i <= columns; i++) { if (i > 1) { sb.append(" | "); } column.setLength(0); - column.append(metaData.getColumnName(i)); + column.append(metadata.getColumnName(i)); column.append("("); - column.append(metaData.getColumnTypeName(i)); + column.append(metadata.getColumnTypeName(i)); column.append(")"); sb.append(trimOrPad(column)); @@ -81,9 +81,9 @@ static void logResultSetMetadata(ResultSet rs, Logger logger) throws SQLExceptio } static void logResultSetData(ResultSet rs, Logger log) throws SQLException { - ResultSetMetaData metaData = rs.getMetaData(); + ResultSetMetaData metadata = rs.getMetaData(); - int columns = metaData.getColumnCount(); + int columns = metadata.getColumnCount(); while (rs.next()) { log.info(rowAsString(rs, columns)); @@ -91,8 +91,8 @@ static void logResultSetData(ResultSet rs, Logger log) throws SQLException { } static String resultSetCurrentData(ResultSet rs) throws SQLException { - ResultSetMetaData metaData = rs.getMetaData(); - return rowAsString(rs, metaData.getColumnCount()); + ResultSetMetaData metadata = rs.getMetaData(); + return rowAsString(rs, metadata.getColumnCount()); } private static String rowAsString(ResultSet rs, int columns) throws SQLException { @@ -122,14 +122,14 @@ private static StringBuilder trimOrPad(StringBuilder buffer) { } public static void logLikeCLI(ResultSet rs, Logger logger) throws SQLException { - ResultSetMetaData metaData = rs.getMetaData(); - int columns = metaData.getColumnCount(); + ResultSetMetaData metadata = rs.getMetaData(); + int columns = metadata.getColumnCount(); List cols = new ArrayList<>(columns); for (int i = 1; i <= columns; i++) { - cols.add(new ColumnInfo(metaData.getTableName(i), metaData.getColumnName(i), metaData.getColumnTypeName(i), - metaData.getColumnDisplaySize(i))); + cols.add(new ColumnInfo(metadata.getTableName(i), metadata.getColumnName(i), metadata.getColumnTypeName(i), + metadata.getColumnDisplaySize(i))); } diff --git a/x-pack/plugin/sql/qa/src/main/java/org/elasticsearch/xpack/sql/qa/jdbc/ResultSetMetaDataTestCase.java b/x-pack/plugin/sql/qa/src/main/java/org/elasticsearch/xpack/sql/qa/jdbc/ResultSetMetaDataTestCase.java index b8ab74bd33f5c..a5f3a2737de81 100644 --- a/x-pack/plugin/sql/qa/src/main/java/org/elasticsearch/xpack/sql/qa/jdbc/ResultSetMetaDataTestCase.java +++ b/x-pack/plugin/sql/qa/src/main/java/org/elasticsearch/xpack/sql/qa/jdbc/ResultSetMetaDataTestCase.java @@ -47,10 +47,10 @@ private void doWithQuery(String query, CheckedConsumer } } - private void assertColumnNamesAndLabels(ResultSetMetaData metaData, String[] names) throws SQLException { + private void assertColumnNamesAndLabels(ResultSetMetaData metadata, String[] names) throws SQLException { for(int i = 0; i < fieldsNames.length; i++) { - assertEquals(names[i], metaData.getColumnName(i + 1)); - assertEquals(names[i], metaData.getColumnLabel(i + 1)); + assertEquals(names[i], metadata.getColumnName(i + 1)); + assertEquals(names[i], metadata.getColumnLabel(i + 1)); } } } From 9f4566e1ed95ab86b8171e1087e0552ff3ef8607 Mon Sep 17 00:00:00 2001 From: Mark Tozzi Date: Thu, 16 Apr 2020 12:36:32 -0400 Subject: [PATCH 13/15] Fix list of supported types for MinAggregator --- .../search/aggregations/metrics/MinAggregatorTests.java | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/MinAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/MinAggregatorTests.java index 6d9f3fbc31fa2..cbfe33a1b7f75 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/MinAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/MinAggregatorTests.java @@ -954,7 +954,10 @@ private void testC @Override protected List getSupportedValuesSourceTypes() { - return Collections.singletonList(CoreValuesSourceType.NUMERIC); + return Arrays.asList( + CoreValuesSourceType.NUMERIC, + CoreValuesSourceType.DATE, + CoreValuesSourceType.BOOLEAN); } @Override From 495a20cab76cc671e84d9a306802fb9eceb911c5 Mon Sep 17 00:00:00 2001 From: Mark Tozzi Date: Thu, 16 Apr 2020 15:06:09 -0400 Subject: [PATCH 14/15] Fix list of supported types for Avg --- .../search/aggregations/metrics/AvgAggregatorTests.java | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/AvgAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/AvgAggregatorTests.java index 8f98421a85bcb..949c8b8800f94 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/AvgAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/AvgAggregatorTests.java @@ -715,7 +715,10 @@ public void testScriptCaching() throws IOException { @Override protected List getSupportedValuesSourceTypes() { - return singletonList(CoreValuesSourceType.NUMERIC); + return Arrays.asList( + CoreValuesSourceType.NUMERIC, + CoreValuesSourceType.DATE, + CoreValuesSourceType.BOOLEAN); } @Override From c83b31353e1145caff36eb35bd3699a29c98c643 Mon Sep 17 00:00:00 2001 From: Mark Tozzi Date: Thu, 16 Apr 2020 16:03:37 -0400 Subject: [PATCH 15/15] remove unused import --- .../search/aggregations/metrics/AvgAggregatorTests.java | 1 - 1 file changed, 1 deletion(-) diff --git a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/AvgAggregatorTests.java b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/AvgAggregatorTests.java index 949c8b8800f94..2b3ad8e294b6f 100644 --- a/server/src/test/java/org/elasticsearch/search/aggregations/metrics/AvgAggregatorTests.java +++ b/server/src/test/java/org/elasticsearch/search/aggregations/metrics/AvgAggregatorTests.java @@ -69,7 +69,6 @@ import java.util.function.Function; import static java.util.Collections.singleton; -import static java.util.Collections.singletonList; import static org.elasticsearch.index.query.QueryBuilders.termQuery; public class AvgAggregatorTests extends AggregatorTestCase {