add benchmark to test xcontent parser filter performance #80069

weizijun · 2021-10-29T08:30:25Z

add a benchmark to test xcontent parser filter performance.
First add the PR.Then it will use to benchmark the performance before and after the improvement.

Here is the benchmark result:

Benchmark                                                  (fieldCount)  (inclusive)         (type)  Mode  Cnt        Score         Error  Units
FilterContentBenchmark.filterWithNewParserConfig               10_field         true  cluster_stats  avgt    3   121751.756 ±    6677.781  ns/op
FilterContentBenchmark.filterWithNewParserConfig               10_field         true    index_stats  avgt    3    22111.200 ±    4800.055  ns/op
FilterContentBenchmark.filterWithNewParserConfig               10_field         true     node_stats  avgt    3    24794.636 ±     416.450  ns/op
FilterContentBenchmark.filterWithNewParserConfig             half_field         true  cluster_stats  avgt    3   677657.665 ±  116475.739  ns/op
FilterContentBenchmark.filterWithNewParserConfig             half_field         true    index_stats  avgt    3    70578.713 ±   24811.400  ns/op
FilterContentBenchmark.filterWithNewParserConfig             half_field         true     node_stats  avgt    3    83243.011 ±   13569.090  ns/op
FilterContentBenchmark.filterWithNewParserConfig              all_field         true  cluster_stats  avgt    3  1131390.967 ±   30398.510  ns/op
FilterContentBenchmark.filterWithNewParserConfig              all_field         true    index_stats  avgt    3   119874.269 ±    7573.592  ns/op
FilterContentBenchmark.filterWithNewParserConfig              all_field         true     node_stats  avgt    3   141095.377 ±    5641.358  ns/op
FilterContentBenchmark.filterWithNewParserConfig         wildcard_field         true  cluster_stats  avgt    3   155739.206 ±   11728.617  ns/op
FilterContentBenchmark.filterWithNewParserConfig         wildcard_field         true    index_stats  avgt    3    16200.205 ±     654.411  ns/op
FilterContentBenchmark.filterWithNewParserConfig         wildcard_field         true     node_stats  avgt    3    19778.015 ±    1080.422  ns/op
FilterContentBenchmark.filterWithNewParserConfig      10_wildcard_field         true  cluster_stats  avgt    3   174019.416 ±   12869.374  ns/op
FilterContentBenchmark.filterWithNewParserConfig      10_wildcard_field         true    index_stats  avgt    3    27546.299 ±    2884.471  ns/op
FilterContentBenchmark.filterWithNewParserConfig      10_wildcard_field         true     node_stats  avgt    3    31921.524 ±    5231.016  ns/op
FilterContentBenchmark.filterWithParserConfigCreated           10_field         true  cluster_stats  avgt    3    97386.963 ±    8619.284  ns/op
FilterContentBenchmark.filterWithParserConfigCreated           10_field         true    index_stats  avgt    3    13974.582 ±    1150.768  ns/op
FilterContentBenchmark.filterWithParserConfigCreated           10_field         true     node_stats  avgt    3    16251.244 ±     992.268  ns/op
FilterContentBenchmark.filterWithParserConfigCreated         half_field         true  cluster_stats  avgt    3   351844.281 ±   17316.449  ns/op
FilterContentBenchmark.filterWithParserConfigCreated         half_field         true    index_stats  avgt    3    28799.525 ±    3432.223  ns/op
FilterContentBenchmark.filterWithParserConfigCreated         half_field         true     node_stats  avgt    3    31724.064 ±    5446.765  ns/op
FilterContentBenchmark.filterWithParserConfigCreated          all_field         true  cluster_stats  avgt    3   534004.297 ±   14777.255  ns/op
FilterContentBenchmark.filterWithParserConfigCreated          all_field         true    index_stats  avgt    3    49554.125 ±   35262.023  ns/op
FilterContentBenchmark.filterWithParserConfigCreated          all_field         true     node_stats  avgt    3    64189.587 ±   47764.156  ns/op
FilterContentBenchmark.filterWithParserConfigCreated     wildcard_field         true  cluster_stats  avgt    3   163864.604 ±   33860.263  ns/op
FilterContentBenchmark.filterWithParserConfigCreated     wildcard_field         true    index_stats  avgt    3    15874.515 ±     819.058  ns/op
FilterContentBenchmark.filterWithParserConfigCreated     wildcard_field         true     node_stats  avgt    3    18673.215 ±    1994.484  ns/op
FilterContentBenchmark.filterWithParserConfigCreated  10_wildcard_field         true  cluster_stats  avgt    3   178781.841 ±   20911.495  ns/op
FilterContentBenchmark.filterWithParserConfigCreated  10_wildcard_field         true    index_stats  avgt    3    23243.409 ±    2686.786  ns/op
FilterContentBenchmark.filterWithParserConfigCreated  10_wildcard_field         true     node_stats  avgt    3    28781.655 ±   13758.599  ns/op

weizijun · 2021-10-29T08:33:18Z

@nik9000 I separate the improve and the benchmark.
First merged this PR. Then test the performance before and after the improvement.

nik9000 · 2021-11-01T13:35:31Z

@elasticmachine, test this please

nik9000 · 2021-11-01T14:20:34Z

First merged this PR. Then test the performance before and after the improvement.

👍

benchmarks/src/main/java/org/elasticsearch/benchmark/xcontent/FilterContentBenchmark.java

nik9000 · 2021-11-03T13:38:58Z

benchmarks/src/main/java/org/elasticsearch/benchmark/xcontent/FilterContentBenchmark.java

+    @Param({ "10_field", "half_field", "all_field", "wildcard_field", "10_wildcard_field" })
+    private String fieldCount;
+
+    @Param({ "true", "false" })


I think in the short term you should set this to just "true". Because of the jackson bug. I feel bad about having this known issue but we have a fix in flight. It's just tangled in something else which is tangled in some other thing and all stuck....

nik9000 · 2021-11-03T14:40:36Z

Awesome. Code looks right to me. I'm going to run it locally because jenkins doesn't do that.

nik9000 · 2021-11-03T15:26:28Z

@elasticmachine test this please

elasticmachine · 2021-11-03T15:27:21Z

Pinging @elastic/es-analytics-geo (Team:Analytics)

nik9000 · 2021-11-03T15:34:59Z

@weizijun it looks like precommit is failing. Could you fix it? I think it's just an unused import.

nik9000 · 2021-11-03T16:02:41Z

@elasticmhachine, test this please

FilterPathBasedFilter is used for filter XContent. Old FilterPath is implemented with a linked list. When it has many patterns. The filter performance is lower. I implement a new FilterPath. It use a Tree to filter XContent. The FilterPath Tree has two kind of children. - One is a hashmap for term pattern. It can find the match field in O(1). - Another is a hashmap for wildcard pattern.It will check all node of hashmap. The benchmark I added in #80069 shows runtime halved in the non-wildcard cases. The wildcardcases have pretty similar runtime. There is an increase in runtime when newly building the filter each time. This is acceptable because the cases where we have to be as fast as possible we're already reusing the filter. ``` Benchmark (fieldCount) (data) Mode Cnt Before ns/op After ns/op NewParserConfig 10_field cluster_stats avgt 3 120303.585 120724.363 NewParserConfig 10_field index_stats avgt 3 22163.350 25700.478 NewParserConfig 10_field node_stats avgt 3 25838.040 25455.610 NewParserConfig half_field cluster_stats avgt 3 1052058.673 551504.323 NewParserConfig half_field index_stats avgt 3 99443.623 64639.860 NewParserConfig half_field node_stats avgt 3 119383.384 78739.087 NewParserConfig all_field cluster_stats avgt 3 1850317.142 949238.665 NewParserConfig all_field index_stats avgt 3 135923.737 102391.586 NewParserConfig all_field node_stats avgt 3 177303.659 134091.199 NewParserConfig wildcard_field cluster_stats avgt 3 242419.000 158402.969 NewParserConfig wildcard_field index_stats avgt 3 19677.926 16604.226 NewParserConfig wildcard_field node_stats avgt 3 21932.954 20135.669 NewParserConfig 10_wildcard_field cluster_stats avgt 3 217022.657 186846.661 NewParserConfig 10_wildcard_field index_stats avgt 3 33292.234 27895.644 NewParserConfig 10_wildcard_field node_stats avgt 3 41748.139 31325.693 ParserConfigReused 10_field cluster_stats avgt 3 120882.036 94980.708 ParserConfigReused 10_field index_stats avgt 3 16474.120 12325.762 ParserConfigReused 10_field node_stats avgt 3 20947.328 14945.972 ParserConfigReused half_field cluster_stats avgt 3 393069.494 182547.734 ParserConfigReused half_field index_stats avgt 3 28891.619 17536.035 ParserConfigReused half_field node_stats avgt 3 32433.385 22170.274 ParserConfigReused all_field cluster_stats avgt 3 551217.667 210148.246 ParserConfigReused all_field index_stats avgt 3 45367.850 20630.002 ParserConfigReused all_field node_stats avgt 3 55899.647 24953.435 ParserConfigReused wildcard_field cluster_stats avgt 3 159949.383 173188.846 ParserConfigReused wildcard_field index_stats avgt 3 15713.951 17187.918 ParserConfigReused wildcard_field node_stats avgt 3 19171.149 19932.483 ParserConfigReused 10_wildcard_field cluster_stats avgt 3 172137.559 173138.035 ParserConfigReused 10_wildcard_field index_stats avgt 3 24635.608 21296.013 ParserConfigReused 10_wildcard_field node_stats avgt 3 28866.104 26567.965 ```

add benchmark to test xcontent parser filter performance

51fdcdf

elasticsearchmachine added v8.1.0 external-contributor Pull request authored by a developer outside the Elasticsearch team labels Oct 29, 2021

weizijun added 2 commits October 29, 2021 16:35

add benchmark to test xcontent parser filter performance

64b81dd

improve

6163930

nik9000 requested changes Nov 1, 2021

View reviewed changes

weizijun added 3 commits November 2, 2021 11:31

improve

44f7e54

improve

fb9ffbf

improve

ac7d4dd

nik9000 reviewed Nov 3, 2021

View reviewed changes

improve

6a40271

nik9000 mentioned this pull request Nov 3, 2021

Add better support for metric data types (TSDB) #74660

Closed

nik9000 added :StorageEngine/TSDB You know, for Metrics auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) v8.0.0-beta1 labels Nov 3, 2021

elasticmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Nov 3, 2021

nik9000 removed the v8.0.0-beta1 label Nov 3, 2021

nik9000 approved these changes Nov 3, 2021

View reviewed changes

fixed unimport

ff1b777

elasticsearchmachine merged commit 6c33d83 into elastic:master Nov 3, 2021

pugnascotia added the >test Issues or PRs that are addressing/adding tests label Jan 31, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add benchmark to test xcontent parser filter performance #80069

add benchmark to test xcontent parser filter performance #80069

weizijun commented Oct 29, 2021 •

edited

Loading

weizijun commented Oct 29, 2021

nik9000 commented Nov 1, 2021

nik9000 commented Nov 1, 2021

nik9000 Nov 3, 2021

weizijun Nov 3, 2021

nik9000 commented Nov 3, 2021

nik9000 commented Nov 3, 2021

elasticmachine commented Nov 3, 2021

nik9000 commented Nov 3, 2021

nik9000 commented Nov 3, 2021

add benchmark to test xcontent parser filter performance #80069

add benchmark to test xcontent parser filter performance #80069

Conversation

weizijun commented Oct 29, 2021 • edited Loading

weizijun commented Oct 29, 2021

nik9000 commented Nov 1, 2021

nik9000 commented Nov 1, 2021

nik9000 Nov 3, 2021

Choose a reason for hiding this comment

weizijun Nov 3, 2021

Choose a reason for hiding this comment

nik9000 commented Nov 3, 2021

nik9000 commented Nov 3, 2021

elasticmachine commented Nov 3, 2021

nik9000 commented Nov 3, 2021

nik9000 commented Nov 3, 2021

weizijun commented Oct 29, 2021 •

edited

Loading