Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP for parsing composite patterns with Format.parseObject #40364

Closed
wants to merge 15 commits into from

Conversation

pgomulka
Copy link
Contributor

@pgomulka pgomulka commented Mar 22, 2019

a work in progress now. making sure this would pass all the tests. The performance should be good of this:

Benchmark                             Mode  Cnt     Score    Error  Units
DateFormatterBenchmark.parseJavaDate  avgt   30   742.159 ± 30.878  ns/op
DateFormatterBenchmark.parseJodaDate  avgt   30  1018.573 ± 26.481  ns/op

relates #40100

@pgomulka pgomulka added WIP :Core/Infra/Core Core issues without another label labels Mar 22, 2019
@pgomulka pgomulka self-assigned this Mar 22, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra

@pgomulka
Copy link
Contributor Author

Rally results on this PR

pgomulka@przemek-gomulka:~/workspace/pgomulka/elasticsearch$ esrally --runtime-jdk=8 --car="4gheap" --pipeline=from-sources-complete --revision=be2820a67fe8e2a0462c6f51ce2a935e22535b41 --track=http_logs --challenge=append-no-conflicts-index-only

    ____        ____
   / __ \____ _/ / /_  __
  / /_/ / __ `/ / / / / /
 / _, _/ /_/ / / / /_/ /
/_/ |_|\__,_/_/_/\__, /
                /____/

[INFO] Preparing for race ...
[INFO] Racing on track [http_logs], challenge [append-no-conflicts-index-only] and car ['4gheap'] with version [8.0.0-SNAPSHOT].

Running delete-index                                                           [100% done]
Running create-index                                                           [100% done]
Running check-cluster-health                                                   [100% done]
Running index-append                                                           [100% done]
Running refresh-after-index                                                    [100% done]
Running force-merge                                                            [100% done]
Running refresh-after-force-merge                                              [100% done]

------------------------------------------------------
    _______             __   _____
   / ____(_)___  ____ _/ /  / ___/_________  ________
  / /_  / / __ \/ __ `/ /   \__ \/ ___/ __ \/ ___/ _ \
 / __/ / / / / / /_/ / /   ___/ / /__/ /_/ / /  /  __/
/_/   /_/_/ /_/\__,_/_/   /____/\___/\____/_/   \___/
------------------------------------------------------
            
|   Lap |                                                         Metric |         Task |     Value |   Unit |
|------:|---------------------------------------------------------------:|-------------:|----------:|-------:|
|   All |                     Cumulative indexing time of primary shards |              |    139.01 |    min |
|   All |             Min cumulative indexing time across primary shards |              |  0.414117 |    min |
|   All |          Median cumulative indexing time across primary shards |              |   1.33825 |    min |
|   All |             Max cumulative indexing time across primary shards |              |   20.5687 |    min |
|   All |            Cumulative indexing throttle time of primary shards |              |         0 |    min |
|   All |    Min cumulative indexing throttle time across primary shards |              |         0 |    min |
|   All | Median cumulative indexing throttle time across primary shards |              |         0 |    min |
|   All |    Max cumulative indexing throttle time across primary shards |              |         0 |    min |
|   All |                        Cumulative merge time of primary shards |              |    38.988 |    min |
|   All |                       Cumulative merge count of primary shards |              |       104 |        |
|   All |                Min cumulative merge time across primary shards |              |         0 |    min |
|   All |             Median cumulative merge time across primary shards |              |         0 |    min |
|   All |                Max cumulative merge time across primary shards |              |   8.25908 |    min |
|   All |               Cumulative merge throttle time of primary shards |              |   24.2958 |    min |
|   All |       Min cumulative merge throttle time across primary shards |              |         0 |    min |
|   All |    Median cumulative merge throttle time across primary shards |              |         0 |    min |
|   All |       Max cumulative merge throttle time across primary shards |              |   5.54963 |    min |
|   All |                      Cumulative refresh time of primary shards |              |   11.4224 |    min |
|   All |                     Cumulative refresh count of primary shards |              |       707 |        |
|   All |              Min cumulative refresh time across primary shards |              | 0.0384333 |    min |
|   All |           Median cumulative refresh time across primary shards |              |   0.12515 |    min |
|   All |              Max cumulative refresh time across primary shards |              |   1.69878 |    min |
|   All |                        Cumulative flush time of primary shards |              |    4.8389 |    min |
|   All |                       Cumulative flush count of primary shards |              |       110 |        |
|   All |                Min cumulative flush time across primary shards |              |   0.00085 |    min |
|   All |             Median cumulative flush time across primary shards |              | 0.0193333 |    min |
|   All |                Max cumulative flush time across primary shards |              |    1.0165 |    min |
|   All |                                               Median CPU usage |              |     485.8 |      % |
|   All |                                             Total Young Gen GC |              |    95.181 |      s |
|   All |                                               Total Old Gen GC |              |     3.459 |      s |
|   All |                                                     Store size |              |   20.2474 |     GB |
|   All |                                                  Translog size |              |   14.6098 |     GB |
|   All |                                                     Index size |              |   34.2891 |     GB |
|   All |                                                  Total written |              |   107.927 |     GB |
|   All |                                         Heap used for segments |              |    108.11 |     MB |
|   All |                                       Heap used for doc values |              |  0.130257 |     MB |
|   All |                                            Heap used for terms |              |    95.232 |     MB |
|   All |                                            Heap used for norms |              | 0.0367432 |     MB |
|   All |                                           Heap used for points |              |   5.77796 |     MB |
|   All |                                    Heap used for stored fields |              |   6.93297 |     MB |
|   All |                                                  Segment count |              |       602 |        |
|   All |                                                 Min Throughput | index-append |    127642 | docs/s |
|   All |                                              Median Throughput | index-append |    135027 | docs/s |
|   All |                                                 Max Throughput | index-append |    156514 | docs/s |
|   All |                                        50th percentile latency | index-append |    263.56 |     ms |
|   All |                                        90th percentile latency | index-append |   411.534 |     ms |
|   All |                                        99th percentile latency | index-append |   1341.69 |     ms |
|   All |                                      99.9th percentile latency | index-append |      3060 |     ms |
|   All |                                     99.99th percentile latency | index-append |   3882.94 |     ms |
|   All |                                       100th percentile latency | index-append |    4844.4 |     ms |
|   All |                                   50th percentile service time | index-append |    263.56 |     ms |
|   All |                                   90th percentile service time | index-append |   411.534 |     ms |
|   All |                                   99th percentile service time | index-append |   1341.69 |     ms |
|   All |                                 99.9th percentile service time | index-append |      3060 |     ms |
|   All |                                99.99th percentile service time | index-append |   3882.94 |     ms |
|   All |                                  100th percentile service time | index-append |    4844.4 |     ms |
|   All |                                                     error rate | index-append |         0 |      % |

@pgomulka
Copy link
Contributor Author

rally results from master

pgomulka@przemek-gomulka:~/workspace/pgomulka/elasticsearch$ esrally --runtime-jdk=8 --car="4gheap" --pipeline=from-sources-complete --revision=209e08eb02df2ee86e2ce264ed6e7f74fe2f3490 --track=http_logs --challenge=append-no-conflicts-index-only

    ____        ____
   / __ \____ _/ / /_  __
  / /_/ / __ `/ / / / / /
 / _, _/ /_/ / / / /_/ /
/_/ |_|\__,_/_/_/\__, /
                /____/

[INFO] Preparing for race ...
[INFO] Racing on track [http_logs], challenge [append-no-conflicts-index-only] and car ['4gheap'] with version [8.0.0-SNAPSHOT].

Running delete-index                                                           [100% done]
Running create-index                                                           [100% done]
Running check-cluster-health                                                   [100% done]
Running index-append                                                           [100% done]
Running refresh-after-index                                                    [100% done]
Running force-merge                                                            [100% done]
Running refresh-after-force-merge                                              [100% done]

------------------------------------------------------
    _______             __   _____
   / ____(_)___  ____ _/ /  / ___/_________  ________
  / /_  / / __ \/ __ `/ /   \__ \/ ___/ __ \/ ___/ _ \
 / __/ / / / / / /_/ / /   ___/ / /__/ /_/ / /  /  __/
/_/   /_/_/ /_/\__,_/_/   /____/\___/\____/_/   \___/
------------------------------------------------------
            
|   Lap |                                                         Metric |         Task |     Value |   Unit |
|------:|---------------------------------------------------------------:|-------------:|----------:|-------:|
|   All |                     Cumulative indexing time of primary shards |              |   139.603 |    min |
|   All |             Min cumulative indexing time across primary shards |              |  0.383283 |    min |
|   All |          Median cumulative indexing time across primary shards |              |   1.31263 |    min |
|   All |             Max cumulative indexing time across primary shards |              |   20.9492 |    min |
|   All |            Cumulative indexing throttle time of primary shards |              |         0 |    min |
|   All |    Min cumulative indexing throttle time across primary shards |              |         0 |    min |
|   All | Median cumulative indexing throttle time across primary shards |              |         0 |    min |
|   All |    Max cumulative indexing throttle time across primary shards |              |         0 |    min |
|   All |                        Cumulative merge time of primary shards |              |   36.8138 |    min |
|   All |                       Cumulative merge count of primary shards |              |       111 |        |
|   All |                Min cumulative merge time across primary shards |              |         0 |    min |
|   All |             Median cumulative merge time across primary shards |              |         0 |    min |
|   All |                Max cumulative merge time across primary shards |              |   7.72227 |    min |
|   All |               Cumulative merge throttle time of primary shards |              |   22.7227 |    min |
|   All |       Min cumulative merge throttle time across primary shards |              |         0 |    min |
|   All |    Median cumulative merge throttle time across primary shards |              |         0 |    min |
|   All |       Max cumulative merge throttle time across primary shards |              |   5.18448 |    min |
|   All |                      Cumulative refresh time of primary shards |              |   11.1123 |    min |
|   All |                     Cumulative refresh count of primary shards |              |       715 |        |
|   All |              Min cumulative refresh time across primary shards |              | 0.0307833 |    min |
|   All |           Median cumulative refresh time across primary shards |              |  0.128933 |    min |
|   All |              Max cumulative refresh time across primary shards |              |   1.62187 |    min |
|   All |                        Cumulative flush time of primary shards |              |   5.11853 |    min |
|   All |                       Cumulative flush count of primary shards |              |       110 |        |
|   All |                Min cumulative flush time across primary shards |              |    0.0007 |    min |
|   All |             Median cumulative flush time across primary shards |              |   0.02155 |    min |
|   All |                Max cumulative flush time across primary shards |              |   1.03015 |    min |
|   All |                                               Median CPU usage |              |    477.25 |      % |
|   All |                                             Total Young Gen GC |              |    94.804 |      s |
|   All |                                               Total Old Gen GC |              |     3.602 |      s |
|   All |                                                     Store size |              |   20.9125 |     GB |
|   All |                                                  Translog size |              |   14.6144 |     GB |
|   All |                                                     Index size |              |   34.2855 |     GB |
|   All |                                                  Total written |              |   107.963 |     GB |
|   All |                                         Heap used for segments |              |   108.913 |     MB |
|   All |                                       Heap used for doc values |              |  0.146904 |     MB |
|   All |                                            Heap used for terms |              |   96.0431 |     MB |
|   All |                                            Heap used for norms |              | 0.0327148 |     MB |
|   All |                                           Heap used for points |              |   5.77322 |     MB |
|   All |                                    Heap used for stored fields |              |   6.91753 |     MB |
|   All |                                                  Segment count |              |       536 |        |
|   All |                                                 Min Throughput | index-append |    124989 | docs/s |
|   All |                                              Median Throughput | index-append |    132852 | docs/s |
|   All |                                                 Max Throughput | index-append |    148503 | docs/s |
|   All |                                        50th percentile latency | index-append |   264.455 |     ms |
|   All |                                        90th percentile latency | index-append |   417.722 |     ms |
|   All |                                        99th percentile latency | index-append |   1442.01 |     ms |
|   All |                                      99.9th percentile latency | index-append |   2992.54 |     ms |
|   All |                                     99.99th percentile latency | index-append |   3602.25 |     ms |
|   All |                                       100th percentile latency | index-append |   4376.98 |     ms |
|   All |                                   50th percentile service time | index-append |   264.455 |     ms |
|   All |                                   90th percentile service time | index-append |   417.722 |     ms |
|   All |                                   99th percentile service time | index-append |   1442.01 |     ms |
|   All |                                 99.9th percentile service time | index-append |   2992.54 |     ms |
|   All |                                99.99th percentile service time | index-append |   3602.25 |     ms |
|   All |                                  100th percentile service time | index-append |   4376.98 |     ms |
|   All |                                                     error rate | index-append |         0 |      % |


----------------------------------
[INFO] SUCCESS (took 2117 seconds)

@pgomulka
Copy link
Contributor Author

pgomulka commented Mar 22, 2019

master

Median Throughput | index-append |    132852 | docs/s |

vs PR

Median Throughput | index-append |    135027 | docs/s |

return (TemporalAccessor) object;
}
}
throw new DateTimeParseException("Failed to parse with all parsers from a composite parser", input, 0);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's implicit that this is a composite parser, as the format will be returned in the IAE?

return firstParser().parse(input);
}

private boolean parssingSucceeded(Object object, String input, ParsePosition pos) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

useless nit: one s less?

@pgomulka pgomulka force-pushed the poc/format_parse_object branch from e979460 to cf76d67 Compare March 25, 2019 08:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Core/Infra/Core Core issues without another label WIP
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants