Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix for errors in prometheus connector when backquotes are used for fieldnames in aggregations. #1041

Merged
merged 1 commit into from
Nov 8, 2022

Conversation

vmmusings
Copy link
Member

@vmmusings vmmusings commented Nov 4, 2022

Signed-off-by: vamsi-amazon [email protected]

Description

Backquotes in stats groupby list is causing errors in prometheus connector.
Eg command:

source=prometheus.promhttp_metric | stats avg(@value) by span(@timestamp, 15s), `instance`

Issues Resolved

[List any issues this PR will resolve]

Check List

  • New functionality includes testing.
    • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented.
    • New functionality has javadoc added
    • New functionality has user manual doc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@vmmusings vmmusings marked this pull request as ready for review November 4, 2022 22:18
@vmmusings vmmusings requested a review from a team as a code owner November 4, 2022 22:18
@codecov-commenter
Copy link

codecov-commenter commented Nov 4, 2022

Codecov Report

Merging #1041 (77588fd) into 2.4 (4a9cef3) will decrease coverage by 2.57%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##                2.4    #1041      +/-   ##
============================================
- Coverage     98.26%   95.68%   -2.58%     
- Complexity     3326     3332       +6     
============================================
  Files           324      334      +10     
  Lines          8400     9071     +671     
  Branches        553      672     +119     
============================================
+ Hits           8254     8680     +426     
- Misses          142      334     +192     
- Partials          4       57      +53     
Flag Coverage Δ
query-workbench 62.76% <ø> (?)
sql-engine 98.26% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...us/storage/model/PrometheusResponseFieldNames.java 100.00% <ø> (ø)
...ch/sql/prometheus/response/PrometheusResponse.java 100.00% <100.00%> (ø)
...rage/implementor/PrometheusDefaultImplementor.java 100.00% <100.00%> (ø)
.../storage/querybuilder/AggregationQueryBuilder.java 100.00% <100.00%> (ø)
workbench/public/components/SQLPage/SQLPage.tsx 100.00% <0.00%> (ø)
workbench/public/components/PPLPage/PPLPage.tsx 56.52% <0.00%> (ø)
...ublic/components/QueryResults/QueryResultsBody.tsx 68.32% <0.00%> (ø)
...ch/public/components/QueryResults/QueryResults.tsx 61.60% <0.00%> (ø)
workbench/public/application.tsx 0.00% <0.00%> (ø)
workbench/public/components/app.tsx 0.00% <0.00%> (ø)
... and 4 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@Yury-Fridlyand
Copy link
Collaborator

Backquotes could be used to wrap any identifier. Could you, please, test a more complex scenario?

source=`prometheus`.`promhttp_metric` | stats avg(@value) as `agg` by span(@timestamp, 15s), `instance`, `handler`, `job`

@vmmusings vmmusings force-pushed the backquote-fix branch 2 times, most recently from 63df9f9 to 2c6d0d2 Compare November 4, 2022 22:43
@vmmusings
Copy link
Member Author

Backquotes could be used to wrap any identifier. Could you, please, test a more complex scenario?

source=`prometheus`.`promhttp_metric` | stats avg(@value) as `agg` by span(@timestamp, 15s), `instance`, `handler`, `job`

Sure

ps48
ps48 previously approved these changes Nov 8, 2022
Copy link
Collaborator

@penghuo penghuo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/opensearch-project/sql/blob/2.x/ppl/src/main/java/org/opensearch/sql/ppl/parser/AstExpressionBuilder.java#L365

Is it covered by Parser?

Yes, Kind of covered.

NamedExpression

{
 name :  \`instance\`
 refExpression : {
  attr : "instance"
  }
 }

I can rely on ReferenceExpression instead of stripping the backquotes.

@dai-chen dai-chen added the bug Something isn't working label Nov 8, 2022
YANG-DB
YANG-DB previously approved these changes Nov 8, 2022
anirudha
anirudha previously approved these changes Nov 8, 2022
@vmmusings vmmusings dismissed stale reviews from anirudha, YANG-DB, and ps48 via 77588fd November 8, 2022 22:00
Signed-off-by: vamsi-amazon <[email protected]>
@vmmusings vmmusings merged commit d3bb902 into opensearch-project:2.4 Nov 8, 2022
opensearch-trigger-bot bot pushed a commit that referenced this pull request Nov 8, 2022
Signed-off-by: vamsi-amazon <[email protected]>
(cherry picked from commit d3bb902)
vmmusings added a commit that referenced this pull request Nov 8, 2022
Signed-off-by: vamsi-amazon <[email protected]>
(cherry picked from commit d3bb902)

Co-authored-by: vamsi-amazon <[email protected]>
dai-chen added a commit that referenced this pull request Nov 9, 2022
* Fix `FLOAT` -> `DOUBLE` cast. (#1025)

Signed-off-by: Yury-Fridlyand <[email protected]>

* Fix error messaging from prometheus. (#1029) (#1037)

Signed-off-by: vamsi-amazon <[email protected]>
(cherry picked from commit 4a9cef3)

Co-authored-by: vamsi-amazon <[email protected]>

* Add `query` function as alternate syntax to `query_string` function (#1010)

This maintains backwards compatibility with the v1 engine.

* Update DATE and TIME functions to parse string input as datetime (#991)

Add option to accept datetime like string in both TIME and DATE (eg. accept "1999-01-02 12:12:12" for both TIME and DATE.

Strict check on date for testing for valid dates (eg. Don't accept Feb 30th as a valid date) and throws a SemanticCheckException.

Co-authored-by: Yury-Fridlyand <[email protected]>
Signed-off-by: MitchellGale-BitQuill <[email protected]>

* back quote fix (#1041) (#1050)

Signed-off-by: vamsi-amazon <[email protected]>
(cherry picked from commit d3bb902)

Co-authored-by: vamsi-amazon <[email protected]>

* Catalog to Datasource changes (#1027) (#1049)

Signed-off-by: vamsi-amazon <[email protected]>
(cherry picked from commit 3e30379)

* Bump jackson to 2.14.0 (#1058)

Signed-off-by: Joshua Li <[email protected]>
(cherry picked from commit 5a1adb2)

* Add valueOf() to Expression (#1055)

Signed-off-by: Joshua Li <[email protected]>

Signed-off-by: Yury-Fridlyand <[email protected]>
Signed-off-by: MitchellGale-BitQuill <[email protected]>
Signed-off-by: Joshua Li <[email protected]>
Signed-off-by: Chen Dai <[email protected]>
Co-authored-by: Yury-Fridlyand <[email protected]>
Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>
Co-authored-by: vamsi-amazon <[email protected]>
Co-authored-by: GabeFernandez310 <[email protected]>
Co-authored-by: MitchellGale-BitQuill <[email protected]>
Co-authored-by: Joshua Li <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants