Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mappings traversal bug fix #669

Merged
merged 10 commits into from
Nov 9, 2022

Conversation

petardz
Copy link
Contributor

@petardz petardz commented Nov 9, 2022

Issue #, if available:

Description of changes:

Fixed few bugs with index mappings traversal which resulted in monitors not working.

CheckList:
[ ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Petar Dzepina <[email protected]>
Signed-off-by: Petar Dzepina <[email protected]>
Signed-off-by: Petar Dzepina <[email protected]>
Signed-off-by: Petar Dzepina <[email protected]>
@petardz petardz requested a review from a team November 9, 2022 02:13
Signed-off-by: Petar Dzepina <[email protected]>
@codecov-commenter
Copy link

codecov-commenter commented Nov 9, 2022

Codecov Report

Merging #669 (f75899f) into main (0fb9dd0) will increase coverage by 0.22%.
The diff coverage is 80.64%.

@@             Coverage Diff              @@
##               main     #669      +/-   ##
============================================
+ Coverage     76.03%   76.26%   +0.22%     
  Complexity      116      116              
============================================
  Files           124      124              
  Lines          6606     6618      +12     
  Branches        980      982       +2     
============================================
+ Hits           5023     5047      +24     
+ Misses         1087     1071      -16     
- Partials        496      500       +4     
Impacted Files Coverage Δ
...opensearch/alerting/util/DocLevelMonitorQueries.kt 80.35% <80.64%> (+0.35%) ⬆️
...pensearch/percolator/PercolatorFieldMapperExt.java 76.33% <0.00%> (+1.78%) ⬆️
.../kotlin/org/opensearch/alerting/util/IndexUtils.kt 72.91% <0.00%> (+2.08%) ⬆️
.../alerting/transport/TransportIndexMonitorAction.kt 62.57% <0.00%> (+2.63%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

* From given index mapping node, extracts fieldName -> fieldProperties pair
*/
fun extractField(node: MutableMap<String, Any>, currentPath: String): Pair<String, MutableMap<String, Any>> {
fun traverseMappingsAndUpdate(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add comments in the different logic flows of this function? It is kind of hard to follow what it is doing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Signed-off-by: Petar Dzepina <[email protected]>
Signed-off-by: Petar Dzepina <[email protected]>
@eirsep eirsep merged commit 0afcc4a into opensearch-project:main Nov 9, 2022
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.x 2.x
# Navigate to the new working tree
cd .worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-669-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 0afcc4ae15519ab1eef6a8edfebb6b7e19fa78d4
# Push it to GitHub
git push --set-upstream origin backport/backport-669-to-2.x
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-669-to-2.x.

@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.4 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.4 2.4
# Navigate to the new working tree
cd .worktrees/backport-2.4
# Create a new branch
git switch --create backport/backport-669-to-2.4
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 0afcc4ae15519ab1eef6a8edfebb6b7e19fa78d4
# Push it to GitHub
git push --set-upstream origin backport/backport-669-to-2.4
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.4

Then, create a pull request where the base branch is 2.4 and the compare/head branch is backport/backport-669-to-2.4.

petardz added a commit to petardz/alerting that referenced this pull request Nov 9, 2022
petardz added a commit to petardz/alerting that referenced this pull request Nov 9, 2022
petardz added a commit to petardz/alerting that referenced this pull request Nov 9, 2022
eirsep pushed a commit to eirsep/alerting that referenced this pull request Nov 9, 2022
eirsep pushed a commit that referenced this pull request Nov 9, 2022
Signed-off-by: Petar Dzepina <[email protected]>

Signed-off-by: Petar Dzepina <[email protected]>
petardz added a commit to petardz/alerting that referenced this pull request Jan 4, 2023
eirsep pushed a commit that referenced this pull request Jan 4, 2023
* Added support for "nested" mappings (#645)

* example

Signed-off-by: Petar Dzepina <[email protected]>

* fixed updating mappings for queryIndex

Signed-off-by: Petar Dzepina <[email protected]>

Signed-off-by: Petar Dzepina <[email protected]>

* mappings traversal bug fix (#669)



Signed-off-by: Petar Dzepina <[email protected]>

Signed-off-by: Petar Dzepina <[email protected]>
@lezzago lezzago mentioned this pull request Jan 6, 2023
lezzago pushed a commit to lezzago/alerting-opensearch that referenced this pull request Jun 8, 2023
* Added support for "nested" mappings (opensearch-project#645)

* example

Signed-off-by: Petar Dzepina <[email protected]>

* fixed updating mappings for queryIndex

Signed-off-by: Petar Dzepina <[email protected]>

Signed-off-by: Petar Dzepina <[email protected]>

* mappings traversal bug fix (opensearch-project#669)

Signed-off-by: Petar Dzepina <[email protected]>

Signed-off-by: Petar Dzepina <[email protected]>
lezzago added a commit that referenced this pull request Jun 8, 2023
* Added exception check once the .opendistro-alerting-config index is b… (#650)

* Added exception check once the .opendistro-alerting-config index is being created

During .opendistro-alerting-config index creation, if ResourceAlreadyExists exception is being raised, the flow will check first if the index is in yellow state and then it will re-try to index monitor

Signed-off-by: Stevan Buzejic <[email protected]>

* Formating of the file fixed

Signed-off-by: Stevan Buzejic <[email protected]>

Signed-off-by: Stevan Buzejic <[email protected]>

* refactored DeleteMonitor Action to be synchronious (#628) (#630)

* refactored DeleteMonitor Action to be synchronious

Signed-off-by: Petar Dzepina <[email protected]>

Signed-off-by: Petar Dzepina <[email protected]>
Co-authored-by: Petar Dzepina <[email protected]>

* [Backport 2.x] QueryIndex rollover when field mapping limit is reached (#729)

Signed-off-by: Petar Dzepina <[email protected]>

* Mappings fix backport 2.x (#730)

* Added support for "nested" mappings (#645)

* example

Signed-off-by: Petar Dzepina <[email protected]>

* fixed updating mappings for queryIndex

Signed-off-by: Petar Dzepina <[email protected]>

Signed-off-by: Petar Dzepina <[email protected]>

* mappings traversal bug fix (#669)

Signed-off-by: Petar Dzepina <[email protected]>

Signed-off-by: Petar Dzepina <[email protected]>

* Added unwrapping exception from core; added more debug logs (#728)

Signed-off-by: Petar Dzepina <[email protected]>

* Add DataSources test for future backports

Signed-off-by: Ashish Agrawal <[email protected]>

* fix  percolator mapping error when having field name 'type' (#726)

Signed-off-by: Raj Chakravarthi <[email protected]>

* [BUG] ExecuteMonitor inserting metadata doc during dry run (#758)

* execute monitor bugfix

Signed-off-by: Petar Dzepina <[email protected]>

* added IT

Signed-off-by: Petar Dzepina <[email protected]>

* fixed created retval when skipIndex=true

Signed-off-by: Petar Dzepina <[email protected]>

---------

Signed-off-by: Petar Dzepina <[email protected]>

* fix for ERROR alert state generation in doc-level monitors (#768)

Signed-off-by: Subhobrata Dey <[email protected]>

* Adjusting max field index setting dynamically for query index (#776)

* added adjusting max field index setting dynamicly for query index

Signed-off-by: Petar Dzepina <[email protected]>

* Multiple indices support in DocLevelMonitorInput (#784)

Signed-off-by: Petar Dzepina <[email protected]>

* Doc transform 2.x backport (#853)

* conflict resovle - backport from main to 2.x

Signed-off-by: Petar Dzepina <[email protected]>

* fixed module class names

Signed-off-by: Petar Dzepina <[email protected]>

---------

Signed-off-by: Petar Dzepina <[email protected]>

* Update config index schema if needed at the start of each monitor execution (#849)

* Update config index schema if needed at the start of each monitor execution

Signed-off-by: Ashish Agrawal <[email protected]>

* Mappings parsing fix (#851)

* fixed mappings parsing when field name named "properties" exists in mappings

Signed-off-by: Petar Dzepina <[email protected]>

* message typo fix

Signed-off-by: Petar Dzepina <[email protected]>

---------

Signed-off-by: Petar Dzepina <[email protected]>

* [Backport 2.x] Notification security fix (#861)

* Notification security fix (#852)

* added injecting whole user object in threadContext before calling notification APIs so that backend roles are available to notification plugin

Signed-off-by: Petar Dzepina <[email protected]>

* compile fix

Signed-off-by: Petar Dzepina <[email protected]>

* refactored user_info injection to use InjectSecurity

Signed-off-by: Petar Dzepina <[email protected]>

* ktlint fix

Signed-off-by: Petar Dzepina <[email protected]>

---------

Signed-off-by: Petar Dzepina <[email protected]>
(cherry picked from commit e0b7a5a)

* remove unneeded import

Signed-off-by: Ashish Agrawal <[email protected]>

---------

Signed-off-by: Ashish Agrawal <[email protected]>
Co-authored-by: Petar Dzepina <[email protected]>
Co-authored-by: Ashish Agrawal <[email protected]>

* Fixed a bug that prevented alerts from being generated for doc level monitors that use wildcard characters in index names. (#894)

Signed-off-by: AWSHurneyt <[email protected]>

* fixed security tests (#484) (#794)

* fixed security tests

Signed-off-by: Raj Chakravarthi <[email protected]>
(cherry picked from commit c51940f)

---------

Signed-off-by: Stevan Buzejic <[email protected]>
Signed-off-by: Petar Dzepina <[email protected]>
Signed-off-by: Ashish Agrawal <[email protected]>
Signed-off-by: Raj Chakravarthi <[email protected]>
Signed-off-by: Subhobrata Dey <[email protected]>
Signed-off-by: AWSHurneyt <[email protected]>
Co-authored-by: Stevan Buzejic <[email protected]>
Co-authored-by: Surya Sashank Nistala <[email protected]>
Co-authored-by: Petar Dzepina <[email protected]>
Co-authored-by: RAJ CHAKRAVARTHI <[email protected]>
Co-authored-by: Subhobrata Dey <[email protected]>
Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>
Co-authored-by: AWSHurneyt <[email protected]>
Co-authored-by: RAJ CHAKRAVARTHI <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants