-
Notifications
You must be signed in to change notification settings - Fork 113
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
added support for mustache scripting of rollup.target_index field #435
added support for mustache scripting of rollup.target_index field #435
Conversation
Signed-off-by: Petar Dzepina <[email protected]>
Looks like the test and build workflows are failing due to code style errors.
Which shouldn't be hard to fix |
Signed-off-by: Petar Dzepina <[email protected]>
Signed-off-by: Petar Dzepina <[email protected]>
We allow source index names to have wildcards, so we can have scenarios like this: To prevent these failures at runtime and clarify the behavior let's validate that the user isn't using both wildcard source indices and a template in their target index. I think an easy way would just be to temporarily resolve the target index name in the init block of Rollup.kt and validate that it doesn't contain any wildcards. I don't mean updating the target index name with the resolved template, just resolving here to perform input validation. Feel free to implement it however you find best though! |
...n/org/opensearch/indexmanagement/rollup/action/mapping/TransportUpdateRollupMappingAction.kt
Show resolved
Hide resolved
...n/org/opensearch/indexmanagement/rollup/action/mapping/TransportUpdateRollupMappingAction.kt
Outdated
Show resolved
Hide resolved
...main/kotlin/org/opensearch/indexmanagement/rollup/util/RollupFieldValueExpressionResolver.kt
Outdated
Show resolved
Hide resolved
...main/kotlin/org/opensearch/indexmanagement/rollup/util/RollupFieldValueExpressionResolver.kt
Outdated
Show resolved
Hide resolved
src/test/kotlin/org/opensearch/indexmanagement/indexstatemanagement/action/RollupActionIT.kt
Outdated
Show resolved
Hide resolved
Signed-off-by: Petar Dzepina <[email protected]>
…etIndex on Rollup init; added test for wildcards Signed-off-by: Petar Dzepina <[email protected]>
Signed-off-by: Petar Dzepina <[email protected]>
...rg/opensearch/indexmanagement/indexstatemanagement/step/rollup/AttemptCreateRollupJobStep.kt
Outdated
Show resolved
Hide resolved
...main/kotlin/org/opensearch/indexmanagement/rollup/action/index/TransportIndexRollupAction.kt
Outdated
Show resolved
Hide resolved
Thanks Petar, I am happy to approve after these. |
Signed-off-by: Petar Dzepina <[email protected]>
Signed-off-by: Petar Dzepina <[email protected]>
Signed-off-by: Petar Dzepina <[email protected]>
Signed-off-by: Petar Dzepina <[email protected]>
Signed-off-by: Petar Dzepina <[email protected]>
Signed-off-by: Petar Dzepina <[email protected]>
dbb884d
to
6dc3385
Compare
Signed-off-by: Petar Dzepina <[email protected]>
* added support for mustache scripting of rollup.target_index field Signed-off-by: Petar Dzepina <[email protected]> * defekt fixes Signed-off-by: Petar Dzepina <[email protected]> * tests Signed-off-by: Petar Dzepina <[email protected]> * small refactor/improvements Signed-off-by: Petar Dzepina <[email protected]> * added wildcard check when creating rollup job; removed resolving targetIndex on Rollup init; added test for wildcards Signed-off-by: Petar Dzepina <[email protected]> * lint fixes Signed-off-by: Petar Dzepina <[email protected]> * moved target_index validation in getRollup resp handler Signed-off-by: Petar Dzepina <[email protected]> * added using toMap() Signed-off-by: Petar Dzepina <[email protected]> * removed catch block Signed-off-by: Petar Dzepina <[email protected]> * exception fix Signed-off-by: Petar Dzepina <[email protected]> * linter fix Signed-off-by: Petar Dzepina <[email protected]> * fixed IT fail Signed-off-by: Petar Dzepina <[email protected]> * added Exception catch block Signed-off-by: Petar Dzepina <[email protected]> (cherry picked from commit e858ab2)
…) (#444) * added support for mustache scripting of rollup.target_index field Signed-off-by: Petar Dzepina <[email protected]> * defekt fixes Signed-off-by: Petar Dzepina <[email protected]> * tests Signed-off-by: Petar Dzepina <[email protected]> * small refactor/improvements Signed-off-by: Petar Dzepina <[email protected]> * added wildcard check when creating rollup job; removed resolving targetIndex on Rollup init; added test for wildcards Signed-off-by: Petar Dzepina <[email protected]> * lint fixes Signed-off-by: Petar Dzepina <[email protected]> * moved target_index validation in getRollup resp handler Signed-off-by: Petar Dzepina <[email protected]> * added using toMap() Signed-off-by: Petar Dzepina <[email protected]> * removed catch block Signed-off-by: Petar Dzepina <[email protected]> * exception fix Signed-off-by: Petar Dzepina <[email protected]> * linter fix Signed-off-by: Petar Dzepina <[email protected]> * fixed IT fail Signed-off-by: Petar Dzepina <[email protected]> * added Exception catch block Signed-off-by: Petar Dzepina <[email protected]> (cherry picked from commit e858ab2) Co-authored-by: Petar Dzepina <[email protected]>
…ensearch-project#435) * tests * small refactor/improvements * added wildcard check when creating rollup job; removed resolving targetIndex on Rollup init; added test for wildcards * lint fixes * moved target_index validation in getRollup resp handler * removed catch block * exception fix * linter fix * fixed IT fail * added Exception catch block Signed-off-by: Petar Dzepina <[email protected]>
…ensearch-project#435) * tests * small refactor/improvements * added wildcard check when creating rollup job; removed resolving targetIndex on Rollup init; added test for wildcards * lint fixes * moved target_index validation in getRollup resp handler * removed catch block * exception fix * linter fix * fixed IT fail * added Exception catch block Signed-off-by: Petar Dzepina <[email protected]>
…ensearch-project#435) (opensearch-project#444) * added support for mustache scripting of rollup.target_index field Signed-off-by: Petar Dzepina <[email protected]> * defekt fixes Signed-off-by: Petar Dzepina <[email protected]> * tests Signed-off-by: Petar Dzepina <[email protected]> * small refactor/improvements Signed-off-by: Petar Dzepina <[email protected]> * added wildcard check when creating rollup job; removed resolving targetIndex on Rollup init; added test for wildcards Signed-off-by: Petar Dzepina <[email protected]> * lint fixes Signed-off-by: Petar Dzepina <[email protected]> * moved target_index validation in getRollup resp handler Signed-off-by: Petar Dzepina <[email protected]> * added using toMap() Signed-off-by: Petar Dzepina <[email protected]> * removed catch block Signed-off-by: Petar Dzepina <[email protected]> * exception fix Signed-off-by: Petar Dzepina <[email protected]> * linter fix Signed-off-by: Petar Dzepina <[email protected]> * fixed IT fail Signed-off-by: Petar Dzepina <[email protected]> * added Exception catch block Signed-off-by: Petar Dzepina <[email protected]> (cherry picked from commit e858ab2) Co-authored-by: Petar Dzepina <[email protected]> Signed-off-by: Angie Zhang <[email protected]>
* added support for mustache scripting of rollup.target_index field (#435) * tests * small refactor/improvements * added wildcard check when creating rollup job; removed resolving targetIndex on Rollup init; added test for wildcards * lint fixes * moved target_index validation in getRollup resp handler * removed catch block * fixed IT fail * added Exception catch block Signed-off-by: Petar Dzepina <[email protected]>
* added support for mustache scripting of rollup.target_index field (#435) * tests * small refactor/improvements * added wildcard check when creating rollup job; removed resolving targetIndex on Rollup init; added test for wildcards * lint fixes * moved target_index validation in getRollup resp handler * removed catch block * fixed IT fail * added Exception catch block Signed-off-by: Petar Dzepina <[email protected]> (cherry picked from commit 70cf4ea)
* added support for mustache scripting of rollup.target_index field (#435) * tests * small refactor/improvements * added wildcard check when creating rollup job; removed resolving targetIndex on Rollup init; added test for wildcards * lint fixes * moved target_index validation in getRollup resp handler * removed catch block * fixed IT fail * added Exception catch block Signed-off-by: Petar Dzepina <[email protected]> (cherry picked from commit 70cf4ea)
* added support for mustache scripting of rollup.target_index field (#435) * tests * small refactor/improvements * added wildcard check when creating rollup job; removed resolving targetIndex on Rollup init; added test for wildcards * lint fixes * moved target_index validation in getRollup resp handler * removed catch block * fixed IT fail * added Exception catch block Signed-off-by: Petar Dzepina <[email protected]> (cherry picked from commit 70cf4ea) Co-authored-by: Petar Dzepina <[email protected]>
* added support for mustache scripting of rollup.target_index field (#435) * tests * small refactor/improvements * added wildcard check when creating rollup job; removed resolving targetIndex on Rollup init; added test for wildcards * lint fixes * moved target_index validation in getRollup resp handler * removed catch block * fixed IT fail * added Exception catch block Signed-off-by: Petar Dzepina <[email protected]> (cherry picked from commit 70cf4ea) Co-authored-by: Petar Dzepina <[email protected]>
…ensearch-project#435) (opensearch-project#444) * added support for mustache scripting of rollup.target_index field Signed-off-by: Petar Dzepina <[email protected]> * defekt fixes Signed-off-by: Petar Dzepina <[email protected]> * tests Signed-off-by: Petar Dzepina <[email protected]> * small refactor/improvements Signed-off-by: Petar Dzepina <[email protected]> * added wildcard check when creating rollup job; removed resolving targetIndex on Rollup init; added test for wildcards Signed-off-by: Petar Dzepina <[email protected]> * lint fixes Signed-off-by: Petar Dzepina <[email protected]> * moved target_index validation in getRollup resp handler Signed-off-by: Petar Dzepina <[email protected]> * added using toMap() Signed-off-by: Petar Dzepina <[email protected]> * removed catch block Signed-off-by: Petar Dzepina <[email protected]> * exception fix Signed-off-by: Petar Dzepina <[email protected]> * linter fix Signed-off-by: Petar Dzepina <[email protected]> * fixed IT fail Signed-off-by: Petar Dzepina <[email protected]> * added Exception catch block Signed-off-by: Petar Dzepina <[email protected]> (cherry picked from commit e858ab2) Co-authored-by: Petar Dzepina <[email protected]>
…ch-project#586) * added support for mustache scripting of rollup.target_index field (opensearch-project#435) * tests * small refactor/improvements * added wildcard check when creating rollup job; removed resolving targetIndex on Rollup init; added test for wildcards * lint fixes * moved target_index validation in getRollup resp handler * removed catch block * fixed IT fail * added Exception catch block Signed-off-by: Petar Dzepina <[email protected]> (cherry picked from commit 70cf4ea) Co-authored-by: Petar Dzepina <[email protected]>
…ensearch-project#435) (opensearch-project#444) * added support for mustache scripting of rollup.target_index field Signed-off-by: Petar Dzepina <[email protected]> * defekt fixes Signed-off-by: Petar Dzepina <[email protected]> * tests Signed-off-by: Petar Dzepina <[email protected]> * small refactor/improvements Signed-off-by: Petar Dzepina <[email protected]> * added wildcard check when creating rollup job; removed resolving targetIndex on Rollup init; added test for wildcards Signed-off-by: Petar Dzepina <[email protected]> * lint fixes Signed-off-by: Petar Dzepina <[email protected]> * moved target_index validation in getRollup resp handler Signed-off-by: Petar Dzepina <[email protected]> * added using toMap() Signed-off-by: Petar Dzepina <[email protected]> * removed catch block Signed-off-by: Petar Dzepina <[email protected]> * exception fix Signed-off-by: Petar Dzepina <[email protected]> * linter fix Signed-off-by: Petar Dzepina <[email protected]> * fixed IT fail Signed-off-by: Petar Dzepina <[email protected]> * added Exception catch block Signed-off-by: Petar Dzepina <[email protected]> (cherry picked from commit e858ab2) Co-authored-by: Petar Dzepina <[email protected]> Signed-off-by: Ronnak Saxena <[email protected]>
…ch-project#586) * added support for mustache scripting of rollup.target_index field (opensearch-project#435) * tests * small refactor/improvements * added wildcard check when creating rollup job; removed resolving targetIndex on Rollup init; added test for wildcards * lint fixes * moved target_index validation in getRollup resp handler * removed catch block * fixed IT fail * added Exception catch block Signed-off-by: Petar Dzepina <[email protected]> (cherry picked from commit 70cf4ea) Co-authored-by: Petar Dzepina <[email protected]> Signed-off-by: Ronnak Saxena <[email protected]>
Issue #, if available: 61
Description of changes:
Added capability to use mustache scripting as target_index field value. Only supported param right now is source_index: target_index: rollup_{{ctx.source_index}}
CheckList:
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.