Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Backport 2.x] Bugfix/202 transform date add date conversion #803

Merged
merged 1 commit into from
May 31, 2023

Conversation

opensearch-trigger-bot[bot]
Copy link
Contributor

Backport 42833b1 from #622

* 202: Added format property when specifying the date histogram

Signed-off-by: Stevan Buzejic <[email protected]>

* 202: Added component responsible for building the target index mapping once the transform is being triggered.

Signed-off-by: Stevan Buzejic <[email protected]>

* 202: date_histogram considered in the case of the creating the target index for the date fields when transform is executed

Signed-off-by: Stevan Buzejic <[email protected]>

* 202: Enabled target index date field mappings if those fields are used in aggregations or as a term aggregation for defining the buckets

Signed-off-by: Stevan Buzejic <[email protected]>

* Updated code according to comments. Added targetIndexMapping when transform preview action is triggered

Signed-off-by: Stevan Buzejic <[email protected]>

* Updated schema versions

Signed-off-by: Stevan Buzejic <[email protected]>

* Addressed the comments

Signed-off-by: Stevan Buzejic <[email protected]>

* Refactored transform tests related with aggregation based on a date field.
Updated transform preview action to consider target index mapping when using a date field. Kept formatting of the date field in target index.

Signed-off-by: Stevan Buzejic <[email protected]>

* detekt fix

Signed-off-by: Stevan Buzejic <[email protected]>

* Added zone in IT

Signed-off-by: Stevan Buzejic <[email protected]>

* Added function for creating target index mapping that considers transform mapping json. Target date field mappings are generated after transform validation when running transform. Removed target index date field values formatting. emoved default format for date_histogram because of the rollup. Updated schema version in test.

Signed-off-by: Stevan Buzejic <[email protected]>

---------

Signed-off-by: Stevan Buzejic <[email protected]>
(cherry picked from commit 42833b1)
@codecov
Copy link

codecov bot commented May 31, 2023

Codecov Report

Merging #803 (ad07b41) into 2.x (fa7e46f) will decrease coverage by 0.16%.
The diff coverage is 72.04%.

@@             Coverage Diff              @@
##                2.x     #803      +/-   ##
============================================
- Coverage     76.07%   75.92%   -0.16%     
- Complexity     2862     2901      +39     
============================================
  Files           364      366       +2     
  Lines         16426    16554     +128     
  Branches       2365     2387      +22     
============================================
+ Hits          12496    12568      +72     
- Misses         2559     2608      +49     
- Partials       1371     1378       +7     
Impacted Files Coverage Δ
...h/indexmanagement/transform/util/TransformUtils.kt 0.00% <0.00%> (ø)
...earch/indexmanagement/transform/TransformRunner.kt 82.24% <66.66%> (+0.32%) ⬆️
...indexmanagement/transform/util/TransformContext.kt 70.00% <66.66%> (-5.00%) ⬇️
...xmanagement/transform/TargetIndexMappingService.kt 69.14% <69.14%> (ø)
.../action/preview/TransportPreviewTransformAction.kt 85.71% <72.00%> (-1.25%) ⬇️
...management/common/model/dimension/DateHistogram.kt 77.55% <77.77%> (-0.48%) ⬇️
...arch/indexmanagement/transform/TransformIndexer.kt 69.09% <85.71%> (ø)
...pensearch/indexmanagement/IndexManagementPlugin.kt 90.23% <100.00%> (+0.03%) ⬆️
...ndexmanagement/transform/TransformSearchService.kt 71.42% <100.00%> (+1.17%) ⬆️

... and 13 files with indirect coverage changes

@bowenlan-amzn bowenlan-amzn merged commit c48129e into 2.x May 31, 2023
@github-actions github-actions bot deleted the backport/backport-622-to-2.x branch May 31, 2023 22:39
ronnaksaxena pushed a commit to ronnaksaxena/index-management that referenced this pull request Jul 19, 2023
opensearch-project#803)

* 202: Added format property when specifying the date histogram

Signed-off-by: Stevan Buzejic <[email protected]>

* 202: Added component responsible for building the target index mapping once the transform is being triggered.

Signed-off-by: Stevan Buzejic <[email protected]>

* 202: date_histogram considered in the case of the creating the target index for the date fields when transform is executed

Signed-off-by: Stevan Buzejic <[email protected]>

* 202: Enabled target index date field mappings if those fields are used in aggregations or as a term aggregation for defining the buckets

Signed-off-by: Stevan Buzejic <[email protected]>

* Updated code according to comments. Added targetIndexMapping when transform preview action is triggered

Signed-off-by: Stevan Buzejic <[email protected]>

* Updated schema versions

Signed-off-by: Stevan Buzejic <[email protected]>

* Addressed the comments

Signed-off-by: Stevan Buzejic <[email protected]>

* Refactored transform tests related with aggregation based on a date field.
Updated transform preview action to consider target index mapping when using a date field. Kept formatting of the date field in target index.

Signed-off-by: Stevan Buzejic <[email protected]>

* detekt fix

Signed-off-by: Stevan Buzejic <[email protected]>

* Added zone in IT

Signed-off-by: Stevan Buzejic <[email protected]>

* Added function for creating target index mapping that considers transform mapping json. Target date field mappings are generated after transform validation when running transform. Removed target index date field values formatting. emoved default format for date_histogram because of the rollup. Updated schema version in test.

Signed-off-by: Stevan Buzejic <[email protected]>

---------

Signed-off-by: Stevan Buzejic <[email protected]>
(cherry picked from commit 42833b1)

Co-authored-by: Stevan Buzejic <[email protected]>
Signed-off-by: Ronnak Saxena <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants