Allow updation/deletion of custom log type if custom rule index is missing #767

goyamegh · 2023-12-04T21:02:48Z

Description

In the case when the custom rule index is missing, we want to allow the update/deletion of custom log types to be successful instead of throwing an error. The detector index may not be present, and the custom log types will still get updated/deleted respectively.

Issues Resolved

#700

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Megha Goyal <[email protected]>

codecov · 2023-12-04T21:17:23Z

Codecov Report

Attention: 101 lines in your changes are missing coverage. Please review.

Comparison is base (c0f7bd9) 24.73% compared to head (1e31c99) 24.74%.
Report is 3 commits behind head on main.

Files	Patch %	Lines
...s/transport/TransportIndexCustomLogTypeAction.java	0.00%	54 Missing ⚠️
.../transport/TransportDeleteCustomLogTypeAction.java	0.00%	34 Missing ⚠️
...opensearch/securityanalytics/util/RuleIndices.java	0.00%	12 Missing ⚠️
...analytics/transport/TransportDeleteRuleAction.java	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main     #767      +/-   ##
============================================
+ Coverage     24.73%   24.74%   +0.01%     
- Complexity     1021     1023       +2     
============================================
  Files           275      275              
  Lines         12662    12689      +27     
  Branches       1389     1400      +11     
============================================
+ Hits           3132     3140       +8     
- Misses         9264     9284      +20     
+ Partials        266      265       -1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

eirsep · 2023-12-04T22:30:54Z

...main/java/org/opensearch/securityanalytics/transport/TransportDeleteCustomLogTypeAction.java

                            }

                            @Override
                            public void onFailure(Exception e) {
-                                onFailures(e);
+                                if (e instanceof IndexNotFoundException) {


can we do. an index exists check instead of letting this fail and handling exception.

Thanks for the suggestion. Added the check but still keeping the exception check to account for edge cases.

eirsep · 2023-12-04T22:31:37Z

...main/java/org/opensearch/securityanalytics/transport/TransportDeleteCustomLogTypeAction.java

+                }
+
+                @Override
+                public void onFailure(Exception e) {


log exception with custom message

The function is calling the onFailures() call which is logging this exception. So, skipping it here.

eirsep · 2023-12-04T22:32:01Z

...main/java/org/opensearch/securityanalytics/transport/TransportDeleteCustomLogTypeAction.java

@@ -292,7 +284,7 @@ private void onOperation(DeleteResponse response) {
        }

        private void onFailures(Exception t) {
-            log.error(String.format(Locale.ROOT, "Failed to delete detector"));
+            log.error(String.format(Locale.ROOT, "Failed to delete log type"));


can we also log the name of the log type

The exception being passed would ideally have the log type. Added the stack trace to be logged here as well.

…ssing Signed-off-by: Megha Goyal <[email protected]>

eirsep · 2023-12-07T19:47:54Z

Looks like delete log type is wrongly wired.

If detector index does NOT exist, we seem to directly delete the log type instead of checking if there is a rule index.

security-analytics/src/main/java/org/opensearch/securityanalytics/transport/TransportDeleteCustomLogTypeAction.java

Line 176 in 8be2159

if (detectorIndices.detectorIndexExists()) {

security-analytics/src/main/java/org/opensearch/securityanalytics/transport/TransportDeleteCustomLogTypeAction.java

Lines 227 to 229 in 8be2159

    
           } else { 
        
               deleteLogType(logType); 
        
           }

Plz add check for rule index here also. extract into a method if redundant.

eirsep · 2023-12-07T19:52:38Z

src/test/java/org/opensearch/securityanalytics/resthandler/CustomLogTypeRestApiIT.java

@@ -547,6 +605,61 @@ public void testDeleteCustomLogTypeFailsAsDetectorExist() throws IOException {
        });
    }

+    @SuppressWarnings("unchecked")
+    public void testDeleteCustomLogTypeWithRuleIndexMissing() throws IOException {


can you add two scenarios:
deleting log type when -

custom rules index exists, detector index exists.

custom rules index exists, detector index does not exist.

Signed-off-by: Megha Goyal <[email protected]>

opensearch-trigger-bot · 2023-12-08T22:37:28Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.x 2.x
# Navigate to the new working tree
cd .worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-767-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 c373343e1981af0bf2795aa2d0ac45f301d8ac3e
# Push it to GitHub
git push --set-upstream origin backport/backport-767-to-2.x
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-767-to-2.x.

opensearch-trigger-bot · 2023-12-08T22:37:33Z

The backport to 2.11 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.11 2.11
# Navigate to the new working tree
cd .worktrees/backport-2.11
# Create a new branch
git switch --create backport/backport-767-to-2.11
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 c373343e1981af0bf2795aa2d0ac45f301d8ac3e
# Push it to GitHub
git push --set-upstream origin backport/backport-767-to-2.11
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.11

Then, create a pull request where the base branch is 2.11 and the compare/head branch is backport/backport-767-to-2.11.

opensearch-trigger-bot · 2023-12-08T22:37:38Z

The backport to 2.11-with-features failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.11-with-features 2.11-with-features
# Navigate to the new working tree
cd .worktrees/backport-2.11-with-features
# Create a new branch
git switch --create backport/backport-767-to-2.11-with-features
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 c373343e1981af0bf2795aa2d0ac45f301d8ac3e
# Push it to GitHub
git push --set-upstream origin backport/backport-767-to-2.11-with-features
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.11-with-features

Then, create a pull request where the base branch is 2.11-with-features and the compare/head branch is backport/backport-767-to-2.11-with-features.

…ssing (opensearch-project#767) Signed-off-by: Megha Goyal <[email protected]>

…ensearch-project#767) * Allow deletion of custom log type if custom rule index is missing Signed-off-by: Megha Goyal <[email protected]> * Allow custom log type name to be updated when custom rule index is missing Signed-off-by: Megha Goyal <[email protected]> * Adding changes for detector index missing Signed-off-by: Megha Goyal <[email protected]> * Update log type when detector index is missing Signed-off-by: Megha Goyal <[email protected]> --------- Signed-off-by: Megha Goyal <[email protected]>

…) (#780) * Allow deletion of custom log type if custom rule index is missing * Allow custom log type name to be updated when custom rule index is missing * Adding changes for detector index missing * Update log type when detector index is missing --------- Signed-off-by: Megha Goyal <[email protected]>

…ssing (#767) (#779) Signed-off-by: Megha Goyal <[email protected]>

opensearch-trigger-bot · 2023-12-19T21:23:29Z

The backport to 2.9 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.9 2.9
# Navigate to the new working tree
cd .worktrees/backport-2.9
# Create a new branch
git switch --create backport/backport-767-to-2.9
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 c373343e1981af0bf2795aa2d0ac45f301d8ac3e
# Push it to GitHub
git push --set-upstream origin backport/backport-767-to-2.9
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.9

Then, create a pull request where the base branch is 2.9 and the compare/head branch is backport/backport-767-to-2.9.

opensearch-trigger-bot · 2023-12-19T21:23:31Z

The backport to 2.9-with-features failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.9-with-features 2.9-with-features
# Navigate to the new working tree
cd .worktrees/backport-2.9-with-features
# Create a new branch
git switch --create backport/backport-767-to-2.9-with-features
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 c373343e1981af0bf2795aa2d0ac45f301d8ac3e
# Push it to GitHub
git push --set-upstream origin backport/backport-767-to-2.9-with-features
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.9-with-features

Then, create a pull request where the base branch is 2.9-with-features and the compare/head branch is backport/backport-767-to-2.9-with-features.

…ensearch-project#767) * Allow deletion of custom log type if custom rule index is missing Signed-off-by: Megha Goyal <[email protected]> * Allow custom log type name to be updated when custom rule index is missing Signed-off-by: Megha Goyal <[email protected]> * Adding changes for detector index missing Signed-off-by: Megha Goyal <[email protected]> * Update log type when detector index is missing Signed-off-by: Megha Goyal <[email protected]> --------- Signed-off-by: Megha Goyal <[email protected]>

…) (#814) * Allow deletion of custom log type if custom rule index is missing * Allow custom log type name to be updated when custom rule index is missing * Adding changes for detector index missing * Update log type when detector index is missing --------- Signed-off-by: Megha Goyal <[email protected]>

opensearch-trigger-bot · 2024-03-11T22:12:48Z

The backport to 2.7 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.7 2.7
# Navigate to the new working tree
cd .worktrees/backport-2.7
# Create a new branch
git switch --create backport/backport-767-to-2.7
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 c373343e1981af0bf2795aa2d0ac45f301d8ac3e
# Push it to GitHub
git push --set-upstream origin backport/backport-767-to-2.7
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.7

Then, create a pull request where the base branch is 2.7 and the compare/head branch is backport/backport-767-to-2.7.

opensearch-trigger-bot · 2024-03-11T22:25:58Z

The backport to 2.5 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.5 2.5
# Navigate to the new working tree
cd .worktrees/backport-2.5
# Create a new branch
git switch --create backport/backport-767-to-2.5
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 c373343e1981af0bf2795aa2d0ac45f301d8ac3e
# Push it to GitHub
git push --set-upstream origin backport/backport-767-to-2.5
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.5

Then, create a pull request where the base branch is 2.5 and the compare/head branch is backport/backport-767-to-2.5.

* Allow deletion of custom log type if custom rule index is missing Signed-off-by: Megha Goyal <[email protected]> * Allow custom log type name to be updated when custom rule index is missing Signed-off-by: Megha Goyal <[email protected]> * Adding changes for detector index missing Signed-off-by: Megha Goyal <[email protected]> * Update log type when detector index is missing Signed-off-by: Megha Goyal <[email protected]> --------- Signed-off-by: Megha Goyal <[email protected]>

Allow deletion of custom log type if custom rule index is missing

d648ed3

Signed-off-by: Megha Goyal <[email protected]>

goyamegh marked this pull request as ready for review December 4, 2023 22:22

goyamegh requested review from amsiglan, AWSHurneyt, getsaurabh02, lezzago, praveensameneni, sbcd90, eirsep and jowg-amazon as code owners December 4, 2023 22:22

eirsep requested changes Dec 4, 2023

View reviewed changes

Allow custom log type name to be updated when custom rule index is mi…

8be2159

…ssing Signed-off-by: Megha Goyal <[email protected]>

goyamegh requested a review from eirsep December 6, 2023 22:06

eirsep reviewed Dec 7, 2023

View reviewed changes

goyamegh added 2 commits December 8, 2023 11:11

Adding changes for detector index missing

cc5c570

Signed-off-by: Megha Goyal <[email protected]>

Update log type when detector index is missing

1e31c99

Signed-off-by: Megha Goyal <[email protected]>

eirsep approved these changes Dec 8, 2023

View reviewed changes

jowg-amazon approved these changes Dec 8, 2023

View reviewed changes

jowg-amazon merged commit c373343 into opensearch-project:main Dec 8, 2023
12 of 16 checks passed

jowg-amazon added backport 2.x backport 2.11 backport 2.11-with-features labels Dec 8, 2023

goyamegh added a commit to goyamegh/security-analytics that referenced this pull request Dec 8, 2023

Allow updation/deletion of custom log type if custom rule index is mi…

af55c64

…ssing (opensearch-project#767) Signed-off-by: Megha Goyal <[email protected]>

goyamegh changed the title ~~Allow deletion of custom log type if custom rule index is missing~~ Allow updation/deletion of custom log type if custom rule index is missing Dec 8, 2023

goyamegh mentioned this pull request Dec 8, 2023

Allow updation/deletion of custom log type if custom rule index is mi… #779

Merged

5 tasks

goyamegh mentioned this pull request Dec 8, 2023

Allow deletion of custom log type if custom rule index is missing (#767) #780

Merged

5 tasks

goyamegh mentioned this pull request Dec 8, 2023

Allow updation/deletion of custom log type if custom rule index is missing (#767) #781

Closed

5 tasks

jowg-amazon pushed a commit that referenced this pull request Dec 8, 2023

Allow updation/deletion of custom log type if custom rule index is mi…

69e5290

…ssing (#767) (#779) Signed-off-by: Megha Goyal <[email protected]>

jowg-amazon added backport 2.9 backports to 2.9 backport 2.9-with-features labels Dec 19, 2023

goyamegh mentioned this pull request Jan 18, 2024

Allow deletion of custom log type if custom rule index is missing (#767) #814

Merged

5 tasks

goyamegh added the backport 2.7 label Mar 11, 2024

goyamegh added the backport 2.5 label Mar 11, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow updation/deletion of custom log type if custom rule index is missing #767

Allow updation/deletion of custom log type if custom rule index is missing #767

goyamegh commented Dec 4, 2023 •

edited

Loading

codecov bot commented Dec 4, 2023 •

edited

Loading

eirsep Dec 4, 2023

goyamegh Dec 6, 2023

eirsep Dec 4, 2023

goyamegh Dec 6, 2023

eirsep Dec 4, 2023

goyamegh Dec 6, 2023

eirsep commented Dec 7, 2023

eirsep Dec 7, 2023 •

edited

Loading

goyamegh Dec 8, 2023

opensearch-trigger-bot bot commented Dec 8, 2023

opensearch-trigger-bot bot commented Dec 8, 2023

opensearch-trigger-bot bot commented Dec 8, 2023

opensearch-trigger-bot bot commented Dec 19, 2023

opensearch-trigger-bot bot commented Dec 19, 2023

opensearch-trigger-bot bot commented Mar 11, 2024

opensearch-trigger-bot bot commented Mar 11, 2024

Allow updation/deletion of custom log type if custom rule index is missing #767

Allow updation/deletion of custom log type if custom rule index is missing #767

Conversation

goyamegh commented Dec 4, 2023 • edited Loading

Description

Issues Resolved

Check List

codecov bot commented Dec 4, 2023 • edited Loading

Codecov Report

eirsep Dec 4, 2023

Choose a reason for hiding this comment

goyamegh Dec 6, 2023

Choose a reason for hiding this comment

eirsep Dec 4, 2023

Choose a reason for hiding this comment

goyamegh Dec 6, 2023

Choose a reason for hiding this comment

eirsep Dec 4, 2023

Choose a reason for hiding this comment

goyamegh Dec 6, 2023

Choose a reason for hiding this comment

eirsep commented Dec 7, 2023

eirsep Dec 7, 2023 • edited Loading

Choose a reason for hiding this comment

goyamegh Dec 8, 2023

Choose a reason for hiding this comment

opensearch-trigger-bot bot commented Dec 8, 2023

opensearch-trigger-bot bot commented Dec 8, 2023

opensearch-trigger-bot bot commented Dec 8, 2023

opensearch-trigger-bot bot commented Dec 19, 2023

opensearch-trigger-bot bot commented Dec 19, 2023

opensearch-trigger-bot bot commented Mar 11, 2024

opensearch-trigger-bot bot commented Mar 11, 2024

goyamegh commented Dec 4, 2023 •

edited

Loading

codecov bot commented Dec 4, 2023 •

edited

Loading

eirsep Dec 7, 2023 •

edited

Loading