Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Session management] Update session cleanup task to start using partial search results #200008

Closed
SiddharthMantri opened this issue Nov 13, 2024 · 1 comment · Fixed by #200912
Closed

[Session management] Update session cleanup task to start using partial search results #200008

SiddharthMantri opened this issue Nov 13, 2024 · 1 comment · Fixed by #200912
Assignees
@SiddharthMantri
Labels
Feature:Security/Session Management Platform Security - Session Management Team:Security Team focused on: Auth, Users, Roles, Spaces, Audit Logging, and more!

Comments

@SiddharthMantri
Copy link
Contributor

SiddharthMantri commented Nov 13, 2024 •
edited
Loading

We are seeing some error logs around session_cleanup, primarily in serverless.

The task uses a Point-in-time query to delete the sessions that require clean up. On investigating, it is possible that the PIT query is trying to run on shards that are unavailable or were first missing when the PIT was created. Relevant function:

https://github.com/elastic/kibana/blob/main/x-pack/plugins/security/server/session_management/session_index.ts#L768

Elasticsearch recently introduced a new option that allows PIT queries to continue execution even if certain shards are missing at the query time. By enabling this flag, we should be able to mitigate the search_phase_execution_exception errors and allow the session_cleanup task to proceed without being interrupted by shard availability issues.

This fix is essential for ensuring more reliable session management and minimizing error log noise, especially in serverless environments.
@SiddharthMantri SiddharthMantri added Feature:Security/Session Management Platform Security - Session Management Team:Security Team focused on: Auth, Users, Roles, Spaces, Audit Logging, and more! labels Nov 13, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-security (Team:Security)

@legrego legrego assigned SiddharthMantri and unassigned legrego Nov 20, 2024
@SiddharthMantri SiddharthMantri mentioned this issue Nov 20, 2024
Merged
2 tasks
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Nov 21, 2024
@SiddharthMantri
[Session management] update cleanup to allow partial search results f…
3ec9035 
…or PIT query (elastic#200912)

Closes elastic#200008

## Summary
This PR updates the session cleanup task to allow partial results in the
point in time query used to delete any invalid sessions.

### Notes
Using a [recently introduced option by
ES](elastic/elasticsearch#111516), the session
cleanup now allows the PIT to run over partial results. This should
reduce the noise we see around session_cleanup errors in our logs.

### Release notes
Fixes error with opening point in time query for session deletion by now
accounting for partial results.

### Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_node:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

(cherry picked from commit 8f12714)
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Nov 21, 2024
@SiddharthMantri
[Session management] update cleanup to allow partial search results f…
a207cef 
…or PIT query (elastic#200912)

Closes elastic#200008

## Summary
This PR updates the session cleanup task to allow partial results in the
point in time query used to delete any invalid sessions.

### Notes
Using a [recently introduced option by
ES](elastic/elasticsearch#111516), the session
cleanup now allows the PIT to run over partial results. This should
reduce the noise we see around session_cleanup errors in our logs.

### Release notes
Fixes error with opening point in time query for session deletion by now
accounting for partial results.

### Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_node:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

(cherry picked from commit 8f12714)
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Nov 21, 2024
@SiddharthMantri
[Session management] update cleanup to allow partial search results f…
011c7df 
…or PIT query (elastic#200912)

Closes elastic#200008

## Summary
This PR updates the session cleanup task to allow partial results in the
point in time query used to delete any invalid sessions.

### Notes
Using a [recently introduced option by
ES](elastic/elasticsearch#111516), the session
cleanup now allows the PIT to run over partial results. This should
reduce the noise we see around session_cleanup errors in our logs.

### Release notes
Fixes error with opening point in time query for session deletion by now
accounting for partial results.

### Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_node:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

(cherry picked from commit 8f12714)
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Nov 21, 2024
@SiddharthMantri
[Session management] update cleanup to allow partial search results f…
062bbdd 
…or PIT query (elastic#200912)

Closes elastic#200008

## Summary
This PR updates the session cleanup task to allow partial results in the
point in time query used to delete any invalid sessions.

### Notes
Using a [recently introduced option by
ES](elastic/elasticsearch#111516), the session
cleanup now allows the PIT to run over partial results. This should
reduce the noise we see around session_cleanup errors in our logs.

### Release notes
Fixes error with opening point in time query for session deletion by now
accounting for partial results.

### Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_node:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

(cherry picked from commit 8f12714)
paulinashakirova pushed a commit to paulinashakirova/kibana that referenced this issue Nov 26, 2024
@SiddharthMantri @paulinashakirova
[Session management] update cleanup to allow partial search results f…
64c0d29 
…or PIT query (elastic#200912)

Closes elastic#200008

## Summary
This PR updates the session cleanup task to allow partial results in the
point in time query used to delete any invalid sessions.


### Notes
Using a [recently introduced option by
ES](elastic/elasticsearch#111516), the session
cleanup now allows the PIT to run over partial results. This should
reduce the noise we see around session_cleanup errors in our logs.


### Release notes
Fixes error with opening point in time query for session deletion by now
accounting for partial results.

### Checklist

Check the PR satisfies following conditions. 

Reviewers should verify this PR satisfies this list as well.


- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_node:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this issue Dec 12, 2024
@SiddharthMantri @CAWilson94
[Session management] update cleanup to allow partial search results f…
2c831c1 
…or PIT query (elastic#200912)

Closes elastic#200008

## Summary
This PR updates the session cleanup task to allow partial results in the
point in time query used to delete any invalid sessions.


### Notes
Using a [recently introduced option by
ES](elastic/elasticsearch#111516), the session
cleanup now allows the PIT to run over partial results. This should
reduce the noise we see around session_cleanup errors in our logs.


### Release notes
Fixes error with opening point in time query for session deletion by now
accounting for partial results.

### Checklist

Check the PR satisfies following conditions. 

Reviewers should verify this PR satisfies this list as well.


- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_node:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@SiddharthMantri SiddharthMantri

Labels
Feature:Security/Session Management Platform Security - Session Management Team:Security Team focused on: Auth, Users, Roles, Spaces, Audit Logging, and more!
Projects
None yet
Milestone
No milestone
3 participants
@SiddharthMantri @legrego @elasticmachine