Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Dedicated threadpool for system index writes #61655

Merged
merged 11 commits into from
Sep 22, 2020
Merged

Dedicated threadpool for system index writes #61655

merged 11 commits into from
Sep 22, 2020

Conversation

jaymode
Copy link
Member

@jaymode jaymode commented Aug 27, 2020

This commit adds a dedicated threadpool for system index write
operations. The dedicated resources for system index writes serves as
a means to ensure that user activity does not block important system
operations from occurring such as the management of users and roles.

@jaymode
Dedicated threadpool for system index writes
ad63f5a 
This commit adds a dedicated threadpool for system index write
operations. The dedicated resources for system index writes serves as
a means to ensure that user activity does not block important system
operations from occurring such as the management of users and roles.
@jaymode jaymode added >enhancement :Distributed Indexing/CRUD A catch all label for issues around indexing, updating and getting a doc by id. Not search. v8.0.0 v7.10.0 labels Aug 27, 2020
@jaymode jaymode requested a review from gwbrown August 27, 2020 20:08
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (:Distributed/CRUD)

@elasticmachine elasticmachine added the Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. label Aug 27, 2020
@jaymode jaymode mentioned this pull request Aug 27, 2020
Open
23 tasks
@Tim-Brooks
Copy link
Contributor

You might want to force execution on IndexingPressure otherwise too much user memory usage will block these system operations.

gwbrown
gwbrown approved these changes Sep 1, 2020
View reviewed changes
Copy link
Contributor

@gwbrown gwbrown left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few minor comments, LGTM otherwise (no need for another round of review once the comments are addressed). You may want a second opinion from someone more familiar with this code though - write threadpools are not my area of expertise.

server/src/main/java/org/elasticsearch/action/bulk/TransportBulkAction.java Show resolved Hide resolved
server/src/main/java/org/elasticsearch/action/support/replication/TransportWriteAction.java Outdated Show resolved Hide resolved
server/src/test/java/org/elasticsearch/action/bulk/TransportBulkActionIngestTests.java Show resolved Hide resolved
@jaymode jaymode requested a review from Tim-Brooks September 18, 2020 16:59
Tim-Brooks
Tim-Brooks approved these changes Sep 21, 2020
View reviewed changes
Copy link
Contributor

@Tim-Brooks Tim-Brooks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - I only looked at the indexing pressure stuff in bulk action and shard bulk action. If you wanted a full review from me, let me know.

@jaymode jaymode merged commit 242083a into elastic:master Sep 22, 2020
@jaymode jaymode deleted the system_index_write_threadpool branch September 22, 2020 18:14
@jaymode jaymode mentioned this pull request Sep 22, 2020
Merged
jaymode added a commit that referenced this pull request Sep 22, 2020
@jaymode
Dedicated threadpool for system index writes (#62792)
cb1dc52 
This commit adds a dedicated threadpool for system index write
operations. The dedicated resources for system index writes serves as
a means to ensure that user activity does not block important system
operations from occurring such as the management of users and roles.

Backport of #61655
@rjernst rjernst mentioned this pull request Oct 6, 2020
Closed
@Mpdreamz Mpdreamz mentioned this pull request Nov 16, 2020
Closed
61 tasks
@stevejgordon stevejgordon mentioned this pull request Dec 17, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gwbrown gwbrown gwbrown approved these changes

@Tim-Brooks Tim-Brooks Tim-Brooks approved these changes

Assignees
No one assigned
Labels
:Distributed Indexing/CRUD A catch all label for issues around indexing, updating and getting a doc by id. Not search. >enhancement Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. v7.10.0 v8.0.0-alpha1
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@jaymode @elasticmachine @Tim-Brooks @gwbrown @jakelandis