Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable remote segment upload backpressure by default #10356

Merged
merged 1 commit into from
Oct 4, 2023

Conversation

ashking94
Copy link
Member

@ashking94 ashking94 commented Oct 4, 2023

Description

This PR makes the change to set the value of remote_store.segment.pressure.enabled as true by default. There are tests around remote store backpressure already.

Related Issues

Resolves #10355

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)
  • GitHub issue/PR created in OpenSearch documentation repo

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions github-actions bot added enhancement Enhancement or improvement to existing feature or request Storage:Durability Issues and PRs related to the durability framework v2.11.0 Issues and PRs related to version 2.11.0 labels Oct 4, 2023
@ashking94 ashking94 marked this pull request as draft October 4, 2023 11:07
@github-actions
Copy link
Contributor

github-actions bot commented Oct 4, 2023

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

github-actions bot commented Oct 4, 2023

Compatibility status:

Checks if related components are compatible with change fe2fa17

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git]

@github-actions
Copy link
Contributor

github-actions bot commented Oct 4, 2023

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

github-actions bot commented Oct 4, 2023

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

github-actions bot commented Oct 4, 2023

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

github-actions bot commented Oct 4, 2023

Gradle Check (Jenkins) Run Completed with:

@codecov
Copy link

codecov bot commented Oct 4, 2023

Codecov Report

Merging #10356 (fe2fa17) into main (d5a95b8) will decrease coverage by 0.01%.
Report is 3 commits behind head on main.
The diff coverage is 96.15%.

@@             Coverage Diff              @@
##               main   #10356      +/-   ##
============================================
- Coverage     71.20%   71.20%   -0.01%     
- Complexity    58298    58299       +1     
============================================
  Files          4832     4832              
  Lines        274711   274756      +45     
  Branches      40031    40037       +6     
============================================
+ Hits         195600   195631      +31     
- Misses        62717    62764      +47     
+ Partials      16394    16361      -33     
Files Coverage Δ
...c/main/java/org/opensearch/index/IndexService.java 75.43% <100.00%> (-0.39%) ⬇️
...arch/index/remote/RemoteStorePressureSettings.java 80.64% <ø> (-6.46%) ⬇️
.../org/opensearch/index/remote/RemoteStoreUtils.java 96.66% <100.00%> (+2.22%) ⬆️
...in/java/org/opensearch/index/shard/IndexShard.java 68.84% <100.00%> (-0.71%) ⬇️
...search/index/shard/RemoteStoreRefreshListener.java 90.90% <100.00%> (+4.88%) ⬆️
...earch/index/store/RemoteSegmentStoreDirectory.java 90.97% <100.00%> (+1.29%) ⬆️
...rg/opensearch/index/translog/RemoteFsTranslog.java 71.63% <100.00%> (-1.80%) ⬇️
.../org/opensearch/index/translog/TranslogConfig.java 100.00% <100.00%> (ø)
...nsearch/index/translog/TruncateTranslogAction.java 78.89% <ø> (-5.51%) ⬇️
...g/transfer/TranslogCheckpointTransferSnapshot.java 79.01% <100.00%> (-6.89%) ⬇️
... and 2 more

... and 456 files with indirect coverage changes

@gbbafna gbbafna merged commit e5024a8 into opensearch-project:main Oct 4, 2023
13 checks passed
@gbbafna gbbafna added the backport 2.x Backport to 2.x branch label Oct 4, 2023
opensearch-trigger-bot bot pushed a commit that referenced this pull request Oct 4, 2023
Signed-off-by: Ashish Singh <[email protected]>
(cherry picked from commit e5024a8)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
andrross pushed a commit that referenced this pull request Oct 4, 2023
(cherry picked from commit e5024a8)

Signed-off-by: Ashish Singh <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@ashking94
Copy link
Member Author

@andrross @gbbafna Should we backport this to 2.11 as well. This is not really a fix though.

deshsidd pushed a commit to deshsidd/OpenSearch that referenced this pull request Oct 9, 2023
vikasvb90 pushed a commit to vikasvb90/OpenSearch that referenced this pull request Oct 10, 2023
@dblock
Copy link
Member

dblock commented Oct 10, 2023

@andrross @gbbafna Should we backport this to 2.11 as well. This is not really a fix though.

Nope, that one has sailed AFAIK.

@ashking94 ashking94 deleted the 10355 branch October 11, 2023 18:24
shiv0408 pushed a commit to Gaurav614/OpenSearch that referenced this pull request Apr 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch enhancement Enhancement or improvement to existing feature or request Storage:Durability Issues and PRs related to the durability framework v2.11.0 Issues and PRs related to version 2.11.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enable remote segments upload backpressure by default
3 participants