CSOT #1215

katcharov · 2023-10-04T22:25:38Z

Description

This PR introduces a unified Client Side Operation Timeout (CSOT) as an Alpha feature to
simplify the multitude of existing timeout settings. This consolidation merges various complex and interacting timeout settings into a single, overarching operation timeout option, enhancing usability and predictability. Existing timeout options will continue to be honored if the operation timeout is not set.

Key Changes:

Implement CSOT across various layers of the driver, applying the new operation
timeout configuration to govern execution time in areas such as Authentication,
Connection Pool and Monitoring, Server Discovery and Monitoring, CRUD, GridFS,
CSFLE, Transactions, Handshake, and Server Selection.
Annotate the CSOT API as Alpha, indicating that this public API element is in
the early stages of development and may undergo incompatible changes.
Centralize the setting of maxTimeMS just before sending commands to the server.
This optimization enhances performance and ensures more precise enforcement of timeouts.
Introduce TimeoutContext to centralize timeout-related logic, enhancing the
clarity and ease of maintenance for timeout management within the driver.
Enable a new retry mode when CSOT is active, allowing operations to retry until the timeout is reached. This improves resiliency compared to the previous behavior of retrying only once.
Enhance MongoClient, MongoDatabase, MongoCollection, and ClientSession to support
setting and inheriting the operation timeout.
Introduce MongoCluster, a new conceptual entity representing server deployments,
configurable with specific writeConcern, timeouts, and read preferences.
Transform timeout errors from external sources into the new MongoOperationTimeoutException,
standardizing error handling when CSOT is enabled.

Note: serverSelectionTimeoutMS and connectTimeoutMS are still honored even if CSOT is enabled.

Reviewers:

@rozza: Since you have made many of the changes in this branch, I have added you as a reviewer. I have addressed and answered some comments on this PR but would appreciate if you could double-check and comment if you have any objections. A full review of the PR is not expected.

@stIncMale: Could you please check if your previous comments have been addressed and resolve them as needed? A full review of the PR is not expected.

For reviewers:

Test Verification: Ensure all tests are passing and all important build variants are tested.
Verify that the ClientSideOperation* test suites does not have any ignored tests without a valid reason.

Context:

This PR focuses on the alpha changes of CSOT and includes changes from the EPIC: JAVA-3828.

Description is updated by @vbabanin.

Initial client side operation timeout (CSOT) work. The CSOT class is passed to all operations and currently encapsulate the following timeouts: - `timeoutMS` the new optional client side operation timeout - `maxTimeMS` the legacy maxTimeMS operation value. Ignored if CSOT is set. - `maxCommitTimeMS` the legacy commit timeout. Ignored if CSOT is set. - `maxAwaitTimeMS` the getMore await timeout. This initial work allows for the CSOT to be available to all operations and later work will pass / apply this timeout where required by the Spec. JAVA-4086

JAVA-4064

# Conflicts: # driver-core/src/main/com/mongodb/ConnectionString.java # driver-core/src/test/unit/com/mongodb/MongoClientSettingsSpecification.groovy

…terface (#1194) JAVA-5112

An immutable class that contains all user configured timeouts. This will eventaually allow access to user configuration down the stack into Bindings / Cluster.selectServer and Connection.command. JAVA-5169

JAVA-5175

In preparation for adding all contexts to OperationContext JAVA-5170

This will allow the timeout settings to be available to the Binding and this will allow the binding to create the TimeoutContext for the operation. JAVA-5170

Now contains RequestContext, SessionContext and TimeoutContext JAVA-5170

JAVA-5170

Operations now supply TimeoutSettings JAVA-5170

Pass OperationContext as a whole instead of RequestContext, SessionContext, TimeoutContext and ServerAPI JAVA-5170

Adds support for Explainable operations JAVA-5172

JAVA-5170

# Conflicts: # driver-core/src/main/com/mongodb/internal/connection/BaseCluster.java # driver-core/src/main/com/mongodb/internal/connection/LoadBalancedCluster.java # driver-core/src/test/functional/com/mongodb/ClusterFixture.java # driver-core/src/test/unit/com/mongodb/internal/connection/BaseClusterSpecification.groovy # driver-reactive-streams/src/main/com/mongodb/reactivestreams/client/internal/ClientSessionBinding.java # driver-reactive-streams/src/test/unit/com/mongodb/reactivestreams/client/internal/ClientSessionBindingSpecification.groovy # driver-sync/src/test/unit/com/mongodb/client/internal/ClientSessionBindingSpecification.groovy

…#1209) JAVA-5184

katcharov · 2023-10-20T14:10:19Z

driver-core/src/main/com/mongodb/internal/time/Timeout.java

-        if (this == o) {
-            return true;
+    static Timeout expiresIn(final long duration, final TimeUnit unit) {
+        // TODO (CSOT) confirm that all usages in final PR are non-negative


Note to address all TODOs prior to merging. (All files in github can be loaded using document.querySelectorAll('.load-diff-button').forEach(node => node.click()) in console, and then ctrl-f for TODO)

I have addressed critical //TODO comments.

JAVA-5212

JAVA-5379 * Remove usages of hasExpired * Replace RetryState's timeout with retryUntilTimeoutThrowsException

JAVA-5379

#1441) JAVA-5469

# Conflicts: # driver-reactive-streams/src/test/functional/com/mongodb/reactivestreams/client/unified/UnifiedServerDiscoveryAndMonitoringTest.java # driver-sync/src/test/functional/com/mongodb/client/unified/UnifiedTest.java

JAVA-5526

stIncMale

The only concern I left here was addressed.

rozza · 2024-07-18T09:15:10Z

Kicked off a full build just to check.

https://spruce.mongodb.com/version/6698dcbb688ab00007faac91/

rozza

Lets replace all the @since CSOT doc annotations before merging into master.

JAVA-5469

vbabanin · 2024-07-19T05:58:57Z

The MongoCluster concept has been introduced in CSOT and is not marked with the @Alpha annotation, except for timeout-related methods like getTimeout and withTimeout. Although it originated in CSOT, it is applicable beyond this context, so the class MongoCluster has not been marked with @Alpha. If there are any objections, please share your feedback.

driver-core/src/main/com/mongodb/internal/operation/SyncOperations.java

...tive-streams/src/main/com/mongodb/reactivestreams/client/internal/OperationExecutorImpl.java

…ions.java Co-authored-by: Ross Lawley <[email protected]>

rozza

LGTM

vbabanin

Approved. LGTM!

rozza and others added 28 commits August 22, 2023 14:43

CSOT: Added timeout to settings & connection string (#1173)

83eeb10

JAVA-4064

Merge remote-tracking branch 'upstream/master' into CSOT

087973e

# Conflicts: # driver-core/src/main/com/mongodb/ConnectionString.java # driver-core/src/test/unit/com/mongodb/MongoClientSettingsSpecification.groovy

Replace Timeout implementation with interface, introduce StartTime in…

8f710f3

…terface (#1194) JAVA-5112

Added TimeoutSettings

432bcb0

An immutable class that contains all user configured timeouts. This will eventaually allow access to user configuration down the stack into Bindings / Cluster.selectServer and Connection.command. JAVA-5169

Refactor to use Timeout for server selection (#1202)

158b69c

JAVA-5175

Checkstyle fix

4b92ec6

Renamed ClientSideOperationTimeout to TimeoutContext

7334f5e

In preparation for adding all contexts to OperationContext JAVA-5170

Added getTimeoutSettings to Operation interfaces

10beb22

This will allow the timeout settings to be available to the Binding and this will allow the binding to create the TimeoutContext for the operation. JAVA-5170

OperationContext promotion

aee66e4

Now contains RequestContext, SessionContext and TimeoutContext JAVA-5170

Cluster Binding now creates the OperationContext

bcdc0eb

JAVA-5170

BindingContext now only contains OperationContext

46f5eba

Operations now supply TimeoutSettings JAVA-5170

Propogate Operation Context

87b7843

Pass OperationContext as a whole instead of RequestContext, SessionContext, TimeoutContext and ServerAPI JAVA-5170

CommandProtocol is now immutable

511a3c3

Add CommandCreator support to CommandReadOperation

0ecc3f5

Adds support for Explainable operations JAVA-5172

Update Tests regarding OperationContext

767bb1c

JAVA-5170

Merge branch 'master' into CSOT-merge

1e769b0

Merge branch 'master' into CSOT

c4ccc24

Merge branch 'master' into CSOT

bdc2abb

Merge branch 'master' into CSOT

c442689

Merge branch 'master' into CSOT

2a0707d

Fix codenarc

35d82be

Spotbugs fixes / exclusions

a9a3b2d

Refactor to obtain timeout from OperationContext for server selection (…

4b002c0

…#1209) JAVA-5184

Fix minor test failure, checkstyle

d65d27f

Merge branch 'master' into CSOT

e8bdab5

Update BindingContext Javadoc

a881808

katcharov commented Oct 20, 2023

View reviewed changes

Obtain waitQueue and connection timeouts from OperationContext (#1228)

ea6c37c

JAVA-5212

katcharov and others added 3 commits June 14, 2024 16:51

Remove hasExpired and expiry checks in retries (#1418)

28c6df9

JAVA-5379 * Remove usages of hasExpired * Replace RetryState's timeout with retryUntilTimeoutThrowsException

Enable ignored tests. (#1428)

b7eed4c

JAVA-5379

Double timeouts in serverless mode to reduce test execution flakiness. (

55f726f

#1441) JAVA-5469

vbabanin self-assigned this Jul 12, 2024

vbabanin added 2 commits July 15, 2024 19:20

Merge branch 'refs/heads/master' into CSOT

86bb039

# Conflicts: # driver-reactive-streams/src/test/functional/com/mongodb/reactivestreams/client/unified/UnifiedServerDiscoveryAndMonitoringTest.java # driver-sync/src/test/functional/com/mongodb/client/unified/UnifiedTest.java

Merge branch 'master' into CSOT

b6d9cb8

vbabanin requested review from stIncMale, vbabanin and rozza July 16, 2024 23:12

vbabanin added 2 commits July 17, 2024 11:42

Unify timeout checks in exception handling methods. (#1449)

27d9dee

JAVA-5526

Merge branch 'master' into CSOT

9363025

vbabanin marked this pull request as ready for review July 17, 2024 23:50

stIncMale approved these changes Jul 18, 2024

View reviewed changes

rozza requested changes Jul 18, 2024

View reviewed changes

Replace "since CSOT" placeholder with "since 5.2" version.

2204a09

JAVA-5469

vbabanin requested a review from rozza July 19, 2024 05:59

evergreen-ci-prod bot mentioned this pull request Jul 19, 2024

Removes the TODO and switchifEmpty stage #1454

Closed

rozza requested changes Jul 19, 2024

View reviewed changes

vbabanin and others added 2 commits July 21, 2024 22:24

Update driver-core/src/main/com/mongodb/internal/operation/SyncOperat…

4177384

…ions.java Co-authored-by: Ross Lawley <[email protected]>

Update driver-core/src/main/com/mongodb/internal/operation/SyncOperat…

6098bce

…ions.java Co-authored-by: Ross Lawley <[email protected]>

vbabanin requested a review from rozza July 22, 2024 05:28

Remove redundant switchIfEmptyStage (#1455)

95bda8f

rozza reviewed Jul 22, 2024

View reviewed changes

rozza approved these changes Jul 22, 2024

View reviewed changes

Merge branch 'master' into CSOT

ee7b306

vbabanin merged commit 01aff5a into master Jul 23, 2024
259 checks passed

vbabanin reviewed Jul 23, 2024

View reviewed changes

vandonr mentioned this pull request Oct 9, 2024

Update Gradle dependencies DataDog/dd-trace-java#7739

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CSOT #1215

CSOT #1215

katcharov commented Oct 4, 2023 •

edited by vbabanin

Loading

katcharov Oct 20, 2023

vbabanin Jul 17, 2024

stIncMale left a comment

rozza commented Jul 18, 2024

rozza left a comment

vbabanin commented Jul 19, 2024 •

edited

Loading

rozza left a comment

vbabanin left a comment •

edited

Loading

CSOT #1215

CSOT #1215

Conversation

katcharov commented Oct 4, 2023 • edited by vbabanin Loading

Description

Key Changes:

Reviewers:

For reviewers:

Context:

katcharov Oct 20, 2023

Choose a reason for hiding this comment

vbabanin Jul 17, 2024

Choose a reason for hiding this comment

stIncMale left a comment

Choose a reason for hiding this comment

rozza commented Jul 18, 2024

rozza left a comment

Choose a reason for hiding this comment

vbabanin commented Jul 19, 2024 • edited Loading

rozza left a comment

Choose a reason for hiding this comment

vbabanin left a comment • edited Loading

Choose a reason for hiding this comment

katcharov commented Oct 4, 2023 •

edited by vbabanin

Loading

vbabanin commented Jul 19, 2024 •

edited

Loading

vbabanin left a comment •

edited

Loading