Spanner - Closing session pool is slow for large pools #19
Assignees
Labels
api: spanner
Issues related to the googleapis/java-spanner API.
type: feature request
‘Nice-to-have’ improvement, new feature or different behavior or design.
Comments
chingor13
added
the
type: feature request
olavloite
added a commit
that referenced
this issue
Jan 9, 2020
Sessions should be closed using an async gRPC call in order to speed up the closing of a large session pool. Instead of using its own executor, which is limited to 8 worker threads, to execute asynchronous delete session calls, the session pool should use the asynchronous call option in gRPC. This allows a larger number of asynchronous delete session calls to be executed in parallel and speeds up closing a session pool with a large number of sessions. Testing against a real Cloud Spanner database with a session pool containing 1,000 sessions shows the following performance for closing the session pool: Before (3 runs): 6603ms 8169ms 8367ms After (3 runs): 1297ms 1710ms 1851ms Fixes #19.
olavloite
added a commit
that referenced
this issue
Jan 14, 2020
Sessions should be closed using an async gRPC call in order to speed up the closing of a large session pool. Instead of using its own executor, which is limited to 8 worker threads, to execute asynchronous delete session calls, the session pool should use the asynchronous call option in gRPC. This allows a larger number of asynchronous delete session calls to be executed in parallel and speeds up closing a session pool with a large number of sessions. Testing against a real Cloud Spanner database with a session pool containing 1,000 sessions shows the following performance for closing the session pool: Before (3 runs): 6603ms 8169ms 8367ms After (3 runs): 1297ms 1710ms 1851ms Fixes #19.
skuruppu
pushed a commit
that referenced
this issue
Jan 23, 2020
* perf: close sessions async Sessions should be closed using an async gRPC call in order to speed up the closing of a large session pool. Instead of using its own executor, which is limited to 8 worker threads, to execute asynchronous delete session calls, the session pool should use the asynchronous call option in gRPC. This allows a larger number of asynchronous delete session calls to be executed in parallel and speeds up closing a session pool with a large number of sessions. Testing against a real Cloud Spanner database with a session pool containing 1,000 sessions shows the following performance for closing the session pool: Before (3 runs): 6603ms 8169ms 8367ms After (3 runs): 1297ms 1710ms 1851ms Fixes #19. * fix: wait for test servers to terminate * remove tracing for async call * do not use directExecutor which could be a gRPC thread * fix: return failed future instead of throwing exception * fix: remove commented code
|
Merge was reverted because of problems with the binary compatibility check. |
google-cloud-label-sync
bot
added
the
api: spanner
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Calling
Spanner#close()can be slow if the underlying session pool contains a large number of sessions.Spanner#close()will close and cleanup all resources that have been used by the instance, and one of these is the session pool. The method will block until all resources have been cleaned up.The session pool deletes all its sessions when it is closed. This is done asynchronously using a fixed 8-worker thread executor. Each worker thread blocks while waiting for a session to be deleted. This means that closing a session pool with 1,000 sessions will have an execution time of
T * 1000 / 8, whereTis the total time needed to delete one session.Session deletion could be further parallelized for large session pools in order to reduce the total execution time of
Spanner#close().The text was updated successfully, but these errors were encountered: