perf: calculate checksum using protobuf values #2848
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
product-auto-label
bot
added
size: l
product-auto-label
bot
added
size: xl
ankiaga
reviewed
Feb 9, 2024
google-cloud-spanner/src/main/java/com/google/cloud/spanner/GrpcResultSet.java
Show resolved
Hide resolved
ankiaga
reviewed
Feb 9, 2024
...panner/src/main/java/com/google/cloud/spanner/connection/ReplaceableForwardingResultSet.java
Show resolved
Hide resolved
ankiaga
approved these changes
Feb 9, 2024
olavloite
added a commit
that referenced
this pull request
Feb 9, 2024
* feat: support lazy decoding of query results Adds an option for lazy decoding of query results. Currently, all values in a query result row are decoded from protobuf values to plain Java objects at the moment that the result set is advanced to the next row. This means that all values are decoded, regardless whether the application actually fetches these or not. Lazy decoding also enables the possibility for (internal) consumers of a result set to access the protobuf value before it is converted to a plain Java object. This for example allows ChecksumResultSet to calculate the checksum based on the protobuf value, instead of a Java object, which can be more efficient. * fix: add null check * perf: calculate checksum using protobuf values (#2848) * perf: calculate checksum using protobuf values * chore: cleanup * test: remove unrelated test * fix: undo change to public API * chore: cleanup|
olavloite
added a commit
that referenced
this pull request
Feb 9, 2024
* refactor: move inner classes to top level Move the gRPC-related inner classes from AbstractResultSet to top-level classes, so they are easier to modify and maintain. This change only contains modifications that are needed to move these inner classes. There are no functional changes. * feat: support lazy decoding of query results (#2847) * feat: support lazy decoding of query results Adds an option for lazy decoding of query results. Currently, all values in a query result row are decoded from protobuf values to plain Java objects at the moment that the result set is advanced to the next row. This means that all values are decoded, regardless whether the application actually fetches these or not. Lazy decoding also enables the possibility for (internal) consumers of a result set to access the protobuf value before it is converted to a plain Java object. This for example allows ChecksumResultSet to calculate the checksum based on the protobuf value, instead of a Java object, which can be more efficient. * fix: add null check * perf: calculate checksum using protobuf values (#2848) * perf: calculate checksum using protobuf values * chore: cleanup * test: remove unrelated test * fix: undo change to public API * chore: cleanup| * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Calculates the read/write transaction checksum for query results in the Connection API using the protobuf values instead of the decoded Java objects. This reduces both memory and CPU usage for the calculation.
The hash algorithm is also changed from
sha-256tomd5. The latter has a slightly higher collision probability, but still widely within the acceptable ranges for the use-case at hand, which is not security related, but only needed in order to determine equality between query results. Switching tomd5also reduces CPU usage.