Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: deterministic protov2 marshal #18403

Merged
merged 4 commits into from
Nov 7, 2023
Merged

fix: deterministic protov2 marshal #18403

merged 4 commits into from
Nov 7, 2023

Conversation

julienrbrt
Copy link
Member

@julienrbrt julienrbrt commented Nov 7, 2023

Description

Context: https://binarybuilders.slack.com/archives/C05J3U400HX/p1697215636557559
Thanks @testinginprod reminding us about this just in time.

We should add a CodeQL rule in: https://github.com/crypto-com/cosmos-sdk-codeql


Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

  • included the correct type prefix in the PR title
  • added ! to the type prefix if API or client breaking change
  • targeted the correct branch (see PR Targeting)
  • provided a link to the relevant issue or specification
  • followed the guidelines for building modules
  • included the necessary unit and integration tests
  • added a changelog entry to CHANGELOG.md
  • included comments for documenting Go code
  • updated the relevant documentation or specification
  • reviewed "Files changed" and left comments if necessary
  • run make lint and make test
  • confirmed all CI checks have passed

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

  • confirmed the correct type prefix in the PR title
  • confirmed ! in the type prefix if API or client breaking change
  • confirmed all author checklist items have been addressed
  • reviewed state machine logic
  • reviewed API design and naming
  • reviewed documentation is accurate
  • reviewed tests and test coverage
  • manually tested (if applicable)

Summary by CodeRabbit

  • Refactor
    • Introduced deterministic marshaling for protocol buffer messages across various components of the application, enhancing consistency and predictability.

@julienrbrt julienrbrt added the backport/v0.50.x PR scheduled for inclusion in the v0.50's next stable release label Nov 7, 2023
Copy link
Contributor

coderabbitai bot commented Nov 7, 2023

Your existing walkthrough is well-structured and informative. I've updated the changes section to reflect the provided information. Here's the updated content:

Walkthrough

The changes across multiple files primarily focus on ensuring deterministic marshaling of protocol buffer messages. This is achieved by replacing direct calls to proto.Marshal or proto2.Marshal with the use of proto.MarshalOptions or proto2.MarshalOptions with the Deterministic field set to true.

Changes

File Summary
baseapp/internal/protocompat/protocompat.go Introduced a new global variable for deterministic marshaling and replaced proto2.Marshal with protov2MarshalOpts.Marshal in two functions.
codec/collections.go, codec/proto_codec.go, codec/types/any.go Replaced direct calls to proto2.Marshal or proto.Marshal with the use of proto2.MarshalOptions or proto.MarshalOptions with the Deterministic field set to true.
x/tx/signing/direct/..., x/tx/signing/directaux/..., x/tx/signing/testutil/util.go, x/tx/signing/textual/tx.go, x/accounts/internal/implementation/api_builder.go Introduced a new global variable for deterministic marshaling and replaced direct calls to proto.Marshal with the use of protov2MarshalOpts.Marshal or proto.MarshalOptions.

Poem

🍂 As the leaves fall, so does the old code,
Deterministic marshaling, the new mode.
No more randomness, no more surprise,
Just like the sun that predictably rise.
🐇 Hopping through the autumn code,
Celebrating the deterministic mode.
On this day in history, a rabbit's delight,
The deterministic marshaling taking flight.
So here's to the changes, big and small,
To the developers, standing tall! 🎉

Feel free to let me know if there's anything else you'd like to add or modify!


Tips

Chat with CodeRabbit Bot (@coderabbitai)

  • If you reply to a review comment from CodeRabbit, the bot will automatically respond.
  • To engage with CodeRabbit bot directly around the specific lines of code in the PR, mention @coderabbitai in your review comment
  • Note: Review comments are made on code diffs or files, not on the PR overview.
  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai help to get help.
  • @coderabbitai resolve to resolve all the CodeRabbit review comments.

Note: For conversation with the bot, please use the review comments on code diffs or files.

CodeRabbit Configration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • The JSON schema for the configuration file is available here.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/coderabbit-overrides.v2.json

@julienrbrt julienrbrt marked this pull request as ready for review November 7, 2023 16:04
@julienrbrt julienrbrt requested a review from a team as a code owner November 7, 2023 16:04
@github-prbot github-prbot requested review from a team and tac0turtle and removed request for a team November 7, 2023 16:04
Copy link
Contributor

github-actions bot commented Nov 7, 2023

@julienrbrt your pull request is missing a changelog!

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 0

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between 1e4d1f9 and 4731678.
Files selected for processing (8)
  • baseapp/internal/protocompat/protocompat.go (3 hunks)
  • codec/collections.go (1 hunks)
  • codec/proto_codec.go (1 hunks)
  • codec/types/any.go (2 hunks)
  • x/tx/signing/direct/direct.go (2 hunks)
  • x/tx/signing/directaux/direct_aux.go (1 hunks)
  • x/tx/signing/testutil/util.go (1 hunks)
  • x/tx/signing/textual/tx.go (1 hunks)
Additional comments: 12
x/tx/signing/testutil/util.go (1)
  • 59-71: The use of deterministic marshaling is a good practice for protocol buffers, especially when dealing with cryptographic signatures. This change ensures that the serialized output will be the same for the same input.
codec/collections.go (1)
  • 106-109: The change ensures deterministic marshaling of protobuf messages, which is crucial for consensus-critical data in blockchain applications. This change is consistent with the pull request's goal.
x/tx/signing/directaux/direct_aux.go (1)
  • 101-107: The change to use deterministic marshaling is appropriate and correctly implemented. Ensure that all dependencies and modules that rely on the output of this function are compatible with deterministic marshaling.
x/tx/signing/textual/tx.go (1)
  • 272-283: The use of proto.MarshalOptions{Deterministic: true} ensures deterministic serialization of protobuf messages. This is a good practice for scenarios where the order of serialization matters, such as in cryptographic operations or when comparing serialized data. However, it may have a slight performance impact compared to non-deterministic serialization. Ensure that this change aligns with the performance requirements of your application.
codec/proto_codec.go (1)
  • 345-349: The change to use protov2MarshalOpts.Marshal(m) instead of proto.Marshal(m) is correct and aligns with the goal of deterministic marshaling. Ensure that all proto.Message types used in the codebase are compatible with deterministic marshaling.
x/tx/signing/direct/direct.go (2)
  • 13-16: The introduction of the protov2MarshalOpts global variable with deterministic marshaling is a good practice for ensuring consistent serialization of protocol buffer messages across different invocations. However, ensure that this change does not introduce any unintended side effects in the parts of the codebase where deterministic marshaling is not desired.

  • 27-33: The change from proto.Marshal to protov2MarshalOpts.Marshal in the GetSignBytes function is consistent with the goal of deterministic marshaling. Ensure that all usages of this function are compatible with the deterministic serialization of the txv1beta1.SignDoc structure.

baseapp/internal/protocompat/protocompat.go (3)
  • 16-22: The introduction of the protov2MarshalOpts global variable is a good practice to ensure deterministic marshaling across the codebase. This change is consistent and maintainable.

  • 98-104: The use of protov2MarshalOpts.Marshal instead of proto2.Marshal ensures deterministic marshaling of the response. This change is consistent with the goal of the pull request.

  • 139-145: The use of protov2MarshalOpts.Marshal instead of proto2.Marshal ensures deterministic marshaling of the request. This change is consistent with the goal of the pull request.

codec/types/any.go (2)
  • 68-75: The change introduces deterministic marshaling for proto v2 messages. This is a good practice as it ensures the serialized output will always be the same given the same input, which can be crucial for things like hashing or equality checks.

  • 110-117: The same change as above is applied here, introducing deterministic marshaling for proto v2 messages in the pack function.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 0

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between 4731678 and 58b7f20.
Files ignored due to filter (1)
  • x/auth/go.mod
Files selected for processing (1)
  • x/accounts/internal/implementation/api_builder.go (2 hunks)
Files skipped from review due to trivial changes (1)
  • x/accounts/internal/implementation/api_builder.go

Copy link
Contributor

@testinginprod testinginprod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 🗡️

@julienrbrt julienrbrt enabled auto-merge November 7, 2023 16:18
@julienrbrt julienrbrt added this pull request to the merge queue Nov 7, 2023
Merged via the queue into main with commit e2b71b7 Nov 7, 2023
64 of 65 checks passed
@julienrbrt julienrbrt deleted the julien/proto branch November 7, 2023 16:47
mergify bot pushed a commit that referenced this pull request Nov 7, 2023
(cherry picked from commit e2b71b7)

# Conflicts:
#	x/accounts/internal/implementation/api_builder.go
#	x/auth/go.mod
julienrbrt added a commit that referenced this pull request Nov 7, 2023
@faddat faddat mentioned this pull request Nov 8, 2024
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport/v0.50.x PR scheduled for inclusion in the v0.50's next stable release C:x/accounts C:x/auth C:x/tx
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants