Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

light-client-verifier: reuse sign_bytes buffer #1413

Merged
merged 3 commits into from
Apr 23, 2024
Merged

Conversation

mina86
Copy link
Contributor

@mina86 mina86 commented Apr 22, 2024

Rather than allocating a new vector each time canonical vote is encoded, allocate a vector in NonAbsentCommitVotes once and then reuse it each time sign_bytes are needed. This saves on doing multiple allocations and deallocations.

For overly verbose description of the length calculation:

  • CanonicalVote. Note that length prefix doesn’t count bytes used for encoding the length, e.g. if the CanonicalVote message is 127 bytes than the length prefix is one byte and thus sign_bytes is 128 bytes. The table assumes all fields are set.

    |                 | tag | value |
    |-----------------+-----+-------|
    | (length)        |     |   1–2 | 2 if (length) is > 127
    | type            |   1 |     1 | SignedMsgType is at most 32
    | height          |   1 |     8 | The field is sfixed64
    | round           |   1 |     8 | The field is sfixed64
    | block_id        |   1 | 73–77 | See CanonicalBlockId below
    | timestamp       |   1 |  7—13 | See Timestamp below
    | chain_id        |   1 |   1+N | Assuming N < 128
    |-----------------+-----+-------|
    | (total)         |   105–116+N | If N≤13 than (length) is
    |                 |             | 1 byte and this is 115+N
    
  • CanonicalBlockId.

    |                 | tag | value |
    |-----------------+-----+-------|
    | (length)        |     |     1 |
    | hash            |   1 |    33 | Assuming 32-byte hash
    | part_set_header |   1 | 37–41 | See CanonicalPartSetHeader
    |-----------------+-----+-------|
    | (total)         |       73–77 |
    
  • CanonicalPartSetHeader.

    |                 | tag | value |
    |-----------------+-----+-------|
    | (length)        |     |     1 |
    | total           |   1 |   1–5 | Varint of u32; at most 5 bytes
    | hash            |   1 |    33 | Assuming 32-byte hash
    |-----------------+-----+-------|
    | (total)         |       37–41 |
    
  • Timestamp. In unlikely event vote was made exactly at a second, nanos is zero and thus the field is not present. Assuming uniform probability of the nanos, in a bit over 73% of cases the filed will occupy it’s maximum size.

    |                 | tag | value |
    |-----------------+-----+-------|
    | (length)        |     |     1 |
    | seconds         |   1 |     5 | Assuming timestamps around now
    | nanos           | 0–1 |   0–5 |
    |-----------------+-----+-------|
    | (total)         |     |  7–13 |
    

  • Referenced an issue explaining the need for the change
  • Updated all relevant documentation in docs
  • Updated all code comments where relevant
  • Wrote tests
  • Added entry in .changelog/

Rather than allocating a new vector each time canonical vote is
encoded, allocate a vector in NonAbsentCommitVotes once and then reuse
it each time sign_bytes are needed.  This saves on doing multiple
allocations and deallocations.

For overly verbose description of the length calculation:

* CanonicalVote.  Note that length prefix doesn’t count bytes used for
  encoding the length, e.g. if the CanonicalVote message is 127 bytes
  than the length prefix is one byte and thus sign_bytes is 128 bytes.
  The table assumes all fields are set.

      |                 | tag | value |
      |-----------------+-----+-------+
      | (length)        |     |   1–2 | 2 if (length) is > 127
      | type            |   1 |     1 | SignedMsgType is at most 32
      | height          |   1 |     8 | The field is sfixed64
      | round           |   1 |     8 | The field is sfixed64
      | block_id        |   1 | 73–77 | See CanonicalBlockId below
      | timestamp       |   1 |  7—13 | See Timestamp below
      | chain_id        |   1 |   1+N | Assuming N < 128
      |-----------------+-----+-------+
      | (total)         |   105–116+N | If N≤13 than (length) is
      |                 |             | 1 byte and this is 115+N

* CanonicalBlockId.

      |                 | tag | value |
      |-----------------+-----+-------+
      | (length)        |     |     1 |
      | hash            |   1 |    33 | Assuming 32-byte hash.
      | part_set_header |   1 | 37–41 | See CanonicalPartSetHeader
      |-----------------+-----+-------+
      | (total)         |       73–77 |

* CanonicalPartSetHeader.

      |                 | tag | value |
      |-----------------+-----+-------+
      | (length)        |     |     1 |
      | total           |   1 |   1–5 | Varint of u32; at most 5 bytes.
      | hash            |   1 |    33 | Assuming 32-byte hash.
      |-----------------+-----+-------+
      | (total)         |       37–41 |

* Timestamp.  In unlikely event vote was made exactly at a second,
  nanos is zero and thus the field is not present.  Assuming uniform
  probability of the nanos, in a bit over 73% of cases the filed will
  occupy it’s maximum size.

      |                 | tag | value |
      |-----------------+-----+-------+
      | (length)        |     |     1 |
      | seconds         |   1 |     5 | Assuming timestamps around now.
      | nanos           | 0–1 |   0–5 |
      |-----------------+-----+-------+
      | (total)         |     |  7–13 |
Copy link
Member

@romac romac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for the PR and the detailed explanations! Looks good, but I left a couple suggestions and believe we should increase the buffer size a little bit to err on the safe side, see my comment above.

light-client-verifier/src/operations/voting_power.rs Outdated Show resolved Hide resolved
light-client-verifier/src/operations/voting_power.rs Outdated Show resolved Hide resolved
Copy link
Member

@romac romac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again so much for the PR!

@romac romac merged commit 38a067c into informalsystems:main Apr 23, 2024
22 checks passed
@romac
Copy link
Member

romac commented Apr 23, 2024

I will release all this in v0.36 later this week.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants