Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: Add Skip() to Decoder #366

Closed
x448 opened this issue Sep 18, 2022 · 4 comments · Fixed by #381
Closed

feature: Add Skip() to Decoder #366

x448 opened this issue Sep 18, 2022 · 4 comments · Fixed by #381
Assignees
Milestone

Comments

@x448
Copy link
Contributor

x448 commented Sep 18, 2022

Is your feature request related to a problem? Please describe.

It can be useful to skip decoding a CBOR data item as discussed in comments for issue #362.

Describe the solution you'd like

Add Skip() to the decoder to skip next data item.

Describe alternatives you've considered

As mentioned by @fxamacker

Until I add a Skip feature, a less efficient way is to decode the messages you want to skip to cbor.RawMessage. cbor.RawMessage copies data item into a byte slice without parsing the data item.

var throwAway cbor.RawMessage
err := decodoer.Decode(&throwAway)
@x448
Copy link
Contributor Author

x448 commented Sep 19, 2022

@espoal mentioned Section 4.3 of CBOR Sequences RFC 8742 which suggests a CBOR byte string can be used to wrap each item in a sequence for efficient skipping.

Protocol designers can consider specifying this in their encoded format specs after considering the pros and cons.

fyi: @fxamacker

@espoal
Copy link

espoal commented Sep 28, 2022

I would like to add that it would be nice to have End Of Sequence error. It would be different from io.EOF because it would cover the case of zero padding of messages, or partial transmission. Example: If I store a sequence using direct IO, or NVMe api, the byte buffer will have a padding of zeros until we reach a multiple of blocksize (usually 4096 bytes). Or, if I'm using quic to send messages, I may get a truncated sequence, with the rest of the sequence in the next packet, or I will have a padding of zeros until the packet size is reached.

ASN.1 implementation in go has the nice property that it returns the rest of the buffer after decoding, so you can inspect it or append the rest of the next message.

@fxamacker
Copy link
Owner

@espoal

I would like to add that it would be nice to have End Of Sequence error. It would be different from io.EOF because it would cover the case of zero padding of messages

A single byte 0x00 represents CBOR integer 0, so each byte of zero padding would be a valid CBOR data item.

One way to handle padding is to create a custom io.Reader for the cbor.Decoder to strip out the padding. The handling of the padding would be defined at the protocol level.

the case of ... or partial transmission

PR #379 (yesterday) updated Decoder.Decode() so it returns io.ErrUnexpectedEOF instead of io.EOF if the current CBOR data item is incomplete.

One way to handle partial transmission is to check for io.ErrUnexpectdEOF and use NumBytesRead() if the underlying data is accessible to you.

@espoal
Copy link

espoal commented Jan 2, 2023

thanks @fxamacker , I will try to write an example and get back to you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants