-
Notifications
You must be signed in to change notification settings - Fork 28
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add optional pledgeinsize
function to transcoding protocol
#239
Conversation
I wonder if it would be appropriate to call this |
I chose the name to match the style of the other functions in the transcoding protocol. This is also slightly stronger than a hint since with Zstd setting the wrong size will cause an error. |
I guess the convention here is not to use Could we make it clearer the order in which these functions will be called? Also do we have a mechanism to pledge the size even in streaming mode? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Document call order and consider potential uses outside of transcode
. For example, what if I have a large file of known size that I plan to stream.
I added some more details to the order of the function calls. The transcoding protocol can be used outside of the |
@mkitti I added some examples of how |
This release adds the optional `pledgeinsize` function to the transcoding protocol (PR #239)
The
pledgeinsize(codec::C, insize::Int64, error::Error)::Symbol
method is usedwhen
transcode
is called to tell thecodec
the total input size.This is called after
startproc
and beforeprocess
. Somecompressors can add this total input size to a header, making
expectedsize
accurate during later decompression. By default this just returns
:ok
.If there is an error, the return code must be
:error
and theerror
argumentmust be set to an exception object. Setting an inaccurate
insize
may cause thecodec to error later on while processing data. A negative
insize
means unknowncontent size.
Ref: JuliaIO/CodecZstd.jl#58
I think this is a nicer alternative to #215