Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose ZFEC functionality via the C-compatible FFI interface #3190

Closed
exarkun opened this issue Jan 18, 2023 · 4 comments
Closed

Expose ZFEC functionality via the C-compatible FFI interface #3190

exarkun opened this issue Jan 18, 2023 · 4 comments
Labels
enhancement Enhancement or new feature
Milestone

Comments

@exarkun
Copy link
Contributor

exarkun commented Jan 18, 2023

It would be nice to use Botan's ZFEC implementation from languages/runtimes other than C++. The obvious way to enable this to add a extern "C" style interface on top of the C++ interface, similar to how other functionality is exposed in src/lib/ffi/.

A reasonably simple interface to use might be something like:

// input gives the primary shares
// size gives the length of the primary shares (all equal)
// output gives a place to write a uint8_t** giving the generated secondary shares
// outputSizes gives a place to write size_t* giving the size of the generated shares (all equal)
// memory ownership of *output and all contained pointers and *outputSizes is given to the caller on success
int botan_zfec_encode(size_t K, size_t N, const uint8_t input[], size_t size, uint8_t ***output, size_t **outputSizes);

// shares gives primary or secondary shares to decode
// share_indexes gives the index of corresponding elements of shares
// num_shares gives the length of shares and share_indexes
// share_size gives the length of each share
// output and outputSizes are like for encode
int botan_zfec_decode(size_t K, size_t N, const uint8_t shares[], const size_t[] share_indexes, size_t num_shares, size_t share_size, uint8_t ***output, size_t **outputSizes);

A different interface could use output_cb_t instead of output buffers which might enable less copying but it's also harder (but maybe not impossible to consume across an FFI interface).

This interface does not preclude the addition of such an interface later on, if it proves desirable enough to do the extra work.

I have a preliminary implementation of botan_fec_encode that I've demonstrated works when called from Haskell. I can finish it (docs, tests, whatever else it needs) and work on botan_fec_decode if this addition is welcome.

@exarkun
Copy link
Contributor Author

exarkun commented Jan 18, 2023

My work on botan_fec_encode is in https://github.com/PrivateStorageio/botan/tree/3190.ffi-zfec.

Is there interest in having this in Botan or should I work on putting this code into a different project?

@randombit
Copy link
Owner

Yes we'd be happy to take this as a PR (probably would need some additional tests/etc before merge but the general approach seems fine)

@exarkun
Copy link
Contributor Author

exarkun commented Jan 19, 2023

I think #3196 is in reasonable shape for a first review. I'm sure I got many things wrong. There is at least a passing test that verifies the round-trip behavior for encode/decode. Looking forward to any feedback.

@randombit randombit added this to the Botan 3.0.0 milestone Jan 31, 2023
@lieser lieser added the enhancement Enhancement or new feature label Feb 21, 2023
@randombit
Copy link
Owner

Done! Will be available in 3.0.0 which is estimated to be released sometime next month.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhancement or new feature
Projects
None yet
Development

No branches or pull requests

3 participants