Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ByteString conversions for JS binary types like ArrayBuffer, Int8Array, Blob, etc. #268

Open
joffrey-bion opened this issue Feb 26, 2024 · 3 comments
Labels
Design enhancement integration An issue related to the integration with other libraries or platform-specific APIs

Comments

@joffrey-bion
Copy link

joffrey-bion commented Feb 26, 2024

In the same vibe as #266, we could add conversions from JS binary types like ArrayBuffer, Blob, Int8Array...

Dealing with binary data in JS is a bit annoying. For example, the ArrayBuffer type is not directly usable and needs to be wrapped in a view. Providing such conversions out of the box in the kotlinx-io-bytestring artifact would be useful for a true multiplatform experience.

Examples:

/**
 * Creates a new [ArrayBuffer] containing the data copied from this [ByteString].
 */
fun ByteString.toArrayBuffer(): ArrayBuffer = toInt8Array().buffer

/**
 * Creates a new [Int8Array] containing the data copied from this [ByteString].
 */
fun ByteString.toInt8Array() = Int8Array(toArrayOfBytes())

private fun ByteString.toArrayOfBytes() = Array(size) { this[it] }

/**
 * Creates a new [ByteString] containing the data copied from this [ArrayBuffer].
 */
fun ArrayBuffer.toByteString(): ByteString = ByteString.wrap(toByteArray())

/**
 * Creates a new [ByteArray] containing the data copied from this [ArrayBuffer].
 */
fun ArrayBuffer.toByteArray(): ByteArray = Int8Array(this).toByteArray()

/**
 * Creates a new [ByteArray] containing the data copied from this [Int8Array].
 */
fun Int8Array.toByteArray() = ByteArray(length) { this[it] } // maybe there is more efficient here
@fzhinkin fzhinkin added enhancement Design integration An issue related to the integration with other libraries or platform-specific APIs labels Feb 26, 2024
@fzhinkin
Copy link
Collaborator

fzhinkin commented Sep 4, 2024

@joffrey-bion Are there any scenarios where these function would be useful right now? Since copying is inevitable, does it make sense to support reading/writing ArrayBuffers from/to Sources/Sinks instead?

@joffrey-bion
Copy link
Author

joffrey-bion commented Sep 4, 2024

Yes, those issues are not hypothetical cases. I needed them in my stomp library (Krossbow), to bridge web socket implementations in different platforms. I need to convert to and from ByteString to different platform-specific types. In this case, it's JS ArrayBuffer.

Since copying is inevitable, does it make sense to support reading/writing ArrayBuffers from/to Sources/Sinks instead?

The goal is to turn it into a ByteString in my case. Wouldn't going through a Source/Sink incur another copy?

Also this would force a dependency on the core library instead of just staying in the "simple" kotlinx-io-bytestring. But I think that's not a very big deal (because if we're "processing stuff", then maybe the core is needed).

@lppedd
Copy link
Contributor

lppedd commented Sep 4, 2024

I just happened to see this issue.

Conversions of a Buffer or ByteString from and to Int8Array and Uint8Array would be a nice addition to the JS interop (only if the data is wrapped tho, and not copied unless it's strictly necessary). We use those two data types when handling TCP sockets.

Well, we mostly use Node's Buffer, but it's just a Uint8Array with some utility functions to operate on the underlying data, just like kotlinx-io's Buffer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Design enhancement integration An issue related to the integration with other libraries or platform-specific APIs
Projects
None yet
Development

No branches or pull requests

3 participants