Skip to content

Commit

Permalink
Make ai.rapids.cudf.HostMemoryBuffer#copyFromStream public. (#17179)
Browse files Browse the repository at this point in the history
This is the first pr of [a larger one](NVIDIA/spark-rapids-jni#2532) to introduce a new serialization format. It make `ai.rapids.cudf.HostMemoryBuffer#copyFromStream` public. For more background, see NVIDIA/spark-rapids-jni#2496

Authors:
  - Renjie Liu (https://github.com/liurenjie1024)
  - Jason Lowe (https://github.com/jlowe)

Approvers:
  - Jason Lowe (https://github.com/jlowe)
  - Alessandro Bellina (https://github.com/abellina)

URL: #17179
  • Loading branch information
liurenjie1024 authored Oct 30, 2024
1 parent eeb4d27 commit 6328ad6
Showing 1 changed file with 4 additions and 2 deletions.
6 changes: 4 additions & 2 deletions java/src/main/java/ai/rapids/cudf/HostMemoryBuffer.java
Original file line number Diff line number Diff line change
Expand Up @@ -255,16 +255,18 @@ public final void copyFromHostBuffer(long destOffset, HostMemoryBuffer srcData,
* @param destOffset offset in bytes in this buffer to start copying to
* @param in input stream to copy bytes from
* @param byteLength number of bytes to copy
* @throws EOFException If there are not enough bytes in the stream to copy.
* @throws IOException If there is an error reading from the stream.
*/
final void copyFromStream(long destOffset, InputStream in, long byteLength) throws IOException {
public final void copyFromStream(long destOffset, InputStream in, long byteLength) throws IOException {
addressOutOfBoundsCheck(address + destOffset, byteLength, "copy from stream");
byte[] arrayBuffer = new byte[(int) Math.min(1024 * 128, byteLength)];
long left = byteLength;
while (left > 0) {
int amountToCopy = (int) Math.min(arrayBuffer.length, left);
int amountRead = in.read(arrayBuffer, 0, amountToCopy);
if (amountRead < 0) {
throw new EOFException();
throw new EOFException("Unexpected end of stream, expected " + left + " more bytes");
}
setBytes(destOffset, arrayBuffer, 0, amountRead);
destOffset += amountRead;
Expand Down

0 comments on commit 6328ad6

Please sign in to comment.