Use stream in tauri-plugin-fs writeFile

[rwv]

Describe the problem

For now I’m using

const dataBuffer = await blob.arrayBuffer();
const data = new Uint8Array(dataBuffer);
await writeFile(filename, data)

to write file.

Describe the solution you'd like

Can we use blob.stream to get a readable stream and pass it to writeFile. Since v2’s IPC is based on Response, we can use new Response() to generate a streaming response to improve performance and reduce memory usage.

Alternatives considered

Can Transferable object improve performance?

Additional context

No response

[rwv]

workaround:

import { writeFile, type WriteFileOptions } from "@tauri-apps/plugin-fs";

export async function writeFileFromStream(
  path: string | URL,
  stream: ReadableStream<Uint8Array>,
  options?: WriteFileOptions & { chunkSize?: number }
) {
  const chunkSize = options?.chunkSize ?? 1024 * 1024; // 1MB in bytes
  const reader = stream.getReader();

  let chunks: Uint8Array[] = [];
  let bytesRead = 0;

  // eslint-disable-next-line no-constant-condition
  while (true) {
    const { done, value } = await reader.read();
    if (done) break;

    chunks.push(value);
    bytesRead += value.length;

    if (bytesRead >= chunkSize) {
      const chunk = new Uint8Array(bytesRead);
      let offset = 0;
      for (const chunkValue of chunks) {
        chunk.set(chunkValue, offset);
        offset += chunkValue.length;
      }

      await writeFile(path, chunk, {
        ...options,
        append: true,
      });
      chunks = [];
      bytesRead = 0;
    }
  }

  if (bytesRead > 0) {
    const chunk = new Uint8Array(bytesRead);
    let offset = 0;
    for (const chunkValue of chunks) {
      chunk.set(chunkValue, offset);
      offset += chunkValue.length;
    }
    await writeFile(path, chunk, {
      ...options,
      append: true,
    });
  }
}

[amrbashir]

I have added support for it in #1964 but I think your workaround could be improved by using open and FileHandle.write APIs

const file = await open(path, options)
for await (const chunk of stream) {
  await file.write(chunk)
}
file.close()

this will stream the data, instead of writing it at one go