unshift stream support

[twiggy]

A few issues have pointed out that the stream becomes broken\altered after to fromStream is called.

its a pretty serious side effect that wouldn't be obvious at all.

we are currently working around this issue by calling stream.read(xxxx) and then stream.unshift. Maybe unshift could be an argument to stream.read() ... or maybe the chunk read could be passed back.

For us we get a stream from the http request, want to detect file type, but then immediately need to pass the stream to AWS's s3 sdk ... bit weird and we could probably just write the file, but unshift seems to work for now.

let leadingChunk = stream.read(4100);
let {ext, mime} = await fileType.fromBuffer(leadingChunk);
stream.unshift(leadingChunk);

[Borewit]

A few issues have pointed out that the stream becomes broken\altered after to fromStream is called.

I think you mean that the concept of a stream has been misunderstood in other issues. fromStream reads from a stream, and then the stream get (partially) consumed. That is how streams work.

At the same time, I do understand the desire to peek ahead on stream, determine the right file type from the stream, and read from the stream as if nothing happened. For that use case stream(readableStream) has been designed. Which is very similar to the example you provided. You directly see limitations, how much data will you buffer (this should be better documented, need to finalize #434).

Related:

• An attempt overcome the limited buffer length in stream(): Make stream() method, stream based, by using teeing #300
• Similar misunderstanding related to consuming streams: fromBuffer and stream methods give different results for same ai file #426

For us we get a stream from the http request, want to detect file type, but then immediately need to pass the stream to AWS's s3 sdk ... bit weird and we could probably just write the file, but unshift seems to work for now.

Writing the file first sounds like a reasonable solution to me. You simply do not know how much of the file you need to determine it's file type. The fact that we can determine the file type for many files with the very first few bytes, does not take away the fact that for some files we need much more data.

[Borewit]

Related to #399.