large file support

[a-tal]

It'd be great if twine could upload files larger than the system's memory. Currently, I don't think this is possible due to reading the entire file into memory

[sigmavirus24]

@dstufft isn't it true that PyPI has a limit on the file size so this is unnecessary?

[dstufft]

PyPI does have a limit, but people have internal PyPI's which don't have a limit. It seems silly to not support that when it's pretty easy to do I think. Just iterate over the file once to get the md5 hash and then seek back to the start and give requests an iterator?

[sigmavirus24]

Except that even 1b14328 won't solve this because you need the MultipartEncoder from the requests-toolbelt. It's the only way to stream a multipart/form-data upload (which is what twine is currently performing).

[dstufft]

Might be time to add a dependency on requests-toolbelt then? Or do you have an opposition to the feature in general?

[sigmavirus24]

None at all. I had approached you a few months ago and you mentioned it not being a problem because of PyPI's limits. =P

[dstufft]

Oh, well I don't recall that :) I'm not actually good at this computer or project leading thing!

[sigmavirus24]

No worries! That's why I pinged you about this. =D

[a-tal]

I added the requests-toolbelt dep to my PR. I think it's all good now if someone wants to double check it.