Calculate and display more descriptive dataset size

[evamaxfield]

Currently dataset / package size is being shown only as the top level directory size. This is really inaccurate for Allen Cell and I would assume many other packages as well.

Example:

The 2.8MB+ is way off target for both the entire package size or even the manifest size and is thus just not really useful. Suggestion by @akarve is to have a lambda that reads the size from the manifest and then displays.

[nl0]

s3 select could be helpful here, but it only works for authenticated users

[akarve]

True. We now have a proxy for the unauthenticated case, so S3 select could be a good fit.

…

On Mon, Sep 9, 2019 at 12:30 PM Alexei Mochalov ***@***.***> wrote: s3 select could be helpful here, but it only works for authenticated users — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1116?email_source=notifications&email_token=AAKC5W27R325SMA3EABTRQLQI2P7FA5CNFSM4IU6ETB2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6IYMYY#issuecomment-529630819>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAKC5W4B4YXIGCUPFNPVNYDQI2P7FANCNFSM4IU6ETBQ> .

-- Aneesh Karve | 765-360-9348 | LinkedIn <http://linkedin.com/in/aneeshkarve> | Twitter <https://twitter.com/akarve> quiltdata.com | Manage data like code <https://blog.quiltdata.com/its-time-to-manage-data-like-source-code-3df04cd312b8>

[evamaxfield]

This is also a pretty big desire to have due to open data bucket size? I know it wouldn't be perfect but at least a good estimate would be great because certain buckets now only have 25 TB of storage space 🙃