Having trouble with large file access from blob storage [context deadline exceeded]

[shawnsullivan-tomtom]

Hello,
We are having issues with access to large files (~250GB). We are using blobfuse2 to mount storage and have tried various configurations with no success. The access ultimately ends with an "Input/output error" related to read of the content. The blobfuse logs reveal the following information...

Jun  6 12:00:03 85846xxx blobfuse2[8098]: LOG_ERR [block_blob.go (688)]: BlockBlob::ReadToFile : Failed to download blob <filename> [context deadline exceeded]
Jun  6 12:00:03 85846xxx blobfuse2[8098]: LOG_ERR [file_cache.go (846)]: FileCache::OpenFile : error downloading file from storage <filename> [context deadline exceeded]
Jun  6 12:00:13 85846xxx blobfuse2[8098]: LOG_ERR [libfuse_handler.go (667)]: Libfuse::libfuse_open : Failed to open <filename> [context deadline exceeded]

filename was redacted to protect the innocent ;)
We thought it might be best to consult the experts here for some guidance.
Any insight is greatly appreciated.
Thanks!
~Shawn

[vibhansa-msft]

[context deadline exceeded] : basically means it took too long to download the contents of this file. By default blobfuse works in file-cache mode where entire file is downloaded on local system and then read/writes are served from that cache. If network or underlying disk are slow then download and local caching will take a lot of time and operation may timeout. If objective is to process this huge file then better switch to "streaming" mde of blobfuse, however that will reduce the performance. If you need high performance then ensure network and local disk speeds are adequate to support such huge file.

[shawnsullivan-tomtom]

Thank you for the feedback! We were able to adjust config to increase timeout and read the data successfully. We have on our backlog to look into other enhancements such as network/disk improvements. Be well!