Cloud media files #141
Cloud media files #141
Conversation
I was just shown this and it seems the URL to the development server is out of date. Fix this.
management/custom_azure.py
Outdated
| from azure.storage.blob import BlobServiceClient, generate_blob_sas | ||
| from storages.backends.azure_storage import AzureStorage | ||
|
|
||
| from liionsden.settings import settings |
There was a problem hiding this comment.
These settings are the standard settings, not the production settings which are in production.py. I'm not sure if it matters, but just in case I better mention it.
There was a problem hiding this comment.
Yeh one question I had, which I forgot to mention, was whether we want to only use the azure storage in the production environment. In principle it sounds sensible, but I'm not sure if it is as simple as just changing settings.py until we sort out the parsing engines - they currently kind of rely on the blob storage being used to retrieve the file.
There was a problem hiding this comment.
I think it makes sense to serve files from Azure only when Liionsden is running in Azure - or possibly the VM. But for local use and development purposes, I don't think we should enforce the need to Azure. Yes, the parsers will need updating.
There was a problem hiding this comment.
Well it will have to be definitely when running on the VM (as we are still using as a development version) because the original issue this solves is that we can't serve media files from there currently.
I agree about local development though so I suggest that we test the implementation in this PR on the VM (once I've addressed the other points) first. Then once we know it fixes the original issue, we can sort out the parsers and make it a production (i.e. VM or Azure) only setting.
|
Hi @dalonsoa - I think I've addressed all the points other than setting this as a production-only setting. As I said, I'd rather do this as a separate step to test that this approach definitely works on the VM. After that, the next PR will be refactoring parsers and allowing the storage backend to be selected based on the environment. Happy to hear any other suggestions though! |
This PR contains most of the changes to move to using Azure storage to serve media files. Overall it should now be possible to download raw data files that have been uploaded by users (and any other media files).
The parsing procedure is now a little different: The file is copied to a temporary local folder, all the parsing routines are carried out, then that temporary file is destroyed. This could be handled better (see #139) but I would prefer to tackle that separately: There are lots of places where "file_path" is used to read the file and it's already started to get a little messy. See file-obj-parsers branch where I've made a start.
Overall, I think we should check these changes solve the problem in #61 for the VM before moving to updating the parsing engines.