GH-391 Add Support for Other Storage Solutions #387
Conversation
� Conflicts: � reposilite-backend/src/main/java/org/panda_lang/reposilite/Reposilite.java � reposilite-backend/src/main/java/org/panda_lang/reposilite/metadata/MetadataService.java � reposilite-backend/src/main/java/org/panda_lang/reposilite/repository/DeployService.java � reposilite-backend/src/main/java/org/panda_lang/reposilite/repository/DiskQuota.java � reposilite-backend/src/main/java/org/panda_lang/reposilite/repository/FileDetailsDto.java � reposilite-backend/src/main/java/org/panda_lang/reposilite/repository/LookupApiEndpoint.java � reposilite-backend/src/main/java/org/panda_lang/reposilite/stats/StatsStorage.java � reposilite-backend/src/main/java/org/panda_lang/reposilite/utils/FilesUtils.java � reposilite-backend/src/test/groovy/org/panda_lang/reposilite/repository/RepositoryServiceTest.groovy � reposilite-backend/src/test/workspace/reposilite.cdn
|
Hey there! :) Interesting changes, it might be kinda tricky to properly rewrite all of it at once. Also, as you can see, there is a lot of places that lost on the readability, for instance:
I think it might be a good idea to write some extra utilities to improve the readability of nio api. Let me know when you feel that PR is ready for review and ask whenever you think it might be helpful to discuss some aspects. |
dzikoysk
changed the title
|
Alright. I was going to do some larger scale abstractions on storage in a separate PR, but doing so now would help clean up a lot of these things, so I'll go ahead and put them here. |
|
I think you could start by reporting issues that address motivations behind these changes. It's easier to work around it 😛 |
…g, but everything seems to basically work. The S3StorageProvider needs to be tested, and a config option of some kind needs to be added to specify which storage provider to use.
Haven-King
changed the title
|
There's still some work that needs to be done before this is done, but most of the framework is there, so I thought I'd push to give you an idea of what I'm working with right now. What still needs to be done:
|
Haven-King
changed the title
|
It's hard to track all of these changes, but tests contain various scenarios and should inform about possible regressions. Tbh there is a little to much tests. Some of them are kinda messy, so if you see something strange that does not make any sense for you, just ask about it. |
dzikoysk
added
the
breaking
…sically work. The S3StorageProvider needs to be tested. Config options have been added, but the no redeploy option has not been implemented yet. The REST API storage provider has not been implemented yet.
Most of the work for StorageProviders is done.
|
Most of the work for StorageProviders is done. Everything seems to basically work. The |
|
Great, remember that you don't have to provide all implementations at once in this PR. For instance, REST based provider could be just supported later in another, dedicated PR. By reviewing latest issues I also wonder if we want to keep compatibility with 2.x branch or we can just drop it and focus on something like 3.x 🤔 |
|
Sure, bumping to 3.x would probably be reasonable. There are certainly a lot of changes in the works. |
|
I've stared rewriting frontend on 3.0 branch (as a draft #465). I'll try to limit changes on backend as much as possibile to relatively easy merge both of our PRs to master. As soon as you'll finish this PR, I'll fork master into 2.x branch and we will continue development of 3.0 on master. |
# Conflicts: # .run/Reposilite.run.xml # reposilite-backend/src/main/java/org/panda_lang/reposilite/Reposilite.java # reposilite-backend/src/main/java/org/panda_lang/reposilite/ReposiliteUtils.java # reposilite-backend/src/main/java/org/panda_lang/reposilite/config/ConfigurationLoader.java # reposilite-backend/src/main/java/org/panda_lang/reposilite/repository/DeployService.java # reposilite-backend/src/main/java/org/panda_lang/reposilite/repository/LookupApiEndpoint.java # reposilite-backend/src/main/java/org/panda_lang/reposilite/repository/LookupService.java # reposilite-backend/src/main/java/org/panda_lang/reposilite/repository/ProxyService.java # reposilite-backend/src/main/java/org/panda_lang/reposilite/repository/RepositoryAuthenticator.java # reposilite-backend/src/main/java/org/panda_lang/reposilite/repository/RepositoryStorage.java # reposilite-backend/src/test/groovy/org/panda_lang/reposilite/ReposiliteUtilsTest.groovy # reposilite-backend/src/test/groovy/org/panda_lang/reposilite/config/ConfigurationLoaderTest.groovy # reposilite-backend/src/test/groovy/org/panda_lang/reposilite/repository/ProxyServiceTest.groovy # reposilite-backend/src/test/groovy/org/panda_lang/reposilite/repository/RepositoryStorageTest.groovy
This pull request adds several storage options as alternatives to storing artifacts on the local disk storage. It does this by abstracting away most of the file manipulation logic into three concrete implementations of the
StorageProviderinterface:FileSystemStorageProvider- Implements behavior identical to the current solution; stores all artifacts to disk.S3StorageProvider- Uploads artifacts and metadata to Amazon S3. Useful for highly available systems, as well as serving artifacts statically.RestAPIStorageProvider- Uploads artifacts and metadata to an external HTTP/HTTPS server which can handle the saving/loading of artifacts itself. This option will be useful for people aiming to customize their Reposilite server to a very high degree.This PR is still a draft.