Refactor/use oras and organize #19
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: vsoch <[email protected]>
I am adding a global oras client, that way we can ensure checking credentials once (and possibly defaulting to a dry run) and using the client across different types of pushes (repo and packages). I also added a "Pusher" class to oras since the actions of creating layers and setting defaults were used between the two places. I am currently maintaining the relatively flat structure of the module, however this could be refactored in the future if needed (I usually prefer to have a main module with the core functionality but I will not push this preference on others! I need to next refactor the cache_packages.py and then add some testing. Signed-off-by: vsoch <[email protected]>
Signed-off-by: vsoch <[email protected]>
Signed-off-by: vsoch <[email protected]>
Signed-off-by: vsoch <[email protected]>
Signed-off-by: vsoch <[email protected]>
this test needs to be extended but for now is setup to do some push and some pull and validate. Signed-off-by: vsoch <[email protected]>
the previous subdir_accessor.py had functions to get info, the index.json, and the package (archive) when given a specific URI. These are now added to the PackageRepo, meaning this class can now take a registry URI (host and username) on init so it functions similarity to the previous class. To be more consistent across the cli and classes, we now take the full registry <host>/<username> as the registry argument instead of separating them. It also did not make sense for the Mirror class to accept multiple channels when we are generally just taking one. Finally, the pull/push classes now are done with parallel tasks, and the structure of tasks is more consistent. There are tests for these new functions, and the upload of a latest for a package is removed. This means when we want to download latest, we determine the latest based on sorted versions and build numbers Signed-off-by: vsoch <[email protected]>
Signed-off-by: vsoch <[email protected]>
…json Signed-off-by: vsoch <[email protected]>
Signed-off-by: vsoch <[email protected]>
This file includes yanked packages, so we use it to do the mirror, although we upload and honor the current repodata.json. In addition, I am adding a helper script to run the registry so the developer does not need to remember / copy paste from the readme, and support for running in serial to the other mirror functions to be consistent! Signed-off-by: vsoch <[email protected]>
wolfv
reviewed
Feb 2, 2023
TODO.md
Outdated
|
|
||
| ### High Priority | ||
|
|
||
| #### Support the new .conda package format. |
There was a problem hiding this comment.
Just removed! Sorry forgot to check the TODOs before I opened!
It would be good to get this merged soon - since I've rebased the slow_upload branch locally I'm not sure I can do it again without messing up the git history here - so if/when we merge I can create a fresh clone/branch to work off of.
Signed-off-by: vsoch <[email protected]>
|
merged! |
|
I should update the github action to use this vs. the old slow upload branch. I'll do that later today and 🙏 that it works :) but it's in a much better state with tests now, so I am very confident! |
1 task
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Also opening this one for comparison - it looks like I might have based this original branch on the main one for conda_oci_mirror!