git(hub) integration feature #791
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sgfost
force-pushed
the
feat/git-mirroring
branch
from
73bee28 to
b25ba7b
Compare
sgfost
force-pushed
the
feat/git-mirroring
branch
from
b25ba7b to
a64a05e
Compare
sgfost
force-pushed
the
refactor/metadata-generation
branch
from
17f1895 to
9cbc597
Compare
sgfost
force-pushed
the
feat/git-mirroring
branch
from
a64a05e to
81edfd5
Compare
this API is responsible for managing a local git repository mirror for a comses codebase. PUBLIC release archives are commits/tags in the history. Release branches are created for each release and only added to if there is an update to metadata `build()` and `append_releases()` are the two main API methods which construct (or rebuild) a git repo and add new releases to the repo, respectively `update_release_branch()` will add a new commit containing changes to a release branch (and update main if they point to the same thing). This will mainly be used for updating metadata
the GithubApi provides access to auth and repository actions adds 3 huey (async) tasks for creating a mirror, updating a mirror, and updating metadata for a single release of a mirror
* /github page to describe the integration features * sidebar element on release detail page will show information about integration status for that codebase, and allow users with edit permissions to create a new mirror
sgfost
force-pushed
the
feat/git-mirroring
branch
2 times, most recently
from
a92874c to
8c42b35
Compare
sgfost
referenced
this pull request
Jan 23, 2025
- include in .env PATH and make docker-compose.yml target depend on .env - replace deprecated usage of self.assertEquals https://docs.python.org/3/whatsnew/3.12.html#id3
sgfost
force-pushed
the
feat/git-mirroring
branch
from
8c42b35 to
baed067
Compare
* use installation access tokens for user repos instead of user access tokens. this is a more secure workflow * add GithubIntegrationAppInstallation model for recording app installations (this will need to be created/updated using webhooks) * CodebaseGitMirror/"mirror" now refers to the local git repository * ^ can have multiple CodebaseGitRemote's which keep track of all the information needed to push to/archive from remote repositories TODO: re-implement views
sgfost
force-pushed
the
feat/git-mirroring
branch
from
baed067 to
d3432cb
Compare
this replaces the simple modal form to give better control over the feature
The distinguishing feature is whether the release has a non-null external_release_package This will be used to 'archive' or pull in releases made on github for synced repos currently, the release assets/package is not stored on the filesystem, instead relying on an external download url, and being only concerned with metadata
App installation tokens did not give access to get/create repos on a user's account. Still trying to avoid storing user access tokens (https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-a-user-access-token-for-a-github-app) so as a workaround, we will direct the submitter to create a new bare repo before continuing * add handler for webhook events for the github sync app * add form/wizard for linking a pre-existing github repo (archive only)
allows setting up a push/archive sync that will automatically have the generated git repo pushed to by providing an empty repo, as well as setting up an archive only sync by providing any github repo in both cases, the submitter needs to: - link their github account with the regular oauth flow (so we have a way to match users with a github account) - install the provided 'GitHub Sync' app on their github account with access to any repository that will be synced
and squash migrations "import(ing)" is the wording I keep finding describes the process the best other potential names and their issues: * publish - same name as the direct publishing, releases need to be manually published after they are imported * pull - git command that is not used * fetch - git command that is not used * ingest - ok and similar to import but not quite as clear
sgfost
force-pushed
the
feat/git-mirroring
branch
from
b4786f6 to
8c05280
Compare
* re-order and clarify the steps to set up a sync (app installation takes place after creating a repo so that permissions can be restricted) * fix push log to actual show useful information * when toggling push back on, do a build/update + push on the spot * better error/success messages
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
this relies on changes in #790
part 1 (1-way mirror):
adds a button that allows model submitters to create an auto-updating, read-only git repository archive which is hosted on a central organization
the 'mirror' git repository consists of commits for each release that are tagged and branched off so that metadata can be updated for individual releases without re-writing history
additions
library.fs.CodebaseGitRepositoryApi: functionality for building/updating a git repository from aCodebaselibrary.github.GithubApi: provides an interface overPyGithubfor interacting with repositories on githublibrary.github.GithubRepoNameValidator: providesvalidate()to make sure a repo name is valid and unusedmirror_codebase()andupdate_mirrored_codebase()huey tasks which call theCodebaseGitRepositoryApito build the git repo on the file system and thenGithubApito create/push to the remoteupdate_mirrored_release_metadata()huey task which is triggered when there is an update to codemeta, and updates the corresponding release branch in git after the submission package is rebuilt/github/configuration steps
comses-model-libraryorganization with the following permissions:.env.envsecrets/