Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow for fetching packages from remote git repositories #255

Merged
merged 3 commits into from
May 30, 2024

Conversation

nabuskey
Copy link
Collaborator

@nabuskey nabuskey commented May 14, 2024

This allows for fetching packages from remote git repositories.

The URL format follows kustomize remote url format.

fixes: #184

example:

idpbuilder create  --package-dir https://github.com/cnoe-io/idpbuilder//examples/basic/package1?version=v0.2.0

This should also reduce the number of git clone operations against local and remote repositories.

@nabuskey nabuskey force-pushed the remote-pkg branch 3 times, most recently from d289b8a to b64b627 Compare May 14, 2024 23:34
@nabuskey
Copy link
Collaborator Author

Seeing some weirdness around re-cloning repositories with depth 1. Seems like it's a bug fixed in a release later than what we are using. I'll need to update it.

@nabuskey nabuskey marked this pull request as draft May 15, 2024 20:47
Signed-off-by: Manabu McCloskey <[email protected]>
@nabuskey nabuskey marked this pull request as ready for review May 16, 2024 18:11
Copy link
Contributor

@nimakaviani nimakaviani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this left a few comments.

I think we should also update the README to go with it. Can you please make the necessary changes?

pkg/controllers/custompackage/controller.go Show resolved Hide resolved
pkg/cmd/get/secrets.go Outdated Show resolved Hide resolved
pkg/cmd/helpers/validation.go Outdated Show resolved Hide resolved
pkg/controllers/custompackage/controller.go Outdated Show resolved Hide resolved
Comment on lines +328 to 350
hash, push, err := addAllAndCommit(repo.Spec.Source.Path, tgtRepository)
if err != nil {
return fmt.Errorf("pushing to git: %w", err)
return fmt.Errorf("add and commit %w", err)
}

if push {
remoteUrl, err := util.FirstRemoteURL(tgtRepository)
if err != nil {
return fmt.Errorf("getting remote url %w", err)
}

logger.V(1).Info("pushing to remote url %s", remoteUrl)
err = pushToRemote(ctx, tgtRepository, creds)
if err != nil {
return fmt.Errorf("pushing to git: %w", err)
}

repo.Status.LatestCommit.Hash = hash.String()
return nil
}

repo.Status.LatestCommit.Hash = commit.String()
repo.Status.LatestCommit.Hash = hash.String()
return nil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is some redundancy between reconcileLocalRepoContent and reconcileRemoteRepoContent. Can we pull them into one function for clarity?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does looks similar but I don't think introducing another function would save us many lines. Maybe like 3 lines. I think it's pretty clear what these functions are doing right now and I'd rather keep it that way.

pkg/controllers/gitrepository/github.go Outdated Show resolved Hide resolved
pkg/controllers/localbuild/controller.go Outdated Show resolved Hide resolved
pkg/controllers/localbuild/controller.go Outdated Show resolved Hide resolved
pkg/controllers/run.go Outdated Show resolved Hide resolved
pkg/util/git_repository.go Show resolved Hide resolved
Signed-off-by: Manabu McCloskey <[email protected]>
@nabuskey
Copy link
Collaborator Author

nabuskey commented May 24, 2024

Updated Readme and addressed review comments. @nimakaviani

README.md Outdated
Idpbuilder supports specifying custom packages using the flag `--package-dir` flag.
This flag expects a directory (local or remote) containing ArgoCD application files.
In case of a remote directory, it must be a directory in a git repository,
and the URL format must be a [kustommize remote URL format](https://github.com/kubernetes-sigs/kustomize/blob/master/examples/remoteBuild.md).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo error: kustommize => kustomize
Why do you force the users to use a kustomize URL instead of HTTPS or SSH git urls ? What is the added value (if there is) to use kustomize url here ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gosh I really should configure my code spell plugin my spelling isn't good.

It actually supports git SSH format. The problem with just plain HTTPS format is that there's no way for us to know where the repository URL ends. For GitHub, it's almost always org-name/repo-name, but that's not the case for other git providers. This is the main reason for me to use the kustomize format.

README.md Show resolved Hide resolved
Signed-off-by: Manabu McCloskey <[email protected]>
Copy link
Contributor

@nimakaviani nimakaviani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

given that we have #264 to add a new flag to handle package references, I am going to approve and merge this, so we can unblock the rest of the work.

@nimakaviani nimakaviani merged commit f556285 into cnoe-io:main May 30, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create new git repository for idpbuilder examples and allow the use of URL for flag -p
3 participants