Flux v2 enable repo integration

[Callisto13]

Why do you want this feature?
Flux v2 (Gitops Toolkit) is 🔥 🔥 and very close to replacing v1.

What feature/behaviour/change do you want?
There should be way to opt-in to bootstrap a v2 repo when I run eksctl enable repo.

I am not sure that we will be able to use the existing flags/config opts which is a bit of a pain: it would be cool to simply give a --use-v2 or whatever and have everything else be the same, but the existing flags don't really line up with the flux cli. This is not a problem for people who only ever bootstrap through eksctl but weird for people who are used to the flux api.
I don't really want to have to go back and have those flags make sense after we remove the v1 integration as that would be super breaking.
Maybe we just have a whole new command and save ourselves a headache? Like eksctl enable gotk?

Honestly, Flux v2 is 1000x simpler to set up than v1 that having an eksctl command is somewhat redundant. The enable repo command is useful because Flux v1 needs a lot of fiddling to get going. v2 is like 1 step (3 if you want to do checks before and after). Being able to configure as part of a general cluster create + file is the main use for eksctl + v2.

Things to consider:

• That flag thing ^^
• The initial set of flags we want to pass through eksctl. We don't want to end up basically exposing all the flux flags on eksctl and pass them through. That would be very irritating to keep in step. I think in the current integration we have the option to pass in a list of extra args?
• Do we even want to have flags? Config only?
• Existing profiles need to be re-written to be compatible with Flux v2
• Note: don't forget about other actions (eg part of the cluster deletion should remove gitops things)

I will start coming up with a UX design for this soon and post here when I have ideas. If anyone else would like to pitch in to how this should look, go nuts!

[Callisto13]

Since none of the new flags line up with the old enable repo command, we are going have a brand new subcommand. What will that subcommand be called? No idea, much discussion needs to be had. But that doesn't mean I can't implement in parallel to those discussions while we figure it out.

So in lieu of a real decision I am calling this eksctl enable gotk (gitops toolkit) for now. (Don't @ me, I need to use something.)

For a first pass I am going to implement a bare set of configuration for running flux bootstrap against a cluster, essentially what the example command in Flux's getting started docs do.

So initial UX would be:

Command (if we want flags at all 🤷‍♀️ giving that we are moving towards a config only world):

eksctl enable gotk \
        --vcs <github/gitlab> \                # required - I know those things are not VCSs but I have not come up with a snappier name yet
        --owner <user/org name> \              # required
        --repo <name of repository, not url> \ # required
        --branch <branch name> \
        --namespace <namespace> \
        --path <dir in repo to store flux stuff>

Config:

git:
  gotk:
    vcs: github
    owner: gitops
    repo: flux-repo
    namespace: flux-system
    branch: main
    path: "flux/"

(So that (git.gotk) looks weird to me but that is probably because I have chosen a stupid placeholder name. I know a lot of discussion went into the git.repo, git.operator and git.bootstrapProfile naming, and I don't want to rock the boat.)

There are of course lots more options which can go into configuring Flux, but I think these are fine to start with and we can create new issues for exposing new functionality. And as I said in the description above, we may not even want to expose all of them. If I try to do everything at once I will invariably miss something, and y'all will have to slog through a mass PR.

Implementation-wise, I am going to simply shell out to Flux. While it is a bummer that we have to ensure users have that binary installed, this is something we already insist on for the aws-cli and kubectl(edit: this is garbage because of course people will need to install flux cli otherwise how will they even use it after bootstrapping 🤦‍♀️ ). Also, the Flux code doesn't have any sort of library (edit: not yet, might ask them about it later), so it is this or copy-pasting ~300 lines of code from that codebase which is daft.

[Callisto13]

Okay so there is a way where we can have eksctl enable repo: we add a flag --use-v2 (or whatever) and that determines which other flags/config options apply.

Pros:

• The UX is nice (Sortof. Users would not be able to throw that flag in with an old command, they would have to alter the other flags if they wanted to switch, not that many will be going back and forth, but maybe in scripts? idk)
• We can keep using enable repo and not have to think of anything else

Cons:

• Code will be a bit gross
• The bit in brackets from my first "pro" above

🤔 is this all just a bit confusing?

Like if a user was going from a v1 config:

git:
  repo:
    url: "[email protected]:myorg/cluster-21.git"
    branch: master
    fluxPath: "flux/"
    user: "gitops"
    email: "[email protected]"
    namespace: flux

To a v2 config, it is not a simple matter of adding the v2 switch, other keys have to change:

git:
  repo:
    v2: true
    vcs: github
    repo: cluster-21 
    branch: main
    path: "flux/"
    owner: gitops
    namespace: flux

Maybe that is fine...? Or is it actually terrible and I'm just tired?