Skip to content

Commit

Permalink
Distro-Agnostic Zarf Registry Bootstrap (#329)
Browse files Browse the repository at this point in the history
Enables Zarf to init any K8s cluster given only a kube context.
  • Loading branch information
jeff-mccoy authored Mar 11, 2022
1 parent 8c3ab91 commit 53376d5
Show file tree
Hide file tree
Showing 63 changed files with 1,764 additions and 856 deletions.
3 changes: 3 additions & 0 deletions .golangci.yml
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
run:
timeout: 5m
skip-files:
- cli/internal/message/logo.go
linters:
enable-all: true
disable:
- exhaustivestruct
- lll
- stylecheck
- wrapcheck
- wsl
Expand Down
34 changes: 15 additions & 19 deletions .vscode/launch.json
Original file line number Diff line number Diff line change
@@ -1,21 +1,17 @@
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"name": "Launch",
"type": "go",
"request": "launch",
"mode": "auto",
"program": "${workspaceFolder}/cli",
"env": {},
"args": [
"connect",
"doom"
]
},

]
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"name": "Launch",
"type": "go",
"request": "launch",
"mode": "auto",
"program": "${workspaceFolder}/cli",
"env": {},
"args": ["init", "--confirm", "-l=trace"]
}
]
}
17 changes: 9 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,11 @@ Zarf runs on [a bunch of operating systems](./docs/supported-oses.md) and aims t

[![asciicast](https://asciinema.org/a/475530.svg)](https://asciinema.org/a/475530)

 
## Why is Zarf Needed?
Most of the software ecosystem assumes your systems have access to the internet. The world (for good reasons) has become more and more dependent upon Software as a Service (SaaS), which assumes a robust connection to the internet and a willingness to inherently trust 3rd party providers. Although this makes sense for most of the world, there are certain SECURE systems that must operate either fully disconnected, semi-disconnected, or might need the ability to disconnect in case of emergencies (like while under an active cyber attack). Although only a small percentage of systems, these SECURE systems make up some of the most vital systems in the world, such as Aerospace and Defense, Finance, Healthcare, Energy, Water, Sewage, and many Federal, Local, and State Government systems.

These SECURE systems need a way to continuously and securely deliver software too. Zarf exists to make it easy for open-source, commercial, and organic software solutions to be delivered to secure and disconnected systems. Although such systems might be small in number, they represent many of the most important systems in the world.

 
## Explain Zarf Like I'm Ten(ish)

Zarf allows you to bundle portions of "the internet" into a single package to be installed later following specific instructions. A Zarf package is really just a single file that includes everything you would need to manage a system or capability while fully disconnected. Think of a disconnected system as a system that always is or sometimes is on airplane mode.
Expand All @@ -27,7 +25,6 @@ Such packages also become highly distributable, as they can now run on edge, emb

Zarf makes DevSecOps for air gap possible.

 
<!--
##########
# This block is about LEARNING TO USE Zarf
Expand Down Expand Up @@ -84,9 +81,6 @@ Zarf makes DevSecOps for air gap possible.
</tbody>
</table>

&nbsp;


<!--
##########
# This block is about DEVELOPING Zarf
Expand Down Expand Up @@ -270,11 +264,18 @@ Zarf is written entirely in [go](https://go.dev/), except for a single 400Kb bin
- Currently the Registry and Git servers _are not HA_, see [#375](https://github.com/defenseunicorns/zarf/issues/376) and [#376](https://github.com/defenseunicorns/zarf/issues/376) for discussion on this
- In order to avoid TLS issues, Zarf binds to `127.0.0.1:319999` on each node as a [NodePort](https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport) to allow all nodes to access the pod(s) in the cluster
- Until [#306](https://github.com/defenseunicorns/zarf/pull/306) is merged, during helm install/upgrade a [Helm PostRender](https://helm.sh/docs/topics/advanced/#post-rendering) function is called to mutate images and [ImagePullSecrets](https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod) so the deployed resoures use the NodePort binding
- Once [#329](https://github.com/defenseunicorns/zarf/pull/329) is merged, Zarf will use a new injector system to bootstrap a new cluster:
- Zarf uses a custom injector system to bootstrap a new cluster. See the PR [#329](https://github.com/defenseunicorns/zarf/pull/329) and [ADR](docs/adr/0003-image-injection-into-remote-clusters-without-native-support.md) for more details on how we came to this solution. The general steps are listed below:
- Get a list images in the cluster
- Attempt to create an ephemeral pod using an image from the lsit
- A small rust binary that is compiled using [musl](https://www.musl-libc.org/) to keep the size the max binary size of ~ 672 KBs is injected into the pod
- The mini zarf registry binary and `docker:2` images are put in a tar archive and split into 512 KB chunks, larger sizes tended to cause latency issues on low-resource control planes
- An init container runs the rust binary to reassabmle and extract the zarf binary and registry image
- The container then starts and runs the zarf binary to host the registry image in an embedded docker registry
- After this the main docker registry chart is deployed, pulls the image from the ephemeral pod and destroys the created configmaps, pod and service
- After this the main docker registry chart is deployed, pulls the image from the ephemeral pod and destroys the created configmaps, pod and service

&nbsp;
### Zarf Architecture
![Architecture Diagram](./docs/architecture.drawio.svg)


[Source DrawIO](docs/architecture.drawio.svg)
4 changes: 3 additions & 1 deletion assets/charts/registry-values-seed.yaml
Original file line number Diff line number Diff line change
@@ -1,2 +1,4 @@
persistence:
enabled: false
image:
repository: "###ZARF_SEED_REGISTRY###/library/registry"
repository: "###ZARF_SEED_REGISTRY###/library/registry"
1 change: 1 addition & 0 deletions assets/charts/registry-values.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
persistence:
enabled: true
storageClass: "###ZARF_STORAGE_CLASS###"
size: 20Gi
image:
repository: "###ZARF_REGISTRY###/library/registry"
secrets:
Expand Down
202 changes: 0 additions & 202 deletions cli/LICENSE

This file was deleted.

6 changes: 6 additions & 0 deletions cli/cmd/connect.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ var (
connectResourceType string
connectLocalPort int
connectRemotePort int
cliOnly bool

connectCmd = &cobra.Command{
Use: "connect <REGISTRY|LOGGING|GIT>",
Expand All @@ -22,6 +23,10 @@ var (
target = args[0]
}
tunnel := k8s.NewTunnel(connectNamespace, connectResourceType, connectResourceName, connectLocalPort, connectRemotePort)
// If the cliOnly flag is false (default), enable auto-open
if !cliOnly {
tunnel.EnableAutoOpen()
}
tunnel.Connect(target, true)
},
}
Expand All @@ -34,4 +39,5 @@ func init() {
connectCmd.Flags().StringVar(&connectResourceType, "type", k8s.SvcResource, "Specify the resource type. E.g. type=svc or type=pod")
connectCmd.Flags().IntVar(&connectLocalPort, "local-port", 0, "(Optional, autogenerated if not provided) Specify the local port to bind to. E.g. local-port=42000")
connectCmd.Flags().IntVar(&connectRemotePort, "remote-port", 0, "Specify the remote port of the resource to bind to. E.g. remote-port=8080")
connectCmd.Flags().BoolVar(&cliOnly, "cli-only", false, "Disable browser auto-open")
}
Loading

0 comments on commit 53376d5

Please sign in to comment.