Skip to content

Commit

Permalink
Merge pull request #145 from chuckha/docs
Browse files Browse the repository at this point in the history 
Update docs
  • Loading branch information
Jessica Yao authored Nov 14, 2017
2 parents 0003c5f + 09f6fe9 commit ec42a23
Show file tree
Hide file tree
Showing 2 changed files with 64 additions and 47 deletions.
7 changes: 4 additions & 3 deletions docs/configuration.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Once the configs are loaded (in either case), Sonobuoy parses them and gathers d
| | Overview|Path on Cluster Node|[STANDALONE]<br>JSON example(s)|[CONTAINERIZED]<br>YAML manifest example(s)
|---|---|---|---|---|
|*Data configuration*| What Sonobuoy records, how, and where. |*ANY of the following*:<br>(1) `config.json` in the directory where `sonobuoy` is executed<br>(2) `/etc/sonobuoy/config.json`<br>(3) `$SONOBUOY_CONFIG`<br><br>|[`config.json`][10]|<br> [`examples/ksonnet/components/10-configmaps.jsonnet`][11]<br><br>*The jsonnet file is basically a wrapper for the `config.json` file, which allows it to be properly mounted onto the cluster's Sonobuoy pod.* <br><br>
|*Plugin configuration*|Settings for each plugin integration.|*ANY of the following*:<br>(1) `/etc/sonobuoy/plugins.d`<br>(2) `$HOME/.sonobuoy/plugins.d`<br>(3) `./plugins.d`<br>(4) `PluginSearchPath` (override from the data configuration) <br><br>| There is a YAML config for each plugin:<br>(1) [`plugins.d/e2e.yaml`][16]<br>(2)[`plugins.d/systemdlogs.yaml`][17]|<br>[`examples/ksonnet/components/10-configmaps.jsonnet`][11]<br><br>*Same comment about the jsonnet file as above.*
|*Plugin configuration*|Settings for each plugin integration.|*ANY of the following*:<br>(1) `/etc/sonobuoy/plugins.d`<br>(2) `$HOME/.sonobuoy/plugins.d`<br>(3) `./plugins.d`<br>(4) `PluginSearchPath` (override from the data configuration) <br><br>| There is a templatized YAML config for each plugin:<br>(1) [`plugins.d/e2e.tmpl`][16]<br>(2) [`plugins.d/systemd_logs.tmpl`][17]<br>(3) [`plugins.d/heptio-e2e.tmpl`][20]|<br>[`examples/ksonnet/components/10-configmaps.jsonnet`][11]<br><br>*Same comment about the jsonnet file as above.*
|*Kubernetes component definitions*|The various K8s objects that need to be defined for Sonobuoy to run as a containerized app.|N/A (manifest only)|N/A|The example splits this into two manifests:<br>(1) [`examples/ksonnet/components/00-rbac.jsonnet`][12]<br>(2) [`examples/ksonnet/components/20-pod.jsonnet`][13]|


Expand Down Expand Up @@ -188,7 +188,8 @@ However, ensure that your pod declaration has addressed the following aspects, w
[13]: /examples/ksonnet/components/20-pod.jsonnet
[14]: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#label-selectors
[15]: https://kubernetes.io/docs/tasks/configure-pod-container/configure-persistent-volume-storage/
[16]: /plugins.d/e2e.yaml
[17]: plugins.d/systemdlogs.yaml
[16]: /plugins.d/e2e.tmpl
[17]: /plugins.d/systemd_logs.tmpl
[18]: /examples/quickstart.yaml
[19]: /examples/ksonnet/components
[20]: /plugins.d/heptio-e2e.tmpl
104 changes: 60 additions & 44 deletions docs/plugins.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,22 +19,31 @@ There are two main components that specify plugin behavior:

*These configs are defined by the **end user**.*

1. **Plugin Definition**: A structured YAML definition that describes a plugin's features, method of launch, and other configurations.
1. **Plugin Definition**: Templatized YAML that defines a Kubernetes API resource like a Pod. Plugin metadata--such as a plugin's features, method of launch, and other configurations--are specified with annotations.

*This YAML is defined by the plugin **developer**, and can be taken as a given by the end user.*

The remainder of this document focuses on **Plugin Definition**.

## Plugin Definition

The *plugin definition* is a YAML file (raw or wrapped in a ConfigMap) that describes the core parts of your custom plugin. It should contain the following fields:
The *plugin definition* is a YAML file (raw or wrapped in a ConfigMap) that describes the core parts of your custom plugin. This YAML defines a Kubernetes API resource like a Pod.

| Field | Description | Example Values |
Some of the values in this YAML file are template variables that will be filled in by Sonobuoy. The template will have access to several variables:

| Variable | Source | Description |
| --- | --- | --- |
| `name` | A name that is used to identify the plugin (e.g. in the Plugin Selection described above). | "e2e" |
| `driver` | Sonobuoy implements *plugin drivers* that define different modes of operation.<br><br>(1) **"Job" driver**: The plugin will run on a single node (e.g. master).<br>(2) **"DaemonSet" driver**: The plugin runs on each cluster node.<br><br>You can find the implementations [here][7]. | "Job&#124;DaemonSet" |
| `resultType` | The name of the subdirectory that this plugin's results are saved in. With a `resultType` of "e2e", results are written into `plugins/e2e/...` (within the tarball output).<br><br>This value is typically the same as the plugin `name`. | "e2e" |
| `spec` | The Pod specification (e.g. network settings, container settings, volume definitions, etc.) | See [the parameter spec][4] below for reference. |
| `{{.SessionID}}` | Autogenerated by Sonobuoy | A generated token used to be able to uniquely identify all pieces of a single sonobuoy run. |
| `{{.Namespace}}` | Populated from the PluginNamespace value in config.json | The Kubernetes namespace the plugins will run in. This value is populated by the `PluginNamespace` value in `config.json`. |
| `{{.MasterAddress}}` | Autogenerated by Sonobuoy | The address the sonobuoy-workers use to communicate with the sonobuoy master process. It is dynamically generated. |

The plugin system relies on some annotations being present in a plugin definition:

| Annotation | Description | Example Values |
| --- | --- | --- |
| `sonobuoy-driver` | Sonobuoy implements *plugin drivers* that define different modes of operation.<br><br>(1) **"Job" driver**: The plugin runs on a single node (e.g. master).<br>(2) **"DaemonSet" driver**: The plugin runs on each cluster node.<br><br>You can find the implementations [here][7]. | "Job&#124;DaemonSet" |
| `sonobuoy-plugin` | A name that is used to identify the plugin (e.g. in the [Plugin Selection][18]). | "e2e", "systemd_logs" |
| `sonobuoy-result-type` | The name of the subdirectory that this plugin's results are saved in. With a `resultType` of "e2e", results are written into `plugins/e2e/...` (within the tarball output).<br><br>This value is typically the same as the `sonobuoy-plugin` annotation. | "e2e" |

Sonobuoy searches for these definitions in three locations by default:

Expand Down Expand Up @@ -76,63 +85,69 @@ To allow your two containers to interact (i.e. for the consumer to read and publ
### Example
Below is the Pod `spec` component from the [`e2e` plugin definition][9]. It follows the two-container-paradigm. One container runs the e2e tests and writes to a temporary folder, and the other container (a Sonobuoy worker) reads those results and sends them back to the master Sonobuoy instance.
Below is the Pod template from the [`e2e` plugin definition][9]. It follows the two-container-paradigm. One container runs the e2e tests and writes to a temporary folder, and the other container (a Sonobuoy worker) reads those results and sends them back to the master Sonobuoy instance.
See [Parameter Reference][4] for descriptions of each of the important config keys.
```
---
apiVersion: v1
kind: Pod
metadata:
annotations:
sonobuoy-driver: Job
sonobuoy-plugin: e2e
sonobuoy-result-type: e2e
labels:
component: sonobuoy
sonobuoy-run: '{{.SessionID}}'
tier: analysis
name: sonobuoy-e2e-job-{{.SessionID}}
namespace: '{{.Namespace}}'
spec:
serviceAccountName: sonobuoy-serviceaccount
tolerations:
- key: node-role.kubernetes.io/master
operator: Exists
effect: NoSchedule
- key: CriticalAddonsOnly
operator: Exists
restartPolicy: Never
containers:
# (1) This container runs the actual end-to-end tests.
- name: e2e
- env:
- name: E2E_FOCUS
value: Pods should be submitted and removed
image: gcr.io/heptio-images/kube-conformance:latest
imagePullPolicy: Always
# NOTE: Full conformance can take a while depending on your cluster size.
# As a result, only a single test is set atm to verify correctness.
# Operators that want the complete test results can comment out the
# env section.
env:
- name: E2E_FOCUS
value: "Pods should be submitted and removed"
name: e2e
volumeMounts:
- name: results
mountPath: /tmp/results
# (2) This container is the Sonobuoy worker that handles uploads.
- name: sonobuoy-worker
command:
- mountPath: /tmp/results
name: results
readOnly: false
- command:
- sh
- -c
- /sonobuoy worker global -v 5 --logtostderr
env:
- name: NODE_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: spec.nodeName
- name: RESULTS_DIR
value: /tmp/results
image: gcr.io/heptio-images/sonobuoy:latest
- name: MASTER_URL
value: '{{.MasterAddress}}'
- name: RESULT_TYPE
value: e2e
image: gcr.io/heptio-images/sonobuoy:master
imagePullPolicy: Always
name: sonobuoy-worker
volumeMounts:
- name: config
mountPath: /etc/sonobuoy
- name: results
mountPath: /tmp/results
- mountPath: /tmp/results
name: results
readOnly: false
restartPolicy: Never
serviceAccountName: sonobuoy-serviceaccount
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/master
- key: CriticalAddonsOnly
operator: Exists
volumes:
- name: results
emptyDir: {}
- name: config
configMap:
# This will be rewritten when the JobPlugin driver goes to launch the pod.
name: __SONOBUOY_CONFIGMAP__
- emptyDir: {}
name: results
```
### Parameter Reference
Expand All @@ -143,7 +158,7 @@ spec:
| `restartPolicy` | Specifies whether your plugin should retry on failure.
| `container.image`, `container.imagePullPolicy` | What and how often a new image should be pulled. |
| `container.env` | Set environmental variables here. These variables can be used to configure plugin behavior.<br><br>For DaemonSet plugins (e.g. `systemdlogs`), the `sonobuoy worker` consumer container needs a `NODE_NAME` variable to know which node the results should be uploaded for.|
| `container.volumeMounts`, `volumes` | <br>It is important to set up volumes and mount them properly, so that the container(s) can:<br><br>(1) **Get necessary configs** from locations like `/etc/sonobuoy`. While the Sonobuoy master creates the ConfigMap, the inbuilt Sonobuoy drivers actually substitute it into the `__SONOBUOY_CONFIGMAP__` pseudo-template. The ConfigMap's name isn't predictable because it only lasts for one run.<br><br>(2) **Write results locally**, such that the Sonobuoy worker can find the files it needs to upload to the Sonobuoy master. Typically the same `emptyDir` `results` directory is shared by both the "plugin" container and the Sonobuoy worker container.<br><br> |
| `container.volumeMounts`, `volumes` | **Write results locally**, such that the Sonobuoy worker can find the files it needs to upload to the Sonobuoy master. Typically the same `emptyDir` `results` directory is shared by both the "plugin" container and the Sonobuoy worker container.<br><br> |
## Available Plugins
Expand All @@ -165,7 +180,7 @@ See the [`/build`][14] directory for the source code used to build these plugins
[6]: img/sonobuoy-plugins.png
[7]: /pkg/plugin/driver
[8]: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
[9]: /plugins.d/e2e.yaml
[9]: /plugins.d/e2e.tmpl
[10]: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#taints-and-tolerations-beta-feature
[11]: /plugins.d/systemdlogs.yaml
[12]: /examples/quickstart.yaml
Expand All @@ -174,3 +189,4 @@ See the [`/build`][14] directory for the source code used to build these plugins
[15]: conformance-testing.md#integration-with-sonobuoy
[16]: https://github.com/heptio/sonobuoy-plugin-systemd-logs
[17]: https://github.com/heptio/kube-conformance
[18]: /docs/configuration.md#plugin-configuration

0 comments on commit ec42a23

Please sign in to comment.