Skip to content

Commit

Permalink
docs: port over existing docs to hugo site (#334)
Browse files Browse the repository at this point in the history
## Description

Port over existing repo docs to the Hugo site.

AC: Ensure that all content is accurately and completely transferred,
maintaining its original meaning and structure. The documentation should
be consistently formatted, adhering to the Hugo site’s style guidelines,
with all links, images, and code snippets functioning correctly.
Additionally, the documentation must be easily navigable, responsive
across various devices, and fully compatible with different browsers,
ensuring a seamless user experience.

Will need to update
[`README.md`](https://github.com/defenseunicorns/pepr/blob/b6aadddc570fb5ac63e3c98a72eb81b0bf98b078/README.md?plain=1#L3)
when site goes live


## Related Issue

Fixes #332 
<!-- or -->
Relates to #324 (If any docs change, they need to change here)

## Type of change

- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [x] Other (security config, docs update, etc)

## Checklist before merging

- [x] Test, docs, adr added or updated as needed
- [x] [Contributor Guide
Steps](https://github.com/defenseunicorns/pepr/blob/main/CONTRIBUTING.md#submitting-a-pull-request)
followed

---------

Signed-off-by: Case Wylie <[email protected]>
  • Loading branch information
cmwylie19 authored Oct 23, 2023
1 parent b6aaddd commit 286e468
Show file tree
Hide file tree
Showing 8 changed files with 659 additions and 1 deletion.
2 changes: 1 addition & 1 deletion website/content/en/docs/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@ menu: {main: {weight: 1}}

## What is Pepr?

Let us know...
Pepr functions as both a Mutating and Validating Admission Webhook, offering a fluent API that facilitates real-time resource alterations and the establishment and enforcement of policies on Kubernetes resources. Its design is not only lightweight but also includes an integrated Kubernetes client, ensuring a straightforward and effective user experience for additional interactions with Kubernetes.
65 changes: 65 additions & 0 deletions website/content/en/docs/cli.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
---
title: CLI
linkTitle: CLI
---

# Pepr CLI

## `pepr init`

Initialize a new Pepr Module.

**Options:**

- `-l, --log-level [level]` - Log level: debug, info, warn, error (default: "info")
- `--skip-post-init` - Skip npm install, git init and VSCode launch

---

## `pepr update`

Update the current Pepr Module to the latest SDK version and update the global Pepr CLI to the same version.

**Options:**

- `-l, --log-level [level]` - Log level: debug, info, warn, error (default: "info")
- `--skip-template-update` - Skip updating the template files

---

## `pepr dev`

Connect a local cluster to a local version of the Pepr Controller to do real-time debugging of your module. Note
the `pepr dev` assumes a K3d cluster is running by default. If you are working with Kind or another docker-based
K8s distro, you will need to pass the `--host host.docker.internal` option to `pepr dev`. If working with a remote
cluster you will have to give Pepr a host path to your machine that is reachable from the K8s cluster.

**Options:**

- `-l, --log-level [level]` - Log level: debug, info, warn, error (default: "info")
- `-h, --host [host]` - Host to listen on (default: "host.k3d.internal")
- `--confirm` - Skip confirmation prompt

---

## `pepr deploy`

Deploy the current module into a Kubernetes cluster, useful for CI systems. Not recommended for production use.

**Options:**

- `-l, --log-level [level]` - Log level: debug, info, warn, error (default: "info")
- `-i, --image [image]` - Override the image tag
- `--confirm` - Skip confirmation prompt

---

## `pepr build`

Create a [zarf.yaml](https://zarf.dev) and K8s manifest for the current module. This includes everything needed to deploy Pepr and the current module into production environments.

**Options:**

- `-r, --registry-info [<registry>/<username>]` - Registry Info: Image registry and username. Note: You must be signed into the registry
- `-rm, --rbac-mode [admin|scoped]` - Rbac Mode: admin, scoped (default: admin)
- `-l, --log-level [level]` - Log level: debug, info, warn, error (default: "info")
File renamed without changes.
238 changes: 238 additions & 0 deletions website/content/en/docs/concepts.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,238 @@
---
title: Concepts
linkTitle: Concepts
---

# Pepr Module

Each Pepr Module is it's own Typescript project, produced by [`pepr init`](./cli.md#pepr-init). Typically a module is maintained by a unique group or system. For example, a module for internal [Zarf](https://zarf.dev/) mutations would be different from a module for [Big Bang](https://p1.dso.mil/products/big-bang). An important idea with modules is that they are _wholly independent of one another_. This means that 2 different modules can be on completely different versions of Pepr and any other dependencies; their only interaction is through the standard K8s interfaces like any other webhook or controller.

## Module development lifecycle

1. **Create the module**:

Use [`pepr init`](./cli.md#pepr-init) to generate a new module.

1. **Quickly validate system setup**:

Every new module includes a sample Pepr Capability called `HelloPepr`. By default,
this capability is deployed and monitoring the `pepr-demo` namespace. There is a sample
yaml also included you can use to see Pepr in your cluster. Here's the quick steps to do
that after `pepr init`:

```bash
# cd to the newly-created Pepr module folder
cd my-module-name

# If you don't already have a local K8s cluster, you can set one up with k3d
npm run k3d-setup

# Launch pepr dev mode
# If using another local K8s distro instead of k3d, use `pepr dev --host host.docker.internal`
pepr dev

# From another terminal, apply the sample yaml
kubectl apply -f capabilities/hello-pepr.samples.yaml

# Verify the configmaps were transformed using kubectl, k9s or another tool
```

1. **Create your custom Pepr Capabilities**

Now that you have confirmed Pepr is working, you can now create new [capabilities](./capabilities.md). You'll also want to disable the `HelloPepr` capability in your module (`pepr.ts`) before pushing to production. You can disable by commenting out or deleting the `HelloPepr` variable below:

```typescript
new PeprModule(cfg, [
// Remove or comment the line below to disable the HelloPepr capability
HelloPepr,

// Your additional capabilities go here
]);
```

_Note: if you also delete the `capabilities/hello-pepr.ts` file, it will be added again on the next [`pepr update`](./cli.md#pepr-update) so you have the latest examples usages from the Pepr SDK. Therefore, it is sufficient to remove the entry from your `pepr.ts` module
config._

1. **Build and deploy the Pepr Module**

Most of the time, you'll likely be iterating on a module with `pepr dev` for real-time feedback and validation Once you are ready to move beyond the local dev environment, Pepr provides deployment and build tools you can use.

`pepr deploy` - you can use this command to build your module and deploy it into any K8s cluster your current `kubecontext` has access to. This setup is ideal for CI systems during testing, but is not recommended for production use. See [`pepr deploy`](./cli.md#pepr-deploy) for more info.

## Advanced Module Configuration

By default, when you run `pepr init`, the module is not configured with any additional options. Currently, there are 3 options you can configure:

- `deferStart` - if set to `true`, the module will not start automatically. You will need to call `start()` manually. This is useful if you want to do some additional setup before the module controller starts. You can also use this to change the default port that the controller listens on.

- `beforeHook` - an optional callback that will be called before every request is processed. This is useful if you want to do some additional logging or validation before the request is processed.

- `afterHook` - an optional callback that will be called after every request is processed. This is useful if you want to do some additional logging or validation after the request is processed.

You can configure each of these by modifying the `pepr.ts` file in your module. Here's an example of how you would configure each of these options:

```typescript
const module = new PeprModule(
cfg,
[
// Your capabilities go here
],
{
deferStart: true,

beforeHook: req => {
// Any actions you want to perform before the request is processed, including modifying the request.
},

afterHook: res => {
// Any actions you want to perform after the request is processed, including modifying the response.
},
}
);

// Do any additional setup before starting the controller
module.start();
```

# Capabilities

A capability is set of related [actions](./actions.md) that work together to achieve a specific transformation or operation on Kubernetes resources. Capabilities are user-defined and can include one or more actions. They are defined within a Pepr module and can be used in both MutatingWebhookConfigurations and ValidatingWebhookConfigurations. A Capability can have a specific scope, such as mutating or validating, and can be reused in multiple Pepr modules.

When you [`pepr init`](./cli.md#pepr-init), a `capabilities` directory is created for you. This directory is where you will define your capabilities. You can create as many capabilities as you need, and each capability can contain one or more actions. Pepr also automatically creates a `HelloPepr` capability with a number of example actions to help you get started.

## Creating a Capability

Define a new capability can be done via a [VSCode Snippet](https://code.visualstudio.com/docs/editor/userdefinedsnippets) generated during [`pepr init`](./cli.md#pepr-init).

1. Create a new file in the `capabilities` directory with the name of your capability. For example, `capabilities/my-capability.ts`.

1. Open the new file in VSCode and type `create` in the file. A suggestion should prompt you to generate the content from there.

https://user-images.githubusercontent.com/882485/230897379-0bb57dff-9832-479f-8733-79e103703135.mp4

_If you prefer not to use VSCode, you can also modify or copy the `HelloPepr` capability to meet your needs instead._


## Reusable Capabilities

Pepr has an NPM org managed by Defense Unicorns, `@pepr`, where capabilities are published for reuse in other Pepr Modules. You can find a list of published capabilities [here](https://www.npmjs.com/search?q=@pepr). You can also publish your own Pepr capabilities to NPM and import them. A couple of things you'll want to be aware of:

- Reuseable capability versions should use the format `0.x.x` or `0.12.x` as examples to determine compatibility with other reusable capabilities. Before `1.x.x`, we recommend binding to `0.x.x` if you can for maximum compatibility.

- `pepr.ts` will still be used for local development, but you'll need to also publish an `index.ts` that exports your capabilities. When you build & publish the capability to NPM, you can use `npx pepr build -e index.ts` to generate the code needed for reuse by other Pepr modules.

- See [Pepr Istio](https://github.com/defenseunicorns/pepr-istio) for an example of a reusable capability.


# Actions

An action is a discrete set of behaviors defined in a single function that acts on a given Kubernetes GroupVersionKind (GVK) passed in during the admission controller lifecycle. Actions are the atomic operations that are performed on Kubernetes resources by Pepr.

For example, an action could be responsible for adding a specific label to a Kubernetes resource, or for modifying a specific field in a resource's metadata. Actions can be grouped together within a Capability to provide a more comprehensive set of operations that can be performed on Kubernetes resources.

Actions are `Mutate()`, `Validate()`, or `Watch()`. Both Mutate and Validate actions run during the admission controller lifecycle, while Watch actions run in a separate controller that tracks changes to resources, including existing resources.

Let's look at some example actions that are included in the `HelloPepr` capability that is created for you when you [`pepr init`](./cli.md#pepr-init):

---

In this first example, Pepr is adding a label and annotation to a ConfigMap with tne name `example-1` when it is created. Comments are added to each line to explain in more detail what is happening.

```ts
// When(a.<Kind>) filters which GroupVersionKind (GVK) this action should act on.
When(a.ConfigMap)
// This limits the action to only act on new resources.
.IsCreated()
// This limits the action to only act on resources with the name "example-1".
.WithName("example-1")
// Mutate() is where we define the actual behavior of this action.
.Mutate(request => {
// The request object is a wrapper around the K8s resource that Pepr is acting on.
request
// Here we are adding a label to the ConfigMap.
.SetLabel("pepr", "was-here")
// And here we are adding an annotation.
.SetAnnotation("pepr.dev", "annotations-work-too");

// Note that we are not returning anything here. This is because Pepr is tracking the changes in each action automatically.
});
```

---

In this example, a Validate action rejects any ConfigMap in the `pepr-demo` namespace that has no data.

```ts
When(a.ConfigMap)
.IsCreated()
.InNamespace("pepr-demo")
// Validate() is where we define the actual behavior of this action.
.Validate(request => {
// If data exists, approve the request.
if (request.Raw.data) {
return request.Approve();
}

// Otherwise, reject the request with a message and optional code.
return request.Deny("ConfigMap must have data");
});
```

---

In this example, a Watch action on the name and phase of any ConfigMap.Watch actions run in a separate controller that tracks changes to resources, including existing resources so that you can react to changes in real-time. It is important to note that Watch actions are not run during the admission controller lifecycle, so they cannot be used to modify or validate resources. They also may run multiple times for the same resource, so it is important to make sure that your Watch actions are idempotent. In a future release, Pepr will provide a better way to control when a Watch action is run to avoid this issue.

```ts
When(a.ConfigMap)
// Watch() is where we define the actual behavior of this action.
.Watch((cm, phase) => {
Log.info(cm, `ConfigMap ${cm.metadata.name} was ${phase}`);
});
```

```
There are many more examples in the `HelloPepr` capability that you can use as a reference when creating your own actions. Note that each time you run [`pepr update`](./cli.md#pepr-update), Pepr will automatically update the `HelloPepr` capability with the latest examples and best practices for you to reference and test directly in your Pepr Module.
```

# Pepr Store: A Lightweight Key-Value Store for Pepr Modules

The nature of admission controllers and general watch operations (the `Mutate`, `Validate` and `Watch` actions in Pepr) make some types of complex and long-running operations difficult. There are also times when you need to share data between different actions. While you could manually create your own K8s resources and manage their cleanup, this can be very hard to track and keep performant at scale.

The Pepr Store solves this by exposing a simple, [Web Storage API](https://developer.mozilla.org/en-US/docs/Web/API/Storage)-compatible mechanism for use within capabilities. Additionally, as Pepr runs multiple replicas of the admission controller along with a watch controller, the Pepr Store provides a unique way to share data between these different instances automatically.

Each Pepr Capability has a `Store` instance that can be used to get, set and delete data as well as subscribe to any changes to the Store. Behind the scenes, all capability store instances in a single Pepr Module are stored within a single CRD in the cluster. This CRD is automatically created when the Pepr Module is deployed. Care is taken to make the read and write operations as efficient as possible by using K8s watches, batch processing and patch operations for writes.

## Key Features

- **Asynchronous Key-Value Store**: Provides an asynchronous interface for storing small amounts of data, making it ideal for sharing information between various actions and capabilities.
- **Web Storage API Compatibility**: The store's API is aligned with the standard [Web Storage API](https://developer.mozilla.org/en-US/docs/Web/API/Storage), simplifying the learning curve.
- **Real-time Updates**: The `.subscribe()` and `onReady()` methods enable real-time updates, allowing you to react to changes in the data store instantaneously.

- **Automatic CRD Management**: Each Pepr Module has its data stored within a single Custom Resource Definition (CRD) that is automatically created upon deployment.
- **Efficient Operations**: Pepr Store uses Kubernetes watches, batch processing, and patch operations to make read and write operations as efficient as possible.

## Quick Start

```typescript
// Example usage for Pepr Store
Store.setItem("example-1", "was-here");
Store.setItem("example-1-data", JSON.stringify(request.Raw.data));
Store.onReady(data => {
Log.info(data, "Pepr Store Ready");
});
const unsubscribe = Store.subscribe(data => {
Log.info(data, "Pepr Store Updated");
unsubscribe();
});
```

## API Reference

### Methods

- `getItem(key: string)`: Retrieves a value by its key. Returns `null` if the key doesn't exist.
- `setItem(key: string, value: string)`: Sets a value for a given key. Creates a new key-value pair if the key doesn't exist.
- `removeItem(key: string)`: Deletes a key-value pair by its key.
- `clear()`: Clears all key-value pairs from the store.
- `subscribe(listener: DataReceiver)`: Subscribes to store updates.
- `onReady(callback: DataReceiver)`: Executes a callback when the store is ready.
File renamed without changes.
74 changes: 74 additions & 0 deletions website/content/en/docs/metrics.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
---
title: Metrics
linkTitle: Metrics
---

# `/metrics` Endpoint Documentation

The `/metrics` endpoint provides metrics for the application that are collected via the `MetricsCollector` class. It uses the `prom-client` library and performance hooks from Node.js to gather and expose the metrics data in a format that can be scraped by Prometheus.

## Metrics Exposed

The `MetricsCollector` exposes the following metrics:

- `pepr_errors`: A counter that increments when an error event occurs in the application.
- `pepr_alerts`: A counter that increments when an alert event is triggered in the application.
- `pepr_Mutate`: A summary that provides the observed durations of mutation events in the application.
- `pepr_Validate`: A summary that provides the observed durations of validation events in the application.

## API Details

**Method:** GET

**URL:** `/metrics`

**Response Type:** text/plain

**Status Codes:**
- 200 OK: On success, returns the current metrics from the application.

**Response Body:**
The response body is a plain text representation of the metrics data, according to the Prometheus exposition formats. It includes the metrics mentioned above.

## Examples

### Request

```plaintext
GET /metrics
```

### Response
```plaintext
`# HELP pepr_errors Mutation/Validate errors encountered
# TYPE pepr_errors counter
pepr_errors 5
# HELP pepr_alerts Mutation/Validate bad api token received
# TYPE pepr_alerts counter
pepr_alerts 10
# HELP pepr_Mutate Mutation operation summary
# TYPE pepr_Mutate summary
pepr_Mutate{quantile="0.01"} 100.60707900021225
pepr_Mutate{quantile="0.05"} 100.60707900021225
pepr_Mutate{quantile="0.5"} 100.60707900021225
pepr_Mutate{quantile="0.9"} 100.60707900021225
pepr_Mutate{quantile="0.95"} 100.60707900021225
pepr_Mutate{quantile="0.99"} 100.60707900021225
pepr_Mutate{quantile="0.999"} 100.60707900021225
pepr_Mutate_sum 100.60707900021225
pepr_Mutate_count 1
# HELP pepr_Validate Validation operation summary
# TYPE pepr_Validate summary
pepr_Validate{quantile="0.01"} 201.19413900002837
pepr_Validate{quantile="0.05"} 201.19413900002837
pepr_Validate{quantile="0.5"} 201.2137690000236
pepr_Validate{quantile="0.9"} 201.23339900001884
pepr_Validate{quantile="0.95"} 201.23339900001884
pepr_Validate{quantile="0.99"} 201.23339900001884
pepr_Validate{quantile="0.999"} 201.23339900001884
pepr_Validate_sum 402.4275380000472
pepr_Validate_count 2
```
Loading

0 comments on commit 286e468

Please sign in to comment.