diff --git a/charts/hashr/.helmignore b/charts/hashr/.helmignore new file mode 100644 index 00000000..0e8a0eb3 --- /dev/null +++ b/charts/hashr/.helmignore @@ -0,0 +1,23 @@ +# Patterns to ignore when building packages. +# This supports shell glob matching, relative path matching, and +# negation (prefixed with !). Only one pattern per line. +.DS_Store +# Common VCS dirs +.git/ +.gitignore +.bzr/ +.bzrignore +.hg/ +.hgignore +.svn/ +# Common backup files +*.swp +*.bak +*.tmp +*.orig +*~ +# Various IDEs +.project +.idea/ +*.tmproj +.vscode/ diff --git a/charts/hashr/Chart.lock b/charts/hashr/Chart.lock new file mode 100644 index 00000000..2b96170e --- /dev/null +++ b/charts/hashr/Chart.lock @@ -0,0 +1,6 @@ +dependencies: +- name: postgresql + repository: https://charts.bitnami.com/bitnami + version: 14.3.3 +digest: sha256:803fc388f1186ca5e0bf7af8597a7f94714bfe2fb4536d3ab2136e0b5ce1e59c +generated: "2024-04-24T19:58:19.38937249Z" diff --git a/charts/hashr/Chart.yaml b/charts/hashr/Chart.yaml new file mode 100644 index 00000000..2173e8de --- /dev/null +++ b/charts/hashr/Chart.yaml @@ -0,0 +1,27 @@ +apiVersion: v2 +name: hashr +description: A Helm chart for HashR Kubernetes deployments. +version: 0.1.0 +type: application +keywords: +- hashr +- dfir +- analysis +- security +home: "https://github.com/google/hashr" +dependencies: +- condition: postgresql.enabled + name: postgresql + version: 14.3.3 + repository: https://charts.bitnami.com/bitnami +maintainers: + - name: Open Source DFIR + email: osdfir-maintainers@googlegroups.com + url: https://github.com/google/osdfir-infrastructure +sources: +- https://github.com/google/hashr +- https://github.com/google/osdfir-infrastructure +appVersion: "latest" +annotations: + category: Security + licenses: Apache-2.0 diff --git a/charts/hashr/README.md b/charts/hashr/README.md new file mode 100644 index 00000000..0bf5f9d6 --- /dev/null +++ b/charts/hashr/README.md @@ -0,0 +1,251 @@ + +# HashR Helm Chart + +HashR allows you to build your own hash sets based on your data sources. It's a +tool that extracts files and hashes out of input sources (e.g. raw disk image, +GCE disk image, ISO file, Windows update package, .tar.gz file, etc.). + +[Overview of HashR](https://github.com/google/hashr) + +[Chart Source Code](https://github.com/google/osdfir-infrastructure) + +## TL;DR + +```console +helm repo add osdfir-charts https://google.github.io/osdfir-infrastructure/ +helm install my-release osdfir-charts/hashr +``` + +> **Tip**: To quickly get started with a local cluster, see +[minikube install docs](https://minikube.sigs.k8s.io/docs/start/). + +## Introduction + +This chart bootstraps a [HashR](https://github.com/google/hashr/tree/main/docker) +deployment on a [Kubernetes](https://kubernetes.io) cluster using the +[Helm](https://helm.sh) package manager. + +## Prerequisites + +- Kubernetes 1.19+ +- Helm 3.2.0+ +- PV provisioner support in the underlying infrastructure + +## Installing the Chart + +The first step is to add the repo and then update to pick up any new changes. + +```console +helm repo add osdfir-charts https://google.github.io/osdfir-infrastructure/ +helm repo update +``` + +To install the chart, specify any release name of your choice. For example, +using `my-release` as the release name, run: + +```console +helm install my-release osdfir-charts/hashr +``` + +The command deploys a PostgreSQL instance to the Kubernetes cluster and +schedules a Kubernetes CronJob to run the configures HashR job. Results of the +HashR job are exported to the PostgreSQL instance. +The [Parameters](#parameters) section lists the parameters that can be +configured during installation. + +## Installing for Production + +Pull the chart locally then cd into `/hashr` and review the `values.yaml` file +for a list of values that will be used for production. + +```console +helm pull osdfir-charts/hashr --untar +``` + +### Configure the HashR importers + +HashR provides different importers. Each importer has its own CronJob and can be +configured separately. Enable and configure all importers you want to use in the +`hashr.importers` section of the `values.yaml` file. + +Ensure that you have setup all requirements for the importers defined in the +HashR project. See [HashR importers](https://github.com/google/hashr?tab=readme-ov-file#setting-up-importers) +for more details. + +### Install chart + +Install the chart with the values in `values.yaml`, then using a release name +such as `my-release`, run: + +```console +helm install my-release ../hashr -f values.yaml +``` + +## Add data for HashR to process + +The HashR CronJob has access to the Persistent Volume (PVC) at `/mnt/hashrvolume`. +See the [Persistence](#persistence) section for more details. + +Each importer that needs local files to procress (e.g. deb, zip, iso9660, etc) +will look in a subfolder of `/mnt/hashrvolume/data/` for files to +process. E.g. `/mnt/hashrvolume/data/deb/`for the deb importer. + +To add data for processing use the `kubectl cp` command and the +`hashr-data-manager` pod. + +Example: + +```console +kubectl cp /deb my-release-hashr-data-manager:/mnt/hashrvolume/data/ +``` + +## Uninstalling the Chart + +To uninstall/delete a Helm deployment with a release name of `my-release`: + +```console +helm uninstall my-release +``` + +> **Tip**: Please update based on the release name chosen. You can list all +releases using `helm list` + +The command removes all the Kubernetes components but Persistent Volumes (PVC) +associated with the chart and deletes the release. + +To delete the PVC's associated with a release name of `my-release`: + +```console +kubectl delete pvc -l release=my-release +``` + +> **Note**: Deleting the PVC's will delete HashR/PostgreSQL data as well. +Please be cautious before doing it. + +## Parameters + +### Global parameters + +| Name | Description | Value | +| ------------------------------- | -------------------------------------------------------------------------------------------- | ------- | +| `global.timesketch.enabled` | Enables the Timesketch deployment (only used in the main OSDFIR Infrastructure Helm chart) | `false` | +| `global.timesketch.servicePort` | Timesketch service port (overrides `timesketch.service.port`) | `nil` | +| `global.turbinia.enabled` | Enables the Turbinia deployment (only used within the main OSDFIR Infrastructure Helm chart) | `false` | +| `global.turbinia.servicePort` | Turbinia API service port (overrides `turbinia.service.port`) | `nil` | +| `global.yeti.enabled` | Enables the Yeti deployment (only used in the main OSDFIR Infrastructure Helm chart) | `false` | +| `global.yeti.servicePort` | Yeti API service port (overrides `yeti.api.service.port`) | `nil` | +| `global.existingPVC` | Existing claim for HashR persistent volume (overrides `persistent.name`) | `""` | +| `global.storageClass` | StorageClass for the HashR persistent volume (overrides `persistent.storageClass`) | `""` | + +### HashR image configuration + +| Name | Description | Value | +| ------------------------ | ------------------------------------------------------------- | ------------------------------------------------------- | +| `image.repository` | HashR image repository | `us-docker.pkg.dev/osdfir-registry/hashr/release/hashr` | +| `image.pullPolicy` | HashR image pull policy | `IfNotPresent` | +| `image.tag` | Overrides the image tag whose default is the chart appVersion | `latest` | +| `image.imagePullSecrets` | Specify secrets if pulling from a private repository | `[]` | + +### HashR Configuration Paramters + + +### Enable/Disable HashR importers + +| Name | Description | Value | +| --------------------------------------- | ------------------------------------------------------------------------------------------------------- | ------------------- | +| `hashr.importers.aws.enabled` | Enables the AWS importer | `false` | +| `hashr.importers.aws.schedule` | sets the CronJob schedule times | `0 9 * * 1` | +| `hashr.importers.gcp.enabled` | Enables the GCP importer | `false` | +| `hashr.importers.gcp.schedule` | sets the CronJob schedule times | `0 3 * * 1` | +| `hashr.importers.gcp.gcp_projects` | sets a comma separated list of cloud projects containing disk images | `""` | +| `hashr.importers.gcp.hashr_gcp_project` | sets GCP project that will be used to store copy of disk images for processing and also run Cloud Build | `""` | +| `hashr.importers.gcp.hashr_gcs_bucket` | sets GCS bucket that will be used to store output of Cloud Build (disk images in .tar.gz format) | `""` | +| `hashr.importers.targz.enabled` | Enables the tar.gz importer | `false` | +| `hashr.importers.targz.schedule` | sets the CronJob schedule times | `0 3 * * 2` | +| `hashr.importers.windows.enabled` | Enables the Windows importer | `false` | +| `hashr.importers.windows.schedule` | sets the CronJob schedule times | `0 3 * * 3` | +| `hashr.importers.wsus.enabled` | Enables the WSUS importer | `false` | +| `hashr.importers.wsus.schedule` | sets the CronJob schedule times | `0 3 * * 4` | +| `hashr.importers.rpm.enabled` | Enables the RPM importer | `false` | +| `hashr.importers.rpm.schedule` | sets the CronJob schedule times | `0 3 * * 5` | +| `hashr.importers.zip.enabled` | Enables the ZIP importer | `false` | +| `hashr.importers.zip.schedule` | sets the CronJob schedule times | `0 3 * * 6` | +| `hashr.importers.gcr.enabled` | Enables the GCR importer | `false` | +| `hashr.importers.gcr.schedule` | sets the CronJob schedule times | `0 3 * * 7` | +| `hashr.importers.iso9660.enabled` | Enables the iso9660 importer | `false` | +| `hashr.importers.iso9660.schedule` | sets the CronJob schedule times | `0 15 * * 1` | +| `hashr.importers.deb.enabled` | Enables the DEB importer | `false` | +| `hashr.importers.deb.schedule` | sets the CronJob schedule times | `0 15 * * 2` | +| `persistence.name` | HashR persistent volume name | `hashrvolume` | +| `persistence.size` | HashR persistent volume size | `50Gi` | +| `persistence.storageClass` | PVC Storage Class for HashR volume | `""` | +| `persistence.accessModes` | PVC Access Mode for HashR volume | `["ReadWriteOnce"]` | + +### Postgresql Configuration Parameters + +| Name | Description | Value | +| ---------------------------------------------- | --------------------------------------------------------------------------- | ------------ | +| `postgresql.enabled` | Enables the Postgresql deployment | `true` | +| `postgresql.architecture` | PostgreSQL architecture (`standalone` or `replication`) | `standalone` | +| `postgresql.auth.username` | Name for a custom PostgreSQL user to create | `postgres` | +| `postgresql.auth.database` | Name for a custom PostgreSQL database to create (overrides `auth.database`) | `hashr` | +| `postgresql.primary.service.type` | PostgreSQL primary service type | `ClusterIP` | +| `postgresql.primary.service.ports.postgresql` | PostgreSQL primary service port | `5432` | +| `postgresql.primary.persistence.size` | PostgreSQL Persistent Volume size | `10Gi` | +| `postgresql.primary.resources.limits` | The resources limits for the PostgreSQL primary containers | `{}` | +| `postgresql.primary.resources.requests.cpu` | The requested cpu for the PostgreSQL primary containers | `250m` | +| `postgresql.primary.resources.requests.memory` | The requested memory for the PostgreSQL primary containers | `256Mi` | + + + +## Persistence + +The HashR deployment stores data at the `/mnt/hashrvolume` path of the +container. + +Persistent Volume Claims are used to keep the data across deployments. This is +known to work in GCP and Minikube. See the Parameters section to configure the +PVC or to disable persistence. + +## Upgrading + +If you need to upgrade an existing release to update a value, such as persistent +volume size or upgrading to a new release, you can run +[helm upgrade](https://helm.sh/docs/helm/helm_upgrade/). +For example, to set a new release and upgrade storage capacity, run: + +```console +helm upgrade my-release ../hashr \ + --set image.tag=latest \ + --set persistence.size=10T +``` + +The above command upgrades an existing release named `my-release` updating the +image tag to `latest` and increasing persistent volume size of an existing +volume to 10 Terabytes. Note that existing data will not be deleted and instead +triggers an expansion of the volume that backs the underlying PersistentVolume. +See [here](https://kubernetes.io/docs/concepts/storage/persistent-volumes/). + +## Troubleshooting + +There is a known issue causing PostgreSQL authentication to fail. This occurs +when you `delete` the deployed Helm chart and then redeploy the Chart without +removing the existing PVCs. When redeploying, please ensure to delete the +underlying PostgreSQL PVC. Refer to [issue 2061](https://github.com/bitnami/charts/issues/2061) +for more details. + +## License + +Copyright © 2024 OSDFIR Infrastructure + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. diff --git a/charts/hashr/templates/_helpers.tpl b/charts/hashr/templates/_helpers.tpl new file mode 100644 index 00000000..09f354e7 --- /dev/null +++ b/charts/hashr/templates/_helpers.tpl @@ -0,0 +1,107 @@ +{{/* +Expand the name of the chart. +*/}} +{{- define "hashr.name" -}} +{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }} +{{- end }} + +{{/* +Create a default fully qualified app name. +We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec). +If release name contains chart name it will be used as a full name. +*/}} +{{- define "hashr.fullname" -}} +{{- if .Values.fullnameOverride }} +{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }} +{{- else }} +{{- $name := default .Chart.Name .Values.nameOverride }} +{{- if contains $name .Release.Name }} +{{- .Release.Name | trunc 63 | trimSuffix "-" }} +{{- else }} +{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }} +{{- end }} +{{- end }} +{{- end }} + +{{/* +Create chart name and version as used by the chart label. +*/}} +{{- define "hashr.chart" -}} +{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }} +{{- end }} + +{{/* +Common labels +*/}} +{{- define "hashr.labels" -}} +helm.sh/chart: {{ include "hashr.chart" . }} +{{ include "hashr.selectorLabels" . }} +{{- if .Chart.AppVersion }} +app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} +{{- end }} +app.kubernetes.io/managed-by: {{ .Release.Service }} +{{- end }} + +{{/* +Selector labels +*/}} +{{- define "hashr.selectorLabels" -}} +app.kubernetes.io/name: {{ include "hashr.name" . }} +app.kubernetes.io/instance: {{ .Release.Name }} +{{- end }} + +{{/* +Create the name of the service account to use +*/}} +{{- define "hashr.serviceAccountName" -}} +{{- if .Values.serviceAccount.create }} +{{- default (include "hashr.fullname" .) .Values.serviceAccount.name }} +{{- else }} +{{- default "default" .Values.serviceAccount.name }} +{{- end }} +{{- end }} + +{{/* +Return the proper persistence volume claim name +*/}} +{{- define "hashr.pvc.name" -}} +{{- $pvcName := .Values.persistence.name -}} +{{- if .Values.global -}} + {{- if .Values.global.existingPVC -}} + {{- $pvcName = .Values.global.existingPVC -}} + {{- end -}} +{{- printf "%s-%s" $pvcName "claim" }} +{{- end -}} +{{- end -}} + +{{/* +Return the proper Storage Class +*/}} +{{- define "hashr.storage.class" -}} +{{- $storageClass := .Values.persistence.storageClass -}} +{{- if .Values.global -}} + {{- if .Values.global.storageClass -}} + {{- $storageClass = .Values.global.storageClass -}} + {{- end -}} +{{- end -}} +{{- if $storageClass -}} + {{- if (eq "-" $storageClass) -}} + {{- printf "storageClassName: \"\"" -}} + {{- else }} + {{- printf "storageClassName: %s" $storageClass -}} + {{- end -}} +{{- end -}} +{{- end -}} + +{{/* +Create the data path. +*/}} +{{- define "hashr.dataPath" -}} +{{- $pvcName := .Values.persistence.name -}} +{{- if .Values.global -}} + {{- if .Values.global.existingPVC -}} + {{- $pvcName = .Values.global.existingPVC -}} + {{- end -}} +{{- printf "/mnt/%s/data" $pvcName }} +{{- end }} +{{- end }} diff --git a/charts/hashr/templates/data-manager.yaml b/charts/hashr/templates/data-manager.yaml new file mode 100644 index 00000000..ebd0bcf1 --- /dev/null +++ b/charts/hashr/templates/data-manager.yaml @@ -0,0 +1,19 @@ +apiVersion: v1 +kind: Pod +metadata: + name: {{ .Release.Name }}-hashr-data-manager +spec: + containers: + - name: hashr-data-manager + image: busybox:latest + imagePullPolicy: IfNotPresent + command: ["sh", "-c", "while true; do sleep 1800; done;"] + volumeMounts: + - name: hashrvolume + mountPath: {{ (include "hashr.dataPath" .) | quote }} + restartPolicy: Always + volumes: + - name: hashrvolume + persistentVolumeClaim: + claimName: {{ include "hashr.pvc.name" . }} + readOnly: false diff --git a/charts/hashr/templates/hashr-deb-cronjob.yaml b/charts/hashr/templates/hashr-deb-cronjob.yaml new file mode 100644 index 00000000..cf03e1d2 --- /dev/null +++ b/charts/hashr/templates/hashr-deb-cronjob.yaml @@ -0,0 +1,54 @@ +{{- if .Values.hashr.importers.deb.enabled -}} +apiVersion: batch/v1 +kind: CronJob +metadata: + name: {{ .Release.Name }}-hashr-deb +spec: + schedule: {{ .Values.hashr.importers.deb.schedule | quote }} + concurrencyPolicy: Forbid + successfulJobsHistoryLimit: 2 + failedJobsHistoryLimit: 1 + jobTemplate: + spec: + template: + spec: + containers: + - name: hashr-deb + image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}" + imagePullPolicy: {{ .Values.image.pullPolicy }} + args: + - --logtostderr=1 + - -storage + - postgres + - -exporters + - postgres + - -postgres_host + - {{ include "common.names.fullname" (dict "Chart" (dict "Name" "postgresql") "Release" .Release "Values" .Values.postgresql) }} + - -postgres_port + - {{ .Values.postgresql.primary.service.ports.postgresql | quote }} + - -postgres_user + - {{ .Values.postgresql.auth.username | quote }} + - -postgres_password + - "$(POSTGRES_PASSWORD)" + - -postgres_db + - {{ .Values.postgresql.auth.database | quote }} + - -importers + - deb + - -deb_repo_path + - {{ (include "hashr.dataPath" .) }}/deb/ + env: + - name: POSTGRES_PASSWORD + valueFrom: + secretKeyRef: + name: {{ include "postgresql.v1.secretName" .Subcharts.postgresql }} + key: {{ include "postgresql.v1.adminPasswordKey" .Subcharts.postgresql }} + volumeMounts: + - name: hashrvolume + mountPath: {{ (include "hashr.dataPath" .) | quote }} + restartPolicy: Never + volumes: + - name: hashrvolume + persistentVolumeClaim: + claimName: {{ include "hashr.pvc.name" . }} + readOnly: false +{{- end }} diff --git a/charts/hashr/templates/hashr-gcp-cronjob.yaml b/charts/hashr/templates/hashr-gcp-cronjob.yaml new file mode 100644 index 00000000..ed80a852 --- /dev/null +++ b/charts/hashr/templates/hashr-gcp-cronjob.yaml @@ -0,0 +1,62 @@ +{{- if .Values.hashr.importers.gcp.enabled -}} +apiVersion: batch/v1 +kind: CronJob +metadata: + name: {{ .Release.Name }}-hashr-gcp +spec: + schedule: {{ .Values.hashr.importers.gcp.schedule | quote }} + concurrencyPolicy: Forbid + successfulJobsHistoryLimit: 2 + failedJobsHistoryLimit: 1 + jobTemplate: + spec: + template: + spec: + containers: + - name: hashr-gcp + image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}" + imagePullPolicy: {{ .Values.image.pullPolicy }} + args: + - --logtostderr=1 + - -storage + - postgres + - -exporters + - postgres + - -postgres_host + - {{ include "common.names.fullname" (dict "Chart" (dict "Name" "postgresql") "Release" .Release "Values" .Values.postgresql) }} + - -postgres_port + - {{ .Values.postgresql.primary.service.ports.postgresql | quote }} + - -postgres_user + - {{ .Values.postgresql.auth.username | quote }} + - -postgres_password + - "$(POSTGRES_PASSWORD)" + - -postgres_db + - {{ .Values.postgresql.auth.database | quote }} + - -importers + - GCP + - -gcp_projects + - {{ .Values.hashr.importers.gcp.gcp_projects | quote }} + - -hashr_gcp_project + - {{ .Values.hashr.importers.gcp.hashr_gcp_project | quote }} + - -hashr_gcs_bucket + - {{ .Values.hashr.importers.gcp.hashr_gcs_bucket | quote }} + env: + - name: GOOGLE_APPLICATION_CREDENTIALS + # Store your SA key in the hashrvolume/creds/ folder via "kubectl cp"! + # chown 999:1000 hashr-sa-private-key.json to prevent permission issues + value: {{ (include "hashr.dataPath" .) }}/creds/hashr-sa-private-key.json + - name: POSTGRES_PASSWORD + valueFrom: + secretKeyRef: + name: {{ include "postgresql.v1.secretName" .Subcharts.postgresql }} + key: {{ include "postgresql.v1.adminPasswordKey" .Subcharts.postgresql }} + volumeMounts: + - name: hashrvolume + mountPath: {{ (include "hashr.dataPath" .) | quote }} + restartPolicy: Never + volumes: + - name: hashrvolume + persistentVolumeClaim: + claimName: {{ include "hashr.pvc.name" . }} + readOnly: false +{{- end }} diff --git a/charts/hashr/templates/hashr-iso9660-cronjob.yaml b/charts/hashr/templates/hashr-iso9660-cronjob.yaml new file mode 100644 index 00000000..9423ec71 --- /dev/null +++ b/charts/hashr/templates/hashr-iso9660-cronjob.yaml @@ -0,0 +1,54 @@ +{{- if .Values.hashr.importers.iso9660.enabled -}} +apiVersion: batch/v1 +kind: CronJob +metadata: + name: {{ .Release.Name }}-hashr-iso9660 +spec: + schedule: {{ .Values.hashr.importers.iso9660.schedule | quote }} + concurrencyPolicy: Forbid + successfulJobsHistoryLimit: 2 + failedJobsHistoryLimit: 1 + jobTemplate: + spec: + template: + spec: + containers: + - name: hashr-iso9660 + image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}" + imagePullPolicy: {{ .Values.image.pullPolicy }} + args: + - --logtostderr=1 + - -storage + - postgres + - -exporters + - postgres + - -postgres_host + - {{ include "common.names.fullname" (dict "Chart" (dict "Name" "postgresql") "Release" .Release "Values" .Values.postgresql) }} + - -postgres_port + - {{ .Values.postgresql.primary.service.ports.postgresql | quote }} + - -postgres_user + - {{ .Values.postgresql.auth.username | quote }} + - -postgres_password + - "$(POSTGRES_PASSWORD)" + - -postgres_db + - {{ .Values.postgresql.auth.database | quote }} + - -importers + - iso9660 + - -iso_repo_path + - {{ (include "hashr.dataPath" .) }}/iso9660/ + env: + - name: POSTGRES_PASSWORD + valueFrom: + secretKeyRef: + name: {{ include "postgresql.v1.secretName" .Subcharts.postgresql }} + key: {{ include "postgresql.v1.adminPasswordKey" .Subcharts.postgresql }} + volumeMounts: + - name: hashrvolume + mountPath: {{ (include "hashr.dataPath" .) | quote }} + restartPolicy: Never + volumes: + - name: hashrvolume + persistentVolumeClaim: + claimName: {{ include "hashr.pvc.name" . }} + readOnly: false +{{- end }} diff --git a/charts/hashr/templates/hashr-rpm-cronjob.yaml b/charts/hashr/templates/hashr-rpm-cronjob.yaml new file mode 100644 index 00000000..4bdfc1fb --- /dev/null +++ b/charts/hashr/templates/hashr-rpm-cronjob.yaml @@ -0,0 +1,54 @@ +{{- if .Values.hashr.importers.rpm.enabled -}} +apiVersion: batch/v1 +kind: CronJob +metadata: + name: {{ .Release.Name }}-hashr-rpm +spec: + schedule: {{ .Values.hashr.importers.rpm.schedule | quote }} + concurrencyPolicy: Forbid + successfulJobsHistoryLimit: 2 + failedJobsHistoryLimit: 1 + jobTemplate: + spec: + template: + spec: + containers: + - name: hashr-rpm + image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}" + imagePullPolicy: {{ .Values.image.pullPolicy }} + args: + - --logtostderr=1 + - -storage + - postgres + - -exporters + - postgres + - -postgres_host + - {{ include "common.names.fullname" (dict "Chart" (dict "Name" "postgresql") "Release" .Release "Values" .Values.postgresql) }} + - -postgres_port + - {{ .Values.postgresql.primary.service.ports.postgresql | quote }} + - -postgres_user + - {{ .Values.postgresql.auth.username | quote }} + - -postgres_password + - "$(POSTGRES_PASSWORD)" + - -postgres_db + - {{ .Values.postgresql.auth.database | quote }} + - -importers + - rpm + - -rpm_repo_path + - {{ (include "hashr.dataPath" .) }}/rpm/ + env: + - name: POSTGRES_PASSWORD + valueFrom: + secretKeyRef: + name: {{ include "postgresql.v1.secretName" .Subcharts.postgresql }} + key: {{ include "postgresql.v1.adminPasswordKey" .Subcharts.postgresql }} + volumeMounts: + - name: hashrvolume + mountPath: {{ (include "hashr.dataPath" .) | quote }} + restartPolicy: Never + volumes: + - name: hashrvolume + persistentVolumeClaim: + claimName: {{ include "hashr.pvc.name" . }} + readOnly: false +{{- end }} diff --git a/charts/hashr/templates/hashr-targz-cronjob.yaml b/charts/hashr/templates/hashr-targz-cronjob.yaml new file mode 100644 index 00000000..a519e88b --- /dev/null +++ b/charts/hashr/templates/hashr-targz-cronjob.yaml @@ -0,0 +1,54 @@ +{{- if .Values.hashr.importers.targz.enabled -}} +apiVersion: batch/v1 +kind: CronJob +metadata: + name: {{ .Release.Name }}-hashr-targz +spec: + schedule: {{ .Values.hashr.importers.targz.schedule | quote }} + concurrencyPolicy: Forbid + successfulJobsHistoryLimit: 2 + failedJobsHistoryLimit: 1 + jobTemplate: + spec: + template: + spec: + containers: + - name: hashr-targz + image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}" + imagePullPolicy: {{ .Values.image.pullPolicy }} + args: + - --logtostderr=1 + - -storage + - postgres + - -exporters + - postgres + - -postgres_host + - {{ include "common.names.fullname" (dict "Chart" (dict "Name" "postgresql") "Release" .Release "Values" .Values.postgresql) }} + - -postgres_port + - {{ .Values.postgresql.primary.service.ports.postgresql | quote }} + - -postgres_user + - {{ .Values.postgresql.auth.username | quote }} + - -postgres_password + - "$(POSTGRES_PASSWORD)" + - -postgres_db + - {{ .Values.postgresql.auth.database | quote }} + - -importers + - targz + - -targz_repo_path + - {{ (include "hashr.dataPath" .) }}/targz/ + env: + - name: POSTGRES_PASSWORD + valueFrom: + secretKeyRef: + name: {{ include "postgresql.v1.secretName" .Subcharts.postgresql }} + key: {{ include "postgresql.v1.adminPasswordKey" .Subcharts.postgresql }} + volumeMounts: + - name: hashrvolume + mountPath: {{ (include "hashr.dataPath" .) | quote }} + restartPolicy: Never + volumes: + - name: hashrvolume + persistentVolumeClaim: + claimName: {{ include "hashr.pvc.name" . }} + readOnly: false +{{- end }} diff --git a/charts/hashr/templates/hashr-zip-cronjob.yaml b/charts/hashr/templates/hashr-zip-cronjob.yaml new file mode 100644 index 00000000..efc23466 --- /dev/null +++ b/charts/hashr/templates/hashr-zip-cronjob.yaml @@ -0,0 +1,54 @@ +{{- if .Values.hashr.importers.zip.enabled -}} +apiVersion: batch/v1 +kind: CronJob +metadata: + name: {{ .Release.Name }}-hashr-zip +spec: + schedule: {{ .Values.hashr.importers.zip.schedule | quote }} + concurrencyPolicy: Forbid + successfulJobsHistoryLimit: 2 + failedJobsHistoryLimit: 1 + jobTemplate: + spec: + template: + spec: + containers: + - name: hashr-zip + image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}" + imagePullPolicy: {{ .Values.image.pullPolicy }} + args: + - --logtostderr=1 + - -storage + - postgres + - -exporters + - postgres + - -postgres_host + - {{ include "common.names.fullname" (dict "Chart" (dict "Name" "postgresql") "Release" .Release "Values" .Values.postgresql) }} + - -postgres_port + - {{ .Values.postgresql.primary.service.ports.postgresql | quote }} + - -postgres_user + - {{ .Values.postgresql.auth.username | quote }} + - -postgres_password + - "$(POSTGRES_PASSWORD)" + - -postgres_db + - {{ .Values.postgresql.auth.database | quote }} + - -importers + - zip + - -zip_repo_path + - {{ (include "hashr.dataPath" .) }}/zip/ + env: + - name: POSTGRES_PASSWORD + valueFrom: + secretKeyRef: + name: {{ include "postgresql.v1.secretName" .Subcharts.postgresql }} + key: {{ include "postgresql.v1.adminPasswordKey" .Subcharts.postgresql }} + volumeMounts: + - name: hashrvolume + mountPath: {{ (include "hashr.dataPath" .) | quote }} + restartPolicy: Never + volumes: + - name: hashrvolume + persistentVolumeClaim: + claimName: {{ include "hashr.pvc.name" . }} + readOnly: false +{{- end }} diff --git a/charts/hashr/templates/pvc.yaml b/charts/hashr/templates/pvc.yaml new file mode 100644 index 00000000..ee5924db --- /dev/null +++ b/charts/hashr/templates/pvc.yaml @@ -0,0 +1,18 @@ +{{- if not (.Values.global.existingPVC) }} +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + annotations: + helm.sh/resource-policy: keep + name: {{ include "hashr.pvc.name" . }} + namespace: {{ .Release.Namespace | quote }} +spec: + {{- include "hashr.storage.class" . | nindent 2 }} + accessModes: + {{- range .Values.persistence.accessModes }} + - {{ . | quote }} + {{- end }} + resources: + requests: + storage: {{ .Values.persistence.size | quote }} +{{- end }} diff --git a/charts/hashr/values.yaml b/charts/hashr/values.yaml new file mode 100644 index 00000000..a2f19780 --- /dev/null +++ b/charts/hashr/values.yaml @@ -0,0 +1,250 @@ +## HashR Helm Chart +## Please use this Helm chart for deploying HashR to a Kubernetes environment +## +## @section Global parameters +## Please, note that this will override the parameters configured to use the global value +## +global: + ## Global Persistence Configuration + ## + timesketch: + ## @param global.timesketch.enabled Enables the Timesketch deployment (only used in the main OSDFIR Infrastructure Helm chart) + ## + enabled: false + ## @param global.timesketch.servicePort Timesketch service port (overrides `timesketch.service.port`) + ## + servicePort: + turbinia: + ## @param global.turbinia.enabled Enables the Turbinia deployment (only used within the main OSDFIR Infrastructure Helm chart) + ## + enabled: false + ## @param global.turbinia.servicePort Turbinia API service port (overrides `turbinia.service.port`) + ## + servicePort: + yeti: + ## @param global.yeti.enabled Enables the Yeti deployment (only used in the main OSDFIR Infrastructure Helm chart) + ## + enabled: false + ## @param global.yeti.servicePort Yeti API service port (overrides `yeti.api.service.port`) + ## + servicePort: + ## @param global.existingPVC Existing claim for HashR persistent volume (overrides `persistent.name`) + ## + existingPVC: "" + ## @param global.storageClass StorageClass for the HashR persistent volume (overrides `persistent.storageClass`) + ## + storageClass: "" +## @section HashR image configuration +## +image: + ## @param image.repository HashR image repository + ## + repository: us-docker.pkg.dev/osdfir-registry/hashr/release/hashr + ## @param image.pullPolicy HashR image pull policy + ## ref https://kubernetes.io/docs/concepts/containers/images/#image-pull-policy + ## + pullPolicy: IfNotPresent + ## @param image.tag Overrides the image tag whose default is the chart appVersion + ## + tag: latest + ## @param image.imagePullSecrets Specify secrets if pulling from a private repository + ## ref https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/ + ## e.g. + ## imagePullSecrets: + ## - myRegistryKeySecretName + ## + imagePullSecrets: [] +## @section HashR Configuration Paramters +## +hashr: + ## @section Enable/Disable HashR importers + ## + importers: + ## List of HashR importers and their settings + ## + aws: + # TODO: Add cronjob file! + # https://github.com/google/hashr?tab=readme-ov-file#aws + ## @param hashr.importers.aws.enabled Enables the AWS importer + ## + enabled: false + ## @param hashr.importers.aws.schedule sets the CronJob schedule times + ## + # At 09:00 on Monday + schedule: "0 9 * * 1" + gcp: + # Ensure you have the correct setup before enabling this importer: + # https://github.com/google/hashr?tab=readme-ov-file#gcp-google-cloud-platform + # IMPORTANT: Store your SA key in the hashrvolume via kubectl cp! + # e.g. kubectl cp ~/hashr-sa-private-key.json hashr-data-manager:/mnt/hashrvolume/data/creds/hashr-sa-private-key.json + ## @param hashr.importers.gcp.enabled Enables the GCP importer + ## + enabled: false + ## @param hashr.importers.gcp.schedule sets the CronJob schedule times + ## + # At 03:00 on Monday + schedule: "0 3 * * 1" + ## @param hashr.importers.gcp.gcp_projects sets a comma separated list of cloud projects containing disk images + ## + gcp_projects: "" + ## @param hashr.importers.gcp.hashr_gcp_project sets GCP project that will be used to store copy of disk images for processing and also run Cloud Build + ## + hashr_gcp_project: "" + ## @param hashr.importers.gcp.hashr_gcs_bucket sets GCS bucket that will be used to store output of Cloud Build (disk images in .tar.gz format) + ## + hashr_gcs_bucket: "" + targz: + # https://github.com/google/hashr?tab=readme-ov-file#targz + ## @param hashr.importers.targz.enabled Enables the tar.gz importer + ## + enabled: false + ## @param hashr.importers.targz.schedule sets the CronJob schedule times + ## + # At 03:00 on Tuesday + schedule: "0 3 * * 2" + windows: + # TODO: Add cronjob file! + # https://github.com/google/hashr?tab=readme-ov-file#windows + ## @param hashr.importers.windows.enabled Enables the Windows importer + ## + enabled: false + ## @param hashr.importers.windows.schedule sets the CronJob schedule times + ## + # At 03:00 on Wednesday + schedule: "0 3 * * 3" + wsus: + # TODO: Add cronjob file! + # https://github.com/google/hashr?tab=readme-ov-file#wsus + ## @param hashr.importers.wsus.enabled Enables the WSUS importer + ## + enabled: false + ## @param hashr.importers.wsus.schedule sets the CronJob schedule times + ## + # At 03:00 on Thursday + schedule: "0 3 * * 4" + rpm: + # https://github.com/google/hashr?tab=readme-ov-file#rpm + ## @param hashr.importers.rpm.enabled Enables the RPM importer + ## + enabled: false + ## @param hashr.importers.rpm.schedule sets the CronJob schedule times + ## + # At 03:00 on Friday + schedule: "0 3 * * 5" + zip: + # https://github.com/google/hashr?tab=readme-ov-file#zip-and-other-zip-like-formats + ## @param hashr.importers.zip.enabled Enables the ZIP importer + ## + enabled: false + ## @param hashr.importers.zip.schedule sets the CronJob schedule times + ## + # At 03:00 on Saturday + schedule: "0 3 * * 6" + gcr: + # TODO: Add cronjob file! + # https://github.com/google/hashr?tab=readme-ov-file#gcr-google-container-registry + ## @param hashr.importers.gcr.enabled Enables the GCR importer + ## + enabled: false + ## @param hashr.importers.gcr.schedule sets the CronJob schedule times + ## + # At 03:00 on Sunday + schedule: "0 3 * * 7" + iso9660: + # https://github.com/google/hashr?tab=readme-ov-file#iso-9660 + ## @param hashr.importers.iso9660.enabled Enables the iso9660 importer + ## + enabled: false + ## @param hashr.importers.iso9660.schedule sets the CronJob schedule times + ## + # At 15:00 on Monday + schedule: "0 15 * * 1" + deb: + # https://github.com/google/hashr?tab=readme-ov-file#deb + ## @param hashr.importers.deb.enabled Enables the DEB importer + ## + enabled: false + ## @param hashr.importers.deb.schedule sets the CronJob schedule times + ## + # At 15:00 on Tuesday + schedule: "0 15 * * 2" +## Persistence Storage Parameters +## +persistence: + ## @param persistence.name HashR persistent volume name + ## + name: hashrvolume + ## @param persistence.size HashR persistent volume size + ## + size: 50Gi + ## @param persistence.storageClass PVC Storage Class for HashR volume + ## If set to "-", storageClassName: "", which disables dynamic provisioning + ## If undefined or set to null, no storageClassName spec is + ## set, choosing the default provisioner. (gp2 on AWS, standard on + ## GKE, AWS & OpenStack) + ## ref https://kubernetes.io/docs/concepts/storage/dynamic-provisioning/#using-dynamic-provisioning + ## + storageClass: "" + ## @param persistence.accessModes PVC Access Mode for HashR volume + ## Access mode may need to be updated based on the StorageClass + ## ref https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes + ## + accessModes: + - ReadWriteOnce +## @section Postgresql Configuration Parameters +## IMPORTANT: Postgresql is deployed with Auth enabled by default +## To see a full list of available values, run helm show values charts/postgresql* +## +postgresql: + ## @param postgresql.enabled Enables the Postgresql deployment + ## + enabled: true + ## @param postgresql.architecture PostgreSQL architecture (`standalone` or `replication`) + ## + architecture: standalone + ## PostgreSQL Authentication parameters + ## + auth: + ## @param postgresql.auth.username Name for a custom PostgreSQL user to create + ## + username: "postgres" + ## @param postgresql.auth.database Name for a custom PostgreSQL database to create (overrides `auth.database`) + ## + database: "hashr" + ## PostgreSQL Primary configuration parameters + ## + primary: + ## PostgreSQL Primary service configuration parameters + ## + service: + ## @param postgresql.primary.service.type PostgreSQL primary service type + ## + type: ClusterIP + ## @param postgresql.primary.service.ports.postgresql PostgreSQL primary service port + ## + ports: + postgresql: 5432 + ## PostgreSQL Primary persistence configuration + ## + persistence: + ## @param postgresql.primary.persistence.size PostgreSQL Persistent Volume size + ## + size: 10Gi + ## PostgreSQL primary resource requests and limits + ## @param postgresql.primary.resources.limits The resources limits for the PostgreSQL primary containers + ## @param postgresql.primary.resources.requests.cpu The requested cpu for the PostgreSQL primary containers + ## @param postgresql.primary.resources.requests.memory The requested memory for the PostgreSQL primary containers + ## + resources: + ## Example: + ## limits: + ## cpu: 500m + ## memory: 1Gi + limits: {} + ## Example: + ## requests: + ## cpu: 500m + ## memory: 1Gi + requests: + cpu: 250m + memory: 256Mi