Skip to content
This repository has been archived by the owner on Jul 26, 2022. It is now read-only.

feat: add validation to CRD #208

Merged
merged 4 commits into from
Nov 8, 2019
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 14 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -215,6 +215,17 @@ data:
password: MTIzNA==
```

## Deprecations
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, nice.


A few properties has changed name overtime, we still maintain backwards compatbility with these but they will eventually be removed, and they are not validated using the CRD validation.

| Old | New |
| ----------------------------- | ------------------------------ |
| `secretDescriptor` | `spec` |
| `spec.type` | `spec.template.type` |
| `spec.properties` | `spec.data` |
| `backendType: secretManager` | `backendType: secretsManager` |

## Backends

kubernetes-external-secrets supports both AWS Secrets Manager and AWS System Manager.
Expand All @@ -238,7 +249,7 @@ aws secretsmanager create-secret --region us-west-2 --name hello-service/credent
We can declare which properties we want from hello-service/credentials:

```yml
apiVersion: 'kubernetes-client.io/v1'
apiVersion: kubernetes-client.io/v1
kind: ExternalSecret
metadata:
name: hello-service
Expand All @@ -258,7 +269,7 @@ spec:
alternatively you can use `dataFrom` and get all the values from hello-service/credentials:

```yml
apiVersion: 'kubernetes-client.io/v1'
apiVersion: kubernetes-client.io/v1
kind: ExternalSecret
metadata:
name: hello-service
Expand All @@ -273,7 +284,7 @@ spec:
`data` and `dataFrom` can of course be combined, any naming conflicts will use the last defined, with `data` overriding `dataFrom`

```yml
apiVersion: 'kubernetes-client.io/v1'
apiVersion: kubernetes-client.io/v1
kind: ExternalSecret
metadata:
name: hello-service
Expand Down
8 changes: 7 additions & 1 deletion config/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,20 @@
const kube = require('kubernetes-client')
const KubeRequest = require('kubernetes-client/backends/request')
const pino = require('pino')
const yaml = require('js-yaml')
const fs = require('fs')
const path = require('path')

const awsConfig = require('./aws-config')
const envConfig = require('./environment')
const CustomResourceManager = require('../lib/custom-resource-manager')
const customResourceManifest = require('../custom-resource-manifest.json')
const SecretsManagerBackend = require('../lib/backends/secrets-manager-backend')
const SystemManagerBackend = require('../lib/backends/system-manager-backend')

// Get document, or throw exception on error
// eslint-disable-next-line security/detect-non-literal-fs-filename
const customResourceManifest = yaml.safeLoad(fs.readFileSync(path.resolve(__dirname, '../crd.yaml'), 'utf8'))

const kubeconfig = new kube.KubeConfig()
kubeconfig.loadFromDefault()
const kubeBackend = new KubeRequest({ kubeconfig })
Expand Down
76 changes: 76 additions & 0 deletions crd.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: externalsecrets.kubernetes-client.io
spec:
group: kubernetes-client.io
version: v1
scope: Namespaced

names:
shortNames:
- es
kind: ExternalSecret
plural: externalsecrets
singular: externalsecret

additionalPrinterColumns:
- JSONPath: .status.lastSync
name: Last Sync
type: date
- JSONPath: .status.status
name: status
type: string
- JSONPath: .metadata.creationTimestamp
name: Age
type: date

validation:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you know which version of Kubernetes introduced validation for CRDs? I think we should use validation, but we might need to explicitly call out the minimum version required.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems to be available in 1.12 which is as far back as the official hosted kubernetes docs go now
https://v1-12.docs.kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/#validation
think its a fair requirement :)

Copy link
Member Author

@Flydiverny Flydiverny Nov 8, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heres the list I was looking for https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/
Seems to have been enabled by default as beta in 1.9+ ( CustomResourceValidation )

CustomResourceSubresources requires 1.11+

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome

openAPIV3Schema:
properties:
spec:
type: object
properties:
template:
description: Template which will be deep merged without mutating
any existing fields. into generated secret, can be used to
set for example annotations or type on the generated secret
type: object
backendType:
type: string
enum:
- secretsManager
- systemManager
- vault
dataFrom:
type: array
items:
type: string
data:
type: array
items:
type: object
properties:
key:
description: Secret key in backend
type: string
name:
description: Name set for this key in the generated secret
type: string
property:
description: Property to extract if secret in backend is a JSON object
required:
- name
- key
roleArn:
type: string
required:
- backendType
anyOf:
- required:
- data
- required:
- dataFrom

subresources:
status: {}
40 changes: 0 additions & 40 deletions custom-resource-manifest.json

This file was deleted.

8 changes: 8 additions & 0 deletions examples/data-from-example.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
apiVersion: kubernetes-client.io/v1
kind: ExternalSecret
metadata:
name: data-from-example
spec:
backendType: systemManager
dataFrom:
- /foo/name1
5 changes: 3 additions & 2 deletions examples/dockerconfig-example.yml
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
apiVersion: kubernetes-client.io/v1
kind: ExternalSecret
metadata:
name: dockerhub-secret
name: dockerconfig-example
spec:
backendType: secretsManager
type: kubernetes.io/dockerconfigjson
template:
type: kubernetes.io/dockerconfigjson
data:
- key: /development/dockerhub
name: .dockerconfigjson
8 changes: 7 additions & 1 deletion examples/hello-service-external-secret.yml
Original file line number Diff line number Diff line change
@@ -1,9 +1,15 @@
apiVersion: 'kubernetes-client.io/v1'
apiVersion: kubernetes-client.io/v1
kind: ExternalSecret
metadata:
name: hello-service
spec:
template:
metadata:
annotations:
external-secret: 'Yes please!'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😄

backendType: secretsManager
data:
- key: hello-service/password
name: password
dataFrom:
- hello-service/secret-envs
4 changes: 2 additions & 2 deletions examples/secretsmanager-example.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
apiVersion: 'kubernetes-client.io/v1'
apiVersion: kubernetes-client.io/v1
kind: ExternalSecret
metadata:
name: demo-service
name: secretsmanager-example
spec:
backendType: secretsManager
# optional: specify role to assume when retrieving the data
Expand Down
4 changes: 2 additions & 2 deletions examples/ssm-example.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
apiVersion: 'kubernetes-client.io/v1'
apiVersion: kubernetes-client.io/v1
kind: ExternalSecret
metadata:
name: ssm-secret-key
name: ssm-example
spec:
backendType: systemManager
# optional: specify role to assume when retrieving the data
Expand Down
5 changes: 3 additions & 2 deletions examples/tls-example.yml
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
apiVersion: kubernetes-client.io/v1
kind: ExternalSecret
metadata:
name: dockerhub-secret
name: tls-secret
spec:
backendType: secretsManager
type: kubernetes.io/tls
template:
type: kubernetes.io/tls
data:
- key: /development/certificate
property: crt
Expand Down
17 changes: 9 additions & 8 deletions lib/poller.js
Original file line number Diff line number Diff line change
Expand Up @@ -75,10 +75,15 @@ class Poller {
*/
async _createSecretManifest () {
const spec = this._spec
const template = spec.template
const template = spec.template || {}

// spec.type for backwards compat
const type = template.type || spec.type || 'Opaque'

const data = await this._backends[spec.backendType]
.getSecretManifestData({ spec })
let secretManifest = {

const secretManifest = {
apiVersion: 'v1',
kind: 'Secret',
metadata: {
Expand All @@ -87,15 +92,11 @@ class Poller {
this._ownerReference
]
},
type: spec.type || 'Opaque',
type,
data
}

if (template) {
secretManifest = merge(clonedeep(template), secretManifest)
}

return secretManifest
return merge(clonedeep(template), secretManifest)
}

/**
Expand Down
32 changes: 29 additions & 3 deletions lib/poller.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -183,12 +183,12 @@ describe('Poller', () => {
})
})

it('creates secret manifest - with type', async () => {
it('creates secret manifest - with type (backwards compat)', async () => {
const poller = pollerFactory({
type: 'dummy-test-type',
backendType: 'fakeBackendType',
name: 'fakeSecretName',
properties: [
data: [
'fakePropertyName1',
'fakePropertyName2'
]
Expand All @@ -206,7 +206,7 @@ describe('Poller', () => {
type: 'dummy-test-type',
backendType: 'fakeBackendType',
name: 'fakeSecretName',
properties: [
data: [
'fakePropertyName1',
'fakePropertyName2'
]
Expand All @@ -228,6 +228,32 @@ describe('Poller', () => {
})
})

it('creates secret manifest - with template type (should work with backwards compat type)', async () => {
const poller = pollerFactory({
template: {
type: 'dummy-test-type'
},
backendType: 'fakeBackendType',
name: 'fakeSecretName',
data: []
})

backendMock.getSecretManifestData.resolves({})

const secretManifest = await poller._createSecretManifest()

expect(secretManifest).deep.equals({
apiVersion: 'v1',
kind: 'Secret',
metadata: {
name: 'fakeSecretName',
ownerReferences: [getOwnerReference()]
},
type: 'dummy-test-type',
data: {}
})
})

it('creates secret manifest - with template', async () => {
const poller = pollerFactory({
type: 'dummy-test-type',
Expand Down
2 changes: 1 addition & 1 deletion package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
"dependencies": {
"aws-sdk": "^2.566.0",
"express": "^4.17.1",
"js-yaml": "^3.13.1",
"json-stream": "^1.0.0",
"kubernetes-client": "^8.3.0",
"lodash.clonedeep": "^4.5.0",
Expand Down