Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: set default certstore namespace in notation verifier to uniquely identify certificate store resource #1134

Merged
merged 19 commits into from
Oct 26, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion charts/ratify/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ $ helm upgrade -n gatekeeper-system [RELEASE_NAME] ratify/ratify
| azureWorkloadIdentity.clientId | ClientID of AAD application/Managed identity associated with Workload Identity | `` |
| azureManagedIdentity.clientId | ClientID of Managed identity | `` |
| azureManagedIdentity.tenantId | TenantID of Managed Identity resource | `` |
| akvCertConfig.enabled | Enables/disables Azure Key Vault certificate store | `false` |
| akvCertConfig.enabled | Enables/disables Azure Key Vault certificate store. If you are using a custom chart, certificate store should be referenced through a Verifier CR. References in ConfigMap will not be correctly resolved. | `false` |
| akvCertConfig.vaultURI | Vault URI for AKV configured | `` |
| akvCertConfig.cert1Name | Exact name of the certificate stored in AKV | `` |
| akvCertConfig.cert1Version | Exact version of certificate to use from AKV | `` |
Expand Down
9 changes: 1 addition & 8 deletions charts/ratify/templates/configmap.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -72,14 +72,7 @@ data:
{
"name":"notation",
"artifactTypes" : "application/vnd.cncf.notary.signature",
"verificationCertStores": {
"certs":[
{{- if .Values.akvCertConfig.enabled }}
"certstore-akv"
{{- else }}
"{{ include "ratify.fullname" . }}-notation-inline-cert"
{{- end }}
]
"verificationCertStores": {
susanshi marked this conversation as resolved.
Show resolved Hide resolved
},
"trustPolicyDoc": {
"version": "1.0",
Expand Down
3 changes: 2 additions & 1 deletion cmd/ratify/cmd/verify.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ import (
"fmt"

"github.com/deislabs/ratify/config"
"github.com/deislabs/ratify/internal/constants"
"github.com/deislabs/ratify/internal/logger"
e "github.com/deislabs/ratify/pkg/executor"
ef "github.com/deislabs/ratify/pkg/executor/core"
Expand Down Expand Up @@ -99,7 +100,7 @@ func verify(opts verifyCmdOptions) error {
return err
}

verifiers, err := vf.CreateVerifiersFromConfig(cf.VerifiersConfig, config.GetDefaultPluginPath())
verifiers, err := vf.CreateVerifiersFromConfig(cf.VerifiersConfig, config.GetDefaultPluginPath(), constants.EmptyNamespace)

if err != nil {
return err
Expand Down
4 changes: 3 additions & 1 deletion config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
"path/filepath"
"sync"

"github.com/deislabs/ratify/internal/constants"
"github.com/deislabs/ratify/internal/logger"
exConfig "github.com/deislabs/ratify/pkg/executor/config"
"github.com/deislabs/ratify/pkg/homedir"
Expand Down Expand Up @@ -91,7 +92,8 @@
}
logrus.Infof("stores successfully created. number of stores %d", len(stores))

verifiers, err := vf.CreateVerifiersFromConfig(cf.VerifiersConfig, GetDefaultPluginPath())
// in K8s , verifiers CR are deployed to specific namespace, namespace is not applicable in config file scenario
verifiers, err := vf.CreateVerifiersFromConfig(cf.VerifiersConfig, GetDefaultPluginPath(), constants.EmptyNamespace)

Check warning on line 96 in config/config.go

View check run for this annotation

Codecov / codecov/patch

config/config.go#L95-L96

Added lines #L95 - L96 were not covered by tests

if err != nil {
return nil, nil, nil, errors.Wrap(err, "failed to load verifiers from config")
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
apiVersion: config.ratify.deislabs.io/v1beta1
kind: Verifier
metadata:
name: verifier-notation
spec:
name: notation
artifactTypes: application/vnd.cncf.notary.signature
parameters:
verificationCertStores:
certs:
- default/ratify-notation-inline-cert
trustPolicyDoc:
version: "1.0"
trustPolicies:
- name: default
registryScopes:
- "*"
signatureVerification:
level: strict
trustStores:
- ca:certs
trustedIdentities:
- "*"
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ EOF

## AzureWIAuthProvider Implementation
```
// AzureK8Conf describes the configuration of Azure K8 Auth Provider
// AzureK8Conf describes the configuration of Azure K8s Auth Provider
type AzureWIConf struct {
Name string `json:"name"`
}
Expand Down
6 changes: 3 additions & 3 deletions docs/design/K8s Secrets AuthProvider.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# K8s Secrets AuthProvider
Author: Akash Singhal (@akashsinghal)

Goal: Create a Kubernetes Secret Authentication Provider which will use K8 secrets to resolve registry credentials for an artifact. In the `auth-provider` section of the ORAS plugin configuration, the `k8s-secrets` auth-provider contains a list of `secrets` where each specifies the k8 secret name along with optional `namespace` where the secret resides (the namespace ratify is deployed in will be used as the default value). Along with named secrets being used from the config, the service account linked to the Ratify pod will be queried for associated imagePullSecrets and considered during credential resolution.
Goal: Create a Kubernetes Secret Authentication Provider which will use K8s secrets to resolve registry credentials for an artifact. In the `auth-provider` section of the ORAS plugin configuration, the `k8s-secrets` auth-provider contains a list of `secrets` where each specifies the K8s secret name along with optional `namespace` where the secret resides (the namespace ratify is deployed in will be used as the default value). Along with named secrets being used from the config, the service account linked to the Ratify pod will be queried for associated imagePullSecrets and considered during credential resolution.

The provider will support 2 types of k8s secrets: kubernetes.io/dockercfg, kubernetes.io/dockerconfigjson

Expand Down Expand Up @@ -111,7 +111,7 @@ type k8SecretAuthProviderConf struct {
func init() // init calls Register for our k8s-secrets provider

// Create returns a k8AuthProvider instance after parsing auth config and resolving
// named K8 secrets
// named K8s secrets
func (s *k8SecretProviderFactory) Create(authProviderConfig AuthProviderConfig) (AuthProvider, error) {
// unmarshal the json config for auth provider

Expand All @@ -130,7 +130,7 @@ func (s *k8SecretProviderFactory) Create(authProviderConfig AuthProviderConfig)
func (d *k8SecretAuthProvider) Enabled() bool

// Provide finds the secret corresponding to artifact's registryhost,
// extracts the authentication credentials from k8 secret, and
// extracts the authentication credentials from K8s secret, and
// returns AuthConfig
func (d *k8SecretAuthProvider) Provide(artifact string) (AuthConfig, error) {
// check provider is properly Enabled
Expand Down
2 changes: 1 addition & 1 deletion docs/design/Metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ New Helm values:
- `metricsType`: string, default: "prometheus"
- `metricsPort`: int, default: 8888

Corresponding flags for exporter and port will be added to the ratify `serve` command to enable metrics emission in non K8 scenarios.
Corresponding flags for exporter and port will be added to the ratify `serve` command to enable metrics emission in non K8s scenarios.


## Proposed Metrics
Expand Down
2 changes: 1 addition & 1 deletion docs/discussion/Gatekeeper Timeout Constraint.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ Now in the simple case of a single image signature verification, Ratify complete

## Questions
1. Can we extend the timeout?
Gatekeeper team has advised that this is not feasible since the 3 second timeout is in place to mitigate a k8 leader election issue that occures with a higher timeout. https://github.com/open-policy-agent/gatekeeper/issues/870
Gatekeeper team has advised that this is not feasible since the 3 second timeout is in place to mitigate a K8s leader election issue that occures with a higher timeout. https://github.com/open-policy-agent/gatekeeper/issues/870
3. Can we add a retry? Where would we add the retry?
- kubectl doesn't seem to have retry abilities built in
- Helm might have something we can leverage? (/cc: Sajay)
Expand Down
2 changes: 2 additions & 0 deletions internal/constants/constants.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,5 @@ limitations under the License.
package constants

const RatifyPolicy = "ratify-policy"
const EmptyNamespace = ""
const NamespaceSeperator = "/"
13 changes: 7 additions & 6 deletions pkg/common/oras/authprovider/k8secret_authprovider.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@
"time"

re "github.com/deislabs/ratify/errors"
"github.com/deislabs/ratify/pkg/utils"

"github.com/docker/cli/cli/config"
core "k8s.io/api/core/v1"
meta "k8s.io/apimachinery/pkg/apis/meta/v1"
Expand All @@ -51,7 +53,6 @@
}

const defaultName = "default"
const ratifyNamespaceEnv = "RATIFY_NAMESPACE"
const secretTimeout = time.Hour * 12

// init calls Register for our k8Secrets provider
Expand All @@ -60,7 +61,7 @@
}

// Create returns a k8AuthProvider instance after parsing auth config and resolving
// named K8 secrets
// named K8s secrets
func (s *k8SecretProviderFactory) Create(authProviderConfig AuthProviderConfig) (AuthProvider, error) {
conf := k8SecretAuthProviderConf{}
authProviderConfigBytes, err := json.Marshal(authProviderConfig)
Expand All @@ -87,9 +88,9 @@
}

// get name of namespace ratify is running in
namespace := os.Getenv(ratifyNamespaceEnv)
namespace := os.Getenv(utils.RatifyNamespaceEnvVar)

Check warning on line 91 in pkg/common/oras/authprovider/k8secret_authprovider.go

View check run for this annotation

Codecov / codecov/patch

pkg/common/oras/authprovider/k8secret_authprovider.go#L91

Added line #L91 was not covered by tests
if namespace == "" {
return nil, re.ErrorCodeEnvNotSet.WithComponentType(re.AuthProvider).WithDetail(fmt.Sprintf("environment variable %s not set", ratifyNamespaceEnv))
return nil, re.ErrorCodeEnvNotSet.WithComponentType(re.AuthProvider).WithDetail(fmt.Sprintf("environment variable %s not set", utils.RatifyNamespaceEnvVar))

Check warning on line 93 in pkg/common/oras/authprovider/k8secret_authprovider.go

View check run for this annotation

Codecov / codecov/patch

pkg/common/oras/authprovider/k8secret_authprovider.go#L93

Added line #L93 was not covered by tests
}

return &k8SecretAuthProvider{
Expand All @@ -109,10 +110,10 @@
}

// Provide finds secret corresponding to artifact's registry host name, extracts
// the authentication credentials from k8 secret, and returns AuthConfig
// the authentication credentials from K8s secret, and returns AuthConfig
func (d *k8SecretAuthProvider) Provide(ctx context.Context, artifact string) (AuthConfig, error) {
if !d.Enabled(ctx) {
return AuthConfig{}, fmt.Errorf("k8 auth provider not properly enabled")
return AuthConfig{}, fmt.Errorf("K8s auth provider not properly enabled")

Check warning on line 116 in pkg/common/oras/authprovider/k8secret_authprovider.go

View check run for this annotation

Codecov / codecov/patch

pkg/common/oras/authprovider/k8secret_authprovider.go#L116

Added line #L116 was not covered by tests
}

hostName, err := GetRegistryHostName(artifact)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ import (
core "k8s.io/api/core/v1"
)

// Checks K8 Docker Json Config Secret is properly extracted and
// Checks K8s Docker Json Config Secret is properly extracted and
// credentials returned when Provide is called
func TestProvide_K8SecretDockerConfigJson_ReturnsExpected(t *testing.T) {
var testSecret core.Secret
Expand Down
4 changes: 2 additions & 2 deletions pkg/controllers/certificatestore_controller.go
Original file line number Diff line number Diff line change
Expand Up @@ -65,14 +65,14 @@
func (r *CertificateStoreReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
logger := logrus.WithContext(ctx)

var resource = req.Name
var resource = req.NamespacedName.String()

Check warning on line 68 in pkg/controllers/certificatestore_controller.go

View check run for this annotation

Codecov / codecov/patch

pkg/controllers/certificatestore_controller.go#L68

Added line #L68 was not covered by tests
var certStore configv1beta1.CertificateStore

logger.Infof("reconciling certificate store '%v'", resource)

if err := r.Get(ctx, req.NamespacedName, &certStore); err != nil {
if apierrors.IsNotFound(err) {
logger.Infof("deletion detected, removing certificate store %v", req.Name)
logger.Infof("deletion detected, removing certificate store %v", resource)

Check warning on line 75 in pkg/controllers/certificatestore_controller.go

View check run for this annotation

Codecov / codecov/patch

pkg/controllers/certificatestore_controller.go#L75

Added line #L75 was not covered by tests
delete(certificatesMap, resource)
} else {
logger.Error(err, "unable to fetch certificate store")
Expand Down
34 changes: 31 additions & 3 deletions pkg/controllers/verifier_controller.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,17 @@
"context"
"encoding/json"
"fmt"
"os"

configv1beta1 "github.com/deislabs/ratify/api/v1beta1"
"github.com/deislabs/ratify/config"
re "github.com/deislabs/ratify/errors"
"github.com/deislabs/ratify/pkg/utils"
vr "github.com/deislabs/ratify/pkg/verifier"
vc "github.com/deislabs/ratify/pkg/verifier/config"
vf "github.com/deislabs/ratify/pkg/verifier/factory"
"github.com/deislabs/ratify/pkg/verifier/types"

"github.com/sirupsen/logrus"
apierrors "k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/runtime"
Expand Down Expand Up @@ -62,6 +66,7 @@

var verifier configv1beta1.Verifier
var resource = req.Name

Check warning on line 69 in pkg/controllers/verifier_controller.go

View check run for this annotation

Codecov / codecov/patch

pkg/controllers/verifier_controller.go#L69

Added line #L69 was not covered by tests
verifierLogger.Infof("reconciling verifier '%v'", resource)

if err := r.Get(ctx, req.NamespacedName, &verifier); err != nil {
Expand All @@ -75,7 +80,13 @@
return ctrl.Result{}, client.IgnoreNotFound(err)
}

if err := verifierAddOrReplace(verifier.Spec, resource); err != nil {
namespace, err := getCertStoreNamespace(req.Namespace)
if err != nil {
verifierLogger.Error(err, "unable to get default namespace for certstore specified in verifier crd")
return ctrl.Result{}, err
}

Check warning on line 87 in pkg/controllers/verifier_controller.go

View check run for this annotation

Codecov / codecov/patch

pkg/controllers/verifier_controller.go#L83-L87

Added lines #L83 - L87 were not covered by tests

if err = verifierAddOrReplace(verifier.Spec, resource, namespace); err != nil {

Check warning on line 89 in pkg/controllers/verifier_controller.go

View check run for this annotation

Codecov / codecov/patch

pkg/controllers/verifier_controller.go#L89

Added line #L89 was not covered by tests
verifierLogger.Error(err, "unable to create verifier from verifier crd")
return ctrl.Result{}, err
}
Expand All @@ -85,7 +96,7 @@
}

// creates a verifier reference from CRD spec and add store to map
func verifierAddOrReplace(spec configv1beta1.VerifierSpec, objectName string) error {
func verifierAddOrReplace(spec configv1beta1.VerifierSpec, objectName string, namespace string) error {
verifierConfig, err := specToVerifierConfig(spec)

if err != nil {
Expand All @@ -100,7 +111,7 @@
spec.Address = config.GetDefaultPluginPath()
logrus.Infof("Address was empty, setting to default path: %v", spec.Address)
}
verifierReference, err := vf.CreateVerifierFromConfig(verifierConfig, verifierConfigVersion, []string{spec.Address})
verifierReference, err := vf.CreateVerifierFromConfig(verifierConfig, verifierConfigVersion, []string{spec.Address}, namespace)

if err != nil || verifierReference == nil {
logrus.Error(err, "unable to create verifier from verifier config")
Expand Down Expand Up @@ -143,3 +154,20 @@
For(&configv1beta1.Verifier{}).
Complete(r)
}

// Historically certStore defined in trust policy only contains name which means the CertStore cannot be uniquely identified
// If verifierNamesapce is not empty, this method returns the default cert store namespace else returns the ratify deployed namespace
func getCertStoreNamespace(verifierNamesapce string) (string, error) {
// first, check if we can use the verifier namespace
if verifierNamesapce != "" {
return verifierNamesapce, nil
}

// next, return the ratify deployed namespace
ns, found := os.LookupEnv(utils.RatifyNamespaceEnvVar)
if !found {
return "", re.ErrorCodeEnvNotSet.WithComponentType(re.Verifier).WithDetail(fmt.Sprintf("environment variable %s not set", utils.RatifyNamespaceEnvVar))
}

return ns, nil
}
35 changes: 31 additions & 4 deletions pkg/controllers/verifier_controller_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ import (
"testing"

configv1beta1 "github.com/deislabs/ratify/api/v1beta1"
"github.com/deislabs/ratify/internal/constants"
"github.com/deislabs/ratify/pkg/utils"
vr "github.com/deislabs/ratify/pkg/verifier"
"k8s.io/apimachinery/pkg/runtime"
)
Expand All @@ -39,7 +41,7 @@ func TestVerifierAdd_EmptyParameter(t *testing.T) {
}
var resource = "notation"

if err := verifierAddOrReplace(testVerifierSpec, resource); err != nil {
if err := verifierAddOrReplace(testVerifierSpec, resource, constants.EmptyNamespace); err != nil {
t.Fatalf("verifierAddOrReplace() expected no error, actual %v", err)
}
if len(VerifierMap) != 1 {
Expand All @@ -55,7 +57,7 @@ func TestVerifierAdd_WithParameters(t *testing.T) {

var testVerifierSpec = getDefaultLicenseCheckerSpec()

if err := verifierAddOrReplace(testVerifierSpec, "testObject"); err != nil {
if err := verifierAddOrReplace(testVerifierSpec, "testObject", constants.EmptyNamespace); err != nil {
t.Fatalf("verifierAddOrReplace() expected no error, actual %v", err)
}
if len(VerifierMap) != 1 {
Expand All @@ -70,7 +72,7 @@ func TestVerifier_UpdateAndDelete(t *testing.T) {
var testVerifierSpec = getDefaultLicenseCheckerSpec()

// add a verifier
if err := verifierAddOrReplace(testVerifierSpec, resource); err != nil {
if err := verifierAddOrReplace(testVerifierSpec, resource, constants.EmptyNamespace); err != nil {
t.Fatalf("verifierAddOrReplace() expected no error, actual %v", err)
}
if len(VerifierMap) != 1 {
Expand All @@ -80,7 +82,7 @@ func TestVerifier_UpdateAndDelete(t *testing.T) {
// modify the verifier
var parametersString = "{\"allowedLicenses\":[\"MIT\",\"GNU\"]}"
testVerifierSpec = getLicenseCheckerFromParam(parametersString)
if err := verifierAddOrReplace(testVerifierSpec, resource); err != nil {
if err := verifierAddOrReplace(testVerifierSpec, resource, constants.EmptyNamespace); err != nil {
t.Fatalf("verifierAddOrReplace() expected no error, actual %v", err)
}

Expand All @@ -96,6 +98,31 @@ func TestVerifier_UpdateAndDelete(t *testing.T) {
}
}

func TestGetCertStoreNamespace(t *testing.T) {
// error scenario, everything is empty, expect error
_, err := getCertStoreNamespace("")
if err.Error() == "environment variable" {
t.Fatalf("env not set should trigger an error")
}

ratifyDeployedNamespace := "sample"
os.Setenv(utils.RatifyNamespaceEnvVar, ratifyDeployedNamespace)
defer os.Unsetenv(utils.RatifyNamespaceEnvVar)

// scenario1, when default namespace is provided, then we should expect default
verifierNamespace := "verifierNamespace"
ns, _ := getCertStoreNamespace(verifierNamespace)
if ns != verifierNamespace {
t.Fatalf("default namespace expected")
}

// scenario2, default is empty, should return ratify installed namespace
ns, _ = getCertStoreNamespace("")
if ns != ratifyDeployedNamespace {
t.Fatalf("default namespace expected")
}
}

func resetVerifierMap() {
VerifierMap = map[string]vr.ReferenceVerifier{}
}
Expand Down
2 changes: 2 additions & 0 deletions pkg/utils/utils.go
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@ import (
"github.com/opencontainers/go-digest"
)

const RatifyNamespaceEnvVar = "RATIFY_NAMESPACE"

// ParseDigest parses the given string and returns a validated Digest object.
func ParseDigest(digestStr string) (digest.Digest, error) {
digest, err := digest.Parse(digestStr)
Expand Down
Loading
Loading