Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide more prescriptive guidance on multi-tenancy #4514

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
92 changes: 92 additions & 0 deletions api/v1alpha4/identity_types.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
/*
Copyright 2021 The Kubernetes Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package v1alpha4

import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

// IdentityReference is a reference to an infrastructure
// provider identity to be used to provision cluster resources.
type IdentityReference struct {
// Kind of the identity. Must be supported by the infrastructure
// provider and may be either cluster or namespace-scoped.
// +kubebuilder:validation:MinLength=1
Kind string `json:"kind"`

// Name of the infrastructure identity to be used.
// Must be either a cluster-scoped resource, or namespaced-scoped
// resource the same namespace as the resource(s) being provisioned.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// resource the same namespace as the resource(s) being provisioned.
// resource with the same namespace as the resource(s) being provisioned.

Name string `json:"name"`
}

type InfraClusterResourceIdentitySpec struct {
// IdentityReference is a reference to an infrastructure
// provider identity to be used to provision cluster resources.
IdentityRef *IdentityReference `json:"identityRef,omitempty"`
}

type InfraClusterScopedIdentityCommonSpec struct {
// AllowedNamespaces is used to identify which namespaces are allowed to use the identity from.
// Namespaces can be selected either using an array of namespaces or with label selector.
// A namespace should be either in the NamespaceList or match with Selector to use the identity.
//
// +optional
AllowedNamespaces InfraClusterScopedIdentityAllowedNamespaces `json:"allowedNamespaces"`
}

type InfraClusterScopedIdentitySecretSpec struct {
// SecretRef is the name of the secret in the controller
// namespace containing private information for this
// identity.
SecretRef string `json:"secretRef"`
}

type InfraNamespaceScopedIdentitySecretSpec struct {
// SecretRef is the name of the secret in the same namespace
// containing private information for this identity.
// identity.
SecretRef string `json:"secretRef"`
}

type InfraClusterScopedIdentityAllowedNamespaces struct {
// EmptySelectorMatch defines what occurs when no selectors are provided.
// Valid values are MatchNone (match no namespaces) and MatchAll (match all namespaces).
// The default value is MatchNone
// +kubebuilder:default=MatchNone
// +kubebuilder:validation:Enum=MatchNone;MatchAll
EmptySelectorMatch string `json:"emptySelectorMatch"`

// Selector is a label query over a set of resources. The result of matchLabels and
// matchExpressions are ANDed. An empty label selector matches all objects. A null
// label selector matches no objects.
//
// This field is mutually exclusive with list.
// +optional
Selector metav1.LabelSelector `json:"selector"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably stupid question: Can we differentiate between null and empty when the selector is not a pointer?


// DEPRECATED.
// An nil or empty list indicates that resources cannot use the identity from any namespace.
//
// This field should not be used where Cluster API is deployed to Kubernetes
// v1.21 or above, where the selector should be used instead.
// This field is mutually exclusive with selector.
//
// +optional
// +nullable
List []string `json:"list"`
}
61 changes: 61 additions & 0 deletions api/v1alpha4/identity_webhook.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
/*
Copyright 2021 The Kubernetes Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package v1alpha4

import (
"k8s.io/apimachinery/pkg/util/validation/field"
)

// ValidateCreate validates the creation of InfraClusterScopedIdentityAllowedNamespaces
func (a *InfraClusterScopedIdentityAllowedNamespaces) ValidateCreate() error {
return a.validate(nil)
}

// ValidateUpdate validates the update of InfraClusterScopedIdentityAllowedNamespaces
func (a *InfraClusterScopedIdentityAllowedNamespaces) ValidateUpdate(oldA *InfraClusterScopedIdentityAllowedNamespaces) error {
return a.validate(oldA)
}

func (a *InfraClusterScopedIdentityAllowedNamespaces) validate(oldA *InfraClusterScopedIdentityAllowedNamespaces) error {
if len(a.List) == 0 {
return nil
}
if len(a.Selector.MatchExpressions) != 0 || len(a.Selector.MatchLabels) != 0 {
return field.Invalid(field.NewPath("spec", "allowedNamespaces", "selector"), a.Selector, "selector cannot be set simultaneously with spec.allowedNamespaces.list")
}
return nil
}

// ValidateDelete validates the delete of AllowedNamespaces
func (a *InfraClusterScopedIdentityAllowedNamespaces) ValidateDelete() error {
return nil
}

// ValidateCreate validates the creation of AllowedNamespaces
func (i *InfraClusterScopedIdentityCommonSpec) ValidateCreate() error {
return i.AllowedNamespaces.ValidateCreate()
}

// ValidateUpdate validates the update of AllowedNamespaces
func (i *InfraClusterScopedIdentityCommonSpec) ValidateUpdate(oldA *InfraClusterScopedIdentityCommonSpec) error {
return i.AllowedNamespaces.ValidateUpdate(&i.AllowedNamespaces)
}

// ValidateDelete validates the delete of AllowedNamespaces
func (i *InfraClusterScopedIdentityCommonSpec) ValidateDelete() error {
return nil
}
83 changes: 81 additions & 2 deletions docs/book/src/developer/architecture/controllers/multi-tenancy.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Multi tenancy

Multi tenancy in Cluster API defines the capability of an infrastructure provider to manage different credentials, each
one of them corresponding to an infrastructure tenant.
Multi tenancy in Cluster API defines the capability of an infrastructure provider to manage multiple sets of
credentials, each one of them corresponding to an infrastructure tenant.

## Contract

Expand All @@ -11,3 +11,82 @@ In order to support multi tenancy, the following rule applies:
- Providers SHOULD deploy and run any kind of webhook (validation, admission, conversion)
following Cluster API codebase best practices for the same release.
- Providers MUST create and publish a `{type}-component.yaml` accordingly.
- Providers MUST implement a reference to the identity by including `InfraClusterResourceIdentitySpec`
in the relevant resource, e.g.

```go
type InfraCluster struct {
clusterv1.InfraClusterResourceIdentitySpec `json:",inline"`
}
```

- A Namespace field MUST never be added to the ProviderIdentityReference type to avoid crossing namespace boundaries.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the ProviderIdentityReference type still exist?


- Where identity types use private key material, CRs MUST implement a `secretRef` on their spec of type string and only
read secrets from the same namespace as the CR for namespaced scope resources OR the controller namespace for
cluster-scoped resources.

- Providers MAY support `Secret` as a top-level supported identity type (either on top of custom resources or instead of), but only from the same namespace as the owning `InfraCluster` CR.

- Providers MAY support the use of `identityRef` in other low-level resources, such as Load Balancers.

## Supported RBAC Models

Providers MAY support any combination of cluster-scoped or namespace-scoped resources as follows:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if the secret variant should also be added here


### Cluster-scoped global resources for delegated access

In a common use for multi-tenancy, a cloud admin will want to set up a range of identities, and then delegate them to
individual teams. This is best done using global resources to prevent repetition.

#### Cluster-scoped Contract

- Cluster scoped resources MUST be named with `<Provider>Cluster<Type>Identity`. Examples:
- `FabrikamClusterRoleIdentity`
- `FabrikamClusterStaticIdentity`

- Where identity types use private key material, CRs MUST implement a `secretRef` on their spec of type string and only
read secrets from the controller namespace.

- Cluster scoped resources MUST be delegated using a label selector, present on the spec as:

```go
type InfraClusterIdentity struct {
// ADDITIONAL INFRASTRUCTURE SPECIFIC IDENTITY FIELDS//


clusterv1.InfraClusterScopedResourceIdentityCommonSpec `json:",inline"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does not exist in the code, renamed?

}
```

- Providers MAY support an additional `NamespaceList` string slice in the `AllowedNamespaces` struct, included in
clusterv1.InfraClusterScopedResourceIdentityCommonSpec.
- Providers SHOULD use the webhook validations provided in the clusterv1 package for
clusterv1.InfraClusterScopedResourceIdentityCommonSpec

When Cluster API no longer supports Kubernetes versions older than Kubernetes v1.21, when the NamespaceDefaultLabelName
[feature gate] transitioned to Beta, then:

- Providers MUST remove the `NamespaceList` field.
- Conversion webhooks MUST perform the following:
- Translate the NamespaceList into the following match expression:
- `kubernetes.io/metadata.name in (<comma-separated list of namespaces from <NamespaceList>)`
randomvariable marked this conversation as resolved.
Show resolved Hide resolved

<!-- TODO @randomvariable: Remove this line when this https://github.com/kubernetes-sigs/cluster-api/issues/4941
is resolved -->
These behaviours will be encoded in utility functions in the Cluster API repository at a later date.

### Namespaced scoped resources
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
### Namespaced scoped resources
### Namespace-scoped resources


Namespaced scoped resources are useful most particularly when you want to allow developers to provision clusters on
their own accounts, but may not be suitable for every use case.

#### Namespace-scoped contract

- Namespace scoped resources MUST be named `<Provider><Type>Identity`, e.g.:
- `ContosoCloudRoleIdentity`
- `ContosoCloudStaticIdentity`
- Where identity types use private key material, CRs MUST implement a `secretRef` on their spec of type string and only
read secrets from the same namespace as the CR.

[feature gate]: https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/