-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Provide more prescriptive guidance on multi-tenancy #4514
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,92 @@ | ||
/* | ||
Copyright 2021 The Kubernetes Authors. | ||
|
||
Licensed under the Apache License, Version 2.0 (the "License"); | ||
you may not use this file except in compliance with the License. | ||
You may obtain a copy of the License at | ||
|
||
http://www.apache.org/licenses/LICENSE-2.0 | ||
|
||
Unless required by applicable law or agreed to in writing, software | ||
distributed under the License is distributed on an "AS IS" BASIS, | ||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
See the License for the specific language governing permissions and | ||
limitations under the License. | ||
*/ | ||
|
||
package v1alpha4 | ||
|
||
import ( | ||
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" | ||
) | ||
|
||
// IdentityReference is a reference to an infrastructure | ||
// provider identity to be used to provision cluster resources. | ||
type IdentityReference struct { | ||
// Kind of the identity. Must be supported by the infrastructure | ||
// provider and may be either cluster or namespace-scoped. | ||
// +kubebuilder:validation:MinLength=1 | ||
Kind string `json:"kind"` | ||
|
||
// Name of the infrastructure identity to be used. | ||
// Must be either a cluster-scoped resource, or namespaced-scoped | ||
// resource the same namespace as the resource(s) being provisioned. | ||
Name string `json:"name"` | ||
} | ||
|
||
type InfraClusterResourceIdentitySpec struct { | ||
// IdentityReference is a reference to an infrastructure | ||
// provider identity to be used to provision cluster resources. | ||
IdentityRef *IdentityReference `json:"identityRef,omitempty"` | ||
} | ||
|
||
type InfraClusterScopedIdentityCommonSpec struct { | ||
// AllowedNamespaces is used to identify which namespaces are allowed to use the identity from. | ||
// Namespaces can be selected either using an array of namespaces or with label selector. | ||
// A namespace should be either in the NamespaceList or match with Selector to use the identity. | ||
// | ||
// +optional | ||
AllowedNamespaces InfraClusterScopedIdentityAllowedNamespaces `json:"allowedNamespaces"` | ||
} | ||
|
||
type InfraClusterScopedIdentitySecretSpec struct { | ||
// SecretRef is the name of the secret in the controller | ||
// namespace containing private information for this | ||
// identity. | ||
SecretRef string `json:"secretRef"` | ||
} | ||
|
||
type InfraNamespaceScopedIdentitySecretSpec struct { | ||
// SecretRef is the name of the secret in the same namespace | ||
// containing private information for this identity. | ||
// identity. | ||
SecretRef string `json:"secretRef"` | ||
} | ||
|
||
type InfraClusterScopedIdentityAllowedNamespaces struct { | ||
// EmptySelectorMatch defines what occurs when no selectors are provided. | ||
// Valid values are MatchNone (match no namespaces) and MatchAll (match all namespaces). | ||
// The default value is MatchNone | ||
// +kubebuilder:default=MatchNone | ||
// +kubebuilder:validation:Enum=MatchNone;MatchAll | ||
EmptySelectorMatch string `json:"emptySelectorMatch"` | ||
|
||
// Selector is a label query over a set of resources. The result of matchLabels and | ||
// matchExpressions are ANDed. An empty label selector matches all objects. A null | ||
// label selector matches no objects. | ||
// | ||
// This field is mutually exclusive with list. | ||
// +optional | ||
Selector metav1.LabelSelector `json:"selector"` | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Probably stupid question: Can we differentiate between null and empty when the selector is not a pointer? |
||
|
||
// DEPRECATED. | ||
// An nil or empty list indicates that resources cannot use the identity from any namespace. | ||
// | ||
// This field should not be used where Cluster API is deployed to Kubernetes | ||
// v1.21 or above, where the selector should be used instead. | ||
// This field is mutually exclusive with selector. | ||
// | ||
// +optional | ||
// +nullable | ||
List []string `json:"list"` | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,61 @@ | ||
/* | ||
Copyright 2021 The Kubernetes Authors. | ||
|
||
Licensed under the Apache License, Version 2.0 (the "License"); | ||
you may not use this file except in compliance with the License. | ||
You may obtain a copy of the License at | ||
|
||
http://www.apache.org/licenses/LICENSE-2.0 | ||
|
||
Unless required by applicable law or agreed to in writing, software | ||
distributed under the License is distributed on an "AS IS" BASIS, | ||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
See the License for the specific language governing permissions and | ||
limitations under the License. | ||
*/ | ||
|
||
package v1alpha4 | ||
|
||
import ( | ||
"k8s.io/apimachinery/pkg/util/validation/field" | ||
) | ||
|
||
// ValidateCreate validates the creation of InfraClusterScopedIdentityAllowedNamespaces | ||
func (a *InfraClusterScopedIdentityAllowedNamespaces) ValidateCreate() error { | ||
return a.validate(nil) | ||
} | ||
|
||
// ValidateUpdate validates the update of InfraClusterScopedIdentityAllowedNamespaces | ||
func (a *InfraClusterScopedIdentityAllowedNamespaces) ValidateUpdate(oldA *InfraClusterScopedIdentityAllowedNamespaces) error { | ||
return a.validate(oldA) | ||
} | ||
|
||
func (a *InfraClusterScopedIdentityAllowedNamespaces) validate(oldA *InfraClusterScopedIdentityAllowedNamespaces) error { | ||
if len(a.List) == 0 { | ||
return nil | ||
} | ||
if len(a.Selector.MatchExpressions) != 0 || len(a.Selector.MatchLabels) != 0 { | ||
return field.Invalid(field.NewPath("spec", "allowedNamespaces", "selector"), a.Selector, "selector cannot be set simultaneously with spec.allowedNamespaces.list") | ||
} | ||
return nil | ||
} | ||
|
||
// ValidateDelete validates the delete of AllowedNamespaces | ||
func (a *InfraClusterScopedIdentityAllowedNamespaces) ValidateDelete() error { | ||
return nil | ||
} | ||
|
||
// ValidateCreate validates the creation of AllowedNamespaces | ||
func (i *InfraClusterScopedIdentityCommonSpec) ValidateCreate() error { | ||
return i.AllowedNamespaces.ValidateCreate() | ||
} | ||
|
||
// ValidateUpdate validates the update of AllowedNamespaces | ||
func (i *InfraClusterScopedIdentityCommonSpec) ValidateUpdate(oldA *InfraClusterScopedIdentityCommonSpec) error { | ||
return i.AllowedNamespaces.ValidateUpdate(&i.AllowedNamespaces) | ||
} | ||
|
||
// ValidateDelete validates the delete of AllowedNamespaces | ||
func (i *InfraClusterScopedIdentityCommonSpec) ValidateDelete() error { | ||
return nil | ||
} |
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
@@ -1,7 +1,7 @@ | ||||||
# Multi tenancy | ||||||
|
||||||
Multi tenancy in Cluster API defines the capability of an infrastructure provider to manage different credentials, each | ||||||
one of them corresponding to an infrastructure tenant. | ||||||
Multi tenancy in Cluster API defines the capability of an infrastructure provider to manage multiple sets of | ||||||
credentials, each one of them corresponding to an infrastructure tenant. | ||||||
|
||||||
## Contract | ||||||
|
||||||
|
@@ -11,3 +11,82 @@ In order to support multi tenancy, the following rule applies: | |||||
- Providers SHOULD deploy and run any kind of webhook (validation, admission, conversion) | ||||||
following Cluster API codebase best practices for the same release. | ||||||
- Providers MUST create and publish a `{type}-component.yaml` accordingly. | ||||||
- Providers MUST implement a reference to the identity by including `InfraClusterResourceIdentitySpec` | ||||||
in the relevant resource, e.g. | ||||||
|
||||||
```go | ||||||
type InfraCluster struct { | ||||||
clusterv1.InfraClusterResourceIdentitySpec `json:",inline"` | ||||||
} | ||||||
``` | ||||||
|
||||||
- A Namespace field MUST never be added to the ProviderIdentityReference type to avoid crossing namespace boundaries. | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does the |
||||||
|
||||||
- Where identity types use private key material, CRs MUST implement a `secretRef` on their spec of type string and only | ||||||
read secrets from the same namespace as the CR for namespaced scope resources OR the controller namespace for | ||||||
cluster-scoped resources. | ||||||
|
||||||
- Providers MAY support `Secret` as a top-level supported identity type (either on top of custom resources or instead of), but only from the same namespace as the owning `InfraCluster` CR. | ||||||
|
||||||
- Providers MAY support the use of `identityRef` in other low-level resources, such as Load Balancers. | ||||||
|
||||||
## Supported RBAC Models | ||||||
|
||||||
Providers MAY support any combination of cluster-scoped or namespace-scoped resources as follows: | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not sure if the secret variant should also be added here |
||||||
|
||||||
### Cluster-scoped global resources for delegated access | ||||||
|
||||||
In a common use for multi-tenancy, a cloud admin will want to set up a range of identities, and then delegate them to | ||||||
individual teams. This is best done using global resources to prevent repetition. | ||||||
|
||||||
#### Cluster-scoped Contract | ||||||
|
||||||
- Cluster scoped resources MUST be named with `<Provider>Cluster<Type>Identity`. Examples: | ||||||
- `FabrikamClusterRoleIdentity` | ||||||
- `FabrikamClusterStaticIdentity` | ||||||
|
||||||
- Where identity types use private key material, CRs MUST implement a `secretRef` on their spec of type string and only | ||||||
read secrets from the controller namespace. | ||||||
|
||||||
- Cluster scoped resources MUST be delegated using a label selector, present on the spec as: | ||||||
|
||||||
```go | ||||||
type InfraClusterIdentity struct { | ||||||
// ADDITIONAL INFRASTRUCTURE SPECIFIC IDENTITY FIELDS// | ||||||
|
||||||
|
||||||
clusterv1.InfraClusterScopedResourceIdentityCommonSpec `json:",inline"` | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does not exist in the code, renamed? |
||||||
} | ||||||
``` | ||||||
|
||||||
- Providers MAY support an additional `NamespaceList` string slice in the `AllowedNamespaces` struct, included in | ||||||
clusterv1.InfraClusterScopedResourceIdentityCommonSpec. | ||||||
- Providers SHOULD use the webhook validations provided in the clusterv1 package for | ||||||
clusterv1.InfraClusterScopedResourceIdentityCommonSpec | ||||||
|
||||||
When Cluster API no longer supports Kubernetes versions older than Kubernetes v1.21, when the NamespaceDefaultLabelName | ||||||
[feature gate] transitioned to Beta, then: | ||||||
|
||||||
- Providers MUST remove the `NamespaceList` field. | ||||||
- Conversion webhooks MUST perform the following: | ||||||
- Translate the NamespaceList into the following match expression: | ||||||
- `kubernetes.io/metadata.name in (<comma-separated list of namespaces from <NamespaceList>)` | ||||||
randomvariable marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
|
||||||
<!-- TODO @randomvariable: Remove this line when this https://github.com/kubernetes-sigs/cluster-api/issues/4941 | ||||||
is resolved --> | ||||||
These behaviours will be encoded in utility functions in the Cluster API repository at a later date. | ||||||
|
||||||
### Namespaced scoped resources | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
||||||
Namespaced scoped resources are useful most particularly when you want to allow developers to provision clusters on | ||||||
their own accounts, but may not be suitable for every use case. | ||||||
|
||||||
#### Namespace-scoped contract | ||||||
|
||||||
- Namespace scoped resources MUST be named `<Provider><Type>Identity`, e.g.: | ||||||
- `ContosoCloudRoleIdentity` | ||||||
- `ContosoCloudStaticIdentity` | ||||||
- Where identity types use private key material, CRs MUST implement a `secretRef` on their spec of type string and only | ||||||
read secrets from the same namespace as the CR. | ||||||
|
||||||
[feature gate]: https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.