Skip to content

Commit

Permalink
Fully support generation of K8s RBAC resources
Browse files Browse the repository at this point in the history
These changes address a long-time issue in regards of K8s RBAC resources (see related issues).

These changes allow to generate custom Roles, ClusterRoles, ServiceAccount, and RoleBindings.
Plus, it allows the Kubernetes Client and Kubernetes Config extensions to configure the role binding to generate. 

Fix #16612
Fix #19286
Fix #15422
  • Loading branch information
Sgitario committed Mar 14, 2023
1 parent 110fc5f commit 3817d18
Show file tree
Hide file tree
Showing 46 changed files with 1,470 additions and 137 deletions.
53 changes: 51 additions & 2 deletions docs/src/main/asciidoc/deploying-to-kubernetes.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -860,14 +860,63 @@ implementation("io.quarkus:quarkus-kubernetes-client")
----

To access the API server from within a Kubernetes cluster, some RBAC related resources are required (e.g. a ServiceAccount, a RoleBinding).
So, when the `kubernetes-client` extension is present, the `kubernetes` extension is going to create those resources automatically, so that application will be granted the `view` role.
If more roles are required, they will have to be added manually.
So, when the `kubernetes-client` extension is present, by default the `kubernetes` extension is going to create those resources automatically, so that application will be granted the `view` role. You can fully customize the roles and subjects to use using the properties under `quarkus.kubernetes-client.role-binding`.
If more roles are required, they can create them using the properties under `quarkus.kubernetes.rbac.roles`.

[NOTE]
====
You can disable the RBAC resources generation using the property `quarkus.kubernetes-client.generate-rbac=false`.
====

=== Generating RBAC resources

In some scenarios, it's necessary to generate additional https://kubernetes.io/docs/reference/access-authn-authz/rbac/[RBAC] resources. The Kubernetes extension got you covered with the `quarkus.kubernetes.rbac` properties:

[source,properties]
----
# Generate Role resource with name "my-role" <1>
quarkus.kubernetes.rbac.roles.my-role.policy-rules.0.api-groups=extensions,apps
quarkus.kubernetes.rbac.roles.my-role.policy-rules.0.resources=deployments,deployments/status
quarkus.kubernetes.rbac.roles.my-role.policy-rules.0.verbs=get,watch,list
quarkus.kubernetes.rbac.roles.my-role.policy-rules.0.resources=services
# Generate ServiceAccount resource with name "my-service-account" in namespace "my_namespace" <2>
quarkus.kubernetes.rbac.service-accounts.my-service-account.namespace=my_namespace
# Bind Role "my-role" with ServiceAccount "my-service-account"
quarkus.kubernetes.rbac.role-bindings.my-role-binding.subjects.my-service-account.kind=ServiceAccount
quarkus.kubernetes.rbac.role-bindings.my-role-binding.subjects.my-service-account.namespace=my_namespace
quarkus.kubernetes.rbac.role-bindings.my-role-binding.role-name=my-role
quarkus.kubernetes.rbac.role-bindings.my-role-binding.cluster-wide=false
# Use the service account "my-service-account" in the application
quarkus.kubernetes.service-account=my-service-account <3>
----

<1> In this example, the role "my-role" will be generated with the specified policy rules.
<2> Also, the service account "my-service-account" will be generated.
<3> This property will configure the generated service account "my-service-account" with the generated Deployment resource.

We can also generate the cluster wide role resource of "ClusterRole" kind as follows:

[source,properties]
----
# Generate ClusterRole resource with name "my-cluster-role" <1>
quarkus.kubernetes.rbac.cluster-roles.my-cluster-role.policy-rules.0.api-groups=extensions,apps
quarkus.kubernetes.rbac.cluster-roles.my-cluster-role.policy-rules.0.resources=deployments,deployments/status
quarkus.kubernetes.rbac.cluster-roles.my-cluster-role.policy-rules.0.verbs=get,watch,list
quarkus.kubernetes.rbac.cluster-roles.my-cluster-role.policy-rules.0.resources=services
# Bind Role "my-role" with ServiceAccount "my-service-account"
quarkus.kubernetes.rbac.role-bindings.my-role-binding.subjects.my-service-account.kind=ServiceAccount
quarkus.kubernetes.rbac.role-bindings.my-role-binding.subjects.my-service-account.namespace=my_namespace
quarkus.kubernetes.rbac.role-bindings.my-role-binding.role-name=my-cluster-role
quarkus.kubernetes.rbac.role-bindings.my-role-binding.cluster-wide=true <2>
----

<1> In this example, the cluster role "my-cluster-role" will be generated with the specified policy rules.
<2> We need to specify that the role to bind is a cluster-wide role using the property `cluster-wide`.

=== Deploying to Minikube

https://github.com/kubernetes/minikube[Minikube] is quite popular when a Kubernetes cluster is needed for development purposes. To make the deployment to Minikube
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,9 @@
import io.quarkus.kubernetes.client.runtime.KubernetesClientBuildConfig;
import io.quarkus.kubernetes.client.runtime.KubernetesClientProducer;
import io.quarkus.kubernetes.client.runtime.KubernetesConfigProducer;
import io.quarkus.kubernetes.client.runtime.RoleBindingConfig;
import io.quarkus.kubernetes.spi.KubernetesRoleBindingBuildItem;
import io.quarkus.kubernetes.spi.KubernetesServiceAccountBuildItem;
import io.quarkus.maven.dependency.ArtifactKey;

public class KubernetesClientProcessor {
Expand All @@ -69,6 +71,7 @@ public class KubernetesClientProcessor {
private static final DotName KUBE_SCHEMA = DotName.createSimple(KubeSchema.class.getName());
private static final DotName VISITABLE_BUILDER = DotName.createSimple(VisitableBuilder.class.getName());
private static final DotName CUSTOM_RESOURCE = DotName.createSimple(CustomResource.class.getName());
private static final String SERVICE_ACCOUNT = "ServiceAccount";

private static final DotName JSON_FORMAT = DotName.createSimple(JsonFormat.class.getName());
private static final String[] EMPTY_STRINGS_ARRAY = new String[0];
Expand Down Expand Up @@ -106,12 +109,26 @@ public void process(ApplicationIndexBuildItem applicationIndex, CombinedIndexBui
BuildProducer<ReflectiveClassBuildItem> reflectiveClasses,
BuildProducer<ReflectiveHierarchyBuildItem> reflectiveHierarchies,
BuildProducer<IgnoreJsonDeserializeClassBuildItem> ignoredJsonDeserializationClasses,
BuildProducer<KubernetesServiceAccountBuildItem> serviceAccountProducer,
BuildProducer<KubernetesRoleBindingBuildItem> roleBindingProducer,
BuildProducer<ServiceProviderBuildItem> serviceProviderProducer) {

featureProducer.produce(new FeatureBuildItem(Feature.KUBERNETES_CLIENT));
if (kubernetesClientConfig.generateRbac) {
roleBindingProducer.produce(new KubernetesRoleBindingBuildItem("view", true));
RoleBindingConfig rbacConfig = kubernetesClientConfig.roleBinding;
if (SERVICE_ACCOUNT.equals(rbacConfig.subjectKind) && rbacConfig.subjectName.isEmpty()) {
// generate default service account resource
serviceAccountProducer.produce(new KubernetesServiceAccountBuildItem(true));
}

roleBindingProducer.produce(
new KubernetesRoleBindingBuildItem(rbacConfig.name.orElse(null), null, rbacConfig.labels,
new KubernetesRoleBindingBuildItem.RoleRef(rbacConfig.roleName, rbacConfig.clusterWide),
new KubernetesRoleBindingBuildItem.Subject(
rbacConfig.subjectApiGroup.orElse(null),
rbacConfig.subjectKind,
rbacConfig.subjectName.orElse(null),
rbacConfig.subjectNamespace.orElse(null))));
}

// register fully (and not weakly) for reflection watchers, informers and custom resources
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -170,11 +170,17 @@ public class KubernetesClientBuildConfig {
public Optional<String[]> noProxy;

/**
* Enable the generation of the RBAC manifests.
* Enable the generation of the RBAC manifests. If enabled, it will generate
*/
@ConfigItem(defaultValue = "true")
public boolean generateRbac;

/**
* If generation of RBAC is enabled (see property `quarkus.kubernetes-client.generate-rbac`), then it will use the
* RBAC resources as specified under these properties.
*/
public RoleBindingConfig roleBinding;

/**
* Dev Services
*/
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
package io.quarkus.kubernetes.client.runtime;

import java.util.Map;
import java.util.Optional;

import io.quarkus.runtime.annotations.ConfigGroup;
import io.quarkus.runtime.annotations.ConfigItem;

@ConfigGroup
public class RoleBindingConfig {

/**
* Name of the RoleBinding resource to be generated. If not provided, it will use the application name plus the role
* ref name.
*/
@ConfigItem
public Optional<String> name;

/**
* Labels to add into the RoleBinding resource.
*/
@ConfigItem
public Map<String, String> labels;

/**
* The "kind" resource to use by the Subject element in the generated Role Binding resource.
* By default, it's "ServiceAccount" kind.
*/
@ConfigItem(defaultValue = "ServiceAccount")
public String subjectKind;

/**
* The "apiGroup" resource that matches with the "kind" property. By default, it's empty.
*/
@ConfigItem
public Optional<String> subjectApiGroup;

/**
* The "name" resource to use by the Subject element in the generated Role Binding resource.
* By default, it's the application name.
*/
@ConfigItem
public Optional<String> subjectName;

/**
* The "namespace" resource to use by the Subject element in the generated Role Binding resource.
* By default, it will use the same as provided in the generated resources.
*/
@ConfigItem
public Optional<String> subjectNamespace;

/**
* The name of the Role resource to use by the RoleRef element in the generated Role Binding resource.
* By default, it's "view" role name.
*/
@ConfigItem(defaultValue = "view")
public String roleName;

/**
* If the Role sets in the `role-name` property is cluster wide or not.
* By default, it's "true".
*/
@ConfigItem(defaultValue = "true")
public boolean clusterWide;
}
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
package io.quarkus.kubernetes.config.deployment;

import java.util.Collections;
import java.util.List;

import org.jboss.logmanager.Level;
Expand All @@ -15,12 +14,21 @@
import io.quarkus.kubernetes.config.runtime.KubernetesConfigBuildTimeConfig;
import io.quarkus.kubernetes.config.runtime.KubernetesConfigRecorder;
import io.quarkus.kubernetes.config.runtime.KubernetesConfigSourceConfig;
import io.quarkus.kubernetes.config.runtime.SecretsRoleConfig;
import io.quarkus.kubernetes.spi.KubernetesClusterRoleBuildItem;
import io.quarkus.kubernetes.spi.KubernetesRoleBindingBuildItem;
import io.quarkus.kubernetes.spi.KubernetesRoleBuildItem;
import io.quarkus.kubernetes.spi.PolicyRule;
import io.quarkus.runtime.TlsConfig;

public class KubernetesConfigProcessor {

private static final String ANY_TARGET = null;
private static final List<PolicyRule> POLICY_RULE_FOR_ROLE = List.of(new PolicyRule(
List.of(""),
List.of("secrets"),
List.of("get")));

@BuildStep
@Record(ExecutionTime.RUNTIME_INIT)
public RunTimeConfigurationSourceValueBuildItem configure(KubernetesConfigRecorder recorder,
Expand All @@ -36,15 +44,26 @@ public RunTimeConfigurationSourceValueBuildItem configure(KubernetesConfigRecord
public void handleAccessToSecrets(KubernetesConfigSourceConfig config,
KubernetesConfigBuildTimeConfig buildTimeConfig,
BuildProducer<KubernetesRoleBuildItem> roleProducer,
BuildProducer<KubernetesClusterRoleBuildItem> clusterRoleProducer,
BuildProducer<KubernetesRoleBindingBuildItem> roleBindingProducer,
KubernetesConfigRecorder recorder) {
if (buildTimeConfig.secretsEnabled) {
roleProducer.produce(new KubernetesRoleBuildItem("view-secrets", Collections.singletonList(
new KubernetesRoleBuildItem.PolicyRule(
Collections.singletonList(""),
Collections.singletonList("secrets"),
List.of("get")))));
roleBindingProducer.produce(new KubernetesRoleBindingBuildItem("view-secrets", false));
SecretsRoleConfig roleConfig = buildTimeConfig.secretsRoleConfig;
String roleName = roleConfig.name;
if (roleConfig.generate) {
if (roleConfig.clusterWide) {
clusterRoleProducer.produce(new KubernetesClusterRoleBuildItem(roleName,
POLICY_RULE_FOR_ROLE,
ANY_TARGET));
} else {
roleProducer.produce(new KubernetesRoleBuildItem(roleName,
roleConfig.namespace.orElse(null),
POLICY_RULE_FOR_ROLE,
ANY_TARGET));
}
}

roleBindingProducer.produce(new KubernetesRoleBindingBuildItem(roleName, roleConfig.clusterWide));
}

recorder.warnAboutSecrets(config, buildTimeConfig);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,9 @@ public class KubernetesConfigBuildTimeConfig {
*/
@ConfigItem(name = "secrets.enabled", defaultValue = "false")
public boolean secretsEnabled;

/**
* Role configuration to generate if the "secrets-enabled" property is true.
*/
public SecretsRoleConfig secretsRoleConfig;
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
package io.quarkus.kubernetes.config.runtime;

import java.util.Optional;

import io.quarkus.runtime.annotations.ConfigGroup;
import io.quarkus.runtime.annotations.ConfigItem;

@ConfigGroup
public class SecretsRoleConfig {

/**
* The name of the role.
*/
@ConfigItem(defaultValue = "view-secrets")
public String name;

/**
* The namespace of the role.
*/
@ConfigItem
public Optional<String> namespace;

/**
* Whether the role is cluster wide or not. By default, it's not a cluster wide role.
*/
@ConfigItem(defaultValue = "false")
public boolean clusterWide;

/**
* If the current role is meant to be generated or not. If not, it will only be used to generate the role binding resource.
*/
@ConfigItem(defaultValue = "true")
public boolean generate;
}
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
import io.quarkus.kubernetes.spi.CustomProjectRootBuildItem;
import io.quarkus.kubernetes.spi.DecoratorBuildItem;
import io.quarkus.kubernetes.spi.KubernetesAnnotationBuildItem;
import io.quarkus.kubernetes.spi.KubernetesClusterRoleBuildItem;
import io.quarkus.kubernetes.spi.KubernetesCommandBuildItem;
import io.quarkus.kubernetes.spi.KubernetesDeploymentTargetBuildItem;
import io.quarkus.kubernetes.spi.KubernetesEnvBuildItem;
Expand All @@ -46,6 +47,7 @@
import io.quarkus.kubernetes.spi.KubernetesResourceMetadataBuildItem;
import io.quarkus.kubernetes.spi.KubernetesRoleBindingBuildItem;
import io.quarkus.kubernetes.spi.KubernetesRoleBuildItem;
import io.quarkus.kubernetes.spi.KubernetesServiceAccountBuildItem;

public class KindProcessor {

Expand Down Expand Up @@ -111,6 +113,8 @@ public List<DecoratorBuildItem> createDecorators(ApplicationInfoBuildItem applic
Optional<KubernetesHealthReadinessPathBuildItem> readinessPath,
Optional<KubernetesHealthStartupPathBuildItem> startupPath,
List<KubernetesRoleBuildItem> roles,
List<KubernetesClusterRoleBuildItem> clusterRoles,
List<KubernetesServiceAccountBuildItem> serviceAccounts,
List<KubernetesRoleBindingBuildItem> roleBindings,
Optional<CustomProjectRootBuildItem> customProjectRoot) {

Expand All @@ -120,7 +124,7 @@ public List<DecoratorBuildItem> createDecorators(ApplicationInfoBuildItem applic
livenessPath,
readinessPath,
startupPath,
roles, roleBindings, customProjectRoot);
roles, clusterRoles, serviceAccounts, roleBindings, customProjectRoot);
}

@BuildStep
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
import io.quarkus.kubernetes.spi.CustomProjectRootBuildItem;
import io.quarkus.kubernetes.spi.DecoratorBuildItem;
import io.quarkus.kubernetes.spi.KubernetesAnnotationBuildItem;
import io.quarkus.kubernetes.spi.KubernetesClusterRoleBuildItem;
import io.quarkus.kubernetes.spi.KubernetesCommandBuildItem;
import io.quarkus.kubernetes.spi.KubernetesDeploymentTargetBuildItem;
import io.quarkus.kubernetes.spi.KubernetesEnvBuildItem;
Expand All @@ -43,6 +44,7 @@
import io.quarkus.kubernetes.spi.KubernetesResourceMetadataBuildItem;
import io.quarkus.kubernetes.spi.KubernetesRoleBindingBuildItem;
import io.quarkus.kubernetes.spi.KubernetesRoleBuildItem;
import io.quarkus.kubernetes.spi.KubernetesServiceAccountBuildItem;

public class MinikubeProcessor {

Expand Down Expand Up @@ -107,6 +109,8 @@ public List<DecoratorBuildItem> createDecorators(ApplicationInfoBuildItem applic
Optional<KubernetesHealthReadinessPathBuildItem> readinessPath,
Optional<KubernetesHealthStartupPathBuildItem> startupPath,
List<KubernetesRoleBuildItem> roles,
List<KubernetesClusterRoleBuildItem> clusterRoles,
List<KubernetesServiceAccountBuildItem> serviceAccounts,
List<KubernetesRoleBindingBuildItem> roleBindings,
Optional<CustomProjectRootBuildItem> customProjectRoot) {

Expand All @@ -116,6 +120,6 @@ public List<DecoratorBuildItem> createDecorators(ApplicationInfoBuildItem applic
livenessPath,
readinessPath,
startupPath,
roles, roleBindings, customProjectRoot);
roles, clusterRoles, serviceAccounts, roleBindings, customProjectRoot);
}
}
Loading

0 comments on commit 3817d18

Please sign in to comment.