Skip to content

Commit

Permalink
feat: google fixes (#3008)
Browse files Browse the repository at this point in the history 
* fix(google/cert-manager): better conditions on metrics

Signed-off-by: Kevin Lefevre <[email protected]>

* fix(google/thanos): improve service account and conditions

Signed-off-by: Kevin Lefevre <[email protected]>

* fix(google/data): compute project number automatically

Signed-off-by: Kevin Lefevre <[email protected]>

* fix(google/cert-manager): better conditions on metrics

Signed-off-by: Kevin Lefevre <[email protected]>

* fix(google/loki): compute gs service account with data source

Signed-off-by: Kevin Lefevre <[email protected]>

* feat(google/thanos-receive): add Thanos in receiver mode

Signed-off-by: Kevin Lefevre <[email protected]>

* feat(google/kube-prometheus-stack): allow thanos receiver & infer bucket location from datasources

Signed-off-by: Kevin Lefevre <[email protected]>

* feat(google/thanos-receive): add Thanos receiver to global variable

Signed-off-by: Kevin Lefevre <[email protected]>

* chore: update README

Signed-off-by: Kevin Lefevre <[email protected]>

* chore: update pre-commit

Signed-off-by: Kevin Lefevre <[email protected]>

* feat: breaking change before release

BREAKING CHANGE:

* Google: Thanos and Kube prometheus were revamp and some default were
  remove and new condition added, please be careful with changelog when
  upgrading
* Google: By default do not create the SA with workload-identity module
  but let the helm chart create them and annotate them.

Signed-off-by: Kevin Lefevre <[email protected]>

---------

Signed-off-by: Kevin Lefevre <[email protected]>
  • Loading branch information
@ArchiFleKs
ArchiFleKs committed Oct 6, 2024
1 parent eee9acf commit 57145cc
Show file tree
Hide file tree
Showing 14 changed files with 423 additions and 55 deletions.
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,6 @@ repos:
- id: check-merge-conflict
- id: end-of-file-fixer
- repo: https://github.com/renovatebot/pre-commit-hooks
rev: 38.88.0
rev: 38.106.4
hooks:
- id: renovate-config-validator
1 change: 1 addition & 0 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -309,6 +309,7 @@ No modules.
| <a name="input_secrets-store-csi-driver"></a> [secrets-store-csi-driver](#input\_secrets-store-csi-driver) | Customize secrets-store-csi-driver chart, see `secrets-store-csi-driver.tf` for supported values | `any` | `{}` | no |
| <a name="input_thanos"></a> [thanos](#input\_thanos) | Customize thanos chart, see `thanos.tf` for supported values | `any` | `{}` | no |
| <a name="input_thanos-memcached"></a> [thanos-memcached](#input\_thanos-memcached) | Customize thanos chart, see `thanos.tf` for supported values | `any` | `{}` | no |
| <a name="input_thanos-receive"></a> [thanos-receive](#input\_thanos-receive) | Customize thanos chart, see `thanos-receive.tf` for supported values | `any` | `{}` | no |
| <a name="input_thanos-storegateway"></a> [thanos-storegateway](#input\_thanos-storegateway) | Customize thanos chart, see `thanos.tf` for supported values | `any` | `{}` | no |
| <a name="input_thanos-tls-querier"></a> [thanos-tls-querier](#input\_thanos-tls-querier) | Customize thanos chart, see `thanos.tf` for supported values | `any` | `{}` | no |
| <a name="input_tigera-operator"></a> [tigera-operator](#input\_tigera-operator) | Customize tigera-operator chart, see `tigera-operator.tf` for supported values | `any` | `{}` | no |
Expand Down
1 change: 1 addition & 0 deletions modules/aws/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -404,6 +404,7 @@ This module can uses [IRSA](https://aws.amazon.com/blogs/opensource/introducing-
| <a name="input_tags"></a> [tags](#input\_tags) | Map of tags for AWS resources | `map(any)` | `{}` | no |
| <a name="input_thanos"></a> [thanos](#input\_thanos) | Customize thanos chart, see `thanos.tf` for supported values | `any` | `{}` | no |
| <a name="input_thanos-memcached"></a> [thanos-memcached](#input\_thanos-memcached) | Customize thanos chart, see `thanos.tf` for supported values | `any` | `{}` | no |
| <a name="input_thanos-receive"></a> [thanos-receive](#input\_thanos-receive) | Customize thanos chart, see `thanos-receive.tf` for supported values | `any` | `{}` | no |
| <a name="input_thanos-storegateway"></a> [thanos-storegateway](#input\_thanos-storegateway) | Customize thanos chart, see `thanos.tf` for supported values | `any` | `{}` | no |
| <a name="input_thanos-tls-querier"></a> [thanos-tls-querier](#input\_thanos-tls-querier) | Customize thanos chart, see `thanos.tf` for supported values | `any` | `{}` | no |
| <a name="input_tigera-operator"></a> [tigera-operator](#input\_tigera-operator) | Customize tigera-operator chart, see `tigera-operator.tf` for supported values | `any` | `{}` | no |
Expand Down
1 change: 1 addition & 0 deletions modules/azure/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -219,6 +219,7 @@ No modules.
| <a name="input_secrets-store-csi-driver"></a> [secrets-store-csi-driver](#input\_secrets-store-csi-driver) | Customize secrets-store-csi-driver chart, see `secrets-store-csi-driver.tf` for supported values | `any` | `{}` | no |
| <a name="input_thanos"></a> [thanos](#input\_thanos) | Customize thanos chart, see `thanos.tf` for supported values | `any` | `{}` | no |
| <a name="input_thanos-memcached"></a> [thanos-memcached](#input\_thanos-memcached) | Customize thanos chart, see `thanos.tf` for supported values | `any` | `{}` | no |
| <a name="input_thanos-receive"></a> [thanos-receive](#input\_thanos-receive) | Customize thanos chart, see `thanos-receive.tf` for supported values | `any` | `{}` | no |
| <a name="input_thanos-storegateway"></a> [thanos-storegateway](#input\_thanos-storegateway) | Customize thanos chart, see `thanos.tf` for supported values | `any` | `{}` | no |
| <a name="input_thanos-tls-querier"></a> [thanos-tls-querier](#input\_thanos-tls-querier) | Customize thanos chart, see `thanos.tf` for supported values | `any` | `{}` | no |
| <a name="input_tigera-operator"></a> [tigera-operator](#input\_tigera-operator) | Customize tigera-operator chart, see `tigera-operator.tf` for supported values | `any` | `{}` | no |
Expand Down
22 changes: 19 additions & 3 deletions modules/google/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -53,8 +53,11 @@ Provides various Kubernetes addons that are often used on Kubernetes with GCP
| <a name="module_iam_assumable_sa_kube-prometheus-stack_grafana"></a> [iam\_assumable\_sa\_kube-prometheus-stack\_grafana](#module\_iam\_assumable\_sa\_kube-prometheus-stack\_grafana) | terraform-google-modules/kubernetes-engine/google//modules/workload-identity | ~> 33.0 |
| <a name="module_iam_assumable_sa_kube-prometheus-stack_thanos"></a> [iam\_assumable\_sa\_kube-prometheus-stack\_thanos](#module\_iam\_assumable\_sa\_kube-prometheus-stack\_thanos) | terraform-google-modules/kubernetes-engine/google//modules/workload-identity | ~> 33.0 |
| <a name="module_iam_assumable_sa_loki-stack"></a> [iam\_assumable\_sa\_loki-stack](#module\_iam\_assumable\_sa\_loki-stack) | terraform-google-modules/kubernetes-engine/google//modules/workload-identity | ~> 33.0 |
| <a name="module_iam_assumable_sa_thanos"></a> [iam\_assumable\_sa\_thanos](#module\_iam\_assumable\_sa\_thanos) | terraform-google-modules/kubernetes-engine/google//modules/workload-identity | ~> 33.0 |
| <a name="module_iam_assumable_sa_thanos-compactor"></a> [iam\_assumable\_sa\_thanos-compactor](#module\_iam\_assumable\_sa\_thanos-compactor) | terraform-google-modules/kubernetes-engine/google//modules/workload-identity | ~> 33.0 |
| <a name="module_iam_assumable_sa_thanos-receive"></a> [iam\_assumable\_sa\_thanos-receive](#module\_iam\_assumable\_sa\_thanos-receive) | terraform-google-modules/kubernetes-engine/google//modules/workload-identity | ~> 33.0 |
| <a name="module_iam_assumable_sa_thanos-receive-compactor"></a> [iam\_assumable\_sa\_thanos-receive-compactor](#module\_iam\_assumable\_sa\_thanos-receive-compactor) | terraform-google-modules/kubernetes-engine/google//modules/workload-identity | ~> 33.0 |
| <a name="module_iam_assumable_sa_thanos-receive-receive"></a> [iam\_assumable\_sa\_thanos-receive-receive](#module\_iam\_assumable\_sa\_thanos-receive-receive) | terraform-google-modules/kubernetes-engine/google//modules/workload-identity | ~> 33.0 |
| <a name="module_iam_assumable_sa_thanos-receive-sg"></a> [iam\_assumable\_sa\_thanos-receive-sg](#module\_iam\_assumable\_sa\_thanos-receive-sg) | terraform-google-modules/kubernetes-engine/google//modules/workload-identity | ~> 33.0 |
| <a name="module_iam_assumable_sa_thanos-sg"></a> [iam\_assumable\_sa\_thanos-sg](#module\_iam\_assumable\_sa\_thanos-sg) | terraform-google-modules/kubernetes-engine/google//modules/workload-identity | ~> 33.0 |
| <a name="module_iam_assumable_sa_thanos-storegateway"></a> [iam\_assumable\_sa\_thanos-storegateway](#module\_iam\_assumable\_sa\_thanos-storegateway) | terraform-google-modules/kubernetes-engine/google//modules/workload-identity | ~> 33.0 |
| <a name="module_kube-prometheus-stack_grafana-iam-member"></a> [kube-prometheus-stack\_grafana-iam-member](#module\_kube-prometheus-stack\_grafana-iam-member) | terraform-google-modules/iam/google//modules/member_iam | ~> 8.0 |
Expand All @@ -63,6 +66,8 @@ Provides various Kubernetes addons that are often used on Kubernetes with GCP
| <a name="module_loki-stack_bucket"></a> [loki-stack\_bucket](#module\_loki-stack\_bucket) | terraform-google-modules/cloud-storage/google//modules/simple_bucket | ~> 6.0 |
| <a name="module_loki-stack_bucket_iam"></a> [loki-stack\_bucket\_iam](#module\_loki-stack\_bucket\_iam) | terraform-google-modules/iam/google//modules/storage_buckets_iam | ~> 8.0 |
| <a name="module_loki-stack_kms_bucket"></a> [loki-stack\_kms\_bucket](#module\_loki-stack\_kms\_bucket) | terraform-google-modules/kms/google | ~> 3.0 |
| <a name="module_thanos-receive_bucket"></a> [thanos-receive\_bucket](#module\_thanos-receive\_bucket) | terraform-google-modules/cloud-storage/google | ~> 6.0 |
| <a name="module_thanos-receive_kms_bucket"></a> [thanos-receive\_kms\_bucket](#module\_thanos-receive\_kms\_bucket) | terraform-google-modules/kms/google | ~> 3.0 |
| <a name="module_thanos-storegateway_bucket_iam"></a> [thanos-storegateway\_bucket\_iam](#module\_thanos-storegateway\_bucket\_iam) | terraform-google-modules/iam/google//modules/storage_buckets_iam | ~> 8.0 |
| <a name="module_thanos_bucket"></a> [thanos\_bucket](#module\_thanos\_bucket) | terraform-google-modules/cloud-storage/google//modules/simple_bucket | ~> 6.0 |
| <a name="module_thanos_kms_bucket"></a> [thanos\_kms\_bucket](#module\_thanos\_kms\_bucket) | terraform-google-modules/kms/google | ~> 3.0 |
Expand All @@ -84,11 +89,18 @@ Provides various Kubernetes addons that are often used on Kubernetes with GCP
| [google_service_account_iam_policy.admin-account-iam](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/service_account_iam_policy) | resource |
| [google_storage_bucket_iam_member.kube_prometheus_stack_thanos_bucket_objectAdmin_iam_permission](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/storage_bucket_iam_member) | resource |
| [google_storage_bucket_iam_member.kube_prometheus_stack_thanos_bucket_objectViewer_iam_permission](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/storage_bucket_iam_member) | resource |
| [google_storage_bucket_iam_member.thanos-receive-receive_gcs_iam_objectViewer_permissions](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/storage_bucket_iam_member) | resource |
| [google_storage_bucket_iam_member.thanos-receive_compactor_gcs_iam_legacyBucketWriter_permissions](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/storage_bucket_iam_member) | resource |
| [google_storage_bucket_iam_member.thanos-receive_compactor_gcs_iam_objectCreator_permissions](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/storage_bucket_iam_member) | resource |
| [google_storage_bucket_iam_member.thanos-receive_compactor_gcs_iam_objectViewer_permissions](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/storage_bucket_iam_member) | resource |
| [google_storage_bucket_iam_member.thanos-receive_receive_gcs_iam_objectCreator_permissions](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/storage_bucket_iam_member) | resource |
| [google_storage_bucket_iam_member.thanos-receive_sg_gcs_iam_objectCreator_permissions](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/storage_bucket_iam_member) | resource |
| [google_storage_bucket_iam_member.thanos-receive_sg_gcs_iam_objectViewer_permissions](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/storage_bucket_iam_member) | resource |
| [google_storage_bucket_iam_member.thanos_compactor_gcs_iam_legacyBucketWriter_permissions](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/storage_bucket_iam_member) | resource |
| [google_storage_bucket_iam_member.thanos_compactor_gcs_iam_objectCreator_permissions](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/storage_bucket_iam_member) | resource |
| [google_storage_bucket_iam_member.thanos_compactor_gcs_iam_objectViewer_permissions](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/storage_bucket_iam_member) | resource |
| [google_storage_bucket_iam_member.thanos_gcs_iam_objectCreator_permissions](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/storage_bucket_iam_member) | resource |
| [google_storage_bucket_iam_member.thanos_gcs_iam_objectViewer_permissions](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/storage_bucket_iam_member) | resource |
| [google_storage_bucket_iam_member.thanos_receive_gcs_iam_objectCreator_permissions](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/storage_bucket_iam_member) | resource |
| [google_storage_bucket_iam_member.thanos_receive_gcs_iam_objectViewer_permissions](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/storage_bucket_iam_member) | resource |
| [google_storage_bucket_iam_member.thanos_sg_gcs_iam_objectCreator_permissions](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/storage_bucket_iam_member) | resource |
| [google_storage_bucket_iam_member.thanos_sg_gcs_iam_objectViewer_permissions](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/storage_bucket_iam_member) | resource |
| [helm_release.admiralty](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) | resource |
Expand All @@ -113,6 +125,7 @@ Provides various Kubernetes addons that are often used on Kubernetes with GCP
| [helm_release.secrets-store-csi-driver](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) | resource |
| [helm_release.thanos](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) | resource |
| [helm_release.thanos-memcached](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) | resource |
| [helm_release.thanos-receive](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) | resource |
| [helm_release.thanos-storegateway](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) | resource |
| [helm_release.thanos-tls-querier](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) | resource |
| [helm_release.traefik](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) | resource |
Expand Down Expand Up @@ -145,6 +158,7 @@ Provides various Kubernetes addons that are often used on Kubernetes with GCP
| [kubernetes_namespace.sealed-secrets](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/namespace) | resource |
| [kubernetes_namespace.secrets-store-csi-driver](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/namespace) | resource |
| [kubernetes_namespace.thanos](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/namespace) | resource |
| [kubernetes_namespace.thanos-receive](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/namespace) | resource |
| [kubernetes_namespace.traefik](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/namespace) | resource |
| [kubernetes_namespace.velero](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/namespace) | resource |
| [kubernetes_namespace.victoria-metrics-k8s-stack](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/namespace) | resource |
Expand Down Expand Up @@ -234,6 +248,7 @@ Provides various Kubernetes addons that are often used on Kubernetes with GCP
| [tls_self_signed_cert.thanos-tls-querier-ca-cert](https://registry.terraform.io/providers/hashicorp/tls/latest/docs/resources/self_signed_cert) | resource |
| [tls_self_signed_cert.webhook_issuer_tls](https://registry.terraform.io/providers/hashicorp/tls/latest/docs/resources/self_signed_cert) | resource |
| [github_repository.main](https://registry.terraform.io/providers/integrations/github/latest/docs/data-sources/repository) | data source |
| [google_client_config.current](https://registry.terraform.io/providers/hashicorp/google/latest/docs/data-sources/client_config) | data source |
| [google_iam_policy.velero](https://registry.terraform.io/providers/hashicorp/google/latest/docs/data-sources/iam_policy) | data source |
| [google_project.current](https://registry.terraform.io/providers/hashicorp/google/latest/docs/data-sources/project) | data source |
| [http_http.prometheus-operator_crds](https://registry.terraform.io/providers/hashicorp/http/latest/docs/data-sources/http) | data source |
Expand Down Expand Up @@ -286,6 +301,7 @@ Provides various Kubernetes addons that are often used on Kubernetes with GCP
| <a name="input_tags"></a> [tags](#input\_tags) | Map of tags for Google resources | `map(any)` | `{}` | no |
| <a name="input_thanos"></a> [thanos](#input\_thanos) | Customize thanos chart, see `thanos.tf` for supported values | `any` | `{}` | no |
| <a name="input_thanos-memcached"></a> [thanos-memcached](#input\_thanos-memcached) | Customize thanos chart, see `thanos.tf` for supported values | `any` | `{}` | no |
| <a name="input_thanos-receive"></a> [thanos-receive](#input\_thanos-receive) | Customize thanos chart, see `thanos-receive.tf` for supported values | `any` | `{}` | no |
| <a name="input_thanos-storegateway"></a> [thanos-storegateway](#input\_thanos-storegateway) | Customize thanos chart, see `thanos.tf` for supported values | `any` | `{}` | no |
| <a name="input_thanos-tls-querier"></a> [thanos-tls-querier](#input\_thanos-tls-querier) | Customize thanos chart, see `thanos.tf` for supported values | `any` | `{}` | no |
| <a name="input_tigera-operator"></a> [tigera-operator](#input\_tigera-operator) | Customize tigera-operator chart, see `tigera-operator.tf` for supported values | `any` | `{}` | no |
Expand Down
2 changes: 1 addition & 1 deletion modules/google/cert-manager.tf
View file
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ serviceAccount:
iam.gke.io/gcp-service-account: "${local.cert-manager.create_iam_resources && local.cert-manager.enabled ? module.cert_manager_workload_identity[0].gcp_service_account_email : ""}"
prometheus:
servicemonitor:
enabled: ${local.cert-manager.enable_monitoring}
enabled: ${local.kube-prometheus-stack["enabled"] || local.victoria-metrics-k8s-stack["enabled"] || local.cert-manager.enable_monitoring}
honorLabels: true
securityContext:
fsGroup: 1001
Expand Down
2 changes: 2 additions & 0 deletions modules/google/data.tf
View file
Original file line number Diff line number Diff line change
@@ -1 +1,3 @@
data "google_project" "current" {}

data "google_client_config" "current" {}
2 changes: 1 addition & 1 deletion modules/google/external-dns.tf
View file
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ locals {
annotations:
iam.gke.io/gcp-service-account: '${module.external_dns_workload_identity[k].gcp_service_account_email}'
serviceMonitor:
enabled: ${v.enable_monitoring}
enabled: ${local.kube-prometheus-stack["enabled"] || local.victoria-metrics-k8s-stack["enabled"] || v.enable_monitoring}
priorityClassName: ${local.priority-class["create"] ? kubernetes_priority_class.kubernetes_addons[0].metadata[0].name : ""}
VALUES
},
Expand Down
Loading

0 comments on commit 57145cc

Please sign in to comment.