Refactor based on PR suggestions

packs/promtail/README.md

@@ -21,17 +21,18 @@ If no custom configuration file is provided, a default template will be used whi
 |------|-------------|------|---------|:--------:|
 | <a name="input_job_name"></a> [job\_name](#input\_job\_name) | The name to use as the job name which overrides using the pack name. | `string` | `""` | no |
 | <a name="input_service_name"></a> [service\_name](#input\_service\_name) | Name used to register the Consul Service | `string` | `"promtail"` | no |
-| <a name="input_service_check_name"></a> [service\_check\_name](#input\_service\_check\_name) | Name of the service check registered with the Consul Service | `string` | `"Readiness"` | no |
+| <a name="input_namespace"></a> [namespace](#input\_namespace) | The namespace where the job should be placed. | `string` | `"default"` | no |
 | <a name="input_datacenters"></a> [datacenters](#input\_datacenters) | A list of datacenters in the region which are eligible for task placement. | `list(string)` | <pre>[<br>  "dc1"<br>]</pre> | no |
 | <a name="input_region"></a> [region](#input\_region) | The region where the job should be placed. | `string` | `"global"` | no |
 | <a name="input_version_tag"></a> [version\_tag](#input\_version\_tag) | The docker image version. For options, see https://hub.docker.com/grafana/promtail | `string` | `"latest"` | no |
-| <a name="input_http_port"></a> [http\_port](#input\_http\_port) | The Nomad client port that routes to the Promtail. | `number` | `9080` | no |
-| <a name="input_resources"></a> [resources](#input\_resources) | The resource to assign to the promtail service task. | <pre>object({<br>    cpu    = number<br>    memory = number<br>  })</pre> | <pre>{<br>  "cpu": 200,<br>  "memory": 256<br>}</pre> | no |
+| <a name="input_privileged"></a> [privileged](#input\_privileged) | Controls whether the container will be run as a privileged container | `bool` | `false` | no |
 | <a name="input_config_file"></a> [config\_file](#input\_config\_file) | Path to custom Promtail configuration file. | `string` | `""` | no |
-| <a name="input_mount_journal"></a> [mount\_journal](#input\_mount\_journal) | Controls whether /var/log/journal is mounted in the container. If true, container will be run privileged. | `bool` | `true` | no |
-| <a name="input_mount_machine_id"></a> [mount\_machine\_id](#input\_mount\_machine\_id) | Controls whether /etc/machine-id is mounted in the container. If true, container will be run privileged. | `bool` | `true` | no |
-| <a name="input_privileged_container"></a> [privileged\_container](#input\_privileged\_container) | Run as a privileged container. Setting mount\_journal or mount\_machine\_id to true will override this. | `bool` | `false` | no |
 | <a name="input_client_urls"></a> [client\_urls](#input\_client\_urls) | A list of client url's for promtail to send it's data to. | `list(string)` | `[]` | no |
 | <a name="input_journal_max_age"></a> [journal\_max\_age](#input\_journal\_max\_age) | Maximum age of journald entries to scrape. | `string` | `"12h"` | no |
-| <a name="input_log_level"></a> [log\_level](#input\_log\_level) | Promtail log level configuration. | `string` | `"info"` | no |
-| <a name="input_upstreams"></a> [upstreams](#input\_upstreams) | Define Connect Upstreams used by Promtail. | <pre>list(object({<br>    name = string<br>    port = number<br>  }))</pre> | `[]` | no |
+| <a name="input_constraints"></a> [constraints](#input\_constraints) | Constraints to apply to the entire job. | <pre>list(object({<br>    attribute = string<br>    operator  = string<br>    value     = string<br>  }))</pre> | <pre>[<br>  {<br>    "attribute": "${attr.kernel.name}",<br>    "operator": "",<br>    "value": "linux"<br>  }<br>]</pre> | no |
+| <a name="input_promtail_group_network"></a> [promtail\_group\_network](#input\_promtail\_group\_network) | The Promtail network configuration options. | <pre>object({<br>    mode  = string<br>    ports = map(number)<br>  })</pre> | <pre>{<br>  "mode": "bridge",<br>  "ports": {<br>    "http": 9090<br>  }<br>}</pre> | no |
+| <a name="input_promtail_group_services"></a> [promtail\_group\_services](#input\_promtail\_group\_services) | Configuration options of the promtail services and checks. | <pre>list(object({<br>    service_port_label = string<br>    service_name       = string<br>    service_tags       = list(string)<br>    check_enabled      = bool<br>    check_path         = string<br>    check_interval     = string<br>    check_timeout      = string<br>    upstreams = list(object({<br>      name = string<br>      port = number<br>    }))<br>  }))</pre> | <pre>[<br>  {<br>    "check_enabled": true,<br>    "check_interval": "3s",<br>    "check_path": "/ready",<br>    "check_timeout": "1s",<br>    "service_name": "promtail",<br>    "service_port_label": "http",<br>    "service_tags": [],<br>    "upstreams": []<br>  }<br>]</pre> | no |
+| <a name="input_resources"></a> [resources](#input\_resources) | The resource to assign to the promtail service task. | <pre>object({<br>    cpu    = number<br>    memory = number<br>  })</pre> | <pre>{<br>  "cpu": 200,<br>  "memory": 256<br>}</pre> | no |
+| <a name="input_container_args"></a> [container\_args](#input\_container\_args) | Arguments passed to the Promtail docker container | `list(string)` | <pre>[<br>  "-config.file=/etc/promtail/promtail-config.yaml",<br>  "-log.level=info"<br>]</pre> | no |
+| <a name="input_extra_mounts"></a> [extra\_mounts](#input\_extra\_mounts) | Additional mounts to create in the Promtail container | <pre>list(object({<br>    type     = string<br>    source   = string<br>    target   = string<br>    readonly = bool<br>    bind_options = list(object({<br>      name  = string<br>      value = string<br>    }))<br>  }))</pre> | `[]` | no |
+| <a name="input_default_mounts"></a> [default\_mounts](#input\_default\_mounts) | Mounts that are configured when using the default Promtail configuration | <pre>list(object({<br>    type     = string<br>    source   = string<br>    target   = string<br>    readonly = bool<br>    bind_options = list(object({<br>      name  = string<br>      value = string<br>    }))<br>  }))</pre> | <pre>[<br>  {<br>    "bind_options": [<br>      {<br>        "name": "propagation",<br>        "value": "rshared"<br>      }<br>    ],<br>    "readonly": true,<br>    "source": "/var/log/journal",<br>    "target": "/var/log/journal",<br>    "type": "bind"<br>  },<br>  {<br>    "bind_options": [<br>      {<br>        "name": "propagation",<br>        "value": "rshared"<br>      }<br>    ],<br>    "readonly": false,<br>    "source": "/etc/machine-id",<br>    "target": "/etc/machine-id",<br>    "type": "bind"<br>  }<br>]</pre> | no |

packs/promtail/metadata.hcl

@@ -1,6 +1,6 @@
 app {
   url    = "https://grafana.com/docs/promtail/latest/clients/promtail"
-  author = "Ricky Grassmuck"
+  author = "Grafana"
 }
 
 pack {

packs/promtail/templates/_helpers.tpl

@@ -16,36 +16,105 @@ region = [[ .promtail.region | quote]]
 [[- end -]]
 [[- end -]]
 
+[[- define "constraints" -]]
+[[- range $idx, $constraint := . ]]
+  constraint {
+    attribute = [[ $constraint.attribute | quote ]]
+    [[ if $constraint.operator -]]
+    operator  = [[ $constraint.operator | quote ]]
+    [[ end -]]
+    value     = [[ $constraint.value | quote ]]
+  }
+[[- end ]]
+[[- end -]]
+
 // Use provided config file if defined, else render a default
 [[- define "promtail_config" -]]
-[[- if (eq .promtail.config_file "") -]]
-server:
-  http_listen_port: [[ .promtail.http_port ]]
-  log_level: [[ .promtail.log_level ]]
-
-positions:
-  filename: /tmp/positions.yaml
-
-clients:
-  [[ range $idx, $url := .promtail.client_urls ]][[if $idx]]  [[end]][[ $url | printf "- url: %s\n" ]][[ end ]]
-scrape_configs:
-- job_name: journal
-  journal:
-    max_age: [[ .promtail.journal_max_age ]]
-    json: false
-    labels:
-      job: systemd-journal
-  relabel_configs:
-  - source_labels:
-    - __journal__systemd_unit
-    target_label: systemd_unit
-  - source_labels:
-    - __journal__hostname
-    target_label: nodename
-  - source_labels:
-    - __journal_syslog_identifier
-    target_label: syslog_identifier
-[[- else -]]
+[[- if (eq .promtail.config_file "") ]]
+        server:
+          http_listen_port: [[ .promtail.http_port ]]
+          log_level: [[ .promtail.log_level ]]
+
+        positions:
+          filename: /tmp/positions.yaml
+
+        clients:
+          [[ range $idx, $url := .promtail.client_urls ]][[if $idx]]  [[end]][[ $url | printf "- url: %s\n" ]][[ end ]]
+        scrape_configs:
+        - job_name: journal
+          journal:
+            max_age: [[ .promtail.journal_max_age ]]
+            json: false
+            labels:
+              job: systemd-journal
+          relabel_configs:
+          - source_labels:
+            - __journal__systemd_unit
+            target_label: systemd_unit
+          - source_labels:
+            - __journal__hostname
+            target_label: nodename
+          - source_labels:
+            - __journal_syslog_identifier
+            target_label: syslog_identifier
+[[- else ]]
 [[ $config := .promtail.config_file ]][[ fileContents $config ]]
+[[- end ]]
 [[- end -]]
+
+// Generic "service" block template
+[[- define "service" -]]
+[[- range $idx, $service := . ]]
+    service {
+      name = [[ $service.service_name | quote ]]
+      port = [[ $service.service_port_label | quote ]]
+      tags = [ [[- range $idx, $dc := $service.service_tags ]][[if $idx]],[[end]][[ $dc | quote ]][[ end -]] ]
+      [[- if gt (len $service.upstreams) 0 ]] 
+      connect {
+        sidecar_service {
+          proxy {
+            [[- if gt (len $service.upstreams) 0 ]]
+            [[- range $upstream := $service.upstreams ]]
+            upstreams {
+              destination_name = [[ $upstream.name | quote ]]
+              local_bind_port  = [[ $upstream.port ]]
+            }
+            [[- end ]]
+            [[- end ]]
+          }
+        }
+      }
+      [[- end ]]
+      check {
+        type     = "http"
+        path     = [[ $service.check_path | quote ]]
+        interval = [[ $service.check_interval | quote ]]
+        timeout  = [[ $service.check_timeout | quote ]]
+      }
+    }
+[[- end ]]
+[[- end -]]
+
+[[- define "env" -]]
+      env {
+        [[- range $idx, $var := . ]]
+        [[ $var.name | quote ]] = [[ $var.value | quote ]]
+        [[- end ]]
+      }
+[[- end -]]
+
+[[- define "mounts" -]]
+[[- range $idx, $mount := . ]]
+        mount {
+          type = [[ $mount.type | quote ]]
+          target = [[ $mount.target | quote ]]
+          source = [[ $mount.source | quote ]]
+          readonly = [[ $mount.readonly ]]
+          bind_options { 
+            [[- range $idx, $opt := $mount.bind_options ]]
+            [[ $opt.name ]] = [[ $opt.value | quote ]]
+            [[- end ]]
+          }
+        }
+[[- end ]]
 [[- end -]]

packs/promtail/templates/promtail.nomad.tpl

@@ -1,70 +1,40 @@
 job [[ template "job_name" . ]] {
 
   [[ template "region" . ]]
-
   datacenters = [ [[ range $idx, $dc := .promtail.datacenters ]][[if $idx]],[[end]][[ $dc | quote ]][[ end ]] ]
-  type = "system"
+  namespace   = [[ .promtail.namespace | quote ]]
+  type        = "system"
   
-  constraint {
-    attribute = "${attr.kernel.name}"
-    value     = "linux"
-  }
+  [[ template "constraints" .promtail.constraints ]]
 
   group "promtail" {
-    count = 1
-
     network {
-      mode = "bridge"
-      port "http" {
-        to = [[ .promtail.http_port ]]
-      }
-    }
-
-    service {
-      name = [[ .promtail.service_name | quote ]]
-      port = "http"
-      [[- if not (eq (len .promtail.upstreams) 0) ]] // Render Connect upstreams if defined
-      connect {
-        sidecar_service {
-          proxy {
-            [[- range $upstream := .promtail.upstreams ]]
-            upstreams {
-              destination_name = [[ $upstream.name | quote ]]
-              local_bind_port  = [[ $upstream.port ]]
-            }
-            [[- end ]]
-          }
-        }
+      mode = [[ .promtail.promtail_group_network.mode | quote ]]
+      [[- range $label, $to := .promtail.promtail_group_network.ports ]]
+      port [[ $label | quote ]] {
+        to = [[ $to ]]
       }
       [[- end ]]
-      check {
-        name = [[ .promtail.service_check_name | quote ]]
-        port = "http"
-        type = "http"
-        path = "/ready"
-        timeout = "2s"
-        interval = "10s"
-      }
     }
 
+    [[- if .promtail.promtail_task_services ]]
+    [[ template "service" .promtail.promtail_group_services ]]
+    [[- end ]]
+
     task "promtail" {
       driver = "docker"
       
       template {
         destination = "local/promtail-config.yaml"
-        data = <<EOT
-[[ template "promtail_config" . ]]
-EOT
+        data = <<-EOT
+        [[ template "promtail_config" . ]]
+        EOT
       }
 
       config {
         image = "grafana/promtail:[[ .promtail.version_tag ]]"
-        args = [
-          "-config.file=/etc/promtail/promtail-config.yaml",
-          "-log.level=[[ .promtail.log_level ]]"
-        ]
-
-        privileged = [[ if or (.promtail.mount_journal) (.promtail.mount_machine_id) (.promtail.privileged_container) ]]true[[ else ]]false[[ end ]]
+        privileged = [[ if or (eq .promtail.config_file "") (.promtail.privileged) ]]true[[ else ]]false[[ end ]]
+        args = [ [[ range $idx, $dc := .promtail.container_args ]][[if $idx]], [[end]][[ $dc | quote ]][[ end ]] ]
 
         mount {
           type = "bind"
@@ -73,24 +43,15 @@ EOT
           readonly = false
           bind_options { propagation = "rshared" }
         }
-        [[ if .promtail.mount_journal ]]
-        mount {
-          type = "bind"
-          target = "/var/log/journal"
-          source = "/var/log/journal"
-          readonly = true
-          bind_options { propagation = "rshared" }
-        }
-        [[ end ]]
-        [[ if .promtail.mount_machine_id ]]
-        mount {
-          type = "bind"
-          target = "/etc/machine-id"
-          source = "/etc/machine-id"
-          readonly = false
-          bind_options { propagation = "rshared" }
-        }
-        [[ end ]]
+
+        [[- if (eq .promtail.config_file "") ]]
+        [[ template "mounts" .promtail.default_mounts ]]
+        [[- end ]]
+
+        [[- if gt (len .promtail.extra_mounts) 0 ]]
+        [[ template "mounts" .promtail.extra_mounts ]]
+        [[- end ]]
+
       }
       resources {
         cpu    = [[ .promtail.resources.cpu ]]