Support Debian packages

[oneiros-de]

There are packages for debian in testing and backports; it would be very nice if they were used.

[tuxmea]

related to #62 - init.pp specifies URL being the only supported $install_method by now.

[anarcat]

I started working on this in #303, although only with node-exporter support for now. Please test and review.

[anarcat]

update: #303 also works with the main prometheus package - I haven't tested exporters other than the node-exporter. my profiles look like this, on the server:

class profile::prometheus::server {
  class {
    'prometheus::server':
      # no static scrape configurations, collect everything from other jobs
      scrape_configs      => [],
      collect_scrape_jobs => [ { 'job_name' => 'prometheus-node-exporter' } ],
      # follow prom2 defaults
      localstorage        => '/var/lib/prometheus/metrics2',
      storage_retention   => '15d',
  }
  # [...] webserver proxy configuration, authentication and firewall rule exports
}

client:

class profile::prometheus::client(
  Enum['present', 'absent'] $ensure = 'present'
) {
  $package_ensure = $ensure ? { 'present' => 'latest', 'absent' => 'absent' }
  $service_ensure = $ensure ? { 'present' => 'running', 'absent'  => 'stopped' }
  class { 'prometheus::node_exporter':
    install_method => 'package',
    package_name   => 'prometheus-node-exporter',
    package_ensure => $package_ensure,
    service_ensure => $service_ensure,
    # purge_config_dir => true,
  }
  # XXX: should be using apt::pin, but that involves replacing a lot
  # of our custom code
  file {
    '/etc/apt/preferences.d/prometheus-node-exporter.pref':
      ensure  => present,
      content => "# this file is managed through puppet, local changes will be lost
Explanation: Prometheus 2.x is not in Debian stretch, remove when we move to buster
Package: prometheus-node-exporter
Pin: release n=stretch-backports
Pin-Priority: 500
",
      notify  => Package[$::prometheus::node_exporter::package_name],
  }
  # [...] firewall rules
}

what #303 really does could also be accomplished by simply hardcoding this in Hiera:

prometheus::bin_dir: '/usr/bin'
prometheus::shared_dir: '/usr/share/prometheus'
prometheus::install_method: 'package'
prometheus::node_exporter::package_ensure: 'latest'
prometheus::node_exporter::package_name: 'prometheus-node-exporter'
prometheus::node_exporter::service_name: 'prometheus-node-exporter'
prometheus::node_exporter::group: 'prometheus'
prometheus::node_exporter::user: 'prometheus' 

... in fact I wonder if that shouldn't simply be the way to go to fix this issue: just documentation?

feedback welcome...

[anarcat]

ever since I started working on #303, I have successfully used this module with Debian packages, but always by patching the module and severly changing the Hiera parameters, as explained above. I ended up giving up on changing the parameters in the patch, and severely reduced the patchset, from #303 to #527 and #528.

it's still far from clean. here's how I setup a server, for example:

  class {
    'prometheus::server':
      # collect everything from node and apache jobs.
      #
      # when a new job is added here, usually the
      # profile::prometheus::server::rule below also needs to be
      # updated.
      collect_scrape_jobs => $collect_scrape_jobs,
      # monitor prometheus and grafana as well
      scrape_configs      => $scrape_configs,
      storage_retention   => $storage_retention,
      global_config       => lookup('prometheus::global_config') + { 'scrape_interval' => $scrape_interval },
      # follow prom2 defaults
      localstorage        => '/var/lib/prometheus/metrics2',
      # force debian package install parameters, further discussion in:
      # https://github.com/voxpupuli/puppet-prometheus/pull/303
      env_file_path       => '/etc/default',
      bin_dir             => '/usr/bin',
      configname          => 'prometheus.yml',
      install_method      => 'package',
      package_ensure      => 'present', # don't upgrade package through puppet
      init_style          => 'none',
      shared_dir          => '/usr/share/prometheus',
  }

basically, i work around that systemd thing by bypassing the init_style altogether. i use a hack to ensure env_file_path works for the daemon (#527) which is how I pass the daemon args to it. i guess the next step would be to figure out how to make those default if install_method=package and init_style=none?

exporters are way trickier: i basically did a similar hack for two of them in #528, but it's ugly: there I pass a env_vars array which basically populates /etc/default/prometheus-postfix-exporter (say) and use that to parametrize the daemon's arguments through system's EnvironmentFile, which the Debian package ships with. interestingly enough, I don't use the postfix exporter anymore (switched to mtail), but this is how it works for the blackbox exporter:

  class { 'prometheus::blackbox_exporter':
    package_ensure    => $package_ensure,
    service_ensure    => $service_ensure,
    export_scrape_job => true,
    # force debian package install parameters, further discussion in:
    # https://github.com/voxpupuli/puppet-prometheus/pull/303
    install_method    => 'package',
    init_style        => 'none',
    user              => 'prometheus',
    group             => 'prometheus',
    package_name      => 'prometheus-blackbox-exporter',
    service_name      => 'prometheus-blackbox-exporter',
    # purge_config_dir => true,
    config_file       => '/etc/prometheus/blackbox-exporter.yaml',
    env_vars          => {
      'ARGS' => "--config.file='/etc/prometheus/blackbox-exporter.yaml'",
    },
  }

i basically need to do the same for the node exporter, because right now, if you pass options to it with init_style=none and install_method=package, they just don't get through the Debian packaging stuff. this, obviously, might need more work, but it at least gives me a way to use the module with debian packages as well.

so that's my progress so far. I'm pretty happy to be down to only two (small) patches, but it's annoying to have to have those profiles that hardcode those settings in there. ideally, those would automatically be setup if the install_method and init_style are changed, but I can't think of a clean way of doing that in the module itself, so for now i'm monkey-patching it like this.

[anarcat]

update: all of my patches are in. what's left here is to enable the "package mode" by default on Debian, which I'm not even sure we'd want to be doing to be honest. but it's a variation of #401, to expand to cover for all other exporters and the prom daemon. this is what is done on Arch, I believe, and would require a rather convoluted run through the test suite to change all those tiny strings everywhere, so I'm not going to do it in the short term, but it's basically the last hurdle.

[m8t]

I did this task recently (RPM based but the changes would be identical for DEBs) and I came up with something very similar already commented above. I have done it via hiera for the node_exporter:

# Adjust values for installation from package                                                                                                                 
prometheus::node_exporter::install_method: 'package'                                                                                                          
prometheus::node_exporter::package_name: 'golang-github-prometheus-node-exporter' # provided by epel repository                                               
prometheus::node_exporter::env_file_path: '/etc/default' # match package                                                                                      
prometheus::node_exporter::service_name: 'prometheus-node-exporter' # instead of node_exporter                                                                
# systemd unit file and system user provided by package                                                                                                       
prometheus::node_exporter::init_style: 'none'                                                                                                                 
prometheus::node_exporter::manage_user: false                                                                                                                 
prometheus::node_exporter::manage_group: false                                                                                                                

Adjust a few values for the package name, and what is provided by the package.

[TheMeier]

I use package installation on debain like this:

prometheus::package_ensure: '2.51.2-1'
prometheus::version: '2.51.2-1'
prometheus::bin_dir: '/usr/bin'
prometheus::install_method: 'package'
prometheus::manage_group: false
prometheus::manage_user: false
prometheus::shared_dir: '/usr/share/prometheus'

IMHO what @oneiros-de is possible with the right parameter.

package as default installation mode is not an option IMHO.