[journald] Adds user-level unit filtering

[ian28223]

What does this PR do?

• Adds User-level service unit filtering support for Journald log collection via include_user_units and exclude_user_units.
• A wildcard (*) can be used in either exclude_units or exclude_user_units
  if only a particular type of Journald log is desired.

Motivation

• AGENT-7627
• FRAGENT-1752

Additional Notes

• Tested on a Ubuntu 21 vagrant box

Possible Drawbacks / Trade-offs

Describe how to test/QA your changes

### Ubuntu 21 / vagrant
### Create an example service in a non-root user's home directory

cat <<EOF > /home/vagrant/jtest.sh
#!/bin/bash

while true; do
    echo \$(date -u) - \$2.service \$1 | systemd-cat
    sleep 1;
done;
EOF
chown vagrant.vagrant /home/vagrant/jtest.sh
chmod +x ~/jtest.sh


### Create Systemd Unit files

## Creates System-level Unit files

cat << EOF > /etc/systemd/system/foo.service
[Unit]
Description=Foo
[Service]
ExecStart=/home/vagrant/jtest.sh system-level foo
[Install]
WantedBy=multi-user.target
EOF

cat << EOF > /etc/systemd/system/bar.service
[Unit]
Description=Bar
[Service]
ExecStart=/home/vagrant/jtest.sh system-level bar
[Install]
WantedBy=multi-user.target
EOF


## Creates User-level Unit files

cat << EOF > /etc/systemd/user/foo.service
[Unit]
Description=Foo
[Service]
ExecStart=/home/vagrant/jtest.sh user-level foo
[Install]
WantedBy=multi-user.target
EOF

cat << EOF > /etc/systemd/user/bar.service
[Unit]
Description=Bar
[Service]
ExecStart=/home/vagrant/jtest.sh user-level bar
[Install]
WantedBy=multi-user.target
EOF


### Reload and start system-level services

systemctl daemon-reload
systemctl restart foo.service
systemctl restart bar.service


### Reload and start user-level services

## Log on as non-root user
su - vagrant

## Set XDG_RUNTIME_DIR
export XDG_RUNTIME_DIR=/run/user/$(id -u)

## Reload and start the services
systemctl --user daemon-reload
systemctl --user restart foo.service
systemctl --user restart bar.service

systemctl --user status foo.service
systemctl --user status bar.service


### Install / configure the agent

## Set API
sudo sh -c "sed 's/api_key:.*/api_key: XXX/' /etc/datadog-agent/datadog.yaml.example > /etc/datadog-agent/datadog.yaml"
## Enable logs
sudo sh -c "sed -i 's/.*logs_enabled:.*/logs_enabled: true/' /etc/datadog-agent/datadog.yaml"


## Prep journald log collection
usermod -a -G systemd-journal dd-agent
touch /etc/datadog-agent/conf.d/logs.yaml
chown -R dd-agent.dd-agent /etc/datadog-agent/

## Test 1: include SYS foo.service and USR bar.service logs

cat << EOF > /etc/datadog-agent/conf.d/logs.yaml
logs:
- type: journald
  include_units:
   - foo.service
  include_user_units:
   - bar.service
EOF
systemctl restart datadog-agent.service && sleep 2

# Expect in message 'foo.service system-level' and 'bar.service user-level'
datadog-agent stream-logs | grep _SYSTEMD_UNIT | grep -Eo "\w+\.service \w+-level"

## Test 2: exclude SYS foo.service and USR bar.service logs

cat << EOF > /etc/datadog-agent/conf.d/logs.yaml
logs:
- type: journald
  exclude_units:
   - foo.service
  exclude_user_units:
   - bar.service
EOF
systemctl restart datadog-agent.service && sleep 2

# Expect in message 'bar.service system-level' and 'foo.service user-level'
datadog-agent stream-logs | grep _SYSTEMD_UNIT | grep -Eo "\w+\.service \w+-level"

## Test 3: exclude all SYS service

cat << EOF > /etc/datadog-agent/conf.d/logs.yaml
logs:
- type: journald
  exclude_units:
   - '*'
EOF
systemctl restart datadog-agent.service && sleep 2

# Expect in message 'foo.service user-level' and 'bar.service user-level'
datadog-agent stream-logs | grep _SYSTEMD_UNIT | grep -Eo "\w+\.service \w+-level"

## Test 4: exclude all USR service

cat << EOF > /etc/datadog-agent/conf.d/logs.yaml
logs:
- type: journald
  exclude_user_units:
   - '*'
EOF
systemctl restart datadog-agent.service && sleep 2

# Expect in message 'foo.service system-level' and 'bar.service system-level'
datadog-agent stream-logs | grep _SYSTEMD_UNIT | grep -Eo "\w+\.service \w+-level"

Reviewer's Checklist

• If known, an appropriate milestone has been selected; otherwise the Triage milestone is set.
• Use the major_change label if your change either has a major impact on the code base, is impacting multiple teams or is changing important well-established internals of the Agent. This label will be use during QA to make sure each team pay extra attention to the changed behavior. For any customer facing change use a releasenote.
• A release note has been added or the changelog/no-changelog label has been applied.
• Changed code has automated tests for its functionality.
• Adequate QA/testing plan information is provided if the qa/skip-qa label is not applied.
• At least one team/.. label has been applied, indicating the team(s) that should QA this change.
• If applicable, docs team has been notified or an issue has been opened on the documentation repo.
• If applicable, the need-change/operator and need-change/helm labels have been applied.
• If applicable, the config template has been updated.

[apigirl]

👍 release note looks fine

[remeh]

Code change LGTM, @ian28223 please provide a PR description and everything like in the other PRs, see the template if needed. (an extremely important section is the QA one)