Skip to content

Commit

Permalink
[pre-commit.ci] auto fixes from pre-commit.com hooks
Browse files Browse the repository at this point in the history
for more information, see https://pre-commit.ci
  • Loading branch information
pre-commit-ci[bot] committed Sep 3, 2024
1 parent e768903 commit a2be29b
Show file tree
Hide file tree
Showing 8 changed files with 45 additions and 25 deletions.
11 changes: 8 additions & 3 deletions kubernetes-addons/memory-bandwidth-exporter/README.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
# memory bandwidth exporter

Pod/container grained memory bandwidth exporter provides users memory bandwidth metrics of their running containers. The metrics include llc_occupancy, mbm_local_bytes, mbm_total_bytes, cpu utilization and memory usage, and the metrics have been processed. In addition to container-level metrics, it also provides class-level and socket-level metrics. Users can configure the list of metrics to be collected. It serves as an exporter which can be connected to Promethus-like obserbility tools. And it also can be used as a telementry provider.
Pod/container grained memory bandwidth exporter provides users memory bandwidth metrics of their running containers. The metrics include llc_occupancy, mbm_local_bytes, mbm_total_bytes, cpu utilization and memory usage, and the metrics have been processed. In addition to container-level metrics, it also provides class-level and socket-level metrics. Users can configure the list of metrics to be collected. It serves as an exporter which can be connected to Promethus-like obserbility tools. And it also can be used as a telementry provider.

Memory bandwidth exporter makes use of state-of-the-art technologies like NRI to build a resource-efficient and well-maintained solution. This solution provides observability to memory bandwidth to OPEA micro-services. It lays the groundwork of better scaling and auto scaling of OPEA. It can also be deployed separately on end user environments, supporting any cases that memory bandwidth metrics are required.

## Setup

### Enable NRI in Containerd

```sh
# download containerd binary, containerd version v1.7.0 or higher is required
$ wget https://github.com/containerd/containerd/releases/download/v1.7.0/containerd-1.7.0-linux-amd64.tar.gz
Expand Down Expand Up @@ -41,13 +42,15 @@ $ ./build/bin/logger -idx 00
### Enable RDT

Mount resctrl to the directory `/sys/fs/resctrl`:

```sh
$ sudo mount -t resctrl resctrl /sys/fs/resctrl
```

### Setup memory bandwidth exporter

Before setup, you need to configure the runc hook:

```sh
$ ./config/config.sh
```
Expand All @@ -57,7 +60,7 @@ $ ./config/config.sh
```sh
$ make build
$ sudo ./bin/memory-bandwidth-exporter
# e.g., sudo ./bin/memory-bandwidth-exporter --collector.node.name=<node_name> --collector.container.namespaceWhiteList="calico-apiserver,calico-system,kube-system,tigera-operator"
# e.g., sudo ./bin/memory-bandwidth-exporter --collector.node.name=<node_name> --collector.container.namespaceWhiteList="calico-apiserver,calico-system,kube-system,tigera-operator"

# get memory bandwidth metrics
$ curl http://localhost:9100/metrics
Expand Down Expand Up @@ -105,6 +108,7 @@ memory-bandwidth-exporter-zxhdl 1/1 Running 0 3m
```

get memory bandwidth metrics

```sh
$ curl http://<memory_bandwidth_exporter_container_ip>:9100/metrics
```
Expand All @@ -118,6 +122,7 @@ $ make clean
## More flags about memory bandwidth exporter

There are some flags to help users better use memory bandwidth exporter:

```sh
-h, --[no-]help Show context-sensitive help (also try --help-long and --help-man).
--collector.node.name="" Give node name.
Expand All @@ -140,4 +145,4 @@ There are some flags to help users better use memory bandwidth exporter:
--log.level=info Only log messages with the given severity or above. One of: [debug, info, warn, error]
--log.format=logfmt Output format of log messages. One of: [logfmt, json]
--[no-]version Show application version.
```
```
5 changes: 4 additions & 1 deletion kubernetes-addons/memory-bandwidth-exporter/config/config.sh
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
#!/bin/bash

# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

sudo apt install jq
sudo mkdir -p /etc/containers/oci/hooks.d/
sudo cp plugin/etc/containers/oci/hooks.d/* /etc/containers/oci/hooks.d/
sudo cp plugin/usr/local/sbin/* /usr/local/sbin/
sudo cp plugin/usr/local/sbin/* /usr/local/sbin/
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

apiVersion: apps/v1
kind: DaemonSet
metadata:
Expand Down Expand Up @@ -53,4 +56,4 @@ spec:
- name: nri
hostPath:
path: /var/run/nri/
hostPID: true
hostPID: true
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

global:
scrape_interval: 15s

scrape_configs:
- job_name: memory-bandwidth-exporter
static_configs:
- targets: ['localhost:9100']
- targets: ['localhost:9100']
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
{
"version": "1.0.0",
"hook": {
"path": "/usr/local/sbin/create-runtime.sh"
},
"when": {
"always": true
},
"stages": ["createRuntime"]
"version": "1.0.0",
"hook": {
"path": "/usr/local/sbin/create-runtime.sh"
},
"when": {
"always": true
},
"stages": ["createRuntime"]
}
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
{
"version": "1.0.0",
"hook": {
"path": "/usr/local/sbin/post-stop.sh"
},
"when": {
"always": true
},
"stages": ["poststop"]
}
"version": "1.0.0",
"hook": {
"path": "/usr/local/sbin/post-stop.sh"
},
"when": {
"always": true
},
"stages": ["poststop"]
}
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
#!/bin/bash

# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

container_state=$(cat)
container_pid=$(echo "$container_state" | jq -r '.pid')

Expand All @@ -11,4 +14,4 @@ fi

tasks_file="$mon_group_dir/tasks"
echo "$container_pid" | sudo tee -a $tasks_file > /dev/null
echo "CREATE Container: $container_id, Container State: $container_state, Container PID: $container_pid" | sudo tee -a /var/log/container_pids.log > /dev/null
echo "CREATE Container: $container_id, Container State: $container_state, Container PID: $container_pid" | sudo tee -a /var/log/container_pids.log > /dev/null
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
#!/bin/bash

# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

container_state=$(cat)
container_id=$(echo "$container_state" | jq -r '.id')
mon_group_dir="/sys/fs/resctrl/mon_groups/container-$container_id"
echo "DELETE Container: $container_id" >> /var/log/container_pids.log
sudo rmdir $mon_group_dir
sudo rmdir $mon_group_dir

0 comments on commit a2be29b

Please sign in to comment.