Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added liveness and telemetry for telegraf #517

Merged
merged 5 commits into from
Apr 9, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions build/linux/installer/conf/td-agent-bit-prom-side-car.conf
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,18 @@
Parsers_File /etc/opt/microsoft/docker-cimprov/azm-containers-parser.conf
Log_File /var/opt/microsoft/docker-cimprov/log/fluent-bit.log

[INPUT]
Name tail
Tag oms.container.log.flbplugin.terminationlog.*
Path /dev/write-to-traces
DB /var/opt/microsoft/docker-cimprov/state/terminationlog-ai.db
DB.Sync Off
Parser docker
Mem_Buf_Limit 1m
Path_Key filepath
Skip_Long_Lines On
Ignore_Older 2m

[INPUT]
Name tcp
Tag oms.container.perf.telegraf.*
Expand Down
12 changes: 12 additions & 0 deletions build/linux/installer/conf/td-agent-bit-rs.conf
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,18 @@
Parsers_File /etc/opt/microsoft/docker-cimprov/azm-containers-parser.conf
Log_File /var/opt/microsoft/docker-cimprov/log/fluent-bit.log

[INPUT]
Name tail
Tag oms.container.log.flbplugin.terminationlog.*
Path /dev/write-to-traces
DB /var/opt/microsoft/docker-cimprov/state/terminationlog-ai.db
DB.Sync Off
Parser docker
Mem_Buf_Limit 1m
Path_Key filepath
Skip_Long_Lines On
Ignore_Older 2m

[INPUT]
Name tcp
Tag oms.container.perf.telegraf.*
Expand Down
12 changes: 12 additions & 0 deletions build/linux/installer/conf/td-agent-bit.conf
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,18 @@
Skip_Long_Lines On
Ignore_Older 2m

[INPUT]
Name tail
Tag oms.container.log.flbplugin.terminationlog.*
Path /dev/write-to-traces
DB /var/opt/microsoft/docker-cimprov/state/terminationlog-ai.db
DB.Sync Off
Parser docker
Mem_Buf_Limit 1m
Path_Key filepath
Skip_Long_Lines On
Ignore_Older 2m

[INPUT]
Name tcp
Tag oms.container.perf.telegraf.*
Expand Down
9 changes: 9 additions & 0 deletions build/linux/installer/scripts/livenessprobe.sh
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,15 @@ then
exit 1
fi

#test to exit non zero value if telegraf is not running
(ps -ef | grep telegraf | grep -v "grep")
if [ $? -ne 0 ]
then
echo "Telegraf is not running" > /dev/termination-log
echo "Telegraf is not running (controller: ${CONTROLLER_TYPE}, container type: ${CONTAINER_TYPE})" > /dev/write-to-traces # this file is tailed and sent to traces
exit 1
fi

if [ -s "inotifyoutput.txt" ]
then
# inotifyoutput file has data(config map was applied)
Expand Down
4 changes: 4 additions & 0 deletions kubernetes/linux/main.sh
Original file line number Diff line number Diff line change
Expand Up @@ -699,6 +699,10 @@ dpkg -l | grep td-agent-bit | awk '{print $2 " " $3}'



# Write messages from the liveness probe to stdout (so telemetry picks it up)
touch /dev/write-to-traces


echo "stopping rsyslog..."
service rsyslog stop

Expand Down