-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Metricbeat] Metricbeat invokes ptrace leading to AppArmor complaints #6932
Comments
I have not witnessed any
My environment: |
I looked further into this and I can now reproduce it on a local vanilla Ubuntu VM when running Here is the exact command to reproduce the mentioned behaviour.
Here is the complete configuration file,
I can reproduce this when using Notes,
Make sure to disable |
I see the following messages in metricbeat logs,
I am not sure if these are relevant for the issue at hand. |
Not sure if it's related but it reminded me of #6641 Would be interesting to see if you still see the issue with beats master? |
Metricbeat reads from The AppArmor documentation has a page dedicated to /proc and ptrace. It mentions that reading from Side note: I was able to reproduce the behavior by running Metricbeat with
And then I only started seeing the apparmor denied messages after running |
The ptrace My messages looks like,
A requested mask of Does this imply that metricbeat is actually trying to ptrace a PID? I don't know if I am interpreting this correctly. Probably not! |
Hmm. I start seeing the denied messages without doing this. I just have to wait < 1 minute before metricbeat starts doing its stuff. |
Good point. The /proc and ptrace page also mentions that there are files in
No. I don't believe this is the case. I have enabled auditing for the
|
Nice findings 👍 Would it be possible to drop these "spurious" events with Auditbeat? I am wondering what the Thanks. |
As of now Auditbeat is not fully parsing the AppArmor message. I have pull request open to fix the issue. Once that is merged you could filter the message like this:
But until then you could use:
|
We are experiencing same issue on our K8s cluster, we are running Metricbeat 6.2.4 as daemonset and captured the same error log. |
@andrewkroh Hi again! I am unable to drop these AppArmor messages. My Auditbeat 6.2.4 configuration contains the following lines,
However, these AppArmor complaints still make it to an Elasticsearch index. They don't get dropped by Auditbeat. Note: The The full document in Elasticsearch look like,
I tried simplifying the
What could I be missing here? |
I was misunderstanding the semantics for
I think I need multiple |
Yeah, I think that's correct. Combining a top-level |
If you are encountering this issue in running Metricbeat in Kubernetes, you can circumvent it by adding an AppArmor annotation to your workload and adding the SYS_PTRACE capability to the security context of the workload's pod. For example: apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: metricbeat
labels:
app: metricbeat
# SNIP
spec:
template:
metadata:
name: metricbeat
labels:
app: metricbeat
# SNIP
annotations:
# This is needed for the proc module to perform ptrace
container.apparmor.security.beta.kubernetes.io/metricbeat: unconfined
spec:
# SNIP
containers:
- name: metricbeat
# SNIP
securityContext:
runAsUser: 0
capabilities:
add:
# This is needed for the proc module
- SYS_PTRACE |
Presumably this isn't something that can be 'fixed' on the metricbeat side, as we're running into kernel behavior. Should we add a note to the metricset docs? |
Documentation regarding this has been added by#11822 Is there anything else we should do, or can I close this? |
Thank you 👍 |
@kholia I am running Metricbeat and Filebeat (7.4.0) on Docker (Ubuntu 18.04 LTS). Since these |
I drop these messages by installing and configuring Auditbeat. |
I am running Metricbeat 5.6.8 insider a Docker container on Ubuntu 16.04.3 LTS. AppArmor's
docker-default
profile is in effect for this container.While running
dmesg
on the host system, I see,My configuration
metricbeat.yml
file,Disabling the
process
metricset gets rid of these AppArmor complaints.How to reproduce:
4.4.0-108-generic
kernel here.metricbeat version 5.6.8-xexec (amd64), libbeat 5.6.8-xexe
)Docker version 17.12.0-ce, build c97c6d6
docker-default
profile applied.Question: Why is
metricbeat
trying to ptrace stuff? It is required for some functionality?Thanks!
The text was updated successfully, but these errors were encountered: