Beats crashing with glibc 2.35 - Fatal glibc error: rseq registration failed #30576
Comments
belimawr
added
bug
libbeat
Team:Elastic-Agent-Data-Plane
|
Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane) |
|
Hitting this as well because of a specific case. |
@mickymiek, unfortunately, the only workaround we've found so far is to disable CGO, however the journald input depends on CGO. It's not an option for you. You could try my PR with the fix. TLDR; it's as simple (now that we know it) as to allow the |
rseq syscall is available on glibc >= 2.35, and called when CGO is used. If we don't allow rseq, Beats will eventually crash with an glibc error: `Fatal glibc error: rseq registration failed`. Fixes: elastic#30576
|
@mickymiek, we found a way to workaround this issue (thanks to @ph ). It is possible to specify the seccomp policy in the configuration file, so you just need to add The default policy for x86_64 is here: beats/libbeat/common/seccomp/policy_linux_amd64.go Lines 25 to 143 in 577a34c So you can add somehting like this to your configuration file (just make sure you trust the policy before deploying it to your environment): seccomp:
default_action: errno
syscalls:
- action: allow
names:
- accept
- accept4
- access
- arch_prctl
- bind
- brk
- chmod
- chown
- clock_gettime
- clone
- clone3
- close
- connect
- dup
- dup2
- epoll_create
- epoll_create1
- epoll_ctl
- epoll_pwait
- epoll_wait
- exit
- exit_group
- fchdir
- fchmod
- fchmodat
- fchown
- fchownat
- fcntl
- fdatasync
- flock
- fstat
- fstatfs
- fsync
- ftruncate
- futex
- getcwd
- getdents
- getdents64
- geteuid
- getgid
- getpeername
- getpid
- getppid
- getrandom
- getrlimit
- getrusage
- getsockname
- getsockopt
- gettid
- gettimeofday
- getuid
- inotify_add_watch
- inotify_init1
- inotify_rm_watch
- ioctl
- kill
- listen
- lseek
- lstat
- madvise
- mincore
- mkdirat
- mmap
- mprotect
- munmap
- nanosleep
- newfstatat
- open
- openat
- pipe
- pipe2
- poll
- ppoll
- pread64
- pselect6
- pwrite64
- read
- readlink
- readlinkat
- recvfrom
- recvmmsg
- recvmsg
- rename
- renameat
- rseq
- rt_sigaction
- rt_sigprocmask
- rt_sigreturn
- sched_getaffinity
- sched_yield
- sendfile
- sendmmsg
- sendmsg
- sendto
- set_robust_list
- setitimer
- setsockopt
- shutdown
- sigaltstack
- socket
- splice
- stat
- statfs
- sysinfo
- tgkill
- time
- tkill
- uname
- unlink
- unlinkat
- wait4
- waitid
- write
- writev |
|
@belimawr thanks a lot for your answers. We'll give it a try and report back soon |
|
This seems to works as well |
|
Adding seccomp configuration fixed it. Thank you guys! |
rseq syscall is available on glibc >= 2.35, and called when CGO is used. If we don't allow rseq, Beats will eventually crash with an glibc error: `Fatal glibc error: rseq registration failed`. Fixes: #30576
Yes, this works fine. The only "down side" is that it will allow all syscalls, which reduces security. No issues using it, just keep in mind what it means. |
rseq syscall is available on glibc >= 2.35, and called when CGO is used. If we don't allow rseq, Beats will eventually crash with an glibc error: `Fatal glibc error: rseq registration failed`. Fixes: #30576 (cherry picked from commit f02fa32) Co-authored-by: Tiago Queiroz <[email protected]>
rseq syscall is available on glibc >= 2.35, and called when CGO is used. If we don't allow rseq, Beats will eventually crash with an glibc error: `Fatal glibc error: rseq registration failed`. Fixes: #30576 (cherry picked from commit f02fa32) Co-authored-by: Tiago Queiroz <[email protected]>
rseq syscall is available on glibc >= 2.35, and called when CGO is used. If we don't allow rseq, Beats will eventually crash with an glibc error: `Fatal glibc error: rseq registration failed`. Fixes: #30576 (cherry picked from commit f02fa32) Co-authored-by: Tiago Queiroz <[email protected]>
|
In metricbeat.yml:
|
I am seeing some Beats (multiple versions, different builds) crashing shortly after startup on Linux with glibc 2.35. They crash with the following error message:
I looked at the coredump and it only lists some runtime assembly code:
So far I am not sure whether it's a problem with Libbeat or the Go runtime. I opened an issue on Go's issue tracker because it seems to be related to the runtime: golang/go#51315
So far I've seen Filebeat and Auditbeat crashing, it does not matter if I download one of the official releases or build them myself, the result is always the same.
Interestingly, if I build with
CGO_ENALBED=0, they run without any issues.The Elastic-Agent seems to work fine, but the beats under it are all failing:
How to reproduce it
The easiest way to reproduce is to run some beats on a Arch Linux VM, with Vagrant that's pretty easy:
The text was updated successfully, but these errors were encountered: