Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Auditbeat] System socket panics with concurrent map read and write #21192

Closed
adriansr opened this issue Sep 21, 2020 · 1 comment · Fixed by #21690
Closed

[Auditbeat] System socket panics with concurrent map read and write #21192

adriansr opened this issue Sep 21, 2020 · 1 comment · Fixed by #21690
Labels
Auditbeat bug

Comments

@adriansr
Copy link
Contributor

For confirmed bugs, please report:

  • Version: 7.8.1
  • Operating System: Linux
  • Discuss Forum URL: -
  • Steps to Reproduce:

Auditbeat running the system/socket dataset with default config panics with the following stack traces:

fatal error: concurrent map read and map write

goroutine 655 [running]:
runtime.throw(0x3500807, 0x21)
	/usr/local/go/src/runtime/panic.go:774 +0x72 fp=0xc000081b20 sp=0xc000081af0 pc=0x1387072
runtime.mapaccess2_faststr(0x3054b20, 0xc00067f3b0, 0xc000811300, 0xa, 0xa, 0x0)
	/usr/local/go/src/runtime/map_faststr.go:116 +0x48f fp=0xc000081b90 sp=0xc000081b20 pc=0x136ad0f
github.com/elastic/beats/v7/x-pack/auditbeat/module/system/socket.(*process).ResolveIP(...)
	/go/src/github.com/elastic/beats/x-pack/auditbeat/module/system/socket/state.go:242
github.com/elastic/beats/v7/x-pack/auditbeat/module/system/socket.(*flow).toEvent(0xc0018b4900, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/elastic/beats/x-pack/auditbeat/module/system/socket/state.go:994 +0x177c fp=0xc000081d60 sp=0xc000081b90 pc=0x2d289cc
github.com/elastic/beats/v7/x-pack/auditbeat/module/system/socket.(*state).reapLoop(0xc000506500)
	/go/src/github.com/elastic/beats/x-pack/auditbeat/module/system/socket/state.go:485 +0x184 fp=0xc000081fd8 sp=0xc000081d60 pc=0x2d24384
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1357 +0x1 fp=0xc000081fe0 sp=0xc000081fd8 pc=0x13b7da1
created by github.com/elastic/beats/v7/x-pack/auditbeat/module/system/socket.NewState
	/go/src/github.com/elastic/beats/x-pack/auditbeat/module/system/socket/state.go:401 +0xa8

goroutine 1 [semacquire, 3126 minutes]:
sync.runtime_Semacquire(0xc0003c0258)
	/usr/local/go/src/runtime/sema.go:56 +0x42
sync.(*WaitGroup).Wait(0xc0003c0250)
	/usr/local/go/src/sync/waitgroup.go:130 +0x64
github.com/elastic/beats/v7/metricbeat/beater.(*Metricbeat).Run(0xc0006863f0, 0xc00079a280, 0xc00078d888, 0x1)
	/go/src/github.com/elastic/beats/metricbeat/beater/metricbeat.go:258 +0x36e
github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).launch(0xc00079a280, 0x34babf4, 0x9, 0x34babf4, 0x9, 0x0, 0x0, 0x1, 0x0, 0x0, ...)
	/go/src/github.com/elastic/beats/libbeat/cmd/instance/beat.go:469 +0x733
github.com/elastic/beats/v7/libbeat/cmd/instance.Run.func1(0x34babf4, 0x9, 0x34babf4, 0x9, 0x0, 0x0, 0xc0006f9c10, 0xc0005d01a0, 0x0, 0x0)
	/go/src/github.com/elastic/beats/libbeat/cmd/instance/beat.go:200 +0x56b
github.com/elastic/beats/v7/libbeat/cmd/instance.Run(0x34babf4, 0x9, 0x34babf4, 0x9, 0x0, 0x0, 0x1, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/elastic/beats/libbeat/cmd/instance/beat.go:201 +0x148
github.com/elastic/beats/v7/libbeat/cmd.genRunCmd.func1(0xc0005ce580, 0xc00017edc0, 0x0, 0xa)
	/go/src/github.com/elastic/beats/libbeat/cmd/run.go:36 +0x9d
github.com/spf13/cobra.(*Command).execute(0xc0005ce580, 0xc00004e0c0, 0xa, 0xa, 0xc0005ce580, 0xc00004e0c0)
	/go/src/github.com/elastic/beats/vendor/github.com/spf13/cobra/command.go:766 +0x2aa
github.com/spf13/cobra.(*Command).ExecuteC(0xc0005ce580, 0x0, 0x0, 0xc0000a8058)
	/go/src/github.com/elastic/beats/vendor/github.com/spf13/cobra/command.go:852 +0x2ea
github.com/spf13/cobra.(*Command).Execute(...)
	/go/src/github.com/elastic/beats/vendor/github.com/spf13/cobra/command.go:800
main.main()
	/go/src/github.com/elastic/beats/x-pack/auditbeat/main.go:21 +0x2f

goroutine 60 [select]:
github.com/elastic/beats/v7/libbeat/publisher/queue/memqueue.(*bufferingEventLoop).run(0xc0004c9b80)
	/go/src/github.com/elastic/beats/libbeat/publisher/queue/memqueue/eventloop.go:316 +0x1cf
github.com/elastic/beats/v7/libbeat/publisher/queue/memqueue.NewQueue.func1(0xc000686380, 0x38e4260, 0xc0004c9b80)
	/go/src/github.com/elastic/beats/libbeat/publisher/queue/memqueue/broker.go:171 +0x67
created by github.com/elastic/beats/v7/libbeat/publisher/queue/memqueue.NewQueue
	/go/src/github.com/elastic/beats/libbeat/publisher/queue/memqueue/broker.go:169 +0x373

goroutine 61 [select]:
github.com/elastic/beats/v7/libbeat/publisher/queue/memqueue.(*ackLoop).run(0xc0004a5450)
	/go/src/github.com/elastic/beats/libbeat/publisher/queue/memqueue/ackloop.go:60 +0x107
github.com/elastic/beats/v7/libbeat/publisher/queue/memqueue.NewQueue.func2(0xc000686380, 0xc0004a5450)
	/go/src/github.com/elastic/beats/libbeat/publisher/queue/memqueue/broker.go:175 +0x61
created by github.com/elastic/beats/v7/libbeat/publisher/queue/memqueue.NewQueue
	/go/src/github.com/elastic/beats/libbeat/publisher/queue/memqueue/broker.go:173 +0x3a2

goroutine 62 [select]:
github.com/elastic/beats/v7/libbeat/publisher/queue/memqueue.(*consumer).Get(0xc0005d06c0, 0x800, 0x3, 0x2, 0xc00092ad00, 0x0)
	/go/src/github.com/elastic/beats/libbeat/publisher/queue/memqueue/consume.go:65 +0xe9
github.com/elastic/beats/v7/libbeat/publisher/pipeline.(*eventConsumer).loop(0xc0004a54f0, 0x38e42a0, 0xc0005d06c0)
	/go/src/github.com/elastic/beats/libbeat/publisher/pipeline/consumer.go:167 +0x497
created by github.com/elastic/beats/v7/libbeat/publisher/pipeline.newEventConsumer
	/go/src/github.com/elastic/beats/libbeat/publisher/pipeline/consumer.go:77 +0x154

goroutine 27 [chan receive]:
k8s.io/klog.(*loggingT).flushDaemon(0x515d800)
	/go/src/github.com/elastic/beats/vendor/k8s.io/klog/klog.go:1000 +0x8b
created by k8s.io/klog.init.0
	/go/src/github.com/elastic/beats/vendor/k8s.io/klog/klog.go:404 +0x6c

goroutine 10 [syscall, 3126 minutes]:
os/signal.signal_recv(0xc000053720)
	/usr/local/go/src/runtime/sigqueue.go:147 +0x9c
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:23 +0x22
created by os/signal.init.0
	/usr/local/go/src/os/signal/signal_unix.go:29 +0x41

goroutine 63 [select, 3126 minutes]:
github.com/elastic/beats/v7/libbeat/publisher/pipeline.(*retryer).loop(0xc0006c26c0)
	/go/src/github.com/elastic/beats/libbeat/publisher/pipeline/retry.go:137 +0x25a
created by github.com/elastic/beats/v7/libbeat/publisher/pipeline.newRetryer
	/go/src/github.com/elastic/beats/libbeat/publisher/pipeline/retry.go:96 +0x14f

goroutine 64 [select]:
github.com/elastic/beats/v7/libbeat/publisher/pipeline.(*netClientWorker).run(0xc0006c28a0)
	/go/src/github.com/elastic/beats/libbeat/publisher/pipeline/output.go:128 +0xda
created by github.com/elastic/beats/v7/libbeat/publisher/pipeline.makeClientWorker
	/go/src/github.com/elastic/beats/libbeat/publisher/pipeline/output.go:80 +0x1c8

goroutine 65 [syscall, 3126 minutes]:
syscall.Syscall6(0xe8, 0x6, 0xc000365bec, 0x7, 0xffffffffffffffff, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/local/go/src/syscall/asm_linux_amd64.s:44 +0x5
golang.org/x/sys/unix.EpollWait(0x6, 0xc000365bec, 0x7, 0x7, 0xffffffffffffffff, 0x0, 0x0, 0x0)
	/go/src/github.com/elastic/beats/vendor/golang.org/x/sys/unix/zsyscall_linux_amd64.go:1881 +0x72
github.com/fsnotify/fsnotify.(*fdPoller).wait(0xc000194040, 0x0, 0x0, 0x0)
	/go/src/github.com/elastic/beats/vendor/github.com/fsnotify/fsnotify/inotify_poller.go:86 +0x91
github.com/fsnotify/fsnotify.(*Watcher).readEvents(0xc0004a5720)
	/go/src/github.com/elastic/beats/vendor/github.com/fsnotify/fsnotify/inotify.go:198 +0x1f8
created by github.com/fsnotify/fsnotify.NewWatcher
	/go/src/github.com/elastic/beats/vendor/github.com/fsnotify/fsnotify/inotify.go:59 +0x1a5

goroutine 619 [select]:
github.com/elastic/beats/v7/metricbeat/mb/module.(*metricSetWrapper).startPeriodicFetching(0xc000768210, 0x392c9e0, 0xc000010060, 0x39131e0, 0xc00087a780)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:230 +0x219
github.com/elastic/beats/v7/metricbeat/mb/module.(*metricSetWrapper).run(0xc000768210, 0xc0000a8540, 0xc0008214a0)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:208 +0x7e1
github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start.func1(0xc0003c0320, 0xc0000a8540, 0xc0008214a0, 0xc000768210)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:147 +0x2e8
created by github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:135 +0x140

goroutine 623 [chan receive, 3126 minutes]:
github.com/elastic/beats/v7/metricbeat/beater.(*Metricbeat).Run.func2(0xc0003c0250, 0xc0006863f0, 0xc0004a5900)
	/go/src/github.com/elastic/beats/metricbeat/beater/metricbeat.go:226 +0x6b
created by github.com/elastic/beats/v7/metricbeat/beater.(*Metricbeat).Run
	/go/src/github.com/elastic/beats/metricbeat/beater/metricbeat.go:224 +0x32c

goroutine 625 [chan receive, 2 minutes]:
github.com/elastic/beats/v7/metricbeat/mb/module.PublishChannels.func1(0xc000821320)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/publish.go:40 +0x171
created by github.com/elastic/beats/v7/metricbeat/mb/module.PublishChannels
	/go/src/github.com/elastic/beats/metricbeat/mb/module/publish.go:48 +0xd0

goroutine 628 [chan receive, 246 minutes]:
github.com/elastic/beats/v7/metricbeat/mb/module.PublishChannels.func1(0xc0008210e0)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/publish.go:40 +0x171
created by github.com/elastic/beats/v7/metricbeat/mb/module.PublishChannels
	/go/src/github.com/elastic/beats/metricbeat/mb/module/publish.go:48 +0xd0

goroutine 626 [runnable]:
github.com/elastic/beats/v7/metricbeat/mb/module.PublishChannels.func1(0xc0008213e0)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/publish.go:40 +0x171
created by github.com/elastic/beats/v7/metricbeat/mb/module.PublishChannels
	/go/src/github.com/elastic/beats/metricbeat/mb/module/publish.go:48 +0xd0

goroutine 622 [chan receive, 3126 minutes]:
github.com/elastic/beats/v7/metricbeat/beater.(*Metricbeat).Run.func1(0xc0003c0250, 0xc0006863f0, 0x3913260, 0xc0007684b0)
	/go/src/github.com/elastic/beats/metricbeat/beater/metricbeat.go:214 +0x6b
created by github.com/elastic/beats/v7/metricbeat/beater.(*Metricbeat).Run
	/go/src/github.com/elastic/beats/metricbeat/beater/metricbeat.go:212 +0x108

goroutine 629 [select, 6 minutes]:
github.com/Shopify/sarama.(*client).backgroundMetadataUpdater(0xc000229710)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/client.go:779 +0x15d
github.com/Shopify/sarama.withRecover(0xc000807fd0)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/utils.go:43 +0x4d
created by github.com/Shopify/sarama.NewClient
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/client.go:178 +0x546

goroutine 627 [chan receive, 246 minutes]:
github.com/elastic/beats/v7/metricbeat/mb/module.PublishChannels.func1(0xc0008214a0)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/publish.go:40 +0x171
created by github.com/elastic/beats/v7/metricbeat/mb/module.PublishChannels
	/go/src/github.com/elastic/beats/metricbeat/mb/module/publish.go:48 +0xd0

goroutine 630 [chan receive]:
github.com/Shopify/sarama.(*asyncProducer).dispatcher(0xc0005d88c0)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/async_producer.go:326 +0xd3
github.com/Shopify/sarama.withRecover(0xc000807ff0)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/utils.go:43 +0x4d
created by github.com/Shopify/sarama.newAsyncProducer
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/async_producer.go:166 +0x249

goroutine 631 [chan receive, 3126 minutes]:
github.com/Shopify/sarama.(*asyncProducer).retryHandler(0xc0005d88c0)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/async_producer.go:1031 +0x20d
github.com/Shopify/sarama.withRecover(0xc00084e000)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/utils.go:43 +0x4d
created by github.com/Shopify/sarama.newAsyncProducer
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/async_producer.go:167 +0x297

goroutine 633 [chan receive, 3126 minutes]:
github.com/elastic/beats/v7/libbeat/outputs/kafka.(*client).errorWorker(0xc000107800, 0xc0004a8a80)
	/go/src/github.com/elastic/beats/libbeat/outputs/kafka/client.go:243 +0x150
created by github.com/elastic/beats/v7/libbeat/outputs/kafka.(*client).Connect
	/go/src/github.com/elastic/beats/libbeat/outputs/kafka/client.go:109 +0x338

goroutine 634 [chan receive]:
github.com/Shopify/sarama.(*topicProducer).dispatch(0xc000262b80)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/async_producer.go:404 +0x63
github.com/Shopify/sarama.withRecover(0xc00084e190)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/utils.go:43 +0x4d
created by github.com/Shopify/sarama.(*asyncProducer).newTopicProducer
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/async_producer.go:399 +0x1aa

goroutine 632 [chan receive]:
github.com/elastic/beats/v7/libbeat/outputs/kafka.(*client).successWorker(0xc000107800, 0xc0004a8cc0)
	/go/src/github.com/elastic/beats/libbeat/outputs/kafka/client.go:233 +0x139
created by github.com/elastic/beats/v7/libbeat/outputs/kafka.(*client).Connect
	/go/src/github.com/elastic/beats/libbeat/outputs/kafka/client.go:108 +0x2f2

goroutine 588 [chan receive]:
github.com/rcrowley/go-metrics.(*meterArbiter).tick(0x515b8a0)
	/go/src/github.com/elastic/beats/vendor/github.com/rcrowley/go-metrics/meter.go:239 +0x31
created by github.com/rcrowley/go-metrics.NewMeter
	/go/src/github.com/elastic/beats/vendor/github.com/rcrowley/go-metrics/meter.go:46 +0x111

goroutine 637 [select]:
github.com/Shopify/sarama.(*brokerProducer).run(0xc0005d8930)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/async_producer.go:744 +0x1e9
github.com/Shopify/sarama.withRecover(0xc00084e430)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/utils.go:43 +0x4d
created by github.com/Shopify/sarama.(*asyncProducer).newBrokerProducer
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/async_producer.go:688 +0x1fc

goroutine 660 [select]:
github.com/elastic/beats/v7/auditbeat/module/auditd.maintain(0xc0004a8c60, 0xc000a041e0)
	/go/src/github.com/elastic/beats/auditbeat/module/auditd/audit_linux.go:433 +0xef
created by github.com/elastic/beats/v7/auditbeat/module/auditd.(*MetricSet).receiveEvents
	/go/src/github.com/elastic/beats/auditbeat/module/auditd/audit_linux.go:391 +0x182

goroutine 638 [select]:
github.com/Shopify/sarama.(*Broker).sendAndReceive(0xc00079ca80, 0x3940ca0, 0xc000fe75c0, 0x3940d00, 0xc000fe75e0, 0x4000105, 0x0)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/broker.go:774 +0x15a
github.com/Shopify/sarama.(*Broker).Produce(0xc00079ca80, 0xc000fe75c0, 0xc00093ba01, 0x0, 0x0)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/broker.go:342 +0xe8
github.com/Shopify/sarama.(*asyncProducer).newBrokerProducer.func1()
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/async_producer.go:695 +0xb1
github.com/Shopify/sarama.withRecover(0xc00093b240)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/utils.go:43 +0x4d
created by github.com/Shopify/sarama.(*asyncProducer).newBrokerProducer
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/async_producer.go:691 +0x263

goroutine 606 [semacquire, 3126 minutes]:
sync.runtime_Semacquire(0xc0003c0428)
	/usr/local/go/src/runtime/sema.go:56 +0x42
sync.(*WaitGroup).Wait(0xc0003c0420)
	/usr/local/go/src/sync/waitgroup.go:130 +0x64
github.com/elastic/beats/v7/metricbeat/mb/module.PublishChannels(0x3912fa0, 0xc0000f4f00, 0xc0003b57b8, 0x1, 0x1)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/publish.go:51 +0xf0
github.com/elastic/beats/v7/metricbeat/mb/module.(*runner).Start.func1.1(0xc0004a4cd0, 0xc0008210e0)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/runner.go:82 +0xa4
created by github.com/elastic/beats/v7/metricbeat/mb/module.(*runner).Start.func1
	/go/src/github.com/elastic/beats/metricbeat/mb/module/runner.go:80 +0xba

goroutine 621 [semacquire, 3126 minutes]:
sync.runtime_Semacquire(0xc0003c0418)
	/usr/local/go/src/runtime/sema.go:56 +0x42
sync.(*WaitGroup).Wait(0xc0003c0410)
	/usr/local/go/src/sync/waitgroup.go:130 +0x64
github.com/elastic/beats/v7/metricbeat/mb/module.PublishChannels(0x3912fa0, 0xc0000f5200, 0xc00081b7b8, 0x1, 0x1)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/publish.go:51 +0xf0
github.com/elastic/beats/v7/metricbeat/mb/module.(*runner).Start.func1.1(0xc0004a58b0, 0xc0008214a0)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/runner.go:82 +0xa4
created by github.com/elastic/beats/v7/metricbeat/mb/module.(*runner).Start.func1
	/go/src/github.com/elastic/beats/metricbeat/mb/module/runner.go:80 +0xba

goroutine 643 [select]:
github.com/Shopify/sarama.(*brokerProducer).run(0xc0005f21c0)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/async_producer.go:744 +0x1e9
github.com/Shopify/sarama.withRecover(0xc0008e03b0)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/utils.go:43 +0x4d
created by github.com/Shopify/sarama.(*asyncProducer).newBrokerProducer
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/async_producer.go:688 +0x1fc

goroutine 617 [semacquire, 3126 minutes]:
sync.runtime_Semacquire(0xc0003c0308)
	/usr/local/go/src/runtime/sema.go:56 +0x42
sync.(*WaitGroup).Wait(0xc0003c0300)
	/usr/local/go/src/sync/waitgroup.go:130 +0x64
github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start.func2(0xc0003c0300, 0xc0008213e0, 0xc0004a5180)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:153 +0x2f
created by github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:152 +0x18e

goroutine 618 [semacquire, 3126 minutes]:
sync.runtime_Semacquire(0xc0003c03e8)
	/usr/local/go/src/runtime/sema.go:56 +0x42
sync.(*WaitGroup).Wait(0xc0003c03e0)
	/usr/local/go/src/sync/waitgroup.go:130 +0x64
github.com/elastic/beats/v7/metricbeat/mb/module.PublishChannels(0x3912fa0, 0xc0000f5100, 0xc0003b77b8, 0x1, 0x1)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/publish.go:51 +0xf0
github.com/elastic/beats/v7/metricbeat/mb/module.(*runner).Start.func1.1(0xc0004a5270, 0xc0008213e0)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/runner.go:82 +0xa4
created by github.com/elastic/beats/v7/metricbeat/mb/module.(*runner).Start.func1
	/go/src/github.com/elastic/beats/metricbeat/mb/module/runner.go:80 +0xba

goroutine 616 [runnable]:
github.com/elastic/beats/v7/x-pack/auditbeat/module/system/socket.(*MetricSet).Run(0xc00052ec40, 0x3922420, 0xc000df8630)
	/go/src/github.com/elastic/beats/x-pack/auditbeat/module/system/socket/socket_linux.go:186 +0x6b0
github.com/elastic/beats/v7/metricbeat/mb/module.(*metricSetWrapper).run(0xc0006f7ec0, 0xc0000a8420, 0xc0008213e0)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:203 +0x659
github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start.func1(0xc0003c0300, 0xc0000a8420, 0xc0008213e0, 0xc0006f7ec0)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:147 +0x2e8
created by github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:135 +0x140

goroutine 652 [select]:
github.com/Shopify/sarama.(*Broker).sendAndReceive(0xc00079c700, 0x3940ca0, 0xc000fe7480, 0x3940d00, 0xc000fe74a0, 0x8000101, 0x0)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/broker.go:774 +0x15a
github.com/Shopify/sarama.(*Broker).Produce(0xc00079c700, 0xc000fe7480, 0xc00179d201, 0x0, 0x0)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/broker.go:342 +0xe8
github.com/Shopify/sarama.(*asyncProducer).newBrokerProducer.func1()
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/async_producer.go:695 +0xb1
github.com/Shopify/sarama.withRecover(0xc000977dc0)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/utils.go:43 +0x4d
created by github.com/Shopify/sarama.(*asyncProducer).newBrokerProducer
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/async_producer.go:691 +0x263

goroutine 615 [semacquire, 3126 minutes]:
sync.runtime_Semacquire(0xc0003c03c8)
	/usr/local/go/src/runtime/sema.go:56 +0x42
sync.(*WaitGroup).Wait(0xc0003c03c0)
	/usr/local/go/src/sync/waitgroup.go:130 +0x64
github.com/elastic/beats/v7/metricbeat/mb/module.PublishChannels(0x3912fa0, 0xc0000f5080, 0xc00081afb8, 0x1, 0x1)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/publish.go:51 +0xf0
github.com/elastic/beats/v7/metricbeat/mb/module.(*runner).Start.func1.1(0xc0004a5130, 0xc000821320)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/runner.go:82 +0xa4
created by github.com/elastic/beats/v7/metricbeat/mb/module.(*runner).Start.func1
	/go/src/github.com/elastic/beats/metricbeat/mb/module/runner.go:80 +0xba

goroutine 611 [semacquire, 3126 minutes]:
sync.runtime_Semacquire(0xc0003c02d8)
	/usr/local/go/src/runtime/sema.go:56 +0x42
sync.(*WaitGroup).Wait(0xc0003c02d0)
	/usr/local/go/src/sync/waitgroup.go:130 +0x64
github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start.func2(0xc0003c02d0, 0xc000821260, 0xc0004a4e60)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:153 +0x2f
created by github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:152 +0x18e

goroutine 610 [select]:
github.com/elastic/beats/v7/metricbeat/mb/module.(*metricSetWrapper).startPeriodicFetching(0xc0006f7860, 0x392c9e0, 0xc000904010, 0x39131e0, 0xc000df9290)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:230 +0x219
github.com/elastic/beats/v7/metricbeat/mb/module.(*metricSetWrapper).run(0xc0006f7860, 0xc0004a9e60, 0xc000821260)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:208 +0x7e1
github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start.func1(0xc0003c02d0, 0xc0004a9e60, 0xc000821260, 0xc0006f7860)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:147 +0x2e8
created by github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:135 +0x140

goroutine 607 [select]:
github.com/elastic/beats/v7/metricbeat/mb/module.(*metricSetWrapper).startPeriodicFetching(0xc0006f7530, 0x392c9e0, 0xc000904558, 0x39131e0, 0xc0006f7f20)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:230 +0x219
github.com/elastic/beats/v7/metricbeat/mb/module.(*metricSetWrapper).run(0xc0006f7530, 0xc0004a9d40, 0xc0008211a0)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:208 +0x7e1
github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start.func1(0xc0003c02c0, 0xc0004a9d40, 0xc0008211a0, 0xc0006f7530)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:147 +0x2e8
created by github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:135 +0x140

goroutine 620 [semacquire, 3126 minutes]:
sync.runtime_Semacquire(0xc0003c0328)
	/usr/local/go/src/runtime/sema.go:56 +0x42
sync.(*WaitGroup).Wait(0xc0003c0320)
	/usr/local/go/src/sync/waitgroup.go:130 +0x64
github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start.func2(0xc0003c0320, 0xc0008214a0, 0xc0004a52c0)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:153 +0x2f
created by github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:152 +0x18e

goroutine 614 [semacquire, 3126 minutes]:
sync.runtime_Semacquire(0xc0003c02f8)
	/usr/local/go/src/runtime/sema.go:56 +0x42
sync.(*WaitGroup).Wait(0xc0003c02f0)
	/usr/local/go/src/sync/waitgroup.go:130 +0x64
github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start.func2(0xc0003c02f0, 0xc000821320, 0xc0004a4fa0)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:153 +0x2f
created by github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:152 +0x18e

goroutine 613 [select]:
github.com/elastic/beats/v7/metricbeat/mb/module.(*metricSetWrapper).startPeriodicFetching(0xc0006f7b90, 0x392c9e0, 0xc000904260, 0x39131e0, 0xc0007b30b0)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:230 +0x219
github.com/elastic/beats/v7/metricbeat/mb/module.(*metricSetWrapper).run(0xc0006f7b90, 0xc0004a9f80, 0xc000821320)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:208 +0x7e1
github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start.func1(0xc0003c02f0, 0xc0004a9f80, 0xc000821320, 0xc0006f7b90)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:147 +0x2e8
created by github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:135 +0x140

goroutine 624 [chan receive, 245 minutes]:
github.com/elastic/beats/v7/metricbeat/mb/module.PublishChannels.func1(0xc000821260)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/publish.go:40 +0x171
created by github.com/elastic/beats/v7/metricbeat/mb/module.PublishChannels
	/go/src/github.com/elastic/beats/metricbeat/mb/module/publish.go:48 +0xd0

goroutine 612 [semacquire, 3126 minutes]:
sync.runtime_Semacquire(0xc0003c0378)
	/usr/local/go/src/runtime/sema.go:56 +0x42
sync.(*WaitGroup).Wait(0xc0003c0370)
	/usr/local/go/src/sync/waitgroup.go:130 +0x64
github.com/elastic/beats/v7/metricbeat/mb/module.PublishChannels(0x3912fa0, 0xc0000f5000, 0xc0003bb7b8, 0x1, 0x1)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/publish.go:51 +0xf0
github.com/elastic/beats/v7/metricbeat/mb/module.(*runner).Start.func1.1(0xc0004a4f50, 0xc000821260)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/runner.go:82 +0xa4
created by github.com/elastic/beats/v7/metricbeat/mb/module.(*runner).Start.func1
	/go/src/github.com/elastic/beats/metricbeat/mb/module/runner.go:80 +0xba

goroutine 651 [select]:
github.com/Shopify/sarama.(*brokerProducer).run(0xc0005f2d90)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/async_producer.go:744 +0x1e9
github.com/Shopify/sarama.withRecover(0xc0008e0510)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/utils.go:43 +0x4d
created by github.com/Shopify/sarama.(*asyncProducer).newBrokerProducer
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/async_producer.go:688 +0x1fc

goroutine 641 [IO wait]:
internal/poll.runtime_pollWait(0x7f41b474ce20, 0x72, 0xffffffffffffffff)
	/usr/local/go/src/runtime/netpoll.go:184 +0x55
internal/poll.(*pollDesc).wait(0xc0000f5818, 0x72, 0x900, 0x9da, 0xffffffffffffffff)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x45
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc0000f5800, 0xc000716a80, 0x9da, 0x9da, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:169 +0x1cf
net.(*netFD).Read(0xc0000f5800, 0xc000716a80, 0x9da, 0x9da, 0x203000, 0x0, 0x9cd)
	/usr/local/go/src/net/fd_unix.go:202 +0x4f
net.(*conn).Read(0xc000010f50, 0xc000716a80, 0x9da, 0x9da, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:184 +0x68
crypto/tls.(*atLeastReader).Read(0xc000fe7820, 0xc000716a80, 0x9da, 0x9da, 0x53, 0x38c40e0, 0xc0009bba00)
	/usr/local/go/src/crypto/tls/conn.go:780 +0x60
bytes.(*Buffer).ReadFrom(0xc000233ad8, 0x38c3e80, 0xc000fe7820, 0x1361ea5, 0x30f71a0, 0x339cce0)
	/usr/local/go/src/bytes/buffer.go:204 +0xb4
crypto/tls.(*Conn).readFromUntil(0xc000233880, 0x38c7800, 0xc000010f50, 0x5, 0xc000010f50, 0x36)
	/usr/local/go/src/crypto/tls/conn.go:802 +0xec
crypto/tls.(*Conn).readRecordOrCCS(0xc000233880, 0x0, 0x0, 0xc0009bbcc8)
	/usr/local/go/src/crypto/tls/conn.go:609 +0x124
crypto/tls.(*Conn).readRecord(...)
	/usr/local/go/src/crypto/tls/conn.go:577
crypto/tls.(*Conn).Read(0xc000233880, 0xc0006e7000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/crypto/tls/conn.go:1255 +0x161
bufio.(*Reader).Read(0xc0003dd680, 0xc001037988, 0x8, 0x8, 0x1360d5b, 0xc00003c000, 0x30f71a0)
	/usr/local/go/src/bufio/bufio.go:226 +0x26a
github.com/Shopify/sarama.(*bufConn).Read(0xc00093b280, 0xc001037988, 0x8, 0x8, 0xc0009bbda0, 0x1361ea5, 0x30f71a0)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/utils.go:107 +0x4d
io.ReadAtLeast(0x7f41b475da90, 0xc00093b280, 0xc001037988, 0x8, 0x8, 0x8, 0x515d2a0, 0x1cc9101, 0xc001037988)
	/usr/local/go/src/io/io.go:310 +0x87
io.ReadFull(...)
	/usr/local/go/src/io/io.go:329
github.com/Shopify/sarama.(*Broker).readFull(0xc00079d180, 0xc001037988, 0x8, 0x8, 0x32, 0x0, 0x0)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/broker.go:702 +0x105
github.com/Shopify/sarama.(*Broker).responseReceiver(0xc00079d180)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/broker.go:858 +0x174
github.com/Shopify/sarama.withRecover(0xc00084e6b0)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/utils.go:43 +0x4d
created by github.com/Shopify/sarama.(*Broker).Open.func1
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/broker.go:211 +0x87c

goroutine 519 [chan receive, 3126 minutes]:
github.com/elastic/go-perf.(*Event).poll(0xc000138750)
	/go/src/github.com/elastic/beats/vendor/github.com/elastic/go-perf/record.go:248 +0x10a
created by github.com/elastic/go-perf.(*Event).MapRingNumPages
	/go/src/github.com/elastic/beats/vendor/github.com/elastic/go-perf/perf.go:284 +0x296

goroutine 656 [select]:
github.com/elastic/beats/v7/x-pack/auditbeat/module/system/socket.(*state).logStateLoop(0xc000506500)
	/go/src/github.com/elastic/beats/x-pack/auditbeat/module/system/socket/state.go:502 +0x11f
created by github.com/elastic/beats/v7/x-pack/auditbeat/module/system/socket.NewState
	/go/src/github.com/elastic/beats/x-pack/auditbeat/module/system/socket/state.go:402 +0xca

goroutine 658 [IO wait]:
internal/poll.runtime_pollWait(0x7f41b474cc80, 0x72, 0xffffffffffffffff)
	/usr/local/go/src/runtime/netpoll.go:184 +0x55
internal/poll.(*pollDesc).wait(0xc000544398, 0x72, 0x900, 0x9da, 0xffffffffffffffff)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x45
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc000544380, 0xc0007d7500, 0x9da, 0x9da, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:169 +0x1cf
net.(*netFD).Read(0xc000544380, 0xc0007d7500, 0x9da, 0x9da, 0x203000, 0x0, 0x9cd)
	/usr/local/go/src/net/fd_unix.go:202 +0x4f
net.(*conn).Read(0xc000904200, 0xc0007d7500, 0x9da, 0x9da, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:184 +0x68
crypto/tls.(*atLeastReader).Read(0xc000fe76a0, 0xc0007d7500, 0x9da, 0x9da, 0x53, 0x38c40e0, 0xc0009b1a00)
	/usr/local/go/src/crypto/tls/conn.go:780 +0x60
bytes.(*Buffer).ReadFrom(0xc00079dad8, 0x38c3e80, 0xc000fe76a0, 0x1361ea5, 0x30f71a0, 0x339cce0)
	/usr/local/go/src/bytes/buffer.go:204 +0xb4
crypto/tls.(*Conn).readFromUntil(0xc00079d880, 0x38c7800, 0xc000904200, 0x5, 0xc000904200, 0x36)
	/usr/local/go/src/crypto/tls/conn.go:802 +0xec
crypto/tls.(*Conn).readRecordOrCCS(0xc00079d880, 0x0, 0x0, 0xc0009b1cc8)
	/usr/local/go/src/crypto/tls/conn.go:609 +0x124
crypto/tls.(*Conn).readRecord(...)
	/usr/local/go/src/crypto/tls/conn.go:577
crypto/tls.(*Conn).Read(0xc00079d880, 0xc0006e5000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/crypto/tls/conn.go:1255 +0x161
bufio.(*Reader).Read(0xc0003451a0, 0xc001037980, 0x8, 0x8, 0x1360d5b, 0xc00003c000, 0x30f71a0)
	/usr/local/go/src/bufio/bufio.go:226 +0x26a
github.com/Shopify/sarama.(*bufConn).Read(0xc0001483a0, 0xc001037980, 0x8, 0x8, 0xc0009b1da0, 0x1361ea5, 0x30f71a0)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/utils.go:107 +0x4d
io.ReadAtLeast(0x7f41b475da90, 0xc0001483a0, 0xc001037980, 0x8, 0x8, 0x8, 0x515d2a0, 0x1cc9101, 0xc001037980)
	/usr/local/go/src/io/io.go:310 +0x87
io.ReadFull(...)
	/usr/local/go/src/io/io.go:329
github.com/Shopify/sarama.(*Broker).readFull(0xc00079ca80, 0xc001037980, 0x8, 0x8, 0x32, 0x0, 0x0)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/broker.go:702 +0x105
github.com/Shopify/sarama.(*Broker).responseReceiver(0xc00079ca80)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/broker.go:858 +0x174
github.com/Shopify/sarama.withRecover(0xc00084e720)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/utils.go:43 +0x4d
created by github.com/Shopify/sarama.(*Broker).Open.func1
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/broker.go:211 +0x87c

goroutine 646 [select]:
github.com/Shopify/sarama.(*brokerProducer).run(0xc0005f2230)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/async_producer.go:744 +0x1e9
github.com/Shopify/sarama.withRecover(0xc0008e03d0)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/utils.go:43 +0x4d
created by github.com/Shopify/sarama.(*asyncProducer).newBrokerProducer
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/async_producer.go:688 +0x1fc

goroutine 591 [chan receive]:
github.com/Shopify/sarama.(*partitionProducer).dispatch(0xc0003cb020)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/async_producer.go:539 +0x107
github.com/Shopify/sarama.withRecover(0xc0008e0370)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/utils.go:43 +0x4d
created by github.com/Shopify/sarama.(*asyncProducer).newPartitionProducer
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/async_producer.go:505 +0x1a8

goroutine 640 [runnable]:
internal/poll.runtime_pollWait(0x7f41b474cef0, 0x72, 0xffffffffffffffff)
	/usr/local/go/src/runtime/netpoll.go:184 +0x55
internal/poll.(*pollDesc).wait(0xc0000f4e18, 0x72, 0x900, 0x9da, 0xffffffffffffffff)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x45
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc0000f4e00, 0xc000716000, 0x9da, 0x9da, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:169 +0x1cf
net.(*netFD).Read(0xc0000f4e00, 0xc000716000, 0x9da, 0x9da, 0x203000, 0x0, 0x9cd)
	/usr/local/go/src/net/fd_unix.go:202 +0x4f
net.(*conn).Read(0xc0009041f8, 0xc000716000, 0x9da, 0x9da, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:184 +0x68
crypto/tls.(*atLeastReader).Read(0xc000fe7460, 0xc000716000, 0x9da, 0x9da, 0x53, 0x38c40e0, 0xc0009afa00)
	/usr/local/go/src/crypto/tls/conn.go:780 +0x60
bytes.(*Buffer).ReadFrom(0xc00079d758, 0x38c3e80, 0xc000fe7460, 0x1361ea5, 0x30f71a0, 0x339cce0)
	/usr/local/go/src/bytes/buffer.go:204 +0xb4
crypto/tls.(*Conn).readFromUntil(0xc00079d500, 0x38c7800, 0xc0009041f8, 0x5, 0xc0009041f8, 0x36)
	/usr/local/go/src/crypto/tls/conn.go:802 +0xec
crypto/tls.(*Conn).readRecordOrCCS(0xc00079d500, 0x0, 0x0, 0xc0009afcc8)
	/usr/local/go/src/crypto/tls/conn.go:609 +0x124
crypto/tls.(*Conn).readRecord(...)
	/usr/local/go/src/crypto/tls/conn.go:577
crypto/tls.(*Conn).Read(0xc00079d500, 0xc0006e4000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/crypto/tls/conn.go:1255 +0x161
bufio.(*Reader).Read(0xc000344660, 0xc001037870, 0x8, 0x8, 0x1360d5b, 0xc00003c000, 0x30f71a0)
	/usr/local/go/src/bufio/bufio.go:226 +0x26a
github.com/Shopify/sarama.(*bufConn).Read(0xc000977ee0, 0xc001037870, 0x8, 0x8, 0xc0009afda0, 0x1361ea5, 0x30f71a0)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/utils.go:107 +0x4d
io.ReadAtLeast(0x7f41b475da90, 0xc000977ee0, 0xc001037870, 0x8, 0x8, 0x8, 0x515d2a0, 0x1cc9101, 0xc001037870)
	/usr/local/go/src/io/io.go:310 +0x87
io.ReadFull(...)
	/usr/local/go/src/io/io.go:329
github.com/Shopify/sarama.(*Broker).readFull(0xc00079ce00, 0xc001037870, 0x8, 0x8, 0x32, 0x0, 0x0)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/broker.go:702 +0x105
github.com/Shopify/sarama.(*Broker).responseReceiver(0xc00079ce00)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/broker.go:858 +0x174
github.com/Shopify/sarama.withRecover(0xc00084e650)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/utils.go:43 +0x4d
created by github.com/Shopify/sarama.(*Broker).Open.func1
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/broker.go:211 +0x87c

goroutine 593 [chan receive]:
github.com/Shopify/sarama.(*partitionProducer).dispatch(0xc0003cb1a0)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/async_producer.go:539 +0x107
github.com/Shopify/sarama.withRecover(0xc0008e0390)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/utils.go:43 +0x4d
created by github.com/Shopify/sarama.(*asyncProducer).newPartitionProducer
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/async_producer.go:505 +0x1a8

goroutine 647 [select]:
github.com/Shopify/sarama.(*Broker).sendAndReceive(0xc00079d180, 0x3940ca0, 0xc000fe7740, 0x3940d00, 0xc000fe7760, 0x7000102, 0x0)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/broker.go:774 +0x15a
github.com/Shopify/sarama.(*Broker).Produce(0xc00079d180, 0xc000fe7740, 0xc00179d101, 0x0, 0x0)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/broker.go:342 +0xe8
github.com/Shopify/sarama.(*asyncProducer).newBrokerProducer.func1()
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/async_producer.go:695 +0xb1
github.com/Shopify/sarama.withRecover(0xc000977c80)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/utils.go:43 +0x4d
created by github.com/Shopify/sarama.(*asyncProducer).newBrokerProducer
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/async_producer.go:691 +0x263

goroutine 592 [chan receive]:
github.com/Shopify/sarama.(*partitionProducer).dispatch(0xc0003cb0e0)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/async_producer.go:539 +0x107
github.com/Shopify/sarama.withRecover(0xc0008e0380)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/utils.go:43 +0x4d
created by github.com/Shopify/sarama.(*asyncProducer).newPartitionProducer
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/async_producer.go:505 +0x1a8

goroutine 599 [chan receive, 3126 minutes]:
github.com/elastic/beats/v7/metricbeat/beater.(*Metricbeat).Run.func1(0xc0003c0250, 0xc0006863f0, 0x3913260, 0xc000314a80)
	/go/src/github.com/elastic/beats/metricbeat/beater/metricbeat.go:214 +0x6b
created by github.com/elastic/beats/v7/metricbeat/beater.(*Metricbeat).Run
	/go/src/github.com/elastic/beats/metricbeat/beater/metricbeat.go:212 +0x108

goroutine 644 [select]:
github.com/Shopify/sarama.(*Broker).sendAndReceive(0xc00079ce00, 0x3940ca0, 0xc000fe7380, 0x3940d00, 0xc000fe73a0, 0x2000107, 0x0)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/broker.go:774 +0x15a
github.com/Shopify/sarama.(*Broker).Produce(0xc00079ce00, 0xc000fe7380, 0xc00093bb01, 0x0, 0x0)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/broker.go:342 +0xe8
github.com/Shopify/sarama.(*asyncProducer).newBrokerProducer.func1()
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/async_producer.go:695 +0xb1
github.com/Shopify/sarama.withRecover(0xc000977c40)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/utils.go:43 +0x4d
created by github.com/Shopify/sarama.(*asyncProducer).newBrokerProducer
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/async_producer.go:691 +0x263

goroutine 589 [chan receive, 6 minutes]:
github.com/Shopify/sarama.(*Broker).responseReceiver(0xc00020f500)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/broker.go:846 +0x98
github.com/Shopify/sarama.withRecover(0xc0008e02e0)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/utils.go:43 +0x4d
created by github.com/Shopify/sarama.(*Broker).Open.func1
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/broker.go:211 +0x87c

goroutine 586 [chan receive, 3126 minutes]:
github.com/elastic/beats/v7/metricbeat/mb/module.PublishChannels.func1(0xc000821020)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/publish.go:40 +0x171
created by github.com/elastic/beats/v7/metricbeat/mb/module.PublishChannels
	/go/src/github.com/elastic/beats/metricbeat/mb/module/publish.go:48 +0xd0

goroutine 584 [chan receive, 920 minutes]:
github.com/elastic/beats/v7/metricbeat/mb/module.PublishChannels.func1(0xc0008211a0)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/publish.go:40 +0x171
created by github.com/elastic/beats/v7/metricbeat/mb/module.PublishChannels
	/go/src/github.com/elastic/beats/metricbeat/mb/module/publish.go:48 +0xd0

goroutine 590 [chan receive]:
github.com/Shopify/sarama.(*partitionProducer).dispatch(0xc0003caf60)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/async_producer.go:539 +0x107
github.com/Shopify/sarama.withRecover(0xc0008e0360)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/utils.go:43 +0x4d
created by github.com/Shopify/sarama.(*asyncProducer).newPartitionProducer
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/async_producer.go:505 +0x1a8

goroutine 585 [chan receive]:
github.com/elastic/beats/v7/metricbeat/mb/module.PublishChannels.func1(0xc000820f60)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/publish.go:40 +0x171
created by github.com/elastic/beats/v7/metricbeat/mb/module.PublishChannels
	/go/src/github.com/elastic/beats/metricbeat/mb/module/publish.go:48 +0xd0

goroutine 605 [semacquire, 3126 minutes]:
sync.runtime_Semacquire(0xc0003c02a8)
	/usr/local/go/src/runtime/sema.go:56 +0x42
sync.(*WaitGroup).Wait(0xc0003c02a0)
	/usr/local/go/src/sync/waitgroup.go:130 +0x64
github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start.func2(0xc0003c02a0, 0xc0008210e0, 0xc0004a4be0)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:153 +0x2f
created by github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:152 +0x18e

goroutine 576 [select]:
github.com/elastic/beats/v7/libbeat/monitoring/report/log.(*reporter).snapshotLoop(0xc0007684e0)
	/go/src/github.com/elastic/beats/libbeat/monitoring/report/log/log.go:129 +0x410
github.com/elastic/beats/v7/libbeat/monitoring/report/log.MakeReporter.func1(0xc0007684e0)
	/go/src/github.com/elastic/beats/libbeat/monitoring/report/log/log.go:107 +0x5d
created by github.com/elastic/beats/v7/libbeat/monitoring/report/log.MakeReporter
	/go/src/github.com/elastic/beats/libbeat/monitoring/report/log/log.go:105 +0x157

goroutine 602 [semacquire, 3126 minutes]:
sync.runtime_Semacquire(0xc000473698)
	/usr/local/go/src/runtime/sema.go:56 +0x42
sync.(*WaitGroup).Wait(0xc000473690)
	/usr/local/go/src/sync/waitgroup.go:130 +0x64
github.com/elastic/beats/v7/metricbeat/mb/module.PublishChannels(0x3912fa0, 0xc0000f5980, 0xc0003fdfb8, 0x1, 0x1)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/publish.go:51 +0xf0
github.com/elastic/beats/v7/metricbeat/mb/module.(*runner).Start.func1.1(0xc0004a5860, 0xc000821020)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/runner.go:82 +0xa4
created by github.com/elastic/beats/v7/metricbeat/mb/module.(*runner).Start.func1
	/go/src/github.com/elastic/beats/metricbeat/mb/module/runner.go:80 +0xba

goroutine 596 [semacquire, 3126 minutes]:
sync.runtime_Semacquire(0xc000a006f8)
	/usr/local/go/src/runtime/sema.go:56 +0x42
sync.(*WaitGroup).Wait(0xc000a006f0)
	/usr/local/go/src/sync/waitgroup.go:130 +0x64
github.com/elastic/beats/v7/auditbeat/module/auditd.(*MetricSet).Run(0xc00052e700, 0x3922420, 0xc00087a510)
	/go/src/github.com/elastic/beats/auditbeat/module/auditd/audit_linux.go:209 +0x2df
github.com/elastic/beats/v7/metricbeat/mb/module.(*metricSetWrapper).run(0xc0003147e0, 0xc0004a8c60, 0xc000820f60)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:203 +0x659
github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start.func1(0xc0003c0270, 0xc0004a8c60, 0xc000820f60, 0xc0003147e0)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:147 +0x2e8
created by github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:135 +0x140

goroutine 594 [chan receive, 3126 minutes]:
github.com/elastic/beats/v7/libbeat/service.HandleSignals.func1(0xc000820ea0, 0xc000158f50, 0xc000158f10, 0xc0003c0210, 0xc00079ee40)
	/go/src/github.com/elastic/beats/libbeat/service/service.go:50 +0x52
created by github.com/elastic/beats/v7/libbeat/service.HandleSignals
	/go/src/github.com/elastic/beats/libbeat/service/service.go:49 +0x172

goroutine 601 [semacquire, 3126 minutes]:
sync.runtime_Semacquire(0xc0003c0298)
	/usr/local/go/src/runtime/sema.go:56 +0x42
sync.(*WaitGroup).Wait(0xc0003c0290)
	/usr/local/go/src/sync/waitgroup.go:130 +0x64
github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start.func2(0xc0003c0290, 0xc000821020, 0xc0004a5770)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:153 +0x2f
created by github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:152 +0x18e

goroutine 609 [semacquire, 3126 minutes]:
sync.runtime_Semacquire(0xc000473658)
	/usr/local/go/src/runtime/sema.go:56 +0x42
sync.(*WaitGroup).Wait(0xc000473650)
	/usr/local/go/src/sync/waitgroup.go:130 +0x64
github.com/elastic/beats/v7/metricbeat/mb/module.PublishChannels(0x3912fa0, 0xc0000f4f80, 0xc00081a7b8, 0x1, 0x1)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/publish.go:51 +0xf0
github.com/elastic/beats/v7/metricbeat/mb/module.(*runner).Start.func1.1(0xc0004a4e10, 0xc0008211a0)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/runner.go:82 +0xa4
created by github.com/elastic/beats/v7/metricbeat/mb/module.(*runner).Start.func1
	/go/src/github.com/elastic/beats/metricbeat/mb/module/runner.go:80 +0xba

goroutine 600 [select, 3125 minutes]:
github.com/elastic/beats/v7/auditbeat/module/file_integrity.(*MetricSet).Run(0xc0004c5000, 0x3922420, 0xc0007690e0)
	/go/src/github.com/elastic/beats/auditbeat/module/file_integrity/metricset.go:112 +0x1d2
github.com/elastic/beats/v7/metricbeat/mb/module.(*metricSetWrapper).run(0xc0003153b0, 0xc0004a8f60, 0xc000821020)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:203 +0x659
github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start.func1(0xc0003c0290, 0xc0004a8f60, 0xc000821020, 0xc0003153b0)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:147 +0x2e8
created by github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:135 +0x140

goroutine 598 [semacquire, 3126 minutes]:
sync.runtime_Semacquire(0xc000473668)
	/usr/local/go/src/runtime/sema.go:56 +0x42
sync.(*WaitGroup).Wait(0xc000473660)
	/usr/local/go/src/sync/waitgroup.go:130 +0x64
github.com/elastic/beats/v7/metricbeat/mb/module.PublishChannels(0x3912fa0, 0xc0000f5900, 0xc00081d7b8, 0x1, 0x1)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/publish.go:51 +0xf0
github.com/elastic/beats/v7/metricbeat/mb/module.(*runner).Start.func1.1(0xc0004a5680, 0xc000820f60)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/runner.go:82 +0xa4
created by github.com/elastic/beats/v7/metricbeat/mb/module.(*runner).Start.func1
	/go/src/github.com/elastic/beats/metricbeat/mb/module/runner.go:80 +0xba

goroutine 661 [syscall]:
syscall.Syscall6(0x2d, 0x4, 0xc00031a500, 0x231a, 0x0, 0xc0003f1c60, 0xc0003f1c54, 0x8, 0x5, 0x3f)
	/usr/local/go/src/syscall/asm_linux_amd64.s:44 +0x5
syscall.recvfrom(0x4, 0xc00031a500, 0x231a, 0x231a, 0x0, 0xc0003f1c60, 0xc0003f1c54, 0xc0003f1cc8, 0x3, 0x3)
	/usr/local/go/src/syscall/zsyscall_linux_amd64.go:1618 +0xa3
syscall.Recvfrom(0x4, 0xc00031a500, 0x231a, 0x231a, 0x0, 0x0, 0x0, 0x0, 0x0, 0x3)
	/usr/local/go/src/syscall/syscall_unix.go:273 +0xaf
github.com/elastic/go-libaudit.(*NetlinkClient).Receive(0xc0006c2de0, 0x0, 0x35c3050, 0x2d399d9, 0xc000a041e0, 0xc0007db680, 0xd5, 0xc0007db680)
	/go/src/github.com/elastic/beats/vendor/github.com/elastic/go-libaudit/netlink.go:164 +0x7c
github.com/elastic/go-libaudit.(*AuditClient).Receive(0xc000649680, 0x400, 0xc00031a510, 0xd5, 0x230a)
	/go/src/github.com/elastic/beats/vendor/github.com/elastic/go-libaudit/audit.go:399 +0x4c
github.com/elastic/beats/v7/auditbeat/module/auditd.(*MetricSet).receiveEvents.func1(0xc00052e700, 0xc000820240, 0xc000a041e0)
	/go/src/github.com/elastic/beats/auditbeat/module/auditd/audit_linux.go:399 +0x167
created by github.com/elastic/beats/v7/auditbeat/module/auditd.(*MetricSet).receiveEvents
	/go/src/github.com/elastic/beats/auditbeat/module/auditd/audit_linux.go:393 +0x1bb

goroutine 597 [semacquire, 3126 minutes]:
sync.runtime_Semacquire(0xc0003c0278)
	/usr/local/go/src/runtime/sema.go:56 +0x42
sync.(*WaitGroup).Wait(0xc0003c0270)
	/usr/local/go/src/sync/waitgroup.go:130 +0x64
github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start.func2(0xc0003c0270, 0xc000820f60, 0xc0004a5590)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:153 +0x2f
created by github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:152 +0x18e

goroutine 604 [select]:
github.com/elastic/beats/v7/metricbeat/mb/module.(*metricSetWrapper).startPeriodicFetching(0xc0006f7200, 0x392c9e0, 0xc000904050, 0x39131e0, 0xc00067f500)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:230 +0x219
github.com/elastic/beats/v7/metricbeat/mb/module.(*metricSetWrapper).run(0xc0006f7200, 0xc0004a9c20, 0xc0008210e0)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:208 +0x7e1
github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start.func1(0xc0003c02a0, 0xc0004a9c20, 0xc0008210e0, 0xc0006f7200)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:147 +0x2e8
created by github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:135 +0x140

goroutine 603 [chan receive, 3126 minutes]:
github.com/elastic/beats/v7/metricbeat/beater.(*Metricbeat).Run.func1(0xc0003c0250, 0xc0006863f0, 0x3913260, 0xc000315650)
	/go/src/github.com/elastic/beats/metricbeat/beater/metricbeat.go:214 +0x6b
created by github.com/elastic/beats/v7/metricbeat/beater.(*Metricbeat).Run
	/go/src/github.com/elastic/beats/metricbeat/beater/metricbeat.go:212 +0x108

goroutine 520 [chan receive, 3126 minutes]:
github.com/elastic/go-perf.(*Event).poll(0xc0001389c0)
	/go/src/github.com/elastic/beats/vendor/github.com/elastic/go-perf/record.go:248 +0x10a
created by github.com/elastic/go-perf.(*Event).MapRingNumPages
	/go/src/github.com/elastic/beats/vendor/github.com/elastic/go-perf/perf.go:284 +0x296

goroutine 608 [semacquire, 3126 minutes]:
sync.runtime_Semacquire(0xc0003c02c8)
	/usr/local/go/src/runtime/sema.go:56 +0x42
sync.(*WaitGroup).Wait(0xc0003c02c0)
	/usr/local/go/src/sync/waitgroup.go:130 +0x64
github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start.func2(0xc0003c02c0, 0xc0008211a0, 0xc0004a4d20)
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:153 +0x2f
created by github.com/elastic/beats/v7/metricbeat/mb/module.(*Wrapper).Start
	/go/src/github.com/elastic/beats/metricbeat/mb/module/wrapper.go:152 +0x18e

goroutine 654 [IO wait]:
internal/poll.runtime_pollWait(0x7f41b474cd50, 0x72, 0xffffffffffffffff)
	/usr/local/go/src/runtime/netpoll.go:184 +0x55
internal/poll.(*pollDesc).wait(0xc0000f5b18, 0x72, 0x900, 0x9da, 0xffffffffffffffff)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x45
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc0000f5b00, 0xc0007d6a80, 0x9da, 0x9da, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:169 +0x1cf
net.(*netFD).Read(0xc0000f5b00, 0xc0007d6a80, 0x9da, 0x9da, 0x203000, 0x0, 0x9cd)
	/usr/local/go/src/net/fd_unix.go:202 +0x4f
net.(*conn).Read(0xc000010f48, 0xc0007d6a80, 0x9da, 0x9da, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:184 +0x68
crypto/tls.(*atLeastReader).Read(0xc000fe7560, 0xc0007d6a80, 0x9da, 0x9da, 0x53, 0x38c40e0, 0xc00097da00)
	/usr/local/go/src/crypto/tls/conn.go:780 +0x60
bytes.(*Buffer).ReadFrom(0xc000233758, 0x38c3e80, 0xc000fe7560, 0x1361ea5, 0x30f71a0, 0x339cce0)
	/usr/local/go/src/bytes/buffer.go:204 +0xb4
crypto/tls.(*Conn).readFromUntil(0xc000233500, 0x38c7800, 0xc000010f48, 0x5, 0xc000010f48, 0x36)
	/usr/local/go/src/crypto/tls/conn.go:802 +0xec
crypto/tls.(*Conn).readRecordOrCCS(0xc000233500, 0x0, 0x0, 0xc00097dcc8)
	/usr/local/go/src/crypto/tls/conn.go:609 +0x124
crypto/tls.(*Conn).readRecord(...)
	/usr/local/go/src/crypto/tls/conn.go:577
crypto/tls.(*Conn).Read(0xc000233500, 0xc0006e6000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/crypto/tls/conn.go:1255 +0x161
bufio.(*Reader).Read(0xc0003dd620, 0xc001037878, 0x8, 0x8, 0x1360d5b, 0xc00003c000, 0x30f71a0)
	/usr/local/go/src/bufio/bufio.go:226 +0x26a
github.com/Shopify/sarama.(*bufConn).Read(0xc00093b260, 0xc001037878, 0x8, 0x8, 0xc00097dda0, 0x1361ea5, 0x30f71a0)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/utils.go:107 +0x4d
io.ReadAtLeast(0x7f41b475da90, 0xc00093b260, 0xc001037878, 0x8, 0x8, 0x8, 0x515d2a0, 0x1cc9101, 0xc001037878)
	/usr/local/go/src/io/io.go:310 +0x87
io.ReadFull(...)
	/usr/local/go/src/io/io.go:329
github.com/Shopify/sarama.(*Broker).readFull(0xc00079c700, 0xc001037878, 0x8, 0x8, 0x32, 0x0, 0x0)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/broker.go:702 +0x105
github.com/Shopify/sarama.(*Broker).responseReceiver(0xc00079c700)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/broker.go:858 +0x174
github.com/Shopify/sarama.withRecover(0xc0008e1da0)
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/utils.go:43 +0x4d
created by github.com/Shopify/sarama.(*Broker).Open.func1
	/go/src/github.com/elastic/beats/vendor/github.com/Shopify/sarama/broker.go:211 +0x87c

goroutine 657 [runnable]:
syscall.Syscall(0x7, 0xc000433a68, 0x1, 0x1f4, 0x1, 0x1f4, 0x0)
	/usr/local/go/src/syscall/asm_linux_amd64.s:18 +0x5
golang.org/x/sys/unix.poll(0xc000433a68, 0x1, 0x1f4, 0xc000ebcb80, 0x1c, 0xc000fe7f00)
	/go/src/github.com/elastic/beats/vendor/golang.org/x/sys/unix/zsyscall_linux_amd64.go:2537 +0x56
golang.org/x/sys/unix.Poll(0xc000433a68, 0x1, 0x1, 0x1f4, 0xc00052ec40, 0x20, 0x18)
	/go/src/github.com/elastic/beats/vendor/golang.org/x/sys/unix/syscall_linux_amd64.go:180 +0x86
github.com/google/gopacket/afpacket.(*TPacket).pollForFirstPacket(0xc0000cc300, 0x394e800, 0x7f41b1e0a400, 0xc000433b10, 0xc00087f180)
	/go/src/github.com/elastic/beats/vendor/github.com/google/gopacket/afpacket/afpacket.go:456 +0xb4
github.com/google/gopacket/afpacket.(*TPacket).ZeroCopyReadPacketData(0xc0000cc300, 0xc000a0a6c0, 0x0, 0x4, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/elastic/beats/vendor/github.com/google/gopacket/afpacket/afpacket.go:302 +0x136
github.com/elastic/beats/v7/x-pack/auditbeat/module/system/socket/dns/afpacket.(*dnsCapture).run(0xc00052a620, 0x392c320, 0xc00092a8c0, 0xc00098bfc0)
	/go/src/github.com/elastic/beats/x-pack/auditbeat/module/system/socket/dns/afpacket/afpacket.go:158 +0x1e0
created by github.com/elastic/beats/v7/x-pack/auditbeat/module/system/socket/dns/afpacket.(*dnsCapture).Monitor
	/go/src/github.com/elastic/beats/x-pack/auditbeat/module/system/socket/dns/afpacket/afpacket.go:103 +0x5d

goroutine 674 [runnable]:
syscall.Syscall6(0x10f, 0xc000f82ba0, 0x2, 0xc0009aad60, 0x0, 0x0, 0x0, 0x1, 0xc0009aad60, 0x0)
	/usr/local/go/src/syscall/asm_linux_amd64.s:44 +0x5
golang.org/x/sys/unix.ppoll(0xc000f82ba0, 0x2, 0xc0009aad60, 0x0, 0x31ed360, 0x3125d01, 0xc000f82ba0)
	/go/src/github.com/elastic/beats/vendor/golang.org/x/sys/unix/zsyscall_linux_amd64.go:100 +0x88
golang.org/x/sys/unix.Ppoll(0xc000f82ba0, 0x2, 0x2, 0xc0009aad60, 0x0, 0x6, 0xc0009aadc8, 0x135cd89)
	/go/src/github.com/elastic/beats/vendor/golang.org/x/sys/unix/syscall_linux.go:142 +0x9d
github.com/elastic/beats/v7/x-pack/auditbeat/tracing.pollAll(0xc0004c4600, 0x2, 0x40, 0xbebc200, 0x0, 0x2e2bf01, 0xc0009aae00, 0x135c8c5)
	/go/src/github.com/elastic/beats/x-pack/auditbeat/tracing/perfevent.go:494 +0x216
github.com/elastic/beats/v7/x-pack/auditbeat/tracing.(*recordMerger).nextSample(0xc0009aaf48, 0x392da60, 0xc000a0a300, 0x34, 0x142)
	/go/src/github.com/elastic/beats/x-pack/auditbeat/tracing/perfevent.go:433 +0x1af
github.com/elastic/beats/v7/x-pack/auditbeat/tracing.(*PerfChannel).channelLoop(0xc0002642c0)
	/go/src/github.com/elastic/beats/x-pack/auditbeat/tracing/perfevent.go:359 +0x1a0
created by github.com/elastic/beats/v7/x-pack/auditbeat/tracing.(*PerfChannel).Run
	/go/src/github.com/elastic/beats/x-pack/auditbeat/tracing/perfevent.go:285 +0x1b0

goroutine 675 [select]:
github.com/elastic/beats/v7/x-pack/auditbeat/module/system/socket.(*MetricSet).clockSyncLoop(0xc00052ec40, 0x2540be400, 0xc0000a8420)
	/go/src/github.com/elastic/beats/x-pack/auditbeat/module/system/socket/socket_linux.go:430 +0xff
created by github.com/elastic/beats/v7/x-pack/auditbeat/module/system/socket.(*MetricSet).Run
	/go/src/github.com/elastic/beats/x-pack/auditbeat/module/system/socket/socket_linux.go:150 +0x419

goroutine 662 [select]:
github.com/elastic/beats/v7/auditbeat/module/auditd.(*MetricSet).Run.func1(0xc000095c80, 0x3922420, 0xc00087a510, 0xc00052e700)
	/go/src/github.com/elastic/beats/auditbeat/module/auditd/audit_linux.go:170 +0x164
created by github.com/elastic/beats/v7/auditbeat/module/auditd.(*MetricSet).Run
	/go/src/github.com/elastic/beats/auditbeat/module/auditd/audit_linux.go:165 +0x386

goroutine 663 [select]:
github.com/elastic/beats/v7/auditbeat/module/auditd.(*MetricSet).Run.func2(0xc000a006f0, 0x3922420, 0xc00087a510, 0xc000820240, 0xc00052e700)
	/go/src/github.com/elastic/beats/auditbeat/module/auditd/audit_linux.go:200 +0x24b
created by github.com/elastic/beats/v7/auditbeat/module/auditd.(*MetricSet).Run
	/go/src/github.com/elastic/beats/auditbeat/module/auditd/audit_linux.go:197 +0x2bc

goroutine 664 [select]:
github.com/elastic/beats/v7/auditbeat/module/auditd.(*MetricSet).Run.func2(0xc000a006f0, 0x3922420, 0xc00087a510, 0xc000820240, 0xc00052e700)
	/go/src/github.com/elastic/beats/auditbeat/module/auditd/audit_linux.go:200 +0x24b
created by github.com/elastic/beats/v7/auditbeat/module/auditd.(*MetricSet).Run
	/go/src/github.com/elastic/beats/auditbeat/module/auditd/audit_linux.go:197 +0x2bc

goroutine 666 [select, 3126 minutes]:
github.com/elastic/beats/v7/auditbeat/module/file_integrity.(*reader).nextEvent(0xc000546540, 0xc0004a8f60, 0x0)
	/go/src/github.com/elastic/beats/auditbeat/module/file_integrity/eventreader_fsnotify.go:127 +0x167
github.com/elastic/beats/v7/auditbeat/module/file_integrity.(*reader).consumeEvents(0xc000546540, 0xc0004a8f60)
	/go/src/github.com/elastic/beats/auditbeat/module/file_integrity/eventreader_fsnotify.go:116 +0xf6
created by github.com/elastic/beats/v7/auditbeat/module/file_integrity.(*reader).Start
	/go/src/github.com/elastic/beats/auditbeat/module/file_integrity/eventreader_fsnotify.go:93 +0x2c1
@elasticmachine
Copy link
Collaborator

Pinging @elastic/siem (Team:SIEM)

adriansr added a commit to adriansr/beats that referenced this issue Oct 8, 2020
@adriansr
Fix concurrent map read and write in socket dataset
295aeb4 
This fixes a panic caused by a concurrent map read and write in Auditbeat's
system/socket dataset.

Fixes elastic#21192
@adriansr adriansr mentioned this issue Oct 8, 2020
Merged
4 tasks
adriansr added a commit that referenced this issue Oct 12, 2020
@adriansr
Fix concurrent map read and write in socket dataset (#21690)
9ab0a91 
This fixes a panic caused by a concurrent map read and write in Auditbeat's
system/socket dataset.

Fixes #21192
adriansr added a commit to adriansr/beats that referenced this issue Oct 12, 2020
@adriansr
Fix concurrent map read and write in socket dataset (elastic#21690)
9f77c9c 
This fixes a panic caused by a concurrent map read and write in Auditbeat's
system/socket dataset.

Fixes elastic#21192

(cherry picked from commit 9ab0a91)
@adriansr adriansr mentioned this issue Oct 12, 2020
Merged
4 tasks
adriansr added a commit to adriansr/beats that referenced this issue Oct 12, 2020
@adriansr
Fix concurrent map read and write in socket dataset (elastic#21690)
18fd7c3 
This fixes a panic caused by a concurrent map read and write in Auditbeat's
system/socket dataset.

Fixes elastic#21192

(cherry picked from commit 9ab0a91)
@adriansr adriansr mentioned this issue Oct 12, 2020
Merged
4 tasks
adriansr added a commit to adriansr/beats that referenced this issue Oct 12, 2020
@adriansr
Fix concurrent map read and write in socket dataset (elastic#21690)
f943157 
This fixes a panic caused by a concurrent map read and write in Auditbeat's
system/socket dataset.

Fixes elastic#21192

(cherry picked from commit 9ab0a91)
@adriansr adriansr mentioned this issue Oct 12, 2020
Merged
4 tasks
adriansr added a commit that referenced this issue Oct 13, 2020
@adriansr
Fix concurrent map read and write in socket dataset (#21690) (#21731)
922cc2d 
This fixes a panic caused by a concurrent map read and write in Auditbeat's
system/socket dataset.

Fixes #21192

(cherry picked from commit 9ab0a91)
adriansr added a commit that referenced this issue Oct 13, 2020
@adriansr
Fix concurrent map read and write in socket dataset (#21690) (#21730)
0c73fa2 
This fixes a panic caused by a concurrent map read and write in Auditbeat's
system/socket dataset.

Fixes #21192

(cherry picked from commit 9ab0a91)
adriansr added a commit that referenced this issue Oct 13, 2020
@adriansr
Fix concurrent map read and write in socket dataset (#21690) (#21729)
62b30d9 
This fixes a panic caused by a concurrent map read and write in Auditbeat's
system/socket dataset.

Fixes #21192

(cherry picked from commit 9ab0a91)
leweafan pushed a commit to leweafan/beats that referenced this issue Apr 28, 2023
@adriansr
Fix concurrent map read and write in socket dataset (elastic#21690) (e…
2f30e60 
…lastic#21731)

This fixes a panic caused by a concurrent map read and write in Auditbeat's
system/socket dataset.

Fixes elastic#21192

(cherry picked from commit 17e4204)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Auditbeat bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix concurrent map read and write in socket dataset adriansr/beats
2 participants
@adriansr @elasticmachine