Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cherry-pick #20084 to 7.9: Fix terminating pod autodiscover issue #20193

Merged
merged 2 commits into from
Jul 23, 2020
Merged

Cherry-pick #20084 to 7.9: Fix terminating pod autodiscover issue #20193

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6d82a41
Fix terminating pod autodiscover issue (#20084)
ChrsMark Jul 23, 2020
5c530a3
Update CHANGELOG.next.asciidoc
ChrsMark Jul 23, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions CHANGELOG.next.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,21 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- Fix goroutine leak and Elasticsearch output file descriptor leak when output reloading is in use. {issue}10491[10491] {pull}17381[17381]
- Fix Elasticsearch license endpoint URL referenced in error message. {issue}17880[17880] {pull}18030[18030]
- Change `decode_json_fields` processor, to merge parsed json objects with existing objects in the event instead of fully replacing them. {pull}17958[17958]
- [Autodiscover] Check if runner is already running before starting again. {pull}18564[18564]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extra changelogs here 😄

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯

- Fix `keystore add` hanging under Windows. {issue}18649[18649] {pull}18654[18654]
- Fix an issue where error messages are not accurate in mapstriface. {issue}18662[18662] {pull}18663[18663]
- Fix regression in `add_kubernetes_metadata`, so configured `indexers` and `matchers` are used if defaults are not disabled. {issue}18481[18481] {pull}18818[18818]
- Fix potential race condition in fingerprint processor. {pull}18738[18738]
- Add better handling for Kubernetes Update and Delete watcher events. {pull}18882[18882]
- Fix the `translate_sid` processor's handling of unconfigured target fields. {issue}18990[18990] {pull}18991[18991]
- Fixed a service restart failure under Windows. {issue}18914[18914] {pull}18916[18916]
- The `monitoring.elasticsearch.api_key` value is correctly base64-encoded before being sent to the monitoring Elasticsearch cluster. {issue}18939[18939] {pull}18945[18945]
- Fix kafka topic setting not allowing upper case characters. {pull}18854[18854] {issue}18640[18640]
- Fix redis key setting not allowing upper case characters. {pull}18854[18854] {issue}18640[18640]
- Fix config reload metrics (`libbeat.config.module.start/stops/running`). {pull}19168[19168]
- Fix metrics hints builder to avoid wrong container metadata usage when port is not exposed {pull}18979[18979]
- Server-side TLS config now validates certificate and key are both specified {pull}19584[19584]
- Fix terminating pod autodiscover issue. {pull}20084[20084]
- Fix seccomp policy for calls to `chmod` and `chown`. {pull}20054[20054]

*Auditbeat*
Expand Down
18 changes: 17 additions & 1 deletion libbeat/autodiscover/providers/kubernetes/pod.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,14 +138,30 @@ func (p *pod) OnUpdate(obj interface{}) {
switch pod.Status.Phase {
case kubernetes.PodSucceeded, kubernetes.PodFailed:
// If Pod is in a phase where all containers in the have terminated emit a stop event
p.logger.Debugf("Watcher Pod update (terminating): %+v", obj)
p.logger.Debugf("Watcher Pod update (terminated): %+v", obj)
time.AfterFunc(p.config.CleanupTimeout, func() { p.emit(pod, "stop") })
return
case kubernetes.PodPending:
p.logger.Debugf("Watcher Pod update (pending): don't know what to do with this Pod yet, skipping for now: %+v", obj)
return
}

// here handle the case when a Pod is in `Terminating` phase.
// In this case the pod is neither `PodSucceeded` nor `PodFailed` and
// hence requires special handling.
if pod.GetObjectMeta().GetDeletionTimestamp() != nil {
p.logger.Debugf("Watcher Pod update (terminating): %+v", obj)
// Pod is terminating, don't reload its configuration and ignore the event
// if some pod is still running, we will receive more events when containers
// terminate.
for _, container := range pod.Status.ContainerStatuses {
if container.State.Running != nil {
return
}
}
time.AfterFunc(p.config.CleanupTimeout, func() { p.emit(pod, "stop") })
}

p.logger.Debugf("Watcher Pod update: %+v", obj)
p.emit(pod, "stop")
p.emit(pod, "start")
Expand Down