After upgrade to 1.16.0 - [inputs.processes] Error in plugin: stat /proc/2690/stat: no such file or directory

[somera]

System info:

pi@pi-server:~ $ uname -a
Linux pi-server 5.4.51-v7+ #1333 SMP Mon Aug 10 16:45:19 BST 2020 armv7l GNU/Linux
pi@pi-server:~ $ telegraf -version
Telegraf 1.16.0 (git: HEAD c2a53d0f)

Actual behavior:

After upgrade to telegraf 1.16.0 I get [inputs.processes] Error in plugin. I'm using my config without any changes.

telegraf.conf.txt

Additional info:

2020-10-22T15:54:30Z E! [inputs.processes] Error in plugin: stat /proc/2442/stat: no such file or directory
2020-10-22T15:54:31Z D! [outputs.influxdb] Wrote batch of 190 metrics in 86.279731ms
2020-10-22T15:54:31Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2020-10-22T15:54:41Z D! [outputs.influxdb] Wrote batch of 23 metrics in 35.672207ms
2020-10-22T15:54:41Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2020-10-22T15:54:50Z E! [inputs.processes] Error in plugin: stat /proc/2487/stat: no such file or directory
2020-10-22T15:54:51Z D! [outputs.influxdb] Wrote batch of 190 metrics in 132.53046ms
2020-10-22T15:54:51Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2020-10-22T15:55:01Z D! [outputs.influxdb] Wrote batch of 359 metrics in 132.188832ms
2020-10-22T15:55:01Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2020-10-22T15:55:10Z E! [inputs.processes] Error in plugin: stat /proc/2554/stat: no such file or directory
2020-10-22T15:55:11Z D! [outputs.influxdb] Wrote batch of 194 metrics in 88.899655ms
2020-10-22T15:55:11Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2020-10-22T15:55:21Z D! [outputs.influxdb] Wrote batch of 191 metrics in 86.259331ms
2020-10-22T15:55:21Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2020-10-22T15:55:31Z D! [outputs.influxdb] Wrote batch of 191 metrics in 88.904331ms
2020-10-22T15:55:31Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2020-10-22T15:55:41Z D! [outputs.influxdb] Wrote batch of 191 metrics in 87.63566ms
2020-10-22T15:55:41Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2020-10-22T15:55:51Z D! [outputs.influxdb] Wrote batch of 191 metrics in 90.250399ms
2020-10-22T15:55:51Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2020-10-22T15:56:01Z D! [outputs.influxdb] Wrote batch of 191 metrics in 93.767371ms
2020-10-22T15:56:01Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2020-10-22T15:56:10Z E! [inputs.processes] Error in plugin: stat /proc/2690/stat: no such file or directory
2020-10-22T15:56:11Z D! [outputs.influxdb] Wrote batch of 194 metrics in 87.844818ms
2020-10-22T15:56:11Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2020-10-22T15:56:20Z E! [inputs.processes] Error in plugin: stat /proc/2711/stat: no such file or directory
2020-10-22T15:56:21Z D! [outputs.influxdb] Wrote batch of 190 metrics in 90.433826ms
2020-10-22T15:56:21Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics
2020-10-22T15:56:31Z D! [outputs.influxdb] Wrote batch of 191 metrics in 93.08023ms
2020-10-22T15:56:31Z D! [outputs.influxdb] Buffer fullness: 0 / 10000 metrics

[somera]

With

Telegraf 1.15.3 (git: HEAD fac81815)

and same config all works fine.

[PrplHaz4]

Reverting back to 1.15.4 fixed this for me too.

[maunzCache]

Can confirm this issues. Our +200 Server spontaneously fail the telegraf health check with that issue. The fail is not persistent so few hosts recover, few hosts fail that check. The message is, however, true. There is no file in proc with that id so i wonder why telegraf wants to check that.

[ssoroka]

this is fixed now, and will be in the next nightly build, and 1.16.1 when it releases.

[jbarotin]

Hi this issue still happened on Telegraf 1.17.2 (git: HEAD 74011e2) under Ubuntu 16.04

[maunzCache]

Occasionally hit us today on one host yesterday too but seems gone now.

[jbarotin]

It happened only on one server. I disabled it, but if needed I can't try some tests to get further information.

@ssoroka is it possible to reopen this ticket ?