If the log files is twice soft links, the filebeat will faild in stat the file.

[chaoyujiang]

Filebeat error log:
2020-08-10T09:19:39.406Z DEBUG [input] log/input.go:297 stat(/var/log/containers/ratings-v1-65cff55fb8-66gsv_default_istio-init-dd7bff3f2327151597c4922a5ae69dcaa280a9fdca5737635fefac7fffb7a681.log) failed: stat /var/log/containers/ratings-v1-65cff55fb8-66gsv_default_istio-init-dd7bff3f2327151597c4922a5ae69dcaa280a9fdca573763

ll /var/log/containers/filebeat-8ql87_kube-system_filebeat-bd7d5c28e69440b700a81cc2963f2e54cf83b7a0718673387b388d89e5807af8.log
lrwxrwxrwx 1 root root 92 Aug 10 04:17 /var/log/containers/filebeat-8ql87_kube-system_filebeat-bd7d5c28e69440b700a81cc2963f2e54cf83b7a0718673387b388d89e5807af8.log -> /var/log/pods/kube-system_filebeat-8ql87_de1ca431-7aba-4169-943d-c68d5c2a6f67/filebeat/0.log
[root@k8s-worker-1 ~]# ll /var/log/pods/kube-system_filebeat-8ql87_de1ca431-7aba-4169-943d-c68d5c2a6f67/filebeat/0.log
lrwxrwxrwx 1 root root 164 Aug 10 04:17 /var/log/pods/kube-system_filebeat-8ql87_de1ca431-7aba-4169-943d-c68d5c2a6f67/filebeat/0.log -> /volume/docker/containers/bd7d5c28e69440b700a81cc2963f2e54cf83b7a0718673387b388d89e5807af8/bd7d5c28e69440b700a81cc2963f2e54cf83b7a0718673387b388d89e5807af8-json.log
[root@k8s-worker-1 ~]# ll /volume/docker/containers/bd7d5c28e69440b700a81cc2963f2e54cf83b7a0718673387b388d89e5807af8/bd7d5c28e69440b700a81cc2963f2e54cf83b7a0718673387b388d89e5807af8-json.log
-rw-r----- 1 root root 2728463 Aug 10 04:39 /volume/docker/containers/bd7d5c28e69440b700a81cc2963f2e54cf83b7a0718673387b388d89e5807af8/bd7d5c28e69440b700a81cc2963f2e54cf83b7a0718673387b388d89e5807af8-json.log
{
"labels": ["filebeat", "enhancement"]
}

[elasticmachine]

Pinging @elastic/integrations (Team:Integrations)

[chaoyujiang]

I make a mistake, the dst files of soft link is not mount in the filebeat. So this is not a issus, I will close it.

[chaoyujiang]

I had mount the dest path, the issue is still hit. Pls pay a attention!

[chaoyujiang]

Thank you very much for creating this issue. However, we would kindly like to ask you to post all questions and issues on the Discuss forum first. In addition to awesome, knowledgeable community contributors, core Beats developers are on the forums every single day to help you out as well. So, your questions will reach a wider audience there, and if we confirm that there is a bug, then you can reopen this issue with the new information or open a new one.

It can be produced easily. Don't need discuss more.

[andresrc]

Can you provide the specific steps to reproduce? Thanks

[elasticmachine]

Pinging @elastic/integrations-services (Team:Services)

[chaoyujiang]

Can you provide the specific steps to reproduce? Thanks
The k8s version:

[root@k8s-master chao]# kubectl version
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.6", GitCommit:"dff82dc0de47299ab66c83c626e08b245ab19037", GitTreeState:"clean", BuildDate:"2020-07-15T16:58:53Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.6", GitCommit:"dff82dc0de47299ab66c83c626e08b245ab19037", GitTreeState:"clean", BuildDate:"2020-07-15T16:51:04Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
[root@k8s-master chao]#
[root@k8s-master chao]# uname -a
Linux k8s-master 3.10.0-957.el7.x86_64 #1 SMP Thu Nov 8 23:39:32 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
[root@k8s-master chao]#
filebeat.inputs:
paths: /var/log/containers/a.log
But the a.log is a symbolic link( a.log---> b.log ), and the b.log is also a symbolic link(b.log ---> c.log). The c.log file is the real file.
The fileBeat will produce a error message in the debug log.

[andresrc]

Thanks @chaoyujiang and my apologies for the late reply.

I'm afraid we recommend not using soft links for the log files, as they cause many issues.

[jlind23]

Closing it for now until further activity.