promtail: exclude files from static_configs __path__

[rgl]

Is your feature request related to a problem? Please describe.

my /var/log directory has files with different structures, which requires me to use different pipeline_stages for each set of different files.

Describe the solution you'd like

something like this would exclude all files that match the pattern /host/var/log/contained.*.log:

    static_configs:
      - labels:
          job: logwrite
          __path__: /host/var/log/*.log
          __path_exclude__: /host/var/log/contained.*.log

or maybe call it __exclude_path__?

or even modify __path__ to be an array where the array items that start with ! exclude paths?

if you agree, I can prepare a PR to modify the code (and docs) at:

loki/clients/pkg/promtail/targets/file/filetarget.go

Lines 162 to 171 in a9d85de

	matched, err := doublestar.Match(t.path, event.Name)
	if err != nil {
	level.Error(t.logger).Log("msg", "failed to match file", "error", err, "filename", event.Name)
	continue
	}
	if !matched {
	level.Debug(t.logger).Log("msg", "new file does not match glob", "filename", event.Name)
	continue
	}
	t.startTailing([]string{event.Name})

to do a second call to doublestar.Match before calling t.startTailing.

[stale]

This issue has been automatically marked as stale because it has not had any activity in the past 30 days. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[rgl]

Should this be implemented as described?

Can this issue be re open?

[michaelkebe]

Please reopen this feature request to allow excluding specific paths.

[kamaradski]

I would also like this to be re-opened for consideration.

In my case, Promtail is reading all files from /var/log/ but there is one file where the Promtail user does not have access to (on purpose) and this is spawning constant errors in my logs. path exclusion would be my preferred way to deal with this specific scenario.

[jadegouws]

I would also like this option, Please can it be reopened

[joemiller]

+1 for re-opening this

[Brettdah]

I would add that I'm using promtail on a synology NAS and it has a folder named synolog
for now I had somthing like :

- job_name: local
  static_configs:
  - targets:
      - localhost
    labels:
      job: varlogs
      __path__: /var/log/*log

And I found that I my want *.log or _log from that directory, cause _log are the result of the smart tests of my HDD
Or should I just create a new scrape_config for those... :/ that answer is making think xD