Auditbeat: Add exclude_files option to file integrity module #5888
Conversation
adriansr
added
Auditbeat
review
in progress
|
Adding a unit test |
metricbeat/mb/testing/modules.go
Outdated
| return true | ||
| } | ||
|
|
||
| func (r *capturingReporterV2) Done() <-chan struct{} { | ||
| return r.done | ||
| func (r *CapturingReporterV2) Done() <-chan struct{} { |
There was a problem hiding this comment.
exported method CapturingReporterV2.Done should have comment or be unexported
metricbeat/mb/testing/modules.go
Outdated
| } | ||
|
|
||
| func (r *capturingReporterV2) Event(event mb.Event) bool { | ||
| r.events = append(r.events, event) | ||
| func (r *CapturingReporterV2) Event(event mb.Event) bool { |
There was a problem hiding this comment.
exported method CapturingReporterV2.Event should have comment or be unexported
metricbeat/mb/testing/modules.go
Outdated
| type capturingReporterV2 struct { | ||
| events []mb.Event | ||
| done chan struct{} | ||
| type CapturingReporterV2 struct { |
There was a problem hiding this comment.
exported type CapturingReporterV2 should have comment or be unexported
adriansr
force-pushed
the
auditbeat/exclude
branch
from
a70c208 to
e8e8e86
Compare
| ScanRateBytesPerSec uint64 `config:",ignore"` | ||
| Recursive bool `config:"recursive"` // Recursive enables recursive monitoring of directories. | ||
| Exclude []string `config:"exclude_files"` | ||
| ExcludeMatchers []match.Matcher `config:",ignore"` |
There was a problem hiding this comment.
match.Matcher implements ucfg.StringUpacker so it will automatically be populated with the compiled regexes. If any of them fail to compile ucfg should emit an error. So I don't think you need the []string version at all.
| @@ -167,6 +167,13 @@ func (ms *MetricSet) init(reporter mb.PushReporterV2) bool { | |||
| } | |||
|
|
|||
| func (ms *MetricSet) reportEvent(reporter mb.PushReporterV2, event *Event) bool { | |||
| // ignore excluded paths | |||
| for _, matcher := range ms.config.ExcludeMatchers { | |||
There was a problem hiding this comment.
While it is most convenient to apply the filtering here, this wouldn't be much different that using a drop_event processor with a regexp condition. In my mind the benefit of having a dedicated exclude_files option is that the filter can be applied earlier in order to save the CPU and I/O cycles associated with reading the file and hashing it. Would it be feasible to apply this filter earlier?
It would need to be applied to the scanner and fsnotify/fsevents readers. I think the events that are triggered by purgeDeleted would need filtered as well.
| file := filepath.Join(dir, f) | ||
| ioutil.WriteFile(file, []byte("hello world"), 0600) | ||
| } | ||
| for i := 0; i < 100 && len(capture.Events) < 3; i++ { |
There was a problem hiding this comment.
Waiting for a certain number of events is a pretty common task in tests. We should make this easier to do with the mb/testing package.
metricbeat/mb/testing/modules.go
Outdated
| return r.done | ||
| // Done returns the Done channel for this reporter. | ||
| func (r *CapturingReporterV2) Done() <-chan struct{} { | ||
| return r.DoneC | ||
| } | ||
|
|
||
| // RunPushMetricSetV2 run the given push metricset for the specific amount of | ||
| // time and returns all of the events and errors that occur during that period. | ||
| func RunPushMetricSetV2(duration time.Duration, metricSet mb.PushMetricSetV2) []mb.Event { |
There was a problem hiding this comment.
I think it would make sense to add another parameter (maxEvents) to RunPushMetricSetV2 for specifying the maximum number of events to wait for. So then the behavior would be run until you reach duration (probably should rename to timeout) or until the maximum number of events is received (if maxEvents > 0).
| @@ -64,6 +68,12 @@ Linux. | |||
| *`paths`*:: A list of paths (directories or files) to watch. Globs are | |||
| not supported. The specified paths should exist when the metricset is started. | |||
|
|
|||
| *`exclude_files`:: A list of regular expressions used to filter out events | |||
There was a problem hiding this comment.
Based on questions I've seen for Filebeat, I think it would be good to explicitly state that this regular expression is matched against the file's absolute path.
| @@ -107,6 +109,15 @@ func deduplicate(in []string) []string { | |||
| return out | |||
| } | |||
|
|
|||
| func (c *Config) IsExcludedPath(path string) bool { | |||
There was a problem hiding this comment.
exported method Config.IsExcludedPath should have comment or be unexported
adriansr
force-pushed
the
auditbeat/exclude
branch
3 times, most recently
from
dbac025 to
a59deaa
Compare
Added a new config field, `exclude_files` consisting of regular expressions used to exclude files from monitoring by the file integrity module.
adriansr
force-pushed
the
auditbeat/exclude
branch
from
a59deaa to
c01430e
Compare
Added a new config field,
exclude_filesconsisting of regular expressions used to exclude files from monitoring by the file integrity module.See #5342