Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using scan_at_start with include_files does not work #27273

Closed
strawgate opened this issue Aug 7, 2021 · 1 comment · Fixed by #27722
Closed

Using scan_at_start with include_files does not work #27273

strawgate opened this issue Aug 7, 2021 · 1 comment · Fixed by #27722
Assignees
Labels
Auditbeat Team:Elastic-Agent Label for the Agent team

Comments

@strawgate
Copy link
Contributor

strawgate commented Aug 7, 2021

For confirmed bugs, please report:

  • Version: 7.13.1
  • Operating System: Windows
  • Discuss Forum URL:
  • Steps to Reproduce:

auditbeat.yml:

{
   "logging":{
      "level":"debug"
   },
   "auditbeat":{
      "modules":[
         {
            "module":"file_integrity",
            "paths":[
               "C:\\Users"
            ],
            "include_files":[
               "\\\\test\\.txt"
            ],
            "scan_at_start":true,
            "scan_rate_per_sec":"10 MiB",
            "max_file_size":"100 MiB",
            "hash_types":[
               "sha256"
            ],
            "recursive":true,
            "tags":"test"
         }
      ]
   }
}

This should scan test.txt (or do absolutely nothing if my match is bad) and produce a single event on initial scan. It should then produce events when the file is modified.

Instead it scans the entire users directory, outputting an event for every file, and then it monitors test.txt for changes:

Example event outputted that shouldnt have been:

2021-08-07T16:08:01.046-0500	DEBUG	[file_integrity]	file_integrity/metricset.go:291	File changed since it was last seen	{"file_path": "C:\\Users\\scandinavia\\AppData\\Local\\Temp\\yp4ggz2k.fy0\\resources\\app\\ServiceHub\\Services\\Microsoft.Developer.IdentityService\\zh-Hans", "took": 0, "event": {"action": "created", "old": null, "new": {"timestamp":"2021-08-07T21:08:01.0444163Z","path":"C:\\Users\\scandinavia\\AppData\\Local\\Temp\\yp4ggz2k.fy0\\resources\\app\\ServiceHub\\Services\\Microsoft.Developer.IdentityService\\zh-Hans","info":{"inode":281474977567954,"uid":0,"gid":0,"sid":"","owner":"","group":"","size":0,"mtime":"2021-07-20T08:08:30.9743538Z","ctime":"2021-07-07T03:48:23.7612897Z","type":"dir","mode":2147484159,"setuid":false,"setgid":false,"origin":null},"source":"scan","action":"created"}}}
2021-08-07T16:08:01.049-0500	DEBUG	[processors]	processing/processors.go:203	Publish event: {
  "@timestamp": "2021-08-07T21:08:01.044Z",
  "@metadata": {
    "beat": "auditbeat",
    "type": "_doc",
    "version": "7.13.1"
  },
  "host": {
    "name": "scandinavia"
  },
  "agent": {
    "id": "b195b53f-9e49-4d29-bd1e-7dc5ed6381b3",
    "name": "scandinavia",
    "type": "auditbeat",
    "version": "7.13.1",
    "hostname": "scandinavia",
    "ephemeral_id": "942ed043-31fd-472d-89ae-9506f4e0d541"
  },
  "service": {
    "type": "file_integrity"
  },
  "file": {
    "drive_letter": "C",
    "path": "C:\\Users\\scandinavia\\AppData\\Local\\Temp\\yp4ggz2k.fy0\\resources\\app\\ServiceHub\\Services\\Microsoft.Developer.IdentityService\\zh-Hans",
    "inode": "281474977567954",
    "mtime": "2021-07-20T08:08:30.974Z",
    "ctime": "2021-07-07T03:48:23.761Z",
    "type": "dir"
  },
  "event": {
    "module": "file_integrity",
    "dataset": "file",
    "kind": "event",
    "category": [
      "file"
    ],
    "type": [
      "creation"
    ],
    "action": [
      "created"
    ]
  },
  "tags": [
    "test"
  ],
  "ecs": {
    "version": "1.9.0"
  }
}
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Aug 7, 2021
@botelastic
Copy link

botelastic bot commented Aug 7, 2021

This issue doesn't have a Team:<team> label.

@jsoriano jsoriano added the Team:Elastic-Agent Label for the Agent team label Aug 20, 2021
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Aug 20, 2021
@leehinman leehinman self-assigned this Sep 2, 2021
leehinman added a commit to leehinman/beats that referenced this issue Sep 2, 2021
- If included_path is set and file is not an included path then it is
  skipped
- directories are included even if they don't match included path
  because the contents might

Closes elastic#27273
leehinman added a commit that referenced this issue Sep 9, 2021
* [Auditbeat] scanner honor include_files

- If included_path is set and file is not an included path then it is
  skipped
- directories are included even if they don't match included path
  because the contents might
- change test regex to work on linux, mac and windows

Closes #27273
mergify bot pushed a commit that referenced this issue Sep 9, 2021
* [Auditbeat] scanner honor include_files

- If included_path is set and file is not an included path then it is
  skipped
- directories are included even if they don't match included path
  because the contents might
- change test regex to work on linux, mac and windows

Closes #27273

(cherry picked from commit c0b3d73)
mergify bot pushed a commit that referenced this issue Sep 9, 2021
* [Auditbeat] scanner honor include_files

- If included_path is set and file is not an included path then it is
  skipped
- directories are included even if they don't match included path
  because the contents might
- change test regex to work on linux, mac and windows

Closes #27273

(cherry picked from commit c0b3d73)
leehinman added a commit that referenced this issue Sep 9, 2021
* [Auditbeat] scanner honor include_files

- If included_path is set and file is not an included path then it is
  skipped
- directories are included even if they don't match included path
  because the contents might
- change test regex to work on linux, mac and windows

Closes #27273

(cherry picked from commit c0b3d73)

Co-authored-by: Lee E Hinman <[email protected]>
leehinman added a commit that referenced this issue Sep 9, 2021
* [Auditbeat] scanner honor include_files

- If included_path is set and file is not an included path then it is
  skipped
- directories are included even if they don't match included path
  because the contents might
- change test regex to work on linux, mac and windows

Closes #27273

(cherry picked from commit c0b3d73)

Co-authored-by: Lee E Hinman <[email protected]>
Icedroid pushed a commit to Icedroid/beats that referenced this issue Nov 1, 2021
* [Auditbeat] scanner honor include_files

- If included_path is set and file is not an included path then it is
  skipped
- directories are included even if they don't match included path
  because the contents might
- change test regex to work on linux, mac and windows

Closes elastic#27273
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Auditbeat Team:Elastic-Agent Label for the Agent team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants