Cherry-pick #18748 to 7.x: Add initial support for configurable file identity tracking #19885
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Cherry-pick of PR #18748 to 7.x branch. Original message:
What does this PR do?
This PR adds a new option to the
log
input of Filebeat namedfile_identity
. The option lets users configure file identity for state tracking.Available strategies
native
(default): Filebeat identifies files based on their inode and device id.path
: Files are considered different if they have different paths.inode_marker
: A special marker file and the inode is used to tell apart files. It is not supported on Windows.State IDs previously were not saved to the registry file. Now these are persisted on the disk.
Architecture
I introduced a new interface:
file.StateIdentifier
. The responsibility ofStateIdentifier
is to generate an identifier for afile.State
based on the configuration. If someone wants to implement their ownStateIdentifier
method, all they need is to create a struct which satisfies this interface.As every state has an ID, Filebeat just compares the IDs of the two states to decide if they belong to the same file or not.
The scope of the PR does not include strategies which include fingerprinting the contents of the file.
Checklist
CHANGELOG.next.asciidoc
orCHANGELOG-developer.next.asciidoc
.