Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[receiver/filelog] "start_at: end" should be disregarded when loading offsets from storage #12769

Closed
djaglowski opened this issue Jul 28, 2022 · 0 comments · Fixed by #12773
Closed
Assignees
Labels
bug Something isn't working priority:p2 Medium receiver/filelog

Comments

@djaglowski
Copy link
Member

Describe the bug
When using a storage extension, the receiver should be able to seamlessly resume log file consumption after a restart. However, current behavior will skip lines that were added to a file while the collector was not running. In order to correct the behavior, the receiver should use ignore the start_at setting for files who's offsets are retrieved from a storage extension.

Steps to reproduce
Sample configuration:

receivers:
  filelog:
    include: [foo/*.log]

exporters:
  logging:
    loglevel: debug

extensions:
  file_storage:
    directory: foo/storage

service:
  extensions: [file_storage]
  pipelines:
    logs:
      receivers: [filelog]
      exporters: [logging]
  • Start the collector and add some lines to the log file. They should be consumed immediately. Offsets should be written to the storage extension in nearly real time.
  • Stop the collector.
  • Add some lines to the log file. These are the lines that are skipped due to this bug.
  • Start the collector. The new lines should be consumed immediately, but they are not.
  • Add more lines to the log file. These lines are correctly consumed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working priority:p2 Medium receiver/filelog
Projects
None yet
1 participant