Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Journald Reader for Logcollector #426

Open
vikman90 opened this issue Dec 13, 2024 · 1 comment · May be fixed by #495
Open

Implement Journald Reader for Logcollector #426

vikman90 opened this issue Dec 13, 2024 · 1 comment · May be fixed by #495
Assignees
Labels

Comments

@vikman90
Copy link
Member

Description

Develop a Journald Reader for Logcollector that behaves similarly to the journalctl command. The Reader will filter and forward logs based on configurable parameters, ensuring robust integration with the Journald API.

Requirements

Configuration Options

The Journald Reader will support the following configuration:

  • journald: Defines a Journald reader with the following options:

    • field: Selects a field from Journald entries.
    • regex: Regular expression the selected field must match for the log to be transmitted.
    • ignore_if_missing: Boolean; defines whether to ignore the log if the specified field does not exist.
  • file_wait: Defines the log polling interval (e.g., 500ms).

  • reload_interval: Interval for detecting log rotation (e.g., 1m).

Example Configuration:

logcollector:
    enabled: true
    reload_interval: 1m
    file_wait: 500ms
    
    journald:
    - field: _SYSTEMD_UNIT
      regex: ^cron.service$
    - field: PRIORITY
      regex: [0-3]
      ignore_if_missing: true

Tasks

Spike

  1. Review Configuration Options
    • Ensure that all necessary options are defined and consider additional useful features.
    • Confirm compatibility with Logcollector's configuration structure.
  2. Integrate Journald API
    • Utilize the client implementation from Wazuh 4.x.
    • Ensure synchronous interaction with the Journald API.
  3. Define Runner Behavior

Development

  1. Extend Logcollector's Setup Function
    • Parse and validate the Journald configuration options.
  2. Integrate Journald API Client
    • Add Journald API as a dependency to facilitate log retrieval and filtering.
  3. Implement Journald Reader
    • Develop the Reader to process logs based on configured filters.
    • Detect log rotation and handle state persistence for reliable log processing.
  4. Testing
    • Create unit and integration tests to validate the Reader's behavior under various configurations and scenarios.
  5. Documentation
    • Provide comprehensive documentation for the Journald Reader, including configuration examples and usage instructions.

Deliverables

  • Fully implemented Journald Reader in Logcollector.
  • Unit and integration test coverage for all features.
  • Documentation for the Journald Reader.
@MarcelKemp
Copy link
Member

MarcelKemp commented Dec 26, 2024

Work report

December 26

  • Research current 4.X code to know all the functionalities.
  • Done local testing based on sd-journal.h (systemd).
  • Trying to integrate local code with wazuh-agent.

December 27

  • Fixing compilation errors + Adapting current code.

December 30

  • Fixing functionality bugs

January 07

  • Fixing problems with the filter
  • Testing the code

January 08

  • Improving code using functions from the sd-journal.h library and fixing regex problem.

January 09

  • Added cursors, tried to use journald match and modified sync interval.
  • Replacing functionality and changing configuration to use sd_journal_add_match instead of regex.

January 10

  • Trying to fix compile and dependency issues for other OSes.

@MarcelKemp MarcelKemp linked a pull request Jan 9, 2025 that will close this issue
@MarcelKemp MarcelKemp linked a pull request Jan 9, 2025 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: In progress
Development

Successfully merging a pull request may close this issue.

2 participants