Skip to content

15.0.0
Compare
Choose a tag to compare
@ekneg54 ekneg54 released this 06 Jan 13:48
· 5 commits to main since this release
v15.0.0
0c011ab
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.

Breaking

  • drop support for python 3.10 and add support for python 3.13
  • CriticalInputError is raised when the input preprocessor values can't be set, this was so far only true
    for the hmac preprocessor, but is now also applied for all other preprocessors.
  • fix delimiter typo in StringSplitterRule configuration
  • removed the configuration tld_lists in domain_resolver, domain_label_extractor and pseudonymizer as
    the list is now fixed inside the packaged logprep
  • remove SQL feature from generic_adder, fields can only be added from rule config or from file
  • use a single rule tree instead of a generic and a specific rule tree
  • replace the extend_target_list parameter with merge_with_target for improved naming clarity
    and functionality across FieldManager based processors (e.g., FieldManager, Clusterer,
    GenericAdder).

Features

  • configuration of initContainers in logprep helm chart is now possible

Improvements

  • fix requester documentation
  • replace BaseException with Exception for custom errors
  • refactor generic_resolver to validate rules on startup instead of application of each rule
  • regex pattern lists for the generic_resolver are pre-compiled
  • regex matching from lists in the generic_resolver is cached
  • matching in the generic_resolver can be case-insensitive
  • rewrite the helper method add_field_to such that it always raises an FieldExistsWarning instead of return a bool.
  • add new helper method add_fields_to to directly add multiple fields to one event
  • refactored some processors to make use of the new helper methods
  • add pre-commit hooks to the repository, install new dev dependency and run pre-commit install in the root dir
  • the default securityContextfor the pod is now configurable
  • allow TimeParser to get the current time with a specified timezone instead of always using local time and setting the timezone to UTC
  • remove tldextract dependency
  • remove urlextract dependency
  • fix wrong documentation for timestamp_differ
  • add container signatures to images build in ci pipeline
  • add sbom to images build in ci pipeline
  • FieldManager supports merging dictionaries

Bugfix

  • fix confluent_kafka.store_offsets if last_valid_record is None, can happen if a rebalancing happens
    before the first message was pulled.
  • fix pseudonymizer cache metrics not updated
  • fix incorrect timezones for log arrival time and delta time in input preprocessing
  • fix _get_value in FilterExpression so that keys don't match on values
  • fix auto_rule_tester to work with LOGPREP_BYPASS_RULE_TREE enabled
  • fix opensearch_output not draining message_backlog on shutdown
  • silence FieldExists warning in metrics when LOGPREP_APPEND_MEASUREMENT_TO_EVENT is active

Details

  • add dynamic scaling of time representations by @Malutthias in #690
  • Delete real kafka tests by @djkhl in #692
  • fix store_offsets call when last_valid_record is None by @dtrai2 in #693
  • standardize exception inheritance to Exception by @dtrai2 in #695
  • fix pseudonymizer cache metrics by @ekneg54 in #703
  • remove requirements.* files by @ekneg54 in #699
  • refactor validation of generic_resolver rules to startup by @dtrai2 in #694
  • speed up kafka output tests by @ekneg54 in #704
  • speed up s3 output tests by @ekneg54 in #706
  • update add_field_to function for improved error handling by @dtrai2 in #696
  • speed up exporter tests by @ekneg54 in #705
  • Dev unit tests speed up by @fabian-moessner in #698
  • fix requester documentation by @ekneg54 in #702
  • add pre-commit hooks and update documentation by @dtrai2 in #707
  • Delete logprep/util/pre_detector_rule_matching_tester.py by @Malutthias in #708
  • make pod security contex configurable by @djkhl in #711
  • add ability to configure init containers in helm chart by @djkhl in #712
  • Add missing SSL config to kafka admin clients by @ppcad in #710
  • Pre-compile patterns, add cache and support ignoring case in generic resolver by @ppcad in #716
  • Fix filter matching with bypass tree by @ppcad in #719
  • Improve test_http_input execution speed by @dtrai2 in #720
  • remove-tldextract-and-urlextract-dependencies by @ekneg54 in #718
  • Fix log arrival timezone by @ppcad in #715
  • Fix timestamp differ rule doc by @dtrai2 in #723
  • add shut_down method to opensearch output connector by @ekneg54 in #724
  • remove sql from generic_adder by @dtrai2 in #722
  • Fix FieldsExist warning using LOGPREP_APPEND_MEASUREMENT_TO_EVENT by @dtrai2 in #725
  • add signing to action by @dtrai2 in #729
  • add SBOM to images by @dtrai2 in #730
  • replace container build steps with reusable workflow by @dtrai2 in #732
  • fix build-version in dev release by @dtrai2 in #733
  • reduce to one rule tree by @ppcad in #731
  • support dict merging in fieldmanager and add_fields_to method by @dtrai2 in #737
  • Add rule tree configuration validation by @dtrai2 in #736
  • add support for python 3.13 and drop support for python 3.10 by @ekneg54 in #666
  • prepare release 15 by @ekneg54 in #739

New Contributors

Full Changelog: v14.0.0...v15.0.0

Contributors

Malutthias, ppcad, and 4 other contributors
Assets 5
Loading