[Filebeat][New Module] Snyk vulnerabilities and audit logs (#22677)

* filebeat Snyk module MVP initial commit

* initial commit for Snyk filebeat module

* adding timestamp exceptions to module test

* fixing field definition and updating timestamp field for audit

* stashing changes

* updating Snyk module to support httpjson v2 input and fixing some broken field mappings and parsing

* adding CHANGELOG entry

* cleaning up test data

* updating testdata

* removing duplicate entry in manifest

* stashing changes

* updating httpjson delimiters and adding the possibility to set first_interval

CHANGELOG.next.asciidoc

@@ -746,6 +746,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
 - Improve Nats filebeat dashboard. {pull}22726[22726]
 - Add support for UNIX datagram sockets in `unix` input. {issues}18632[18632] {pull}22699[22699]
 - Add new httpjson input features and mark old config ones for deprecation {pull}22320[22320]
+- Add support for Snyk Vulnerability and Audit API. {pull}22677[22677]
 - Add logic for external network.direction in sophos xg fileset {pull}22973[22973]
 - Add `http.request.mime_type` for Elasticsearch audit log fileset. {pull}22975[22975]
 - Add configuration option to set external and internal networks for panw panos fileset {pull}22998[22998]

filebeat/docs/fields.asciidoc

@@ -78,6 +78,7 @@ grouped in the following categories:
 * <<exported-fields-s3>>
 * <<exported-fields-santa>>
 * <<exported-fields-snort>>
+* <<exported-fields-snyk>>
 * <<exported-fields-sonicwall>>
 * <<exported-fields-sophos>>
 * <<exported-fields-squid>>
@@ -126376,6 +126377,364 @@ type: keyword
 --
 This key captures values or decorators used within a registry entry
 
+type: keyword
+
+--
+
+[[exported-fields-snyk]]
+== Snyk fields
+
+Snyk module
+
+
+
+[float]
+=== snyk
+
+Module for parsing Snyk project vulnerabilities.
+
+
+
+*`snyk.projects`*::
++
+--
+Array with all related projects objects.
+
+
+type: flattened
+
+--
+
+*`snyk.related.projects`*::
++
+--
+Array of all the related project ID's.
+
+
+type: keyword
+
+--
+
+[float]
+=== audit
+
+Module for parsing Snyk audit logs.
+
+
+
+*`snyk.audit.org_id`*::
++
+--
+ID of the related Organization related to the event.
+
+
+type: keyword
+
+--
+
+*`snyk.audit.project_id`*::
++
+--
+ID of the project related to the event.
+
+
+type: keyword
+
+--
+
+*`snyk.audit.content`*::
++
+--
+Overview of the content that was changed, both old and new values.
+
+
+type: flattened
+
+--
+
+[float]
+=== vulnerabilities
+
+Module for parsing Snyk project vulnerabilities.
+
+
+
+*`snyk.vulnerabilities.cvss3`*::
++
+--
+CSSv3 scores.
+
+
+type: keyword
+
+--
+
+*`snyk.vulnerabilities.disclosure_time`*::
++
+--
+The time this vulnerability was originally disclosed to the package maintainers.
+
+
+type: date
+
+--
+
+*`snyk.vulnerabilities.exploit_maturity`*::
++
+--
+The Snyk exploit maturity level.
+
+
+type: keyword
+
+--
+
+*`snyk.vulnerabilities.id`*::
++
+--
+The vulnerability reference ID.
+
+
+type: keyword
+
+--
+
+*`snyk.vulnerabilities.is_ignored`*::
++
+--
+If the vulnerability report has been ignored.
+
+
+type: boolean
+
+--
+
+*`snyk.vulnerabilities.is_patchable`*::
++
+--
+If vulnerability is fixable by using a Snyk supplied patch.
+
+
+type: boolean
+
+--
+
+*`snyk.vulnerabilities.is_patched`*::
++
+--
+If the vulnerability has been patched.
+
+
+type: boolean
+
+--
+
+*`snyk.vulnerabilities.is_pinnable`*::
++
+--
+If the vulnerability is fixable by pinning a transitive dependency.
+
+
+type: boolean
+
+--
+
+*`snyk.vulnerabilities.is_upgradable`*::
++
+--
+If the vulnerability fixable by upgrading a dependency.
+
+
+type: boolean
+
+--
+
+*`snyk.vulnerabilities.language`*::
++
+--
+The package's programming language.
+
+
+type: keyword
+
+--
+
+*`snyk.vulnerabilities.package`*::
++
+--
+The package identifier according to its package manager.
+
+
+type: keyword
+
+--
+
+*`snyk.vulnerabilities.package_manager`*::
++
+--
+The package manager.
+
+
+type: keyword
+
+--
+
+*`snyk.vulnerabilities.patches`*::
++
+--
+Patches required to resolve the issue created by Snyk.
+
+
+type: flattened
+
+--
+
+*`snyk.vulnerabilities.priority_score`*::
++
+--
+The CVS priority score.
+
+
+type: long
+
+--
+
+*`snyk.vulnerabilities.publication_time`*::
++
+--
+The vulnerability publication time.
+
+
+type: date
+
+--
+
+*`snyk.vulnerabilities.jira_issue_url`*::
++
+--
+Link to the related Jira issue.
+
+
+type: keyword
+
+--
+
+*`snyk.vulnerabilities.original_severity`*::
++
+--
+The original severity of the vulnerability.
+
+
+type: long
+
+--
+
+*`snyk.vulnerabilities.reachability`*::
++
+--
+If the vulnerable function from the library is used in the code scanned. Can either be No Info, Potentially reachable and Reachable.
+
+
+type: keyword
+
+--
+
+*`snyk.vulnerabilities.title`*::
++
+--
+The issue title.
+
+
+type: keyword
+
+--
+
+*`snyk.vulnerabilities.type`*::
++
+--
+The issue type. Can be either "license" or "vulnerability".
+
+
+type: keyword
+
+--
+
+*`snyk.vulnerabilities.unique_severities_list`*::
++
+--
+A list of related unique severities.
+
+
+type: keyword
+
+--
+
+*`snyk.vulnerabilities.version`*::
++
+--
+The package version this issue is applicable to.
+
+
+type: keyword
+
+--
+
+*`snyk.vulnerabilities.introduced_date`*::
++
+--
+The date the vulnerability was initially found.
+
+
+type: date
+
+--
+
+*`snyk.vulnerabilities.is_fixed`*::
++
+--
+If the related vulnerability has been resolved.
+
+
+type: boolean
+
+--
+
+*`snyk.vulnerabilities.credit`*::
++
+--
+Reference to the person that original found the vulnerability.
+
+
+type: keyword
+
+--
+
+*`snyk.vulnerabilities.semver`*::
++
+--
+One or more semver ranges this issue is applicable to. The format varies according to package manager.
+
+
+type: flattened
+
+--
+
+*`snyk.vulnerabilities.identifiers.alternative`*::
++
+--
+Additional vulnerability identifiers.
+
+
+type: keyword
+
+--
+
+*`snyk.vulnerabilities.identifiers.cwe`*::
++
+--
+CWE vulnerability identifiers.
+
+
 type: keyword
 
 --