Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lowercase mutate filter followed by gsub not working. #152

Open
adison88 opened this issue Mar 29, 2021 · 2 comments
Open

Lowercase mutate filter followed by gsub not working. #152

adison88 opened this issue Mar 29, 2021 · 2 comments

Comments

@adison88
Copy link

adison88 commented Mar 29, 2021

Please post all product and debugging questions on our forum. Your questions will reach our wider community members there, and if we confirm that there is a bug, then we can open a new issue here.

For all general issues, please provide the following details for fast resolution:

  • Version:
Logstash 6.8.5
Elasticsearch & Kibana 6.3
Filebeat 6.3
Winlogbeat 6.0
  • Operating System:
  • Config File (if you have sensitive info, please remove it):

`
mutate {
add_field => { "Static_Url" => "%{Url}" }
}

    mutate {
        gsub => [
            "Static_Url", "\\/[({]?[a-fA-F0-9]{8}[-]?([a-fA-F0-9]{4}[-]?){3}[a-fA-F0-9]{12}[})]?+", "/{uuid}"
        ]
    }

    mutate {
        gsub => [
            "Static_Url", "\\/[0-9]+( \\/|\\/|$)", "/{id}/"
        ]
    }

    mutate {
        lowercase => [ "Static_Url" ]
    }

`

  • Sample Data:

`
{
"_type": "doc",
"_version": 1,
"_score": null,
"_source": {
"log_timestamp": "2021-03-29T02:31:28.000Z",
"type": "lumberjack-json_line",
"Url": "/DeviceServices/Dm.svc/token/jW6Q8/333CB41462686848A9BA73ECA4794B05",
"path": "e:\AirWatch\Logs\IIS\W3SVC1\u_ex210329.log",
"LogSize": 1432,
"sc_bytes": "1097",
"@Version": "1",
"beat": {
"name": "",
"version": "6.8.5",
"hostname": ""
},
"Static_Url": "/DeviceServices/Dm.svc/token/jW6Q8/{uuid}",
"tags": [
"filebeat",
"ws1",
"iis-log",
"us04_ls",
"beats_input_codec_plain_applied",
"lumberjack-json_line-5006"
],
"delay": {
"downstream_logstash_ack": -12.848,
"filebeat_read": "2021-03-29T02:31:31.587Z"
},
"@timestamp": "2021-03-29T02:31:31.587Z",
"WinStatus": "0",
"_logzio_pattern": 3516387,
"Username": "-",
"SubStatusCode": "0",
"fields": {
"filebeat_config_version": "20.12",
"env_name": "CN1258",
"ws1_release": "20.11.0.7"
}
},
"fields": {
"log_timestamp": [
"2021-03-29T02:31:28.000Z"
],
"@timestamp": [
"2021-03-29T02:31:31.587Z"
]
},
"highlight": {
"tags": [
"@kibana-highlighted-field@iis@/kibana-highlighted-field@-@kibana-highlighted-field@log@/kibana-highlighted-field@"
]
},
"sort": [
1616985091587
]
}

`

  • Steps to Reproduce:

I've 2 gsub filters followed by lowercase filter on newly added field. gsub filters work properly however lowercase filter does not work.

@kares
Copy link
Contributor

kares commented Mar 29, 2021

Thanks for the report, LS 6.3 is not longer supported.
Could you try with latest LS 7.x if the issue still persist?

@adison88
Copy link
Author

@kares corrected the LS version is 6.8.5.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants