Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Logs UI] Fix pre-ECS filebeat module message reconstruction rules #30398

Merged

Conversation

weltenwort
Copy link
Member

@weltenwort weltenwort commented Feb 7, 2019

Summary

This fixes the message reconstruction for the pre-ECS formats of several filebeat modules by adding appropriate rules:

Notes on testing

The screenshot below have been created by ingesting the test data used by filebeat 6.7 itself.

Icinga

image

HAProxy

image

MongoDB

image

IIS

image

Logstash

image

Osquery

image

Checklist

For maintainers

@weltenwort weltenwort added WIP Work in progress v7.0.0 [zube]: In Progress Feature:Logs UI Logs UI feature v6.7.0 Team:Infra Monitoring UI - DEPRECATED DEPRECATED - Label for the Infra Monitoring UI team. Use Team:obs-ux-infra_services labels Feb 7, 2019
@weltenwort weltenwort self-assigned this Feb 7, 2019
@elasticmachine
Copy link
Contributor

Pinging @elastic/infrastructure-ui

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

@elasticmachine
Copy link
Contributor

💔 Build Failed

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

@weltenwort weltenwort force-pushed the infra-ui-fix-pre-ecs-filebeat-messages branch 2 times, most recently from d74f3f1 to 4359908 Compare February 11, 2019 12:16
@elasticmachine
Copy link
Contributor

💚 Build Succeeded

@weltenwort weltenwort force-pushed the infra-ui-fix-pre-ecs-filebeat-messages branch from 4359908 to 2a93a2b Compare February 11, 2019 19:00
@elasticmachine
Copy link
Contributor

💔 Build Failed

@weltenwort weltenwort force-pushed the infra-ui-fix-pre-ecs-filebeat-messages branch from 2a93a2b to a0e1bb3 Compare February 12, 2019 09:36
@elasticmachine
Copy link
Contributor

💚 Build Succeeded

@simianhacker simianhacker self-assigned this Feb 12, 2019
@@ -4,6 +4,8 @@
* you may not use this file except in compliance with the Elastic License.
*/

import stringify from 'json-stable-stringify';
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably use this in the Flyout code too.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense 👍 ➡️ #30930

I hope that was what you meant?

@simianhacker
Copy link
Member

From a review standpoint I looked through everything and nothing stands out. What was your testing setup? Is a visual read through good enough?

@weltenwort
Copy link
Member Author

For manual testing I used the *.log test sample files from the 6.7 branch of the filebeat sources. The unit test contain some copy+pasted expected json from the filebeat tests.

Given that there are unit tests for the new rules and that I included screenshots above, I would say checking for typos in the constant parts of the messages and consistency in the field selection would be most valuable.

@weltenwort
Copy link
Member Author

jenkins, test this

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

@weltenwort
Copy link
Member Author

@simianhacker is there anything else you need in order to perform the review?

Copy link
Member

@simianhacker simianhacker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made another visual pass and everything looks good to me. I did load it in Kibana just to make sure nothing fatal happens.

 __         ______     ______   __    __    
/\ \       /\  ___\   /\__  _\ /\ "-./  \   
\ \ \____  \ \ \__ \  \/_/\ \/ \ \ \-./\ \  
 \ \_____\  \ \_____\    \ \_\  \ \_\ \ \_\ 
  \/_____/   \/_____/     \/_/   \/_/  \/_/ 
                                            

@weltenwort weltenwort merged commit fc8370f into elastic:master Feb 19, 2019
weltenwort added a commit to weltenwort/kibana that referenced this pull request Feb 19, 2019
…lastic#30398)

This fixes the message reconstruction for the pre-ECS formats of several filebeat modules by adding appropriate rules.
weltenwort added a commit to weltenwort/kibana that referenced this pull request Feb 19, 2019
…lastic#30398)

This fixes the message reconstruction for the pre-ECS formats of several filebeat modules by adding appropriate rules.
weltenwort added a commit that referenced this pull request Feb 19, 2019
…30398) (#31503)

This fixes the message reconstruction for the pre-ECS formats of several filebeat modules by adding appropriate rules.
weltenwort added a commit that referenced this pull request Feb 19, 2019
…30398) (#31502)

This fixes the message reconstruction for the pre-ECS formats of several filebeat modules by adding appropriate rules.
weltenwort added a commit to weltenwort/kibana that referenced this pull request Feb 20, 2019
…lastic#30398)

This fixes the message reconstruction for the pre-ECS formats of several filebeat modules by adding appropriate rules.
weltenwort added a commit that referenced this pull request Feb 20, 2019
…30398) (#31501)

This fixes the message reconstruction for the pre-ECS formats of several filebeat modules by adding appropriate rules.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment