Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docs] Adds list of type changes to the 7.0 breaking changes docs #17911

Closed
wants to merge 1 commit into from

Conversation

dedemorton
Copy link
Contributor

@dedemorton dedemorton commented Apr 22, 2020

Adds list of type changes requested in #11790.

#11790 has been open for a very long time. I was hoping the dev team would take ownership because the list of changed types is incomplete.

For the sake of getting what we know into the published docs, I'm opening this PR. I think, however, that this kind of documentation should be generated or at last maintained by the dev team when they make changes to field types or names.

@webmat Including you as a reviewer because you opened the original issue.

@urso Adding you to the review so we have a plan going forward to make sure this doesn't happen again.

@dedemorton dedemorton added docs needs_backport PR is waiting to be backported to other branches. v7.0.0 Team:Integrations Label for the Integrations team labels Apr 22, 2020
@dedemorton dedemorton requested review from webmat and urso April 22, 2020 19:10
@dedemorton dedemorton self-assigned this Apr 22, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations (Team:Integrations)

@webmat
Copy link
Contributor

webmat commented Apr 24, 2020

Thanks @dedemorton.

I haven't been following the changes in Beats closely after the lead up to 7.0, nor am I familiar with the field changes that happened before 7.0 that were unrelated to ECS.

But I'll be happy to review once this is ready.

@urso
Copy link

urso commented Apr 24, 2020

Unfortunately I'm not really familiar with all the changes made. I guess we will have to discuss how we can track schema changes better in the future

@dedemorton
Copy link
Contributor Author

dedemorton commented Apr 28, 2020

@webmat @urso Sounds like we have two separate problems. We need to:

  1. Identify schema changes that were not documented for 7.0.
  2. Improve our process so we track and document future schema changes.

I'm not sure where to go from here.

For 1, is there some way to reconstruct the schema changes that are not included in this PR? Is it worth the effort? Can we automate it?

For 2, I wonder if @andresrc can help.

@webmat
Copy link
Contributor

webmat commented Apr 28, 2020

Agreed, that's why I opened the initial issue.

Once again, all of the ECS-related changes have been tracked, as far as I know. It's a matter of looking at what else people were doing at that time, that weren't captured.

I opened that original issue because I can't do it myself. I've worked on a bunch of Filebeat modules for 3 months, once. But that's it. I don't know the code base very much.

@elasticmachine
Copy link
Collaborator

elasticmachine commented Apr 30, 2020

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Branch indexing]

  • Start Time: 2020-05-20T19:51:13.999+0000

  • Duration: 24 min 31 sec (1411457)

@elasticmachine
Copy link
Collaborator

elasticmachine commented May 26, 2020

💔 Build Failed

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Branch indexing]

  • Start Time: 2020-07-01T08:30:36.147+0000

  • Duration: 22 min 5 sec

Steps errors

Expand to view the steps failures

  • Name: Make check
    • Description: make check

    • Duration: 16 min 44 sec

    • Start Time: 2020-07-01T08:36:30.416+0000

    • log

Log output

Expand to view the last 100 lines of log output

[2020-07-01T08:52:18.500Z] Stage "Generators" skipped due to earlier failure(s)
[2020-07-01T08:52:18.501Z] Stage "Kubernetes" skipped due to earlier failure(s)
[2020-07-01T08:52:18.585Z] Stage "Heartbeat" skipped due to earlier failure(s)
[2020-07-01T08:52:18.587Z] Stage "Libbeat" skipped due to earlier failure(s)
[2020-07-01T08:52:18.588Z] Stage "Metricbeat x-pack" skipped due to earlier failure(s)
[2020-07-01T08:52:18.589Z] Stage "Packetbeat" skipped due to earlier failure(s)
[2020-07-01T08:52:18.590Z] Stage "dockerlogbeat" skipped due to earlier failure(s)
[2020-07-01T08:52:18.591Z] Stage "Winlogbeat" skipped due to earlier failure(s)
[2020-07-01T08:52:18.592Z] Stage "Functionbeat" skipped due to earlier failure(s)
[2020-07-01T08:52:18.592Z] Stage "Journalbeat" skipped due to earlier failure(s)
[2020-07-01T08:52:18.593Z] Stage "Generators" skipped due to earlier failure(s)
[2020-07-01T08:52:19.086Z] Failed in branch Elastic Agent x-pack
[2020-07-01T08:52:19.087Z] Failed in branch Elastic Agent x-pack Windows
[2020-07-01T08:52:19.088Z] Failed in branch Elastic Agent Mac OS X
[2020-07-01T08:52:19.088Z] Failed in branch Filebeat oss
[2020-07-01T08:52:19.089Z] Failed in branch Filebeat x-pack
[2020-07-01T08:52:19.090Z] Failed in branch Filebeat Mac OS X
[2020-07-01T08:52:19.090Z] Failed in branch Filebeat x-pack Mac OS X
[2020-07-01T08:52:19.091Z] Failed in branch Filebeat Windows
[2020-07-01T08:52:19.091Z] Failed in branch Filebeat x-pack Windows
[2020-07-01T08:52:19.092Z] Failed in branch Auditbeat oss Linux
[2020-07-01T08:52:19.092Z] Failed in branch Auditbeat crosscompile
[2020-07-01T08:52:19.093Z] Failed in branch Auditbeat oss Mac OS X
[2020-07-01T08:52:19.094Z] Failed in branch Auditbeat oss Windows
[2020-07-01T08:52:19.094Z] Failed in branch Auditbeat x-pack
[2020-07-01T08:52:19.095Z] Failed in branch Auditbeat x-pack Mac OS X
[2020-07-01T08:52:19.095Z] Failed in branch Auditbeat x-pack Windows
[2020-07-01T08:52:19.096Z] Failed in branch Libbeat x-pack
[2020-07-01T08:52:19.097Z] Failed in branch Metricbeat OSS Unit tests
[2020-07-01T08:52:19.097Z] Failed in branch Metricbeat OSS Integration tests
[2020-07-01T08:52:19.098Z] Failed in branch Metricbeat Python integration tests
[2020-07-01T08:52:19.098Z] Failed in branch Metricbeat crosscompile
[2020-07-01T08:52:19.099Z] Failed in branch Metricbeat Mac OS X
[2020-07-01T08:52:19.099Z] Failed in branch Metricbeat x-pack Mac OS X
[2020-07-01T08:52:19.100Z] Failed in branch Metricbeat Windows
[2020-07-01T08:52:19.100Z] Failed in branch Metricbeat x-pack Windows
[2020-07-01T08:52:19.101Z] Failed in branch Winlogbeat Windows x-pack
[2020-07-01T08:52:19.101Z] Failed in branch Kubernetes
[2020-07-01T08:52:19.358Z] Stage "Heartbeat" skipped due to earlier failure(s)
[2020-07-01T08:52:19.360Z] Stage "Libbeat" skipped due to earlier failure(s)
[2020-07-01T08:52:19.361Z] Stage "Metricbeat x-pack" skipped due to earlier failure(s)
[2020-07-01T08:52:19.362Z] Stage "Winlogbeat" skipped due to earlier failure(s)
[2020-07-01T08:52:19.363Z] Stage "Functionbeat" skipped due to earlier failure(s)
[2020-07-01T08:52:19.364Z] Stage "Generators" skipped due to earlier failure(s)
[2020-07-01T08:52:19.419Z] Failed in branch Packetbeat
[2020-07-01T08:52:19.419Z] Failed in branch dockerlogbeat
[2020-07-01T08:52:19.420Z] Failed in branch Journalbeat
[2020-07-01T08:52:19.601Z] Stage "Heartbeat" skipped due to earlier failure(s)
[2020-07-01T08:52:19.603Z] Stage "Libbeat" skipped due to earlier failure(s)
[2020-07-01T08:52:19.604Z] Stage "Functionbeat" skipped due to earlier failure(s)
[2020-07-01T08:52:19.605Z] Stage "Generators" skipped due to earlier failure(s)
[2020-07-01T08:52:19.640Z] Failed in branch Metricbeat x-pack
[2020-07-01T08:52:19.641Z] Failed in branch Winlogbeat
[2020-07-01T08:52:19.814Z] Failed in branch Heartbeat
[2020-07-01T08:52:19.815Z] Failed in branch Libbeat
[2020-07-01T08:52:19.816Z] Failed in branch Functionbeat
[2020-07-01T08:52:19.816Z] Stage "Generators" skipped due to earlier failure(s)
[2020-07-01T08:52:19.888Z] Failed in branch Generators
[2020-07-01T08:52:20.012Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-17911/src/github.com/elastic/beats
[2020-07-01T08:52:20.320Z] + find . -type f -name TEST*.xml -path */build/* -delete
[2020-07-01T08:52:20.332Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-17911/src/github.com/elastic/beats/Lint
[2020-07-01T08:52:20.762Z] + cat
[2020-07-01T08:52:20.762Z] + /usr/local/bin/runbld ./runbld-script
[2020-07-01T08:52:20.762Z] Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
[2020-07-01T08:52:27.359Z] runbld>>> runbld started
[2020-07-01T08:52:27.359Z] runbld>>> 1.6.12/f45d832f2ba0aa2722ab4ec1fda8ad140f027f8b
[2020-07-01T08:52:29.911Z] runbld>>> The following profiles matched the job 'Beats/beats-beats-mbp/PR-17911' in order of occurrence in the config (last value wins).
[2020-07-01T08:52:30.855Z] runbld>>> Debug logging enabled.
[2020-07-01T08:52:30.855Z] runbld>>> Storing result
[2020-07-01T08:52:31.116Z] runbld>>> Store result: created {:total 2, :successful 2, :failed 0} 1
[2020-07-01T08:52:31.117Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200701085230-A91113DF
[2020-07-01T08:52:31.117Z] runbld>>> Adding system facts.
[2020-07-01T08:52:32.064Z] runbld>>> Adding vcs info for the latest commit:  372aca8c8e789796c4fe4c75a3e6e255eb59f7c0
[2020-07-01T08:52:32.064Z] runbld>>> >>>>>>>>>>>> SCRIPT EXECUTION BEGIN >>>>>>>>>>>>
[2020-07-01T08:52:32.064Z] runbld>>> Adding /usr/lib/jvm/java-8-openjdk-amd64/bin to the path.
[2020-07-01T08:52:32.064Z] + echo 'Processing JUnit reports with runbld...'
[2020-07-01T08:52:32.064Z] Processing JUnit reports with runbld...
[2020-07-01T08:52:32.326Z] runbld>>> <<<<<<<<<<<< SCRIPT EXECUTION END <<<<<<<<<<<<
[2020-07-01T08:52:32.326Z] runbld>>> DURATION: 34ms
[2020-07-01T08:52:32.326Z] runbld>>> STDOUT: 40 bytes
[2020-07-01T08:52:32.326Z] runbld>>> STDERR: 49 bytes
[2020-07-01T08:52:32.326Z] runbld>>> WRAPPED PROCESS: SUCCESS (0)
[2020-07-01T08:52:32.326Z] runbld>>> Searching for build metadata in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-17911/src/github.com/elastic/beats
[2020-07-01T08:52:33.270Z] runbld>>> Storing build metadata: 
[2020-07-01T08:52:33.270Z] runbld>>> Adding test report.
[2020-07-01T08:52:33.270Z] runbld>>> Searching for junit test output files with the pattern: TEST-.*\.xml$ in: /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-17911/src/github.com/elastic/beats
[2020-07-01T08:52:34.213Z] runbld>>> Found 0 test output files
[2020-07-01T08:52:34.213Z] runbld>>> Test output logs contained: Errors: 0 Failures: 0 Tests: 0 Skipped: 0
[2020-07-01T08:52:34.213Z] runbld>>> Storing result
[2020-07-01T08:52:34.474Z] runbld>>> Store result: updated {:total 2, :successful 2, :failed 0} 2
[2020-07-01T08:52:34.474Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200701085230-A91113DF
[2020-07-01T08:52:34.474Z] runbld>>> Email notification disabled by environment variable.
[2020-07-01T08:52:40.075Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-17911
[2020-07-01T08:52:40.189Z] [INFO] getVaultSecret: Getting secrets
[2020-07-01T08:52:40.256Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-07-01T08:52:41.017Z] + chmod 755 generate-build-data.sh
[2020-07-01T08:52:41.017Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-17911/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-17911/runs/12 FAILURE 1324609
[2020-07-01T08:52:41.017Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-17911/runs/12/steps/?limit=10000 -o steps-info.json
[2020-07-01T08:52:41.268Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-17911/runs/12/tests/?status=FAILED -o tests-errors.json
[2020-07-01T08:52:41.518Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-17911/runs/12/log/ -o pipeline-log.txt

@dedemorton
Copy link
Contributor Author

This PR is stalled. Should I close it?

@andresrc
Copy link
Contributor

I'm afraid so, sorry for this, let's try to get it better for 8.0

@dedemorton dedemorton closed this Jul 1, 2020
@dedemorton dedemorton removed the needs_backport PR is waiting to be backported to other branches. label Jul 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs review Team:Integrations Label for the Integrations team v7.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants