Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Autodiscover provider for Nomad #14954

Merged
merged 91 commits into from
Jan 7, 2021
Merged
Show file tree
Hide file tree
Changes from 36 commits
Commits
Show all changes
91 commits
Select commit Hold shift + click to select a range
86996ff
Metadata extractor from Nomad
jorgelbg Aug 1, 2019
2d4d434
Add hint generation from the nomad event's meta
jorgelbg Sep 4, 2019
164bf0e
Autodetect the hostname to filter Nomad events
jorgelbg Sep 4, 2019
a03f588
Use GCP region as datacenter
jorgelbg Sep 4, 2019
023765c
Remove duplicated task and job keys from the event. Use the proper fl…
jorgelbg Sep 4, 2019
fa7df7e
Add host from the allocation to the event.
jorgelbg Sep 5, 2019
83884c7
Add host from the allocation to the event.
jorgelbg Sep 5, 2019
6a5e46c
Merge branch 'autodiscovery-nomad' of github.com:jorgelbg/beats into …
jorgelbg Sep 5, 2019
281a384
Fire the proper events depending on the status & update time of the a…
jorgelbg Sep 6, 2019
60c65ef
Rename Flag on the test to a closer name to match the nomad nomenclature
jorgelbg Sep 6, 2019
145a92f
Add support for older nomad versions
jorgelbg Sep 6, 2019
37b509e
Use the internal watcher logger
jorgelbg Sep 9, 2019
372db05
WIP pass the client to the metadata generator
jorgelbg Sep 9, 2019
afccc97
Add cache for the NodeName data
jorgelbg Sep 9, 2019
a30057f
Patch the alloc.NodeName before emitting allocation events
jorgelbg Sep 10, 2019
97ea28f
Add filebeat allocation ID extractor from the log path
jorgelbg Sep 25, 2019
05f7e3f
Fix usage of the of the WaitIndex on the allocation watcher
jorgelbg Sep 25, 2019
af3dfdc
Add add_nomad_metadata processor to libbeat
jorgelbg Sep 25, 2019
bf59c3a
Add the nomad API package to the vendors
jorgelbg Oct 9, 2019
9ec5f00
Glue the add_nomad_metadata processor to the rest of filebeat
jorgelbg Oct 14, 2019
0b6e9d4
Log only the object IDs in the INFO level
jorgelbg Oct 14, 2019
f1b6283
Update NOTICE.txt
jorgelbg Oct 14, 2019
caae329
Fix issue with detecting updated allocations.
jorgelbg Nov 11, 2019
c8b7da7
Split ResourceMetadata and GroupMetadata into two different functions
jorgelbg Nov 14, 2019
a647a0a
Rename some kubernetes ocurrences from the nomad autodiscover beat
jorgelbg Nov 18, 2019
f398d51
Add AllowStale option to the Nomad configuration
jorgelbg Nov 19, 2019
148ba9f
Remove config.Prefix fields from the final event
jorgelbg Nov 19, 2019
4cca342
Add status field to the allocation metadata
jorgelbg Nov 21, 2019
f0c63e0
Allow stale responses from the Nomad client by default
jorgelbg Nov 21, 2019
1d7cc78
Fix the value of the status field
jorgelbg Nov 29, 2019
1da443b
Merge branch 'master' into autodiscovery-nomad
jorgelbg Dec 5, 2019
a03006c
Add nomad API dependency using govendor. Move import of the new packa…
jorgelbg Dec 5, 2019
60ee670
Add status field to the allocation metadata. Fix metadata tests
jorgelbg Dec 5, 2019
446e296
Formatting with mage
jorgelbg Dec 5, 2019
67ffef8
Vendor github.com/hashicorp/nomad/helper
jorgelbg Dec 9, 2019
42e3c17
Move the nomad autodiscover module and processors into x-pack
jorgelbg Dec 17, 2019
1bcee44
Update NOTICE.txt
jorgelbg Dec 17, 2019
141be1d
Remove unused configuration options from the Nomad provider
jorgelbg Jan 9, 2020
f8364c7
Add a first draft of the documentation for the Nomad autodiscover pro…
jorgelbg Jan 20, 2020
e1a8ea2
Add CleanupTimeout and check for goroutines leak on the main provider…
jorgelbg Jan 24, 2020
348fd81
Remove redundant temporary variable when processing the job metadata
jorgelbg Jan 24, 2020
de1b09e
Refactor the allocation watcher and add some tests
jorgelbg Jan 24, 2020
f96964e
Merge remote-tracking branch 'upstream/master' into autodiscovery-nomad
jorgelbg Jan 24, 2020
b48fb54
Improve handling of Nomad events
jorgelbg Jan 28, 2020
6523942
Handle in place allocations update on the Watcher
jorgelbg Feb 3, 2020
8140448
Use the local AllocClientStatus constant
jorgelbg Mar 3, 2020
834c0be
Merge remote-tracking branch 'upstream/master' into autodiscovery-nomad
jorgelbg Mar 3, 2020
4d62dfe
Use go modules for handling the dependencies
jorgelbg Mar 3, 2020
79f6444
Add some changes that were missing from the commit
jorgelbg Mar 3, 2020
5f3451f
Fix formatting of the Elastic License header
jorgelbg Mar 3, 2020
341ce22
Remove some internal Nomad dependencies. Adjusted tests
jorgelbg Mar 3, 2020
96d4ab6
Fix formatting issue
jorgelbg Mar 4, 2020
f098bc3
Merge remote-tracking branch 'upstream/master' into autodiscovery-nomad
jorgelbg Mar 4, 2020
6666499
Remove elastic/beats dependency
jorgelbg Mar 4, 2020
1135c86
Merge remote-tracking branch 'upstream/master' into autodiscovery-nomad
jorgelbg Mar 5, 2020
6acebf8
Fix some remarks from code review
jorgelbg May 8, 2020
66334ed
Merge remote-tracking branch 'upstream/master' into autodiscovery-nomad
jorgelbg Jun 15, 2020
f74f075
Update vendor dependencies
jorgelbg Jun 15, 2020
fd7547b
Fix a bug when an allocation was updated without being reallocated
jorgelbg Jul 20, 2020
ae84e97
Fix some issues from the code review
jorgelbg Jul 20, 2020
5199248
Merge remote-tracking branch 'upstream/master' into autodiscovery-nomad
jorgelbg Jul 21, 2020
91d86a6
Fix typo in license
jorgelbg Jul 21, 2020
c772a41
Remove vendor directory
jorgelbg Jul 21, 2020
8b8fe79
Fix typo on the autodiscover documentation
jorgelbg Jul 21, 2020
6702c69
Fix unreachable code statement
jorgelbg Jul 21, 2020
68b8886
Update NOTICE.txt (via mage)
jorgelbg Jul 21, 2020
118427b
Avoid skipping over stop events
jorgelbg Jul 27, 2020
fc671d4
Refactor the watcher tests into a table test
jorgelbg Jul 27, 2020
8e59cda
Make fields in hints-based autodiscover nomad specific
jorgelbg Nov 16, 2020
847cefe
Merge remote-tracking branch 'upstream/master' into autodiscovery-nomad
jorgelbg Nov 16, 2020
5e456ed
Fix signatuer of the AutodiscoverBuilder function
jorgelbg Nov 16, 2020
b971ec8
Update packetbeat configuration files (via mage update)
jorgelbg Nov 17, 2020
61a3a56
Use client status for the allocation status
jsoriano Dec 29, 2020
4016974
Add add_nomad_metadata to the list of included processors
jsoriano Dec 29, 2020
a53aeef
Small refactors on configuration
jsoriano Dec 29, 2020
758d72f
Add scope to nomad config
jsoriano Dec 29, 2020
1787d54
Refactor meta collection from tasks
jsoriano Dec 30, 2020
f5b23b2
Update docs
jsoriano Dec 30, 2020
8bf00a0
Fix cluster scope
jsoriano Dec 30, 2020
601ebd9
Merge remote-tracking branch 'origin/master' into autodiscover-nomad
jsoriano Dec 30, 2020
434b133
Fix tests
jsoriano Dec 30, 2020
5e40743
Fix docs
jsoriano Dec 30, 2020
4a141e3
Add changelog entry
jsoriano Dec 30, 2020
2115077
Use singular in datacenters field name
jsoriano Dec 30, 2020
6077aed
Reorganize allocation and job fields
jsoriano Dec 30, 2020
bf2c03d
Add fields definition
jsoriano Dec 30, 2020
a1dc3f1
Fix test in Windows
jsoriano Dec 30, 2020
e182038
Merge remote-tracking branch 'origin/master' into autodiscover-nomad
jsoriano Dec 30, 2020
221622b
Merge remote-tracking branch 'upstream/master' into autodiscovery-nomad
jorgelbg Jan 5, 2021
a18e6bf
Remove duplicated entry in changelog
jsoriano Jan 5, 2021
ad49e39
Merge remote-tracking branch 'upstream/master' into autodiscovery-nomad
jorgelbg Jan 6, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 31 additions & 31 deletions NOTICE.txt
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Elastic Beats
Copyright 2014-2019 Elasticsearch BV

This product includes software developed by The Apache Software
This product includes software developed by The Apache Software
Foundation (http://www.apache.org/).

==========================================================================
Expand Down Expand Up @@ -247,7 +247,7 @@ Apache License 2.0

-------NOTICE-----
GoFormation
Copyright 2011-2017 Amazon.com, Inc. or its affiliates. All Rights Reserved.
Copyright 2011-2017 Amazon.com, Inc. or its affiliates. All Rights Reserved.

--------------------------------------------------------------------
Dependency: github.com/Azure/azure-sdk-for-go
Expand Down Expand Up @@ -1074,14 +1074,14 @@ License type (autodetected): Apache-2.0
Apache License 2.0

-------NOTICE-----
Copyright (c) [2009-2011] VMware, Inc. All Rights Reserved.
Copyright (c) [2009-2011] VMware, Inc. All Rights Reserved.

This product is licensed to you under the Apache License, Version 2.0 (the "License").
You may not use this product except in compliance with the License.
This product is licensed to you under the Apache License, Version 2.0 (the "License").
You may not use this product except in compliance with the License.

This product includes a number of subcomponents with
separate copyright notices and license terms. Your use of these
subcomponents is subject to the terms and conditions of the
subcomponents is subject to the terms and conditions of the
subcomponent's license, as noted in the LICENSE file.
--------------------------------------------------------------------
Dependency: github.com/fatih/color
Expand Down Expand Up @@ -1358,7 +1358,7 @@ Mozilla Public License Version 2.0
means any form of the work other than Source Code Form.

1.7. "Larger Work"
means a work that combines Covered Software with other material, in
means a work that combines Covered Software with other material, in
a separate file or files, that is not Covered Software.

1.8. "License"
Expand Down Expand Up @@ -4317,9 +4317,9 @@ The following is courtesy of our legal counsel:


Use and transfer of Docker may be subject to certain restrictions by the
United States and other governments.
United States and other governments.
It is your responsibility to ensure that your use and/or transfer does not
violate applicable laws.
violate applicable laws.

For more information, please see http://www.bis.doc.gov

Expand Down Expand Up @@ -4993,23 +4993,23 @@ Copyright (c) 2012 - 2013 Mat Ryer and Tyler Bunnell

Please consider promoting this project if you find it useful.

Permission is hereby granted, free of charge, to any person
obtaining a copy of this software and associated documentation
files (the "Software"), to deal in the Software without restriction,
including without limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of the Software,
and to permit persons to whom the Software is furnished to do so,
Permission is hereby granted, free of charge, to any person
obtaining a copy of this software and associated documentation
files (the "Software"), to deal in the Software without restriction,
including without limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of the Software,
and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:

The above copyright notice and this permission notice shall be included
in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT
OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT
OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE
OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

--------------------------------------------------------------------
Expand Down Expand Up @@ -5987,13 +5987,13 @@ Copyright (c) 2010-2013 - Gustavo Niemeyer <[email protected]>
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
Expand All @@ -6019,13 +6019,13 @@ Copyright (c) 2010-2012 - Gustavo Niemeyer <[email protected]>
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
Expand Down Expand Up @@ -6142,13 +6142,13 @@ License type (autodetected): BSD-2-Clause
Copyright (c) 2013, Dustin L. Howett. All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
Expand All @@ -6162,7 +6162,7 @@ ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

The views and conclusions contained in the software and documentation are those
of the authors and should not be interpreted as representing official policies,
of the authors and should not be interpreted as representing official policies,
either expressed or implied, of the FreeBSD Project.

--------------------------------------------------------------------------------
Expand Down
5 changes: 3 additions & 2 deletions generator/common/beatgen/beatgen.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,12 @@ import (
"path/filepath"
"strings"

devtools "github.com/elastic/beats/dev-tools/mage"
"github.com/elastic/beats/generator/common/beatgen/setup"
"github.com/magefile/mage/mg"
"github.com/magefile/mage/sh"
"github.com/pkg/errors"

devtools "github.com/elastic/beats/dev-tools/mage"
"github.com/elastic/beats/generator/common/beatgen/setup"
)

// ConfigItem represents a value that must be configured for the custom beat
Expand Down
3 changes: 2 additions & 1 deletion generator/common/beatgen/setup/creator.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,9 @@ import (
"path/filepath"
"strings"

devtools "github.com/elastic/beats/dev-tools/mage"
"github.com/pkg/errors"

devtools "github.com/elastic/beats/dev-tools/mage"
)

// CfgPrefix specifies the env variable prefix used to configure the beat
Expand Down
3 changes: 2 additions & 1 deletion generator/common/beatgen/setup/setup.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,10 @@ import (
"os"
"path/filepath"

devtools "github.com/elastic/beats/dev-tools/mage"
"github.com/magefile/mage/sh"
"github.com/pkg/errors"

devtools "github.com/elastic/beats/dev-tools/mage"
)

// RunSetup runs any remaining setup commands after the vendor directory has been setup
Expand Down
3 changes: 2 additions & 1 deletion magefile.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,12 @@ import (
"os"
"path/filepath"

"github.com/elastic/beats/generator/common/beatgen"
"github.com/magefile/mage/mg"
"github.com/pkg/errors"
"go.uber.org/multierr"

"github.com/elastic/beats/generator/common/beatgen"

devtools "github.com/elastic/beats/dev-tools/mage"
"github.com/elastic/beats/dev-tools/mage/gotool"
)
Expand Down
2 changes: 2 additions & 0 deletions vendor/github.com/gofrs/uuid/go.mod

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 9 additions & 0 deletions vendor/github.com/gorilla/websocket/AUTHORS

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

22 changes: 22 additions & 0 deletions vendor/github.com/gorilla/websocket/LICENSE

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

64 changes: 64 additions & 0 deletions vendor/github.com/gorilla/websocket/README.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading