Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New windows service metricset #5332

Merged
merged 60 commits into from
Nov 2, 2017
Merged
Show file tree
Hide file tree
Changes from 51 commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
1ef849c
Create new metricset
martinscholz83 Sep 20, 2017
d414d86
Add Close function for handler
martinscholz83 Sep 26, 2017
b9878d7
Add description
martinscholz83 Sep 27, 2017
d7d0aa1
make update
martinscholz83 Sep 27, 2017
a5bd839
Add service start types
martinscholz83 Oct 2, 2017
65ea2f7
Add service access rights
martinscholz83 Oct 2, 2017
a0c7f13
Add OpenService function
martinscholz83 Oct 2, 2017
d5be9ed
Add more funtions
martinscholz83 Oct 2, 2017
8fea616
Add errors
martinscholz83 Oct 2, 2017
6fd5d02
Query service config
martinscholz83 Oct 2, 2017
eb3ecd2
Correct return type for QueryServiceConfig
martinscholz83 Oct 5, 2017
73ef011
Remove services config
martinscholz83 Oct 5, 2017
3fe19b7
Add process access rights
martinscholz83 Oct 5, 2017
51f64db
Add process sycall functions
martinscholz83 Oct 5, 2017
ff57f2b
remove process functions
martinscholz83 Oct 5, 2017
df94026
Share run.go for all windows metricsets
martinscholz83 Oct 20, 2017
ae9de49
Cleanup code
martinscholz83 Oct 20, 2017
6ade0fc
Break into smaller functions
martinscholz83 Oct 20, 2017
49430ff
Refactor getting uptime
martinscholz83 Oct 20, 2017
e3b71ce
Calculate service uptime
martinscholz83 Oct 20, 2017
78f4a34
Add TestData integration test
martinscholz83 Oct 23, 2017
8ffd2c6
Remove unnecessary types
martinscholz83 Oct 24, 2017
7dff37d
Add SeDebugPrivilege helper function
martinscholz83 Oct 24, 2017
ed94168
Add fields
martinscholz83 Oct 24, 2017
239d774
Change uptime format
martinscholz83 Oct 24, 2017
8f0dcc6
Move uptime logic into helper
martinscholz83 Oct 24, 2017
89f6ce8
Change return type for service uptime
martinscholz83 Oct 24, 2017
076e376
Change uptime to time.Duration
martinscholz83 Oct 24, 2017
b0dde4e
Refactor getServiceStates
martinscholz83 Oct 24, 2017
999e2b5
Write data.json
martinscholz83 Oct 24, 2017
088389f
Refactor uptime calculating
martinscholz83 Oct 25, 2017
63df59e
Remove new lines
martinscholz83 Oct 25, 2017
75958f9
Remove commented out code
martinscholz83 Oct 25, 2017
57a47a3
Call privilege func once
martinscholz83 Oct 25, 2017
570d828
make update
martinscholz83 Oct 25, 2017
2f98979
make fmt
martinscholz83 Oct 25, 2017
8577fc4
Reorder imports
martinscholz83 Oct 25, 2017
8654ee8
Add miising return value for CheckAndEnableSeDebugPrivilege()
martinscholz83 Oct 25, 2017
fa8ef48
remove named parameter
martinscholz83 Oct 25, 2017
120c8f8
Format fields
martinscholz83 Oct 25, 2017
2a172bb
Remove uptime conversion in helper func
martinscholz83 Oct 25, 2017
e534732
Set oxford comma for lists
martinscholz83 Oct 25, 2017
e504e6f
Only add uptime to event if its not 0
martinscholz83 Oct 25, 2017
884212e
Fix adding uptime to event
martinscholz83 Oct 25, 2017
0489cf8
Update data.json
martinscholz83 Oct 25, 2017
63d735b
Update Makefile
martinscholz83 Oct 26, 2017
4ff7381
Fix formatting
martinscholz83 Oct 26, 2017
12b78b2
make fmt
martinscholz83 Oct 27, 2017
55751ae
make update
martinscholz83 Oct 27, 2017
4eb1120
Build only on windows
martinscholz83 Oct 27, 2017
99fbfd1
Add integration test
martinscholz83 Oct 27, 2017
563f487
Refactor getServiceStates
andrewkroh Oct 27, 2017
9d83f1e
Fix const Running
martinscholz83 Nov 2, 2017
aecd50b
Rename service metricset
martinscholz83 Nov 2, 2017
e9409e8
Rename json output
martinscholz83 Nov 2, 2017
17ac114
make update
martinscholz83 Nov 2, 2017
cb4cf55
go generate
martinscholz83 Nov 2, 2017
e743b68
go test -data
martinscholz83 Nov 2, 2017
5ca5ebe
Fix typo
martinscholz83 Nov 2, 2017
b98d69b
make update
martinscholz83 Nov 2, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion metricbeat/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ configs: python-env
@cat ${ES_BEATS}/metricbeat/_meta/common.reference.yml > _meta/beat.reference.yml
@${PYTHON_ENV}/bin/python ${ES_BEATS}/script/config_collector.py --beat ${BEAT_NAME} --full $(PWD) >> _meta/beat.reference.yml
@rm -rf modules.d && mkdir -p modules.d
@for MODULE in `find module -type d -maxdepth 1 -mindepth 1 -exec basename {} \;`; do cp -a $(PWD)/module/$$MODULE/_meta/config.yml modules.d/$$MODULE.yml.disabled; done
@for MODULE in `find module -maxdepth 1 -mindepth 1 -type d -exec basename {} \;`; do cp -a $(PWD)/module/$$MODULE/_meta/config.yml modules.d/$$MODULE.yml.disabled; done
@chmod go-w modules.d/*
@# Enable system by default:
@if [ -f modules.d/system.yml.disabled ]; then mv modules.d/system.yml.disabled modules.d/system.yml; fi
Expand Down
50 changes: 50 additions & 0 deletions metricbeat/docs/fields.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -11304,6 +11304,56 @@ beta[] Module for Windows




[float]
== services fields

`services` contains the status for windows services.



[float]
=== `windows.services.uptime.ms`

type: long

format: duration

`uptime` contains the service uptime metric.


[float]
=== `windows.services.service_name`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I read through these keys I am thinking we should call this the windows service metricset (not plural). Similar to how the system process metricset is not plural. Then the field names here make more sense.

Additionally, if it is renamed then I would change windows.services.service_name to windows.service.name to removing the stuttering.


type: keyword

The service name.


[float]
=== `windows.services.display_name`

type: keyword

The display name of the service.


[float]
=== `windows.services.start_type`

type: keyword

The start type of the service. The possible values are `ServiceAutoStart`, `ServiceBootStart`, `ServiceDemandStart`, `ServiceDisabled`, and `ServiceSystemStart`.


[float]
=== `windows.services.state`

type: keyword

The actual state of the service. The possible values are `ServiceContinuePending`, `ServicePausePending`, `ServicePaused`, `ServiceRuning`, `ServiceStartPending`, `ServiceStopPending`, and `ServiceStopped`.


[[exported-fields-zookeeper]]
== ZooKeeper fields

Expand Down
4 changes: 4 additions & 0 deletions metricbeat/docs/modules/windows.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -30,5 +30,9 @@ The following metricsets are available:

* <<metricbeat-metricset-windows-perfmon,perfmon>>

* <<metricbeat-metricset-windows-services,services>>

include::windows/perfmon.asciidoc[]

include::windows/services.asciidoc[]

19 changes: 19 additions & 0 deletions metricbeat/docs/modules/windows/services.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
////
This file is generated! See scripts/docs_collector.py
////

[[metricbeat-metricset-windows-services]]
include::../../../module/windows/services/_meta/docs.asciidoc[]


==== Fields

For a description of each field in the metricset, see the
<<exported-fields-windows,exported fields>> section.

Here is an example document generated by this metricset:

[source,json]
----
include::../../../module/windows/services/_meta/data.json[]
----
96 changes: 96 additions & 0 deletions metricbeat/helper/privileges_windows.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
package helper

import (
"sync"
"syscall"

"github.com/pkg/errors"

"github.com/elastic/gosigar/sys/windows"

"github.com/elastic/beats/libbeat/logp"
)

var once sync.Once

// errMissingSeDebugPrivilege indicates that the SeDebugPrivilege is not
// present in the process's token. This is distinct from disabled. The token
// would be missing if the user does not have "Debug programs" rights. By
// default, only administrators and LocalSystem accounts have the privileges to
// debug programs.
var errMissingSeDebugPrivilege = errors.New("Metricbeat is running without " +
"SeDebugPrivilege, a Windows privilege that allows it to collect metrics " +
"from other processes. The user running Metricbeat may not have the " +
"appropriate privileges or the security policy disallows it.")

// enableSeDebugPrivilege enables the SeDebugPrivilege if it is present in
// the process's token.
func enableSeDebugPrivilege() error {
self, err := syscall.GetCurrentProcess()
if err != nil {
return err
}

var token syscall.Token
err = syscall.OpenProcessToken(self, syscall.TOKEN_QUERY|syscall.TOKEN_ADJUST_PRIVILEGES, &token)
if err != nil {
return err
}

if err = windows.EnableTokenPrivileges(token, windows.SeDebugPrivilege); err != nil {
return errors.Wrap(err, "EnableTokenPrivileges failed")
}

return nil
}

// CheckAndEnableSeDebugPrivilege checks if the process's token has the
// SeDebugPrivilege and enables it if it is disabled.
func CheckAndEnableSeDebugPrivilege() error {
Copy link
Member

@andrewkroh andrewkroh Oct 24, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean some simple bool if the check has already run?

I'm thinking this should be checkAndEnableSeDebugPrivilege() and a separate method named CheckAndEnableSeDebugPrivilege exists that calls checkAndEnableSeDebugPrivilege() only once by using sync.Once.

This way only the system module or the windows module will do the initialization, but not both.

var err error
once.Do(func() {
err = checkAndEnableSeDebugPrivilege()
})
return err
}

func checkAndEnableSeDebugPrivilege() error {
info, err := windows.GetDebugInfo()
if err != nil {
return errors.Wrap(err, "GetDebugInfo failed")
}
logp.Info("Metricbeat process and system info: %v", info)

seDebug, found := info.ProcessPrivs[windows.SeDebugPrivilege]
if !found {
return errMissingSeDebugPrivilege
}

if seDebug.Enabled {
logp.Info("SeDebugPrivilege is enabled. %v", seDebug)
return nil
}

if err = enableSeDebugPrivilege(); err != nil {
logp.Warn("Failure while attempting to enable SeDebugPrivilege. %v", err)
}

info, err = windows.GetDebugInfo()
if err != nil {
return errors.Wrap(err, "GetDebugInfo failed")
}

seDebug, found = info.ProcessPrivs[windows.SeDebugPrivilege]
if !found {
return errMissingSeDebugPrivilege
}

if !seDebug.Enabled {
return errors.Errorf("Metricbeat failed to enable the "+
"SeDebugPrivilege, a Windows privilege that allows it to collect "+
"metrics from other processes. %v", seDebug)
}

logp.Info("SeDebugPrivilege is now enabled. %v", seDebug)
return nil
}
1 change: 1 addition & 0 deletions metricbeat/include/list.go
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,7 @@ import (
_ "github.com/elastic/beats/metricbeat/module/vsphere/virtualmachine"
_ "github.com/elastic/beats/metricbeat/module/windows"
_ "github.com/elastic/beats/metricbeat/module/windows/perfmon"
_ "github.com/elastic/beats/metricbeat/module/windows/services"
_ "github.com/elastic/beats/metricbeat/module/zookeeper"
_ "github.com/elastic/beats/metricbeat/module/zookeeper/mntr"
)
4 changes: 2 additions & 2 deletions metricbeat/module/docker/helper.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@ package docker
import (
"strings"

"github.com/elastic/beats/libbeat/common"

"github.com/fsouza/go-dockerclient"

"github.com/elastic/beats/libbeat/common"
)

type Container struct {
Expand Down
82 changes: 2 additions & 80 deletions metricbeat/module/system/system_windows.go
Original file line number Diff line number Diff line change
@@ -1,90 +1,12 @@
package system

import (
"syscall"

"github.com/pkg/errors"

"github.com/elastic/beats/libbeat/logp"
"github.com/elastic/gosigar/sys/windows"
"github.com/elastic/beats/metricbeat/helper"
)

// errMissingSeDebugPrivilege indicates that the SeDebugPrivilege is not
// present in the process's token. This is distinct from disabled. The token
// would be missing if the user does not have "Debug programs" rights. By
// default, only administrators and LocalSystem accounts have the privileges to
// debug programs.
var errMissingSeDebugPrivilege = errors.New("Metricbeat is running without " +
"SeDebugPrivilege, a Windows privilege that allows it to collect metrics " +
"from other processes. The user running Metricbeat may not have the " +
"appropriate privileges or the security policy disallows it.")

func initModule() {
if err := checkAndEnableSeDebugPrivilege(); err != nil {
if err := helper.CheckAndEnableSeDebugPrivilege(); err != nil {
logp.Warn("%v", err)
}
}

// checkAndEnableSeDebugPrivilege checks if the process's token has the
// SeDebugPrivilege and enables it if it is disabled.
func checkAndEnableSeDebugPrivilege() error {
info, err := windows.GetDebugInfo()
if err != nil {
return errors.Wrap(err, "GetDebugInfo failed")
}
logp.Info("Metricbeat process and system info: %v", info)

seDebug, found := info.ProcessPrivs[windows.SeDebugPrivilege]
if !found {
return errMissingSeDebugPrivilege
}

if seDebug.Enabled {
logp.Info("SeDebugPrivilege is enabled. %v", seDebug)
return nil
}

if err = enableSeDebugPrivilege(); err != nil {
logp.Warn("Failure while attempting to enable SeDebugPrivilege. %v", err)
}

info, err = windows.GetDebugInfo()
if err != nil {
return errors.Wrap(err, "GetDebugInfo failed")
}

seDebug, found = info.ProcessPrivs[windows.SeDebugPrivilege]
if !found {
return errMissingSeDebugPrivilege
}

if !seDebug.Enabled {
return errors.Errorf("Metricbeat failed to enable the "+
"SeDebugPrivilege, a Windows privilege that allows it to collect "+
"metrics from other processes. %v", seDebug)
}

logp.Info("SeDebugPrivilege is now enabled. %v", seDebug)
return nil
}

// enableSeDebugPrivilege enables the SeDebugPrivilege if it is present in
// the process's token.
func enableSeDebugPrivilege() error {
self, err := syscall.GetCurrentProcess()
if err != nil {
return err
}

var token syscall.Token
err = syscall.OpenProcessToken(self, syscall.TOKEN_QUERY|syscall.TOKEN_ADJUST_PRIVILEGES, &token)
if err != nil {
return err
}

if err = windows.EnableTokenPrivileges(token, windows.SeDebugPrivilege); err != nil {
return errors.Wrap(err, "EnableTokenPrivileges failed")
}

return nil
}
4 changes: 2 additions & 2 deletions metricbeat/module/windows/perfmon/doc.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
package perfmon

//go:generate go run mkpdh_defs.go
//go:generate go run run.go -cmd "go tool cgo -godefs defs_pdh_windows.go" -goarch amd64 -output defs_pdh_windows_amd64.go
//go:generate go run run.go -cmd "go tool cgo -godefs defs_pdh_windows.go" -goarch 386 -output defs_pdh_windows_386.go
//go:generate go run ../run.go -cmd "go tool cgo -godefs defs_pdh_windows.go" -goarch amd64 -output defs_pdh_windows_amd64.go
//go:generate go run ../run.go -cmd "go tool cgo -godefs defs_pdh_windows.go" -goarch 386 -output defs_pdh_windows_386.go
//go:generate go run $GOROOT/src/syscall/mksyscall_windows.go -output zpdh_windows.go pdh_windows.go
//go:generate gofmt -w defs_pdh_windows_amd64.go defs_pdh_windows_386.go zpdh_windows.go
25 changes: 25 additions & 0 deletions metricbeat/module/windows/services/_meta/data.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a TestData function (example) to the package that will generate this file when you run go test -data . from within this package's directory.

"@timestamp": "2016-05-23T08:05:34.853Z",
"@metadata": {
"beat": "noindex",
"type": "doc",
"version": "1.2.3"
},
"beat": {
"name": "host.example.com",
"hostname": "host.example.com"
},
"windows": {
"services": {
"display_name": "AllJoyn-Routerdienst",
"service_name": "AJRouter",
"state": "ServiceStopped",
"start_type": "ServiceDemandStart"
}
},
"metricset": {
"module": "windows",
"name": "services",
"rtt": 115
}
}
14 changes: 14 additions & 0 deletions metricbeat/module/windows/services/_meta/docs.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
=== windows services MetricSet

The `services` metricset of the Windows module reads the status for Windows
Services.

[float]
=== Configuration

[source,yaml]
----
- module: windows
metricsets: ["services"]
period: 10s
----
34 changes: 34 additions & 0 deletions metricbeat/module/windows/services/_meta/fields.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
- name: services
type: group
description: >
`services` contains the status for windows services.
fields:
- name: uptime.ms
type: long
format: duration
input_format: milliseconds
description: >
`uptime` contains the service uptime metric.

- name: service_name
type: keyword
description: >
The service name.

- name: display_name
type: keyword
description: >
The display name of the service.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the fields that are enums it would be nice to document what the possible values are. I would simply add a sentence that says "The possible values are x, y, and z".

Copy link
Contributor Author

@martinscholz83 martinscholz83 Oct 25, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some special format for the values like x, y and z?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you like 👍


- name: start_type
type: keyword
description: >
The start type of the service.
The possible values are `ServiceAutoStart`, `ServiceBootStart`, `ServiceDemandStart`, `ServiceDisabled`, and `ServiceSystemStart`.

- name: state
type: keyword
description: >
The actual state of the service.
The possible values are `ServiceContinuePending`, `ServicePausePending`, `ServicePaused`, `ServiceRuning`, `ServiceStartPending`,
`ServiceStopPending`, and `ServiceStopped`.
Loading