Merge branch 'master' into LdapInput--NewPlugin

* master: (59 commits) Added additional SQL Server performance counters (influxdata#3770) Update changelog Fix ping plugin not reporting zero durations (influxdata#3778) Adjust time of nightly build Update changelog Add TLS support to the mesos input plugin (influxdata#3769) Install new requirements for fpm gem install Update changelog Add additional metrics and reverse metric names option to openldap (influxdata#3722) Update paho mqtt to latest release Update changelog Remove userinfo from url tag in prometheus input (influxdata#3743) Update sample config in contributing docs Run nightly build sequentially Fix Makefile on Windows and use in AppVeyor build (influxdata#3748) Fix example source_override values in wavefront output (influxdata#3744) Update gitignore Update changelog Improve procstat readme Add native Go method for finding pids to procstat (influxdata#3559) ...
mkboudreau · Feb 14, 2018 · c9b3f4f · c9b3f4f
2 parents 5de1a89 + 42ccc9f
commit c9b3f4f
Show file tree

Hide file tree

Showing 72 changed files with 3,127 additions and 1,868 deletions.
diff --git a/.circleci/config.yml b/.circleci/config.yml
@@ -0,0 +1,49 @@
+---
+defaults: &defaults
+  docker:
+    - image: 'circleci/golang:1.9.2'
+  working_directory: '/go/src/github.com/influxdata/telegraf'
+version: 2
+jobs:
+  build:
+    <<: *defaults
+    steps:
+      - checkout
+      - run: 'make ci-test'
+  release:
+    <<: *defaults
+    steps:
+      - checkout
+      - run: './scripts/release.sh'
+      - store_artifacts:
+          path: './artifacts'
+          destination: '.'
+  nightly:
+    <<: *defaults
+    steps:
+      - checkout
+      - run: './scripts/release.sh'
+      - store_artifacts:
+          path: './artifacts'
+          destination: '.'
+workflows:
+  version: 2
+  build_and_release:
+    jobs:
+      - 'build'
+      - 'release':
+          requires:
+            - 'build'
+  nightly:
+    jobs:
+      - 'build'
+      - 'nightly':
+          requires:
+            - 'build'
+    triggers:
+      - schedule:
+          cron: "0 18 * * *"
+          filters:
+            branches:
+              only:
+                - master
diff --git a/.gitignore b/.gitignore
@@ -1,7 +1,5 @@
 build
-tivan
-.vagrant
 /telegraf
-.idea
+/telegraf.gz
 *~
 *#
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -16,6 +16,22 @@
   In environments where TCP connections are terminated the `max_lifetime`
   setting should be set less than the collection `interval` to prevent errors.
 
+- The `sqlserver` input plugin has a new query and data model that can be enabled
+  by setting `query_version = 2`.  It is encouraged to migrate to the new
+  model when possible as the old version is deprecated and will be removed in
+  a future version.
+
+- An option has been added to the `openldap` input plugin that reverses metric
+  name to improve grouping.  This change is enabled when `reverse_metric_names = true`
+  is set.  It is encouraged to enable this option when possible as the old
+  ordering is deprecated.
+
+
+### New Plugins
+
+- [ipset](./plugins/inputs/ipset/README.md) - Thanks to @sajoupa
+- [nats](./plugins/inputs/nats/README.md) - Thanks to @mjs & @levex
+
 ### Features
 
 - [#3551](https://github.com/influxdata/telegraf/pull/3551): Add health status mapping from string to int in elasticsearch input.
@@ -30,20 +46,45 @@
 - [#2846](https://github.com/influxdata/telegraf/pull/2846): Add support for dropwizard input format.
 - [#3666](https://github.com/influxdata/telegraf/pull/3666): Add container health metrics to docker input.
 - [#3687](https://github.com/influxdata/telegraf/pull/3687): Add support for using globs in devices list of diskio input plugin.
+- [#2754](https://github.com/influxdata/telegraf/pull/2754): Allow running as console application on Windows.
+- [#3703](https://github.com/influxdata/telegraf/pull/3703): Add listener counts and node running status to rabbitmq input.
+- [#3674](https://github.com/influxdata/telegraf/pull/3674): Add NATS Monitoring Input Plugin.
+- [#3702](https://github.com/influxdata/telegraf/pull/3702): Add ability to select which queues will be gathered in rabbitmq input.
+- [#3726](https://github.com/influxdata/telegraf/pull/3726): Add support for setting bsd source address to the ping input.
+- [#3346](https://github.com/influxdata/telegraf/pull/3346): Add Ipset input plugin.
+- [#3719](https://github.com/influxdata/telegraf/pull/3719): Add TLS and HTTP basic auth to prometheus_client output.
+- [#3618](https://github.com/influxdata/telegraf/pull/3618): Add new sqlserver output data model.
+- [#3559](https://github.com/influxdata/telegraf/pull/3559): Add native Go method for finding pids to procstat.
+- [#3722](https://github.com/influxdata/telegraf/pull/3722): Add additional metrics and reverse metric names option to openldap.
+- [#3769](https://github.com/influxdata/telegraf/pull/3769): Add TLS support to the mesos input plugin.
 
 ### Bugfixes
 
 - [#1896](https://github.com/influxdata/telegraf/issues/1896): Fix various mysql data type conversions.
 
-## v1.5.2 [unreleased]
+## v1.5.3 [unreleased]
+
+### Bugfixes
+
+- [#3729](https://github.com/influxdata/telegraf/issues/3729): Set path to / if HOST_MOUNT_PREFIX matches full path.
+- [#3739](https://github.com/influxdata/telegraf/issues/3739): Remove userinfo from url tag in prometheus input.
+- [#3778](https://github.com/influxdata/telegraf/issues/3778): Fix ping plugin not reporting zero durations.
+
+## v1.5.2 [2018-01-30]
 
 ### Bugfixes
 
 - [#3684](https://github.com/influxdata/telegraf/pull/3684): Ignore empty lines in Graphite plaintext.
 - [#3604](https://github.com/influxdata/telegraf/issues/3604): Fix index out of bounds error in solr input plugin.
 - [#3680](https://github.com/influxdata/telegraf/pull/3680): Reconnect before sending graphite metrics if disconnected.
-
-## v1.5.1 [2017-01-10]
+- [#3693](https://github.com/influxdata/telegraf/pull/3693): Align aggregator period with internal ticker to avoid skipping metrics.
+- [#3629](https://github.com/influxdata/telegraf/issues/3629): Fix a potential deadlock when using aggregators.
+- [#3697](https://github.com/influxdata/telegraf/issues/3697): Limit wait time for writes in mqtt output.
+- [#3698](https://github.com/influxdata/telegraf/issues/3698): Revert change in graphite output where dot in field key was replaced by underscore.
+- [#3710](https://github.com/influxdata/telegraf/issues/3710): Add timeout to wavefront output write.
+- [#3725](https://github.com/influxdata/telegraf/issues/3725): Exclude master_replid fields from redis input.
+
+## v1.5.1 [2018-01-10]
 
 ### Bugfixes
 

diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -79,7 +79,10 @@ func (s *Simple) Description() string {
 }
 
 func (s *Simple) SampleConfig() string {
-    return "ok = true # indicate if everything is fine"
+    return `
+  ## Indicate if everything is fine
+  ok = true
+`
 }
 
 func (s *Simple) Gather(acc telegraf.Accumulator) error {
@@ -207,7 +210,9 @@ func (s *Simple) Description() string {
 }
 
 func (s *Simple) SampleConfig() string {
-    return "url = localhost"
+    return `
+  ok = true
+`
 }
 
 func (s *Simple) Connect() error {

diff --git a/Godeps b/Godeps
@@ -16,7 +16,7 @@ github.com/docker/go-connections 990a1a1a70b0da4c4cb70e117971a4f0babfbf1a
 github.com/eapache/go-resiliency b86b1ec0dd4209a588dc1285cdd471e73525c0b3
 github.com/eapache/go-xerial-snappy bb955e01b9346ac19dc29eb16586c90ded99a98c
 github.com/eapache/queue 44cc805cf13205b55f69e14bcb69867d1ae92f98
-github.com/eclipse/paho.mqtt.golang d4f545eb108a2d19f9b1a735689dbfb719bc21fb
+github.com/eclipse/paho.mqtt.golang aff15770515e3c57fc6109da73d42b0d46f7f483
 github.com/go-logfmt/logfmt 390ab7935ee28ec6b286364bba9b4dd6410cb3d5
 github.com/go-sql-driver/mysql 2e00b5cd70399450106cec6431c2e2ce3cae5034
 github.com/gobwas/glob bea32b9cd2d6f55753d94a28e959b13f0244797a
@@ -45,6 +45,7 @@ github.com/miekg/dns 99f84ae56e75126dd77e5de4fae2ea034a468ca1
 github.com/mitchellh/mapstructure d0303fe809921458f417bcf828397a65db30a7e4
 github.com/multiplay/go-ts3 07477f49b8dfa3ada231afc7b7b17617d42afe8e
 github.com/naoina/go-stringutil 6b638e95a32d0c1131db0e7fe83775cbea4a0d0b
+github.com/nats-io/gnatsd 393bbb7c031433e68707c8810fda0bfcfbe6ab9b
 github.com/nats-io/go-nats ea9585611a4ab58a205b9b125ebd74c389a6b898
 github.com/nats-io/nats ea9585611a4ab58a205b9b125ebd74c389a6b898
 github.com/nats-io/nuid 289cccf02c178dc782430d534e3c1f5b72af807f

diff --git a/Makefile b/Makefile
@@ -2,6 +2,9 @@ PREFIX := /usr/local
 VERSION := $(shell git describe --exact-match --tags 2>/dev/null)
 BRANCH := $(shell git rev-parse --abbrev-ref HEAD)
 COMMIT := $(shell git rev-parse --short HEAD)
+GOFILES ?= $(shell git ls-files '*.go')
+GOFMT ?= $(shell gofmt -l $(GOFILES))
+
 ifdef GOBIN
 PATH := $(GOBIN):$(PATH)
 else
@@ -16,10 +19,18 @@ ifdef VERSION
 endif
 
 all:
+	$(MAKE) fmtcheck
 	$(MAKE) deps
 	$(MAKE) telegraf
 
+ci-test:
+	$(MAKE) deps
+	$(MAKE) fmtcheck
+	$(MAKE) vet
+	$(MAKE) test
+
 deps:
+	go get -u github.com/golang/lint/golint
 	go get github.com/sparrc/gdm
 	gdm restore
 
@@ -36,92 +47,53 @@ install: telegraf
 test:
 	go test -short ./...
 
+fmt:
+	@gofmt -w $(GOFILES)
+
+fmtcheck:
+	@echo '[INFO] running gofmt to identify incorrectly formatted code...'
+	@if [ ! -z $(GOFMT) ]; then \
+		echo "[ERROR] gofmt has found errors in the following files:"  ; \
+		echo "$(GOFMT)" ; \
+		echo "" ;\
+		echo "Run make fmt to fix them." ; \
+		exit 1 ;\
+	fi
+	@echo '[INFO] done.'
+
+lint:
+	golint ./...
+
 test-windows:
 	go test ./plugins/inputs/ping/...
 	go test ./plugins/inputs/win_perf_counters/...
 	go test ./plugins/inputs/win_services/...
-
-lint:
-	go vet ./...
-
-test-all: lint
+	go test ./plugins/inputs/procstat/...
+
+# vet runs the Go source code static analysis tool `vet` to find
+# any common errors.
+vet:
+	@echo 'go vet $$(go list ./...)'
+	@go vet $$(go list ./...) ; if [ $$? -eq 1 ]; then \
+		echo ""; \
+		echo "go vet has found suspicious constructs. Please remediate any reported errors"; \
+		echo "to fix them before submitting code for review."; \
+		exit 1; \
+	fi
+
+test-all: vet
 	go test ./...
 
 package:
 	./scripts/build.py --package --platform=all --arch=all
 
 clean:
-	-rm -f telegraf
-	-rm -f telegraf.exe
+	rm -f telegraf
+	rm -f telegraf.exe
 
 docker-image:
 	./scripts/build.py --package --platform=linux --arch=amd64
 	cp build/telegraf*$(COMMIT)*.deb .
 	docker build -f scripts/dev.docker --build-arg "package=telegraf*$(COMMIT)*.deb" -t "telegraf-dev:$(COMMIT)" .
 
-# Run all docker containers necessary for integration tests
-docker-run:
-	docker run --name aerospike -p "3000:3000" -d aerospike/aerospike-server:3.9.0
-	docker run --name zookeeper -p "2181:2181" -d wurstmeister/zookeeper
-	docker run --name kafka \
-		--link zookeeper:zookeeper \
-		-e KAFKA_ADVERTISED_HOST_NAME=localhost \
-		-e KAFKA_ADVERTISED_PORT=9092 \
-		-e KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181 \
-		-e KAFKA_CREATE_TOPICS="test:1:1" \
-		-p "9092:9092" \
-		-d wurstmeister/kafka
-	docker run --name elasticsearch -p "9200:9200" -p "9300:9300" -d elasticsearch:5
-	docker run --name mysql -p "3306:3306" -e MYSQL_ALLOW_EMPTY_PASSWORD=yes -d mysql
-	docker run --name memcached -p "11211:11211" -d memcached
-	docker run --name postgres -p "5432:5432" -d postgres
-	docker run --name rabbitmq -p "15672:15672" -p "5672:5672" -d rabbitmq:3-management
-	docker run --name redis -p "6379:6379" -d redis
-	docker run --name nsq -p "4150:4150" -d nsqio/nsq /nsqd
-	docker run --name mqtt -p "1883:1883" -d ncarlier/mqtt
-	docker run --name riemann -p "5555:5555" -d stealthly/docker-riemann
-	docker run --name nats -p "4222:4222" -d nats
-	docker run --name openldap \
-		-e SLAPD_CONFIG_ROOTDN="cn=manager,cn=config" \
-		-e SLAPD_CONFIG_ROOTPW="secret" \
-		-p "389:389" -p "636:636" \
-		-d cobaugh/openldap-alpine
-	docker run --name cratedb \
-		-p "6543:5432" \
-		-d crate:2.2 \
-		-Cnetwork.host=0.0.0.0 \
-		-Ctransport.host=localhost \
-		-Clicense.enterprise=false
-
-# Run docker containers necessary for integration tests; skipping services provided
-# by CircleCI
-docker-run-circle:
-	docker run --name aerospike -p "3000:3000" -d aerospike/aerospike-server:3.9.0
-	docker run --name zookeeper -p "2181:2181" -d wurstmeister/zookeeper
-	docker run --name kafka \
-		--link zookeeper:zookeeper \
-		-e KAFKA_ADVERTISED_HOST_NAME=localhost \
-		-e KAFKA_ADVERTISED_PORT=9092 \
-		-e KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181 \
-		-e KAFKA_CREATE_TOPICS="test:1:1" \
-		-p "9092:9092" \
-		-d wurstmeister/kafka
-	docker run --name elasticsearch -p "9200:9200" -p "9300:9300" -d elasticsearch:5
-	docker run --name nsq -p "4150:4150" -d nsqio/nsq /nsqd
-	docker run --name mqtt -p "1883:1883" -d ncarlier/mqtt
-	docker run --name riemann -p "5555:5555" -d stealthly/docker-riemann
-	docker run --name nats -p "4222:4222" -d nats
-	docker run --name openldap \
-		-e SLAPD_CONFIG_ROOTDN="cn=manager,cn=config" \
-		-e SLAPD_CONFIG_ROOTPW="secret" \
-		-p "389:389" -p "636:636" \
-		-d cobaugh/openldap-alpine
-
-docker-kill:
-	-docker kill aerospike elasticsearch kafka memcached mqtt mysql nats nsq \
-		openldap postgres rabbitmq redis riemann zookeeper cratedb
-	-docker rm aerospike elasticsearch kafka memcached mqtt mysql nats nsq \
-		openldap postgres rabbitmq redis riemann zookeeper cratedb
-
-.PHONY: deps telegraf telegraf.exe install test test-windows lint test-all \
-	package clean docker-run docker-run-circle docker-kill docker-image
+.PHONY: deps telegraf install test test-windows lint vet test-all package clean docker-image fmtcheck
diff --git a/README.md b/README.md
@@ -159,6 +159,7 @@ configuration options.
 * [interrupts](./plugins/inputs/interrupts)
 * [ipmi_sensor](./plugins/inputs/ipmi_sensor)
 * [iptables](./plugins/inputs/iptables)
+* [ipset](./plugins/inputs/ipset)
 * [jolokia](./plugins/inputs/jolokia) (deprecated, use [jolokia2](./plugins/inputs/jolokia2))
 * [jolokia2](./plugins/inputs/jolokia2)
 * [kapacitor](./plugins/inputs/kapacitor)
@@ -171,6 +172,7 @@ configuration options.
 * [minecraft](./plugins/inputs/minecraft)
 * [mongodb](./plugins/inputs/mongodb)
 * [mysql](./plugins/inputs/mysql)
+* [nats](./plugins/inputs/nats)
 * [net_response](./plugins/inputs/net_response)
 * [nginx](./plugins/inputs/nginx)
 * [nginx_plus](./plugins/inputs/nginx_plus)

diff --git a/agent/accumulator.go b/agent/accumulator.go
@@ -26,7 +26,7 @@ type MetricMaker interface {
 func NewAccumulator(
 	maker MetricMaker,
 	metrics chan telegraf.Metric,
-) *accumulator {
+) telegraf.Accumulator {
 	acc := accumulator{
 		maker:     maker,
 		metrics:   metrics,