Update V2 stress test

CHANGELOG.md

@@ -72,6 +72,7 @@ With this release the systemd configuration files for InfluxDB will use the syst
 - [#6882](https://github.com/influxdata/influxdb/pull/6882): Remove a double lock in the tsm1 index writer.
 - [#6883](https://github.com/influxdata/influxdb/pull/6883): Rename dumptsmdev to dumptsm in influx_inspect.
 - [#6864](https://github.com/influxdata/influxdb/pull/6864): Allow a non-admin to call "use" for the influx cli.
+- [#6855](https://github.com/influxdata/influxdb/pull/6855): Update `stress/v2` to work with clusters, ssl, and username/password auth. Code cleanup
 
 ## v0.13.0 [2016-05-12]
 

cmd/influx_stress/influx_stress.go

@@ -36,7 +36,7 @@ func main() {
 		if *config != "" {
 			v2.RunStress(*config)
 		} else {
-			v2.RunStress("stress/v2/file.iql")
+			v2.RunStress("stress/v2/iql/file.iql")
 		}
 	} else {
 

stress/v2/DESIGN.md

@@ -17,9 +17,9 @@ The tool has the following components:
   - `QUERY` - Runs a given query or generates sample queries given a companion `INSERT` statement
   - `SET` - Changes the test parameters. Defaults are listed in the `README.md`
   - `WAIT` - Required after a `GO` statement. Blocks till all proceeding statements finish.
-* Clients - The statement, results and InfluxDB clients. This code lives in `v2/ponyExpress`
-  - `Storefront` - The `Statement` client. Also contains the results client.
-  - `ponyExpress` - A performant InfluxDB client. Makes `GET /query` and `POST /write` requests. Forwards the results to the results client.
+* Clients - The statement, results and InfluxDB clients. This code lives in `v2/stress_client`
+  - `StressTest` - The `Statement` client. Also contains the results client.
+  - `stressClient` - A performant InfluxDB client. Makes `GET /query` and `POST /write` requests. Forwards the results to the results client.
 
 ![Influx Stress Design](./influx_stress_v2.png)
 
@@ -28,20 +28,20 @@ The tool has the following components:
 `Statement` is an interface defined in `v2/statement/statement.go`:
 ```go
 type Statement interface {
-	Run(s *ponyExpress.StoreFront)
-	Report(s *ponyExpress.StoreFront) string
+	Run(s *stressClient.StressTest)
+	Report(s *stressClient.StressTest) string
 	SetID(s string)
 }
 ```
 * `Run` prompts the statement to carry out it's instructions. See the run functions of the various statements listed above for more information. 
 * `Report` retrieves and collates all recorded test data from the reporting InfluxDB instance.
 * `SetID` gives the statement an ID. Used in the parser. Each `statementID` is an 8 character random string used for reporting.
 
-### `Statement` -> `Storefront`
+### `Statement` -> `StressTest`
 
-`Statement`s send `Packages` (queries or writes to the target database) or `Directives` (for changing test state) through the `StoreFront` to the `ponyExpress` where they are processed.
+`Statement`s send `Package`s (queries or writes to the target database) or `Directives` (for changing test state) through the `StressTest` to the `stressClient` where they are processed.
 ```go
-// v2/ponyExpress/package.go
+// v2/stress_client/package.go
 
 // T is Query or Write
 // StatementID is for reporting
@@ -52,9 +52,9 @@ type Package struct {
 	Tracer      *Tracer
 }
 
-// v2/ponyExpress/directive.go
+// v2/stress_client/directive.go
 
-// Property is test state to change
+// Property is test state variable to change
 // Value is the new value
 type Directive struct {
 	Property string
@@ -63,23 +63,23 @@ type Directive struct {
 }
 ```
 
-The `Tracer` on both of these packages contains a `sync.WaitGroup` that prevents `Statement`s from returning before all their operations are finished. This `WaitGroup` is incremented in the `Run()` of the statement and decremented in `*StoreFront.resultsListen()` after results are recorded in the database. This is well documented with inline comments. `Tracer`s also carry optional tags for reporting purposes.
+The `Tracer` on both of these packages contains a `sync.WaitGroup` that prevents `Statement`s from returning before all their operations are finished. This `WaitGroup` is incremented in the `Run()` of the statement and decremented in `*StressTest.resultsListen()` after results are recorded in the database. This is well documented with inline comments. `Tracer`s also carry optional tags for reporting purposes.
 
 ```go
-//  v2/ponyExpress/tracer.go
+//  v2/stress_client/tracer.go
 type Tracer struct {
 	Tags map[string]string
 
 	sync.WaitGroup
 }
 ```
 
-### `StoreFront`
+### `StressTest`
 
-The `StoreFront` is the client for the statements through the `*StoreFront.SendPackage()` and `*StoreFront.SendDirective()` functions. It also contains some test state and the `ResultsClient`. 
+The `StressTest` is the client for the statements through the `*StressTest.SendPackage()` and `*StressTest.SendDirective()` functions. It also contains some test state and the `ResultsClient`. 
 
 ```go
-type StoreFront struct {
+type StressTest struct {
 	TestID   string
 	TestName string
 
@@ -101,19 +101,19 @@ type StoreFront struct {
 
 ### Reporting Client
 
-The `ResultsClient` turns raw responses from InfluxDB into properly tagged points containing any relevant information for storage in another InfluxDB instance. The code for creating those points lives in `v2/ponyExpress/reporting.go`
+The `ResultsClient` turns raw responses from InfluxDB into properly tagged points containing any relevant information for storage in another InfluxDB instance. The code for creating those points lives in `v2/stress_client/reporting.go`
 
 ### InfluxDB Instance (reporting)
 
-This is `localhost:8086` by default. The results are currently stored in the `_DefaultTestName` database. This is going to be changed.
+This is `localhost:8086` by default. The results are currently stored in the `_stressTest` database.
 
-### `ponyExpress`
+### `stressClient`
 
-An InfluxDB client designed for speed. `ponyExpress` also holds most test state. 
+An InfluxDB client designed for speed. `stressClient` also holds most test state. 
 
 ```go
-// v2/ponyExpress/ponyExpress.go
-type ponyExpress struct {
+// v2/stress_client/stress_client.go
+type stressClient struct {
 	testID string
 
 	// State for the Stress Test
@@ -144,18 +144,18 @@ type ponyExpress struct {
 	rc *ConcurrencyLimiter
 }
 ```
-Code for handling the write path is in `v2/ponyExpress/ponyExpress_write.go` while the query path is in `v2/ponyExpress/ponyExpress_query.go`.
+Code for handling the write path is in `v2/stress_client/stress_client_write.go` while the query path is in `v2/stress_client/stress_client_query.go`.
 
 ### InfluxDB Instance (stress test target)
 
 The InfluxDB which is being put under stress.
 
 ### response data
 
-`Response`s carry points from `ponyExpress` to the `ResultsClient`.
+`Response`s carry points from `stressClient` to the `ResultsClient`.
 
 ```go
-// v2/ponyExpress/response.go
+// v2/stress_client/response.go
 type Response struct {
 	Point  *influx.Point
 	Tracer *Tracer

stress/v2/README.md

@@ -1,35 +1,32 @@
 # Influx Stress tool
 
-Blockers to finishing:
-* Finalize reporting
-  - Decide on how to incorporate TestName (db[difficult], measurement[refactor], tag[easy])
-  - Get feedback on reporting syntax
-  - Pull addition data from queries
-* Documentation is sorely lacking. 
-  - Parser behavior and proper `.iql` syntax
-  - How the templated query generation works
-  - Collection of tested `.iql` files to simulate different loads
-
-Commune is potentially blocking writes, look into performance
-
 This stress tool works from list of InfluxQL-esque statements. The language has been extended to allow for some basic templating of fields, tags and measurements in both line protocol and query statements.
 
 By default the test outputs a human readable report to `STDOUT` and records test statistics in an active installation of InfluxDB at `localhost:8086`.
 
 To set state variables for the test such as the address of the Influx node use the following syntax:
 
 ```
+# The values listed below are the default values for each of the parameters
+
 # Pipe delineated list of addresses. For cluster: [192.168.0.10:8086|192.168.0.2:8086|192.168.0.3:8086]
-# Queries currently hit only the first node in a list. Writes are round robin.
+# Queries and writes are round-robin to the configured addresses.
 SET Addresses [localhost:8086]
 
-# Influx instance to store results
-SET ResultsAddress [localhost:8086]
+# False (default) uses http, true uses https
+SET SSL [false]
+
+# Username for targeted influx server or cluster
+SET Username []
+
+# Password for targeted influx server or cluster
+SET Password []
 
 # Database to target for queries and writes. Works like the InfluxCLI USE
-SET Database [thing2]
+SET Database [stress]
 
 # Precision for the data being written
+# Only s and ns supported
 SET Precision [s]
 
 # Date the first written point will be timestamped
@@ -157,3 +154,19 @@ WAIT -> 624.585319ms
 [√] "DROP DATABASE thing" -> 991.088464ms
 [√] "DROP DATABASE thing2" -> 421.362831ms
 ```
+
+### Next Steps:
+
+##### Reporting
+- Only use one database for reporting
+- Get feedback on reporting syntax
+- Pull addition data from queries
+
+##### Documentation
+- Parser behavior and proper `.iql` syntax
+- How the templated query generation works
+- Collection of tested `.iql` files to simulate different loads
+
+##### Performance
+- `Commune` is potentially blocking writes, look into performance.
+- Templated query generation is currently in a quazi-working state.

stress/v2/iql/default.iql

@@ -0,0 +1,13 @@
+CREATE DATABASE stress
+
+GO INSERT cpu
+cpu,
+host=server-[int inc(0) 100000],location=us-west
+value=[int rand(100) 0]
+10000000 10s 
+
+GO QUERY cpu
+SELECT count(value) FROM cpu WHERE %t
+DO 250
+
+WAIT

stress/v2/main.go

@@ -6,15 +6,15 @@ import (
 	"time"
 
 	influx "github.com/influxdata/influxdb/client/v2"
-	"github.com/influxdata/influxdb/stress/v2/ponyExpress"
+	"github.com/influxdata/influxdb/stress/v2/stress_client"
 	"github.com/influxdata/influxdb/stress/v2/stressql"
 )
 
 // RunStress takes a configFile and kicks off the stress test
 func RunStress(file string) {
 
 	// Spin up the Client
-	s := ponyExpress.NewStoreFront()
+	s := stressClient.NewStressTest()
 
 	// Parse the file into Statements
 	stmts, err := stressql.ParseStatements(file)
@@ -40,7 +40,7 @@ func RunStress(file string) {
 	}
 }
 
-func blankResponse() ponyExpress.Response {
+func blankResponse() stressClient.Response {
 	// Points must have at least one field
 	fields := map[string]interface{}{"done": true}
 	// Make a 'blank' point
@@ -50,10 +50,10 @@ func blankResponse() ponyExpress.Response {
 		log.Fatalf("Error creating blank response point\n  error: %v\n", err)
 	}
 	// Add a tracer to prevent program from returning too early
-	tracer := ponyExpress.NewTracer(make(map[string]string))
+	tracer := stressClient.NewTracer(make(map[string]string))
 	// Add to the WaitGroup
 	tracer.Add(1)
 	// Make a new response with the point and the tracer
-	resp := ponyExpress.NewResponse(p, tracer)
+	resp := stressClient.NewResponse(p, tracer)
 	return resp
 }