JMETER Elasticsearch Tests

This repo comes with 2 sets jmeter test plans and associated bash scripts :

• One that performs ingestion, query , scroll queries at specified throughput and logs latencies on a CSV file and reports latencies on a CSV file. These are es_test.bash and setup_es_test.bash
• Another that lets you replay logs at given throughput. JMeter will read (already available) raw logs and replay them onto a specific folder at a specified pace. So you could have a filebeat shipping them to ES or LS and monitor performance.

Limitations

The logs replayer only works on linux based systems (tested MacOS) Not tested/adapted for Windows

Pre-requisites

Install Jmeter 3.1 Have a ES running cluster accessible from JM if using es_test.bash

If planning to use the setup_es_test.bash (generate sample bulk ingest scripts) you need : 1. npm 2. coffeescript (see http://coffeescript.org/) 3. run npm install from within the coffee folder

If planning to perform scan and scroll queries from Jmeter : 4. you need python

Setup data and queries

• the ./input folder contains bulk indexing JSON files that can be generated by running the setup.bash (as a demo) That's where you need to put your bulk requests files that will be used by JMeter.
• If you want jmeter to send queries, there should be a query input csv (see format below) in ./queries
• If you want jmeter to sens scroll queries, there should be a scroll input csv (see format below) in ./queries

vars.bash

Modify the test parameters in the script as follows :

JMETER_PATH : The path of your JMeter install USER= Shield User PASS= Shield Pass HOST= Target Elastic hostname/ IP address PORT= Target Elastic port (ex : 9200) INDEX= Indices being queried (ex: apachelogs-* ) QUERY_CSV= the file containing queries input in relative to the csv folder (ex:input1K1h.csv) SCROLL_CSV=the file containing scroll input relative to the csv folder (ex:inputScroll.csv) SHIELD_ENABLED= is shied enabled DOCS_PER_BULK= How many events on each bulk request NB_FILES=Nb of bulk request to generate. set -1 to generate all bulks for 300MB raw data.

cleanup.bash

This script cleans up all files generated by the tests including results.csv

setup_es_test.bash

Use setup_es_test.bash script to setup test data from the (already GROK parsed) sample apache logs located in ./ingest/logs.json.gz This is output from an LS instance ready to feed into ES

This script will generate in ./input 20 BULK requests with 500 docs in each request. Tweak the parameters in the script to create more bulks / more docs per bulk

es_test.bash

Set the required parameters for the test :

• Througput in requests per minute

QUERY_THROUGHPUT=5.0 SCROLL_THROUGHPUT=20.0

• Ingestion Througput in raw MB/s .

This throughput takes is currentely based on the avg size of docs in one file in the input folder INGEST_THROUGHPUT=1

• test tag : include a tag on each result line for this test

TEST_TAG=T0602

Run this script to perfome the JMETER test

elk_stress.jmx runs in the background and takes as additional params :

-JtestScroll=true/false : wether we enable the scroll queries to run during the test -JtestIngest=true/false : "" "" "" ingestion "" "" -JtestQuery=true/false : "" "" "" querying

• if ingestion enabled :

JMeter will iterate the files in ./input (for ever) and send the bulk queries at specified throughput

• if scroll enabled :

JMeter will iterate the scroll CSV (for ever) and send scroll queries at specified throughput

• if query enabled :

JMeter will iterate the query CSV (for ever) and send queries at specified throughput

if shield not used then you have to disable the authentication managers in the Jmeter test plan (using a client). not tested with an authentication manager without shield

• Format of the query input CSV :

elk_stress.jmx comes with a generic ES query sampler. This query sampler takes as an input a line in the QUERY_CSV file and inserts each value in the corresponding query. json body

example (3 lines):

time1,time2,country_code,queryFileName 440772151510,1440775751510,,query1.json 1441972718913,1441976318913,,query1.json ,,US,query2.json

Note the variables time1, time2 referenced in the corresponding query1.json and country_code in query2.json

You can refer to multiple queries in the CSV.. Make sure the CSV headers properly match each CSV values on each rows , ex:

Each query will be sent iteratively by JMeter, and the global throughput will be QUERY_THROUGHPUT

• genDateIntervals.coffee can be used to generate random timestamp intervals.

supported parameters:

date1 = date min date2 = date max interval = width of the interval unit = time unit (s,m,h,d) nbSamples = number of intervals to generate example : coffee genDateIntervals.coffee "22/08/2015 23:13:42" "21/09/2015 14:00:00" 10 m 5000 > input5K10m.csv This will generate 5000 intervals of 10 minutes each between the "22/08/2015 23:13:42" and the "21/09/2015 14:00:00"

1442414393929,1442420393929 1441669056891,1441675056891 1442289841447,1442295841447 1440405924480,1440411924480 1440843795433,1440849795433 1441414408742,1441420408742 1441073914319,1441079914319 1440498223420,1440504223420 .....

Test results

Are located in results/results.csv the latency in ms is the csv file

setup_play_logs.bash

This simply extracts the sample raw log for the ingest/gzip file and puts it on the ./input folder

play_logs.bash

This will launch a test plan that reads from input/. raw log files and spits each line of log onto the ./logs folder. You can specify the number of threads (aka simulating the number of processes generating logs simultaneously) and the pace. You can then use Filebeat or Logstash to

Useful scripts ###adds ,request1.json at the end of each line cat input1d10.csv | awk '{print $0 ",request1.json"}' > input1d5000R1.csv