added fix to support es5

[sarakumap]

To fix error to support ES5 issue with id while indexing "Error: elastalert an id must be provided if version type or value are set"

[Tyoshi14]

I am sorry that the error still remains.

INFO:elastalert:Queried rule IES crash rate Alert from 2017-01-12 02:16 CST to 2017-01-12 02:31 CST: 4960 / 4960 hits
Traceback (most recent call last):
File "/usr/local/bin/elastalert-test-rule", line 11, in
load_entry_point('elastalert==0.1.4', 'console_scripts', 'elastalert-test-rule')()
File "/usr/local/lib/python2.7/site-packages/elastalert-0.1.4-py2.7.egg/elastalert/test_rule.py", line 322, in main
test_instance.run_rule_test()
File "/usr/local/lib/python2.7/site-packages/elastalert-0.1.4-py2.7.egg/elastalert/test_rule.py", line 317, in run_rule_test
self.run_elastalert(rule_yaml, conf, args)
File "/usr/local/lib/python2.7/site-packages/elastalert-0.1.4-py2.7.egg/elastalert/test_rule.py", line 236, in run_elastalert
client.run_rule(rule, endtime, starttime)
File "/usr/local/lib/python2.7/site-packages/elastalert-0.1.4-py2.7.egg/elastalert/elastalert.py", line 528, in run_rule
if not self.run_query(rule, rule['starttime'], tmp_endtime):
File "/usr/local/lib/python2.7/site-packages/elastalert-0.1.4-py2.7.egg/elastalert/elastalert.py", line 380, in run_query
data = self.get_hits(rule, start, end, index, scroll)
File "/usr/local/lib/python2.7/site-packages/elastalert-0.1.4-py2.7.egg/elastalert/elastalert.py", line 271, in get_hits
hits = self.process_hits(rule, hits)
File "/usr/local/lib/python2.7/site-packages/elastalert-0.1.4-py2.7.egg/elastalert/elastalert.py", line 211, in process_hits
set_es_key(hit['_source'], rule['timestamp_field'], rule'ts_to_dt')
File "/usr/local/lib/python2.7/site-packages/elastalert-0.1.4-py2.7.egg/elastalert/util.py", line 108, in ts_to_dt
dt = dateutil.parser.parse(timestamp)
File "/usr/local/lib/python2.7/site-packages/dateutil/parser.py", line 743, in parse
return DEFAULTPARSER.parse(timestr, **kwargs)
File "/usr/local/lib/python2.7/site-packages/dateutil/parser.py", line 300, in parse
res = self._parse(timestr, **kwargs)
File "/usr/local/lib/python2.7/site-packages/dateutil/parser.py", line 358, in _parse
l = _timelex.split(timestr)
File "/usr/local/lib/python2.7/site-packages/dateutil/parser.py", line 139, in split
return list(cls(s))
File "/usr/local/lib/python2.7/site-packages/dateutil/parser.py", line 136, in next
return self.next() # Python 2.x support
File "/usr/local/lib/python2.7/site-packages/dateutil/parser.py", line 130, in next
token = self.get_token()
File "/usr/local/lib/python2.7/site-packages/dateutil/parser.py", line 61, in get_token
nextchar = self.instream.read(1)

[Qmando]

@Tyoshi14 That is a completely different issue.

PR looks good. Thanks.

[Qmando]

Oops, I was too eager. This changes doesn't make sense to me. Index API is different for Elasticsearchand indices API. This is for creating an index.

Can you show an actual traceback with this line?

[sarakumap]

please find the traceback before the fix.

ERROR:root:Error querying for last run: TransportError(400, u'parsing_exception', u'Unknown key for a START_OBJECT in [filter].')
WARNING:elasticsearch:POST http://localhost:9200/pms_elastalert/elastalert_error?op_type=create [status:400 request:0.008s]
ERROR:root:Error writing alert info to Elasticsearch: TransportError(400, u'action_request_validation_exception', u'Validation Failed: 1: an id must be provided if version type or value are set;')
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/elastalert-0.1.4-py2.7.egg/elastalert/elastalert.py", line 1021, in writeback
doc_type=doc_type, body=body)
File "build/bdist.macosx-10.6-intel/egg/elasticsearch/client/utils.py", line 69, in _wrapped
return func(*args, params=params, **kwargs)
File "build/bdist.macosx-10.6-intel/egg/elasticsearch/client/init.py", line 248, in create
return self.index(index, doc_type, body, id=id, params=params, op_type='create')
File "build/bdist.macosx-10.6-intel/egg/elasticsearch/client/utils.py", line 69, in _wrapped
return func(*args, params=params, **kwargs)
File "build/bdist.macosx-10.6-intel/egg/elasticsearch/client/init.py", line 279, in index
_make_path(index, doc_type, id), params=params, body=body)
File "build/bdist.macosx-10.6-intel/egg/elasticsearch/transport.py", line 327, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "build/bdist.macosx-10.6-intel/egg/elasticsearch/connection/http_requests.py", line 84, in perform_request
self._raise_error(response.status_code, raw_data)
File "build/bdist.macosx-10.6-intel/egg/elasticsearch/connection/base.py", line 113, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
RequestError: TransportError(400, u'action_request_validation_exception', u'Validation Failed: 1: an id must be provided if version type or value are set;')

---

after this fix

$ elastalert --verbose --rule example_rules/pms_frequency.yaml
INFO:elastalert:Starting up
INFO:elastalert:Queried rule Test:PMS Error Frequency rule from 2017-01-13 10:30 JST to 2017-01-13 10:45 JST: 0 / 0 hits

[Qmando]

elastalert.py and create_index.py are not the same thing. These functions are not the same thing.

In support_es5, that line has already been changed to use index. https://github.com/Yelp/elastalert/blob/support_es5/elastalert/elastalert.py#L1050

@sarakumap are you using the support_es5 branch??

Edit: You are using v0.1.4 not support_es5.

[sarakumap]

yes I forked from support_es5 and created a pull request. it works fine now.

[Qmando]

You are not running that branch. From your traceback: File "/Library/Python/2.7/site-packages/elastalert-0.1.4-py2.7.egg/elastalert/elastalert.py", line 1021. Version 0.1.4 is completely separate from support_es5

[Qmando]

You'll need to uninstall 0.1.4 and reinstall the support_es5 branch.

sudo pip uninstall elastalert
cd elastalert/
git checkout support_es5
sudo pip install -e .

[sarakumap]

yes you are correct. it doesn't have the latest changes. let me pull the latest support_es5 and check it. will confirm in a while

[sarakumap]

just did that. support_es5 works fine. thanks!