-
Notifications
You must be signed in to change notification settings - Fork 5.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Elasticsearch query input plugin support for ES v7 #12099
Comments
next steps: look into what is required to support v7 |
Since (un)fortunately the data source I want to query is in Elasticsearch 7 and would like to have telegraf support it so we keep the tool we use for collecting all metrics. I tried to fiddle around quickly with it today and could get it to run against my datasource using the changes in this commit with the following configs:
I am not sure though if the changes I made to Any updates on the topic? |
Thanks for taking a stab at this.
I do not think this will work, per this response and the readme which states the API version of the library needs to be the same as the version of the target elasticsearch server. I think we are currently getting lucky with using v5 with the v5+v6 server. I think a PR to resolve this would need to add a config option to state which version of elasticsearch to target, with v5 as the default. And then based on that config option, create the correct client. @srebhan thoughts? |
@powersj and @aymanrb there are multiple issues I see here. First of all, as @powersj said, you should additionally elastic7 "github.com/olivere/elastic/v7" and not replace the existing client. While I'm not 100% sure about the compatibility I guess there is a reason why we see client versions matching server versions... So given that we need to use a matching client version and the required migration of libraries, we should
elastic5 "github.com/elastic/go-elasticsearch/v5"
elastic6 "github.com/elastic/go-elasticsearch/v6"
elastic7 "github.com/elastic/go-elasticsearch/v7"
elastic8 "github.com/elastic/go-elasticsearch/v8"
What do you think? |
Yes sure, I definitely didn't mean we should remove the older version clients (was curious already about how did v5 work for v6 servers, yes absolutely we need to keep the support for all versions). What I have in that commit was more of a quick check on what's needed (not an actual implementation, I even left the import alias as elastic5 to reduce the PoC changes), and as I found out it's not only the client package version that needs to be changed, I had to do changes to the What @srebhan suggested here sounds like a good plan (assuming the query code doesn't need changes besides using the right client).
Maybe just instead of this point, we can actually use a config value as @powersj suggested avoiding creating a client, connecting to ElasticSearch, and querying with the wrong client version already just to destroy it and create the proper one? but generally speaking, that should work yes, again with the assumption that the query implementation doesn't require any adaptation between the different versions. |
Having an option also sounds good. |
Ok and I guess we will need the aggregation query changes, unfortunately. And generally speaking, the whole plugin needs to be rewritten (to some extent) if we want to use the official Elasticsearch client ( |
If a user provides a PR with support for v7 I think we are in agreement that it should be with the new client and probably a new plugin. |
I totally agree as well, especially since v8 anyway can't be supported by the old/deprecated client. So rewriting is inevitable on the very short term even. |
Hello, |
@milank78git someone would need to create a pull-request and add support... Help is appreciated! |
Unfortunately I can't program that well and I'm not from a big company :-(
I love the telegraph it's a shame great tool thanks
Dne út 19. 3. 2024 10:18 uživatel Sven Rebhan ***@***.***>
napsal:
… @milank78git <https://github.com/milank78git> someone would need to
create a pull-request and add support... Help is appreciated!
—
Reply to this email directly, view it on GitHub
<#12099 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AOB5Y2SL4XH5BS5H5N76TU3YY77HBAVCNFSM6AAAAAARNXWVVOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBWGQ2TGOJRGQ>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Use Case
Querying logs stored in Elasticsearch version 7
Expected behavior
The connection works to Elasticsearch v7 and the query is executed
Actual behavior
I get the following error:
Additional info
No response
The text was updated successfully, but these errors were encountered: