Skip to content
This repository has been archived by the owner on Aug 2, 2022. It is now read-only.

Add stats for custom scoring feature #233

Merged

Conversation

jmazanec15
Copy link
Member

Issue #, if available:

Description of changes:
Adds a few statistics for custom scoring feature. The following statistics are added:

script_compilations

The number of times the knn script is compiled. This value should only be 0 or 1 most of the time. However, if the cache containing the compiled scripts is filled, it may cause the script to be recompiled.

script_compilation_errors

The number of errors during script compilation.

script_query_requests

The number of query requests that use the k-NN score script.

script_query_errors

The number of errors that have occurred during use of the k-NN score script.

Along with adding the stats, I updated the README and added another stats integration test to verify the changes.

I think in the future, it will probably make sense to refactor the stats api into different sections for custom scoring and approximate k Nearest Neighbor search. I will create an issue to do this.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@jmazanec15 jmazanec15 added the Enhancements Improvement on existing component label Sep 21, 2020
@jmazanec15 jmazanec15 requested a review from vamshin September 21, 2020 20:41
/**
* Test checks that script stats are properly updated
*/
public void testScriptStats() throws Exception {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add similar test with multiple shards?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

#### script_query_requests
The number of query requests that use the k-NN score script.

#### script_query_errors
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these errors are counted at shard level. We could update the documentation accordingly. Also you might want to check if all the above metrics at shard level or index level

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, they are at shard level. I updated README

Copy link
Member

@vamshin vamshin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for the stats.

@vamshin vamshin merged commit c2e84fd into opendistro-for-elasticsearch:master Sep 22, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Enhancements Improvement on existing component
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants