Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check for null before overriding task settings #102918

Merged
merged 2 commits into from
Dec 4, 2023

Conversation

davidkyle
Copy link
Member

Non issue as this fixes a bug in unreleased code

@elasticsearchmachine elasticsearchmachine added the Team:ML Meta label for the ML team label Dec 4, 2023
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/ml-core (Team:ML)

@@ -84,8 +84,11 @@ public ExecutableAction accept(OpenAiActionVisitor creator, Map<String, Object>
}

public OpenAiEmbeddingsModel overrideWith(Map<String, Object> taskSettings) {
var requestTaskSettings = OpenAiEmbeddingsRequestTaskSettings.fromMap(taskSettings);
if (taskSettings == null || taskSettings.isEmpty()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really understand this behavior.

If I call model.overrideWith(Map.of()), I expect the settings to be cleared, not to be preserved. Why this unintuitiuve behavior?

Copy link
Member Author

@davidkyle davidkyle Dec 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code is hit when the inference API is called, the inference endpoint allows the user to override the default model task settings either for experimentation or to change the behaviour in certain situations

POST _inference/text_embedding/my_model 
{
  input: ["....],
  task_settings: {
    "temperature": 5
  }
}

For an setting to be overridden it must be explicitly set.

"task_settings" : {
  "temperature": 5
}

Calling the API with empty task_settings does not explicitly overwrite any options, in this case the user probably didn't mean wipe all the task settings, what does it mean to wipe the defaults anyway as what would we replace them with.

"task_settings" : {
}

public void testOverrideWith_NullOrEmptyMap() {
var model = createModel("url", "org", "api_key", "model_name", null);

var requestTaskSettingsMap = randomBoolean() ? null : Map.<String, Object>of();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this kind of randomization is considered bad practice, see: https://github.com/elastic/elasticsearch/blob/main/TESTING.asciidoc#bad-practices

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to ensure that a specific code path was taken and was less concerned about which clause in the conditional test was triggered. I created separate test cases for each to be more explicit and the naming of the tests also aids understandability.

Copy link
Contributor

@jan-elastic jan-elastic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@davidkyle davidkyle merged commit af30fe4 into elastic:main Dec 4, 2023
14 checks passed
@davidkyle davidkyle deleted the null-task-settings branch December 4, 2023 14:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:ml Machine learning >non-issue Team:ML Meta label for the ML team v8.12.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants