Fix non-deterministic behaviour in ModelLoadingServiceTests #55008
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Pinging @elastic/ml-core (:ml) |
davidkyle
commented
Apr 9, 2020
| @@ -353,9 +353,9 @@ private void auditIfNecessary(String modelId, MessageSupplier msg) { | |||
| logger.trace(() -> new ParameterizedMessage("[{}] {}", modelId, msg.get().getFormattedMessage())); | |||
| return; | |||
| } | |||
| auditor.warning(modelId, msg.get().getFormattedMessage()); | |||
| auditor.info(modelId, msg.get().getFormattedMessage()); | |||
There was a problem hiding this comment.
A model being evicted from the cache is a routine event and not a warning level event. I would prefer this to be debug as it really is a non-issue (the model will be reloaded if required) but there is very little insight into the internals of the cache (which models are loaded etc). Once we have a model management system this should be debug or trace.
I'm raising here for discussion before I forget but I'd be happy to revert the change for this PR
benwtrent
approved these changes
Apr 9, 2020
davidkyle
force-pushed
the
fix-model-loading-test
branch
from
47e800c to
42baf3f
Compare
davidkyle
added a commit
to davidkyle/elasticsearch
that referenced
this pull request
Apr 15, 2020
davidkyle
added a commit
that referenced
this pull request
Apr 15, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
ModelLoadingServiceTests::testMaxCachedLimitReachedis testing models get evicted from the cache when the total size of all the models is too large for the cache. There is some non-deterministic behaviour when the models are first loaded as it it known what order they are loaded in and therefore which will be evicted by a later load. This change adjusts the assertions to account for that uncertainty.I believe the non-deterministic behaviour comes from a loss of ordering going from a list of IDs to a set. Model loading is mocked so there shouldn't be a race there.
Closes #54986