Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor tests that require direct access to model system index #1365

Closed
jmazanec15 opened this issue Dec 29, 2023 · 0 comments
Closed

Refactor tests that require direct access to model system index #1365

jmazanec15 opened this issue Dec 29, 2023 · 0 comments
Labels
integ-test-failure Integration test failures

Comments

@jmazanec15
Copy link
Member

Description

Some of our rest tests make direct calls to the MODEL_SYSTEM_INDEX to initialize/check the state for a given test. However, the system index should not be directly accessed by the user for any reason. So, it does not make sense to access it for rest test cases. This can lead to flaky test behavior.

I did a quick scan of the tests that access the system index:
KNNRestTestCase.createModelSystemIndex:

  1. RestDeleteModelHandlerIT.testDeleteModelExists
  2. RestDeleteModelHandlerIT.testDeleteTrainingModel
  3. RestDeleteModelHandlerIT.testDeleteModelFailsInvalid
  4. RestDeleteModelHandlerIT.testTrainingDeletedModel
  5. RestGetModelHandlerIT.testGetModelExists
  6. RestGetModelHandlerIT.testGetModelExistsWithFilter
  7. RestGetModelHandlerIT.testGetModelFailsInvalid
  8. RestGetModelHandlerIT.testGetModelFailsBlank
  9. RestKNNStatsHandlerIT.testModelIndexHealthMetricsStats
  10. RestSearchModelHandlerIT.testNotSupportedParams
  11. RestSearchModelHandlerIT.testNoModelExists
  12. RestSearchModelHandlerIT.testSizeValidationFailsInvalidSize
  13. RestSearchModelHandlerIT.testSearchModelExists
  14. RestSearchModelHandlerIT.testSearchModelWithoutSource
  15. RestSearchModelHandlerIT.testSearchModelWithSourceFilteringIncludes
  16. RestSearchModelHandlerIT.testSearchModelWithSourceFilteringExcludes

KNNRestTestCase.systemIndexExists

  1. RestKNNStatsHandlerIT.testModelIndexHealthMetricsStats

MODEL_INDEX_NAME

  1. RestDeleteModelHandlerIT.testTrainingDeletedModel
  2. RestKNNStatsHandlerIT.testModelIndexHealthMetricsStats

Solution

For the given tests above, refactor to not directly access the system index. If it does not make sense to refactor, remove test and test functionality through unit tests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
integ-test-failure Integration test failures
Projects
None yet
Development

No branches or pull requests

1 participant