Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failing test: Observability AI Assistant API Integration tests (enterprise).x-pack/test/observability_ai_assistant_api_integration/tests/knowledge_base/knowledge_base_migration·spec·ts - Observability AI Assistant API tests knowledge_base/knowledge_base_migration.spec.ts When there are knowledge base entries (from 8.15 or earlier) that does not contain semantic_text embeddings after migrating the docs have semantic_text embeddings #202926

Closed
kibanamachine opened this issue Dec 4, 2024 · 7 comments · Fixed by #206324
Closed

Failing test: Observability AI Assistant API Integration tests (enterprise).x-pack/test/observability_ai_assistant_api_integration/tests/knowledge_base/knowledge_base_migration·spec·ts - Observability AI Assistant API tests knowledge_base/knowledge_base_migration.spec.ts When there are knowledge base entries (from 8.15 or earlier) that does not contain semantic_text embeddings after migrating the docs have semantic_text embeddings #202926

kibanamachine opened this issue Dec 4, 2024 · 7 comments · Fixed by #206324
Assignees
@viduni94
Labels
failed-test A test failure on a tracked branch, potentially flaky-test Team:Obs AI Assistant Observability AI Assistant

Comments

@kibanamachine
Copy link
Contributor

kibanamachine commented Dec 4, 2024 •
edited
Loading

A test failed on a tracked branch

Error: expected false to equal true
    at Assertion.assert (expect.js:100:11)
    at Assertion.apply (expect.js:227:8)
    at Assertion.be (expect.js:69:22)
    at Context.<anonymous> (knowledge_base_migration.spec.ts:95:46)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at Object.apply (wrap_function.js:74:16)

First failure: kibana-on-merge - main
@kibanamachine kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Dec 4, 2024
@botelastic botelastic bot added the needs-team Issues missing a team label label Dec 4, 2024
@kibanamachine kibanamachine added the Team:Obs AI Assistant Observability AI Assistant label Dec 4, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/obs-ai-assistant (Team:Obs AI Assistant)

@botelastic botelastic bot removed the needs-team Issues missing a team label label Dec 4, 2024
@kibanamachine
Copy link
Contributor Author

New failure: kibana-elasticsearch-snapshot-verify - 8.x

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.x

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@neptunian neptunian mentioned this issue Jan 3, 2025
Closed
@viduni94 viduni94 self-assigned this Jan 6, 2025
@viduni94
Copy link
Contributor

viduni94 commented Jan 6, 2025 •
edited
Loading

This test doesn't fail locally for me. Trying the flaky test runner - https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7646

Update: The test passes in the Flaky test runner too - #205610 (comment)

@viduni94 viduni94 assigned viduni94 and unassigned viduni94 Jan 7, 2025
@viduni94 viduni94 mentioned this issue Jan 9, 2025
Merged
2 tasks
@viduni94
Copy link
Contributor

viduni94 commented Jan 9, 2025

Reproduced via the flaky test runner - #205897 (comment)

@viduni94 viduni94 self-assigned this Jan 9, 2025
viduni94 added a commit to viduni94/kibana that referenced this issue Jan 10, 2025
@viduni94
[Obs AI Assistant] Attempt to fix flaky KB migration test (elastic#20…
c2872ae 
…2926)
@viduni94 viduni94 mentioned this issue Jan 10, 2025
Merged
2 tasks
viduni94 added a commit to viduni94/kibana that referenced this issue Jan 10, 2025
@viduni94
[Obs AI Assistant] Revert changes to the stateful test (elastic#202926)
b14c8bd
viduni94 added a commit to viduni94/kibana that referenced this issue Jan 13, 2025
@viduni94
[Obs AI Assistant] Attempt to fix flaky KB migration test (elastic#20…
dc4a603 
…2926)
viduni94 added a commit to viduni94/kibana that referenced this issue Jan 13, 2025
@viduni94
[Obs AI Assistant] Revert changes to the stateful test (elastic#202926)
36b628e
viduni94 added a commit to viduni94/kibana that referenced this issue Jan 13, 2025
@viduni94
[Obs AI Assistant] Attempt to fix flaky KB migration test (elastic#20…
e3c7e43 
…2926)
viduni94 added a commit to viduni94/kibana that referenced this issue Jan 13, 2025
@viduni94
[Obs AI Assistant] Revert changes to the stateful test (elastic#202926)
5a735cf
viduni94 added a commit that referenced this issue Jan 13, 2025
@viduni94
[Obs AI Assistant] Attempt to fix flaky KB migration test (#206324)
95b76dc 
## Summary

Closes #202926

### Problem
The KB migration test appears to be flaky sometimes, because the
migration takes a few seconds to run. Therefore, when the test expects
to have semantic_text for all entries, it fails.

### Solution
Adding a retry statement to avoid the failure in cases where the
migration takes a little while to run.

### Checklist

- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
viduni94 added a commit to viduni94/kibana that referenced this issue Jan 23, 2025
@viduni94
[Obs AI Assistant] Attempt to fix flaky KB migration test (elastic#20…
6d5190f 
…6324)

## Summary

Closes elastic#202926

### Problem
The KB migration test appears to be flaky sometimes, because the
migration takes a few seconds to run. Therefore, when the test expects
to have semantic_text for all entries, it fails.

### Solution
Adding a retry statement to avoid the failure in cases where the
migration takes a little while to run.

### Checklist

- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@viduni94 viduni94

Labels
failed-test A test failure on a tracked branch, potentially flaky-test Team:Obs AI Assistant Observability AI Assistant
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Obs AI Assistant] Attempt to fix flaky KB migration test viduni94/kibana
3 participants
@elasticmachine @viduni94 @kibanamachine