-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[8.x] Security solutions upgrade test fixes #130750
Conversation
@elasticmachine merge upstream |
Looks like this is possibly unrelated, this PR may fix it: #130771 |
@elasticmachine merge upstream |
Looks like all the tests pass now, @MadameSheema can you give this a review please? |
cy.get(HOST_NAME).should('have.text', alert.hostName); | ||
cy.get(REASON).contains(expectedReason); | ||
// TODO: Needs data-test-subj | ||
// cy.get(HOST_NAME).should('have.text', alert.hostName); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you please confirm In which version are you missing a data-test-subj? I checked 8.x and the data-test-subj value is present.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@MadameSheema I added a comment TODO: Needs data-test-subj
to the ones that were missing. There are ones in import_cases.spec.ts
and threshold_rule.spec.ts
and it will be the element below the comment that needs it.
Thanks for trying to fix the tests @liza-mae. I checked and all the data-test-subj exists on the code but may be updated on the tests. Please let me know if you are ok with me pushing changes on your PR to fix the tests. Thanks. |
@elasticmachine merge upstream |
@MadameSheema yes feel free to push any changes. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for all the efforts done here, really appreciate it ❤️
Rerunning the cloud upgrade tests with latest changes, pending results. |
@MadameSheema unfortunately the results failed with re-enabling some of the assertions: https://internal-ci.elastic.co/view/Stack%20Tests/job/elastic+estf-cloud-upgrade-tests/228/ |
This reverts commit 5783eae.
@MadameSheema I reverted the last commit. Let me know how you want to proceed, shall we try to fix these failing assertions in this PR or do another PR later? |
@liza-mae merge it to unblock the current issue and we can continue working on a separate PR in order to bring the assertions back. Can you please let me know which is the upgrade path that you used (original version > new version) for executing the tests? I executed the tests on my local before pushing the latest changes and everything seemed to work, so I guess the original version is going to be important to bring all the assertions back :) |
@MadameSheema sounds good, the report link shows the failing jobs testing this PR: https://internal-ci.elastic.co/view/Stack%20Tests/job/elastic+estf-cloud-upgrade-tests/228/ which can be mapped to versions here: https://raw.githubusercontent.com/elastic/elastic-stack-testing/main/ci/upgrade/ess/upgrade_paths.json. In this description I wrote what I saw as inconsistent behavior with asserting participants, so I can refile another issue with the details to get the commented assertions added back. Thanks! |
💚 Build SucceededMetrics [docs]
History
To update your PR or re-run it, just comment with: cc @liza-mae |
Just for reference, here are the results from the revert: https://internal-ci.elastic.co/view/Stack%20Tests/job/elastic+estf-cloud-upgrade-tests/lastCompletedBuild/testReport/ Good results - There are only a handful of failures now, which I believe are due to flaky tests, so we can address those along with adding back the removed assertions in separate issues/PRs. |
* Fix upgrade tests * More fixes * Remove unused * Comment out ones missing data-test-subj * Remove participant verification * Remove unused vars * brings back assertions * Revert "brings back assertions" This reverts commit 5783eae. Co-authored-by: Kibana Machine <[email protected]> Co-authored-by: Gloria Hornero <[email protected]> (cherry picked from commit 2c9f2aa)
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
* Fix upgrade tests * More fixes * Remove unused * Comment out ones missing data-test-subj * Remove participant verification * Remove unused vars * brings back assertions * Revert "brings back assertions" This reverts commit 5783eae. Co-authored-by: Kibana Machine <[email protected]> Co-authored-by: Gloria Hornero <[email protected]> (cherry picked from commit 2c9f2aa) Co-authored-by: liza-mae <[email protected]>
* Fix upgrade tests * More fixes * Remove unused * Comment out ones missing data-test-subj * Remove participant verification * Remove unused vars * brings back assertions * Revert "brings back assertions" This reverts commit 5783eae. Co-authored-by: Kibana Machine <[email protected]> Co-authored-by: Gloria Hornero <[email protected]>
Fixes https://github.com/elastic/elastic-stack-testing/issues/1224. Latest report shows these are now passing: https://internal-ci.elastic.co/view/Stack%20Tests/job/elastic+estf-cloud-upgrade-tests/225/testReport/
I commented out a couple of checks and added a TODO since they either in need of a data-test-subj or the verification was not working for some reason and it needs to be investigated, a summary is below.
We can keep these steps commented out, attempt to fix or remove them.
For
import_case.spec.ts
For
threshold_rule.spec.ts