[Cases] Improve functional tests #150117

jonathan-buttner · 2023-02-01T22:06:32Z

This PR tries to improve how often our functional tests succeeds. I also tried cleaning up a few things that seemed to be slowing the tests down and also causing errors when the tests were run individually.

Fixes: #145271

Notable changes:

I added a value to the property-actions* in most places so that the functional tests can distinguish between a description, comment, or the case ellipses this seems to work consistently where other methods have not

Flaky test run: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1871

jonathan-buttner · 2023-02-01T22:08:47Z

x-pack/plugins/cases/public/components/property_actions/index.tsx


 PropertyActions.displayName = 'PropertyActions';
+
+const makeDataTestSubjPrepend = (customDataTestSubj?: string) => {


This allows us to pass in a specific portion of the data test subject

jonathan-buttner · 2023-02-01T22:09:39Z

x-pack/plugins/cases/public/components/case_action_bar/actions.tsx

@@ -84,7 +84,7 @@ const ActionsComponent: React.FC<CaseViewActions> = ({ caseData, currentExternal

  return (
    <EuiFlexItem grow={false} data-test-subj="case-view-actions">
-      <PropertyActions propertyActions={propertyActions} />
+      <PropertyActions propertyActions={propertyActions} customDataTestSubj={'case'} />


For the single case view we pass in the case value so the data test subject ellipses is unique

jonathan-buttner · 2023-02-01T22:10:52Z

x-pack/plugins/cases/public/components/user_actions/property_actions/property_actions.tsx

+const UserActionPropertyActionsComponent: React.FC<Props> = ({
+  isLoading,
+  propertyActions,
+  customDataTestSubj = 'user-action',


Let's default all the user action property actions to user-action. In the future we may want to make these more specific. One particular scenario is the description so that some of the functional tests can be more specific.

jonathan-buttner · 2023-02-01T22:11:12Z

...ugins/cases/public/components/user_actions/property_actions/description_property_actions.tsx

+    <UserActionPropertyActions
+      isLoading={isLoading}
+      propertyActions={propertyActions}
+      customDataTestSubj={'description'}


Adding description to the data test subject for the functional tests

jonathan-buttner · 2023-02-01T22:11:30Z

x-pack/test/functional/services/cases/single_case_view.ts

-      await caseActions.click();
-      await testSubjects.existOrFail('property-actions-trash');
-      await common.clickAndValidate('property-actions-trash', 'confirmModalConfirmButton');
+      await testSubjects.click('property-actions-case-ellipses');


Now we can look specifically for the case ellipses.

jonathan-buttner · 2023-02-01T22:12:43Z

x-pack/test/functional_with_es_ssl/apps/cases/view_case.ts

-            '[data-test-subj*="property-actions-ellipses"]'
-          );
-
-          propertyActions[propertyActions.length - 1].click();


This was trying to account for the description ellipses, now we can call it out directly.

jonathan-buttner · 2023-02-01T22:13:12Z

x-pack/test/functional_with_es_ssl/apps/cases/view_case.ts

@@ -379,7 +363,7 @@ export default ({ getPageObject, getService }: FtrProviderContext) => {
    describe('Assignees field', () => {
      before(async () => {
        await createUsersAndRoles(getService, users, roles);
-        await cases.api.activateUserProfiles([casesAllUser]);
+        await cases.api.activateUserProfiles([casesAllUser, casesAllUser2]);


Without casesAllUser2 the tests below this fail if run by themselves 😬

jonathan-buttner · 2023-02-01T22:15:12Z

x-pack/test/functional/services/cases/list.ts

@@ -81,9 +81,6 @@ export function CasesTableServiceProvider(
        rows = await find.allByCssSelector('[data-test-subj*="cases-table-row-"', 100);
        if (rows.length > 0) {
          await this.bulkDeleteAllCases();
-          // wait for a second
-          await new Promise((r) => setTimeout(r, 1000));


@cnasikas this felt like it was slowly down the tests. Do we need to 1 second wait if we're going to call waitUntilLoadingHasFinished anyway?

Also the call to header.waitUntilLoadingHasFinished(); was happening twice I think, once at like 86 and then it loops since rows.length > 0 to get in this if block and again at line 79 above. So I just removed 86 since it'll happen above anyway.

I am not sure why the timeout was there. I agree with your changes. The flaky test runner will verify that it is ok to remove it.

elasticmachine · 2023-02-02T18:11:01Z

Pinging @elastic/response-ops (Team:ResponseOps)

elasticmachine · 2023-02-02T18:11:03Z

Pinging @elastic/response-ops-cases (Feature:Cases)

cnasikas

Great job! The flaky test runner failed two tests. It seems that the case deletion test is still flaky.

…e-case-row-action

kibana-ci · 2023-02-06T19:18:32Z

💚 Build Succeeded

Buildkite Build
Commit: 6a3dd1b

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`cases`	391.9KB	392.1KB	+241.0B

History

💚 Build #105747 succeeded 9d5f792
💚 Build #105197 succeeded 09e5589
💔 Build #104922 failed d5eb6d9

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

kibanamachine · 2023-02-06T20:45:27Z

💔 All backports failed

Status	Branch	Result
❌	8.6	Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 150117

Questions ?

Please refer to the Backport tool documentation

jonathan-buttner · 2023-02-07T16:17:10Z

💚 All backports created successfully

Status	Branch	Result
✅	8.6

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

This PR tries to improve how often our functional tests succeeds. I also tried cleaning up a few things that seemed to be slowing the tests down and also causing errors when the tests were run individually. Fixes: elastic#145271 Notable changes: - I added a value to the `property-actions*` in most places so that the functional tests can distinguish between a description, comment, or the case ellipses this seems to work consistently where other methods have not Flaky test run: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1871 (cherry picked from commit c3ea5e5) # Conflicts: # x-pack/plugins/cases/public/components/case_action_bar/actions.test.tsx # x-pack/plugins/cases/public/components/case_action_bar/index.test.tsx # x-pack/plugins/cases/public/components/property_actions/index.tsx # x-pack/plugins/cases/public/components/user_actions/index.test.tsx # x-pack/test/functional_with_es_ssl/apps/cases/view_case.ts

# Backport This will backport the following commits from `main` to `8.6`: - [[Cases] Improve functional tests (#150117)](#150117)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)

Fixing tests

d5eb6d9

jonathan-buttner commented Feb 1, 2023

View reviewed changes

Fixing description tests

09e5589

jonathan-buttner marked this pull request as ready for review February 2, 2023 18:10

jonathan-buttner requested a review from a team as a code owner February 2, 2023 18:10

cnasikas approved these changes Feb 6, 2023

View reviewed changes

jonathan-buttner added 3 commits February 6, 2023 10:42

Adding retry

f20c15a

Merge branch 'main' of github.com:elastic/kibana into cases-fix-delet…

9d5f792

…e-case-row-action

reducing wait for trash icon

6a3dd1b

jonathan-buttner merged commit c3ea5e5 into elastic:main Feb 6, 2023

jonathan-buttner deleted the cases-fix-delete-case-row-action branch February 6, 2023 20:38

jonathan-buttner removed backport:prev-minor Backport to (9.0) the previous minor version (i.e. one version back from main) v8.6.2 labels Feb 7, 2023

kibanamachine added the backport:skip This commit does not require backporting label Feb 7, 2023

jonathan-buttner mentioned this pull request Feb 7, 2023

[8.6] [Cases] Improve functional tests (#150117) #150454

Merged

jonathan-buttner added v8.6.2 and removed backport:skip This commit does not require backporting labels Feb 7, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Cases] Improve functional tests #150117

[Cases] Improve functional tests #150117

jonathan-buttner commented Feb 1, 2023 •

edited by kibanamachine

Loading

jonathan-buttner Feb 1, 2023

jonathan-buttner Feb 1, 2023

jonathan-buttner Feb 1, 2023

jonathan-buttner Feb 1, 2023

jonathan-buttner Feb 1, 2023

jonathan-buttner Feb 1, 2023

jonathan-buttner Feb 1, 2023

jonathan-buttner Feb 1, 2023

cnasikas Feb 6, 2023

elasticmachine commented Feb 2, 2023

elasticmachine commented Feb 2, 2023

cnasikas left a comment •

edited

Loading

kibana-ci commented Feb 6, 2023

kibanamachine commented Feb 6, 2023

jonathan-buttner commented Feb 7, 2023


		PropertyActions.displayName = 'PropertyActions';

		const makeDataTestSubjPrepend = (customDataTestSubj?: string) => {

[Cases] Improve functional tests #150117

[Cases] Improve functional tests #150117

Conversation

jonathan-buttner commented Feb 1, 2023 • edited by kibanamachine Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elasticmachine commented Feb 2, 2023

elasticmachine commented Feb 2, 2023

cnasikas left a comment • edited Loading

Choose a reason for hiding this comment

kibana-ci commented Feb 6, 2023

💚 Build Succeeded

Metrics [docs]

Async chunks

History

kibanamachine commented Feb 6, 2023

💔 All backports failed

Manual backport

Questions ?

jonathan-buttner commented Feb 7, 2023

💚 All backports created successfully

Questions ?

jonathan-buttner commented Feb 1, 2023 •

edited by kibanamachine

Loading

cnasikas left a comment •

edited

Loading