Test fails on busy animation (before timeout)

[MarschallSteffen]

In our test, we check if the new page is loaded and visible. This check fails sometimes (1 in 5-10 runs). Screenshots of the failed runs show the application is still in the busy animation and the next page hasn't load yet.

The actual code we use to verify if the control is visible is:

 SolveIssuesListReport: {
        actions: {          
        },
        assertions: {
            theListReportShouldAppear: function () {
                var listreport = element(by.control({
                    id: "c4e-eds-validation-app::sap.suite.ui.generic.template.ListReport.view.ListReport::ValidationResultEntryView"
                }));
               expect(listreport.isPresent()).toBeTruthy();
            }
        }
    },

The timeouts we used are the following, usually the test completes in ~35s, the test-step in approximately 6-8s:
(however we found in instances the test failed, the step usually only took ~4s)

    timeouts: {
        getPageTimeout: '60000',
        allScriptsTimeout: '60000',
        defaultTimeoutInterval: '120000'
    },
    pageLoading: {
        wait: '10000'
    },

The Test log of a failed run.

INFO: @ui5/uiveri5 v1.47.4
INFO: specs defined in both *.conf.js and --specs CLI argument, using --specs argument.
INFO: Resolving specs
INFO: Check for latest version of: chromedriver from: https://chromedriver.storage.googleapis.com/LATEST_RELEASE
(node:27590) [DEP0022] DeprecationWarning: os.tmpDir() is deprecated. Use os.tmpdir() instead.
INFO: Found latest version of chromedriver: 96.0.4664.45
INFO: Found correct binary locally: /usr/local/lib/node_modules/@ui5/uiveri5/selenium/chromedriver-96.0.4664.45
INFO: Executing 1 specs
[14:32:29] I/launcher - Running 1 instances of WebDriver
INFO: Opening webdriver connection with capabilities: {"browserName":"chrome","platform":"MAC","chromeOptions":{"args":["--no-sandbox","--disable-dev-shm-usage","--disable-gpu","disable-infobars"]},"count":1,"loggingPrefs":{"browser":"SEVERE"}}
INFO: Suite started: displayissues
INFO: Spec started: Should display the solve issues app
INFO: Opening: https://xxx/cp.portal/site#Shell-home
INFO: UI5 Version: 1.97.0
INFO: UI5 Timestamp: 202112011147
INFO: Spec finished: Should display the solve issues app with status: PASSED
INFO: Spec started: Should open the solve issues app
INFO: No elements found using by.control locator. This is what control locator last logged: Found no control with the global ID 'c4e-eds-validation-app::sap.suite.ui.generic.template.ListReport.view.ListReport::ValidationResultEntryView' -  sap.ui.test.Opa5
INFO: Expectation FAILED: Expected false to be truthy.
INFO: Spec finished: Should open the solve issues app with status: FAILED
INFO: Spec started: Should logout
INFO: Spec finished: Should logout with status: PASSED
INFO: Suite finished: displayissues with status: FAILED for: 44.866s
INFO: Suite specs summary, total: 4, passed: 3, failed: 1, pending: 0, disabled: 0
INFO: Suite expectations summary, total: 4, passed: 3, failed total: 1, failed with error: 1, failed with image comparison: 0
INFO: Overall status: FAILED for: 44.877s
INFO: Overall suites summary, total: 1, passed: 0, failed: 1
INFO: Overall specs summary, total: 4, passed: 3, failed: 1, pending: 0, disabled: 0
INFO: Overall expectations summary, total: 4, passed: 3, failed total: 1, failed with error: 1, failed with image comparison: 0
[14:33:16] I/launcher - 0 instance(s) of WebDriver still running
[14:33:16] I/launcher - chrome #01 failed 1 test(s)
[14:33:16] I/launcher - overall: 1 failed spec(s)
[14:33:16] E/launcher - Process exited with error code 1

[hmanchev]

Hi,

Could you please try to reproduce it manually? Try to execute the steps several times and I suppose the issue should be reproduced. If the page just stays in the busy animation state, during manual execution, it shows a problem in the app.

Best Regards,
Hristo

[MarschallSteffen]

Hi Hristo,

I reproduced it (i.e. by running uiveri5 locally and watching the test browser window) and found the page is loading normally as expected. Also in the screenshots of the subsequent tests, we see the page has finished loading.

It seems uiveri5 is checking one or two seconds too early on the affected runs.

Best Regards.,
Steffen

[hmanchev]

Hi Steffen,

What version of UI5 is running your application? If you are running with old version, maybe you are hitting an issue which is already fixed.

Also do you have any page refresh in your script?

Best Regards,
Hristo

[MarschallSteffen]

Last failure we had was with version v1.47.4

Also no, we do not perform any page refresh within our test

[hmanchev]

This definitely explains your problem - we had na issue in the synchronization which was fixed somewhere around UI5 version 1.7X.

If you update your UI5 version, the issue should be gone.

[MarschallSteffen]

Was there any change to another repo or similar ?
If I check the package.json in this repository I see 1.47.4 is the current version

  "name": "@ui5/uiveri5",
  "version": "1.47.4",

Also Installing with npm install @ui5/uiveri5 -g installs the same version

[hmanchev]

Here I am pointing the UI5 version which is used in the app under test.
UiVeri5's latest version is 1.47.4, which you are using.

[MarschallSteffen]

Oh sorry, I misread. The App Under Test is running with UI5 version 1.97.0 (202112011147)

[hmanchev]

Ok, in this case I will need more logs from your test execution.
So please increase the log level with adding -v parameter in the console execution command and add in your conf.js the configurations described here: https://github.com/SAP/ui5-uiveri5/blob/master/docs/config/plugins.md

Then please send me the logs as a file (there will be a lot of logs).