Calling Runner::afterEach() should not start a new browser after the last test in the suite

[jan-molak]

Hi there!

As of Protractor 5.1.0 (#4087) custom frameworks, such as Serenity/JS, have to invoke runner.afterEach() after every test so that Protractor can restart the browser should the developer set the restartBrowserBetweenTests flag to true.

However, when runner.afterEach() is invoked after the last test in the suite, Protractor will again restart the browser only to then immediately shut it down.

This behaviour seems incorrect, and it seems to me like it could be resolved by either:

• making sure that the afterEach method only ever invokes the restart if the test is not the last one in the suite
• or probably a bit cleaner, as the only thing that afterEach is doing is restarting the browser:
  • rename afterEach to beforeEach and require the custom frameworks to invoke that method before a test starts. This would also resolve restartBrowserBetweenTests should restart test in beforeEach, not afterEach #4224

Thoughts?
Jan

[jan-molak]

I managed to reproduce this issue using the built-in Jasmine framework and running the example_spec.js from the Protractor repo, so the problem is not limited to custom frameworks.

Even though the example_spec.js only contains 3 tests, 4 browsers are spawned.

I've added a console.log statement to Runner::createBrowser, Runner::replaceBrowser, Browser::restart

Runner::createBrowser
Started
 # Test 1 executed ...
Browser::restart
Runner::replaceBrowser
Runner::createBrowser
 # Test 2 executed ...
Browser::restart
Runner::replaceBrowser
Runner::createBrowser
 # Test 3 executed ...
Browser::restart
Runner::replaceBrowser
Runner::createBrowser
 # Another browser spawned, even though there are no tests left

Please let me know if there's anything else I can do to help?

[simonua]

This is giving us some trouble, too. Our setup consists of Protractor 5.3.1, jasmine 2.8.0, paired with Sauce Labs. We can't set restartBrowserBetweenTests to true because we don't want one monolithic test result with just one status whereas any of our tests could pass or fail individually, but X tests will always result in X + 1 sessions in Sauce Labs. I can't seem to find any part of jasmine that exposes the total number of tests to be run to where one could ensure that an afterEach is not effective after test X.

Furthermore, the new/last browser session's ID is then used in Sauce Lab's updateJob code which uses the erroneous session's ID (X + 1's ID) in the console output for the Sauce Labs result link instead of X's ID.

Tangentially, we have mitigated some of this behavior by creating a custom Sauce Labs reporter for jasmine, which obtains the browser ID in specStarted, then storing it on jasmine's result object as a custom property before retrieving it in specDone and updating Sauce Labs via the Sauce Labs API. If protractor could obtain the ID in a newly-created beforeEach - they only have hooks for afterEach, but it's easily updated - then use that ID when submitting the job, at least the ID will be correct.