ember-testing with scheduled timers fails

[jschilli]

the wait method in ember-testing is checking for scheduledTimers (https://github.com/emberjs/ember.js/blob/master/packages/ember-testing/lib/helpers.js#L102) and returns if any are active.

We have several scheduled timers running in the app via run.later

When a test action (e.g. click) gets beyond the login in our application, the timers start and all subsequent thennables are blocked.

I'm happy to construct a fix (or failing test case) but need to understand what the motivation for checking hasTimers is.

[teddyzeenny]

wait() makes sure all async operations have completed before it resolves. In fact that's what wait is for. run.later is an async operation so it is supposed to block wait.

We need to figure out how to tell wait to skip waiting for specific timers. In the meantime, replacing run.later with setTimeout(function() { Em.run(function() { //.. code here } }); is a possible workaround.

[jschilli]

thx. The workaround you propose does indeed resolve it.

agreed on telling wait to skip timers - not sure where to dig in on that - will review.

[hajee]

Any progress on this one? I have this problem when testing a function that calls Ember data's store.push(). I haven't looked into it yet, but store.push() creates some scheduled timers. The test fails because it wait's indefinitely . Because it is Ember Data code, I cannot use the work around.

[stefanpenner]

@hajee can you open a ED specific issue (with example, so we can replicate) and link it back to this one.

[wagenet]

@hajee What's the status of this?

[hajee]

I've not been able to recreate the symptoms with any test.

[wagenet]

@hajee but you're still seeing the problem?

[hajee]

No. I've changed my test setup.

[balinterdi]

The problem @jschilli mentioned is still valid. In our application, I poll for new notifications every 5 seconds.

The acceptance test will wait indefinitely because Ember.run.hasScheduledTimers() always returns true and thus the wait helper in ember-testing waits 10 more msecs each time. The solution proposed by @teddyzeenny seems a good one.

[mixonic]

We usually have a flag for test environments where we disable nextTick style loops. I don't believe wait needs to change here, but maybe wait should be better documented regarding this behavior?

Unless we can come up with an alternative API.

[balinterdi]

@mixonic Could you give an insight into how you guys do that? I find that the tests that exercise "run-loop scheduling" (e.g Ember.run.later) are super flaky and what you describe might improve on that. Thank you.

[rwjblue]

Would love to see a PR documenting this behavior on wait() helper.

Closing this as the suggestion by both @teddyzeenny (using setTimeout instead) and @mixonic (disabling run.later's in testing) work well.

[bjornstar]

This issue wasted several days of development time as a user new to Ember starting to test an existing Ember app. I had absolutely no idea why the visit promise wasn't resolving, all tests just timed out with no hint as to what might be causing the problem.

It was a horrible experience.

If you put the sample code from Continuous Redrawing of Views into your app you will be unable to run integration tests in your app.

Here's a jsbin that illustrates the problem: http://jsbin.com/renapukufu/1/edit?html,js,output

[sheldonbaker]

I was also bitten by this a couple weeks ago. Some additional documentation in the testing guide would be very helpful.

[ebryn]

I think we might want to add some information about the run loop to the Ember Inspector to help you see what's going on. What do you think @teddyzeenny?

[workmanw]

It does seem like this trips a lot of people up. FWIW, this was our solution: http://discuss.emberjs.com/t/proper-way-to-handler-timers-w-ember-testing/4693/9

EDIT: Though instead of using injection, a service might be perfect for a "config". Just a thought.

[teddyzeenny]

@ebryn I think a run loop tab and an ember-testing tab (now that the inspector works in tests) would be really helpful.

[fishermand46]

I've been bit on this again. Trying to test some code that uses Ember.run.later. I really don't like the idea of running a different code path in testing vs. production. That's a sign that your code isn't testable to begin with.

[oliverwilkie]

This has caught me too as I had a live-updating clock on the page that was being tested.

[durkie]

argh, me too. a chrome app with an auto-refreshing list of serial ports. any documentation about this would be great...i had to really dig down in to andThen() to find out that it had scheduled timers, and that they were a problem.

[PhilLehmann]

I also walked into this trap while calling moment.js in a run.later recursion in order to wait 20 seconds before updating the moment strings.

I am now disabling that recursion if Ember.testing is set to true, as acceptance tests will hopefully not run that long anyway.

Ideal would be to add a parameter to run.later, which would mark it as "unblocking" for tests. Otherwise, we would have to keep a central list of timer ids up to date, that seems error prone... What do you think about this?

[aprescott]

I don't know if a different issue replaced this one, but I think this should be considered a bug. I've recently hit this problem, with tests which time out because I have an run.later call which recursively sets up another run.later. I'd like to use Ember.run to stay within the framework instead of switching to setTimeout. Like another comment above says, codepaths which branch based on testing vs. not testing is generally a bad sign.

[mixonic]

@aprescott unfortunately the current testing API design is simply limited in this way. We definitely cannot change it without a semver violation, and indeed this is just a tricky problem to solve regardless of how free our hand is.

I can suggest using ember-lifeline and pollTask which provides a more testable polling mechanism. Instead of a loop running forever and blocking tests, you must manually tick the polling task during test. I think ember-concurrency has a similar API now or is considering one for the future.

[micky2be]

The only way to make it work for me was to override Ember.run.later and Ember.run.next with a setTimeout counterpart when running my test.
For some reason functions in later or next do not run and cause my tests to time out.

[johnnyshields]

@mixonic Micky above is the second senior dev from my company who wasted significant time dealing with issues due to the unintuitive API design here. Please consider improving this in Ember 3.

[tsing80]

angular has similiar problem as well : angular/protractor#169