Only loading a subset of files

[nello]

We've been using karma-jspm in our build for some time and have recently come to the confused realisation that some of our tests are no longer being run.

The config:

jspm: {
  ...
  loadFiles: ['client/testSupport/*.js', 'client/app/**/*.spec.js'],
  ...
}

loads 1681 tests.

The config:

jspm: {
  ...
  loadFiles: ['client/testSupport/*.js', 'client/app/services/**/*.spec.js'],
  ...
}

loads 428 tests, none of which were loaded in the previous config.

Bizarrely:

jspm: {
  ...
  loadFiles: ['client/testSupport/*.js', 'client/app/services/**/*.spec.js', 'client/app/**/*.spec.js'],
  ...
}

loads all 2109 tests.

And even more confusingly:

jspm: {
  ...
  loadFiles: ['client/testSupport/*.js', 'client/app/services/**/*.spec.js', 'client/app/**/*.spec.js'],
  ...
}

only loads 1681.

No clues here. Is there some kind of strange file limit being reached?

[SerkanSipahi]

@nello please share your karma.conf.js ... i had a similar problem ...

[nello]

Ours came down to a silently-failing error in one of our tests that halted further execution. We figured it out by progressively bisecting the tests being executed.

The first fix we applied was to ensure that the file pattern provided could not miss all our tests. This effectively quarantined the issue.

-      loadFiles: ['client/testSupport/*.js', envValue('KARMA_SPEC_FILE_PATTERN',
-        'client/app/**/*.spec.js')],

+      loadFiles: ['client/testSupport/*.js',  
+        envValue('KARMA_SPEC_FILE_PATTERN', 'client/app/services/**/*.spec.js'),
+        envValue('KARMA_SPEC_FILE_PATTERN', 'client/app/**/*.spec.js')],

The "missing" tests were in client/app/services/**, so then we ran each folder of tests individually, finding a number of failing tests that had not been executing due to this bug. Fixing one of those tests suddenly allowed us to see the rest of the tests in that file and that folder.

I've just gone back and reviewed the fixes done at the time and even knowing what was happening doesn't help. Nothing stands out as being the obvious culprit.

This is just a still-existent bug, waiting to catch the next person unfortunate enough to stumble over it. We now monitor the number of tests we are executing, and if it drops precipitously we get out the microscope. Sigh.