Using Babel as secondary compiler

[Lodin]

Due to lack of support for some features of es6/es7 in Typescript I have to use Typescript + Babel compiling chain. But even if I can make tests with this chain, coverage is not correct. Can the babel be implemented as a secondary optional compiler with proper istanbul remapping?

[kulshekhar]

Could you explain what your transpilation setup looks like

[Lodin]

@kulshekhar My project is an "ejected" version of create-react-app. For the project itself I use webpack with awesome-typescript-loader that already contains babel as secondary compiler. For tests by now I use simple preprocessor that uses ts-jest as main compiler and babel-core as secondary.

const fs = require('fs');
const tsJest = require('ts-jest/preprocessor');
const babel = require('babel-core');
const jestConfig = JSON.parse(fs.readFileSync('./.jestrc', 'utf8'));
const babelConfig = JSON.parse(fs.readFileSync('./.babelrc', 'utf8'));

module.exports = {
  process(src, path) {
    const ts = tsJest.process(src, path, jestConfig);
    return babel.transform(ts, babelConfig).code;
  }
};

And I just send it to the transform property of Jest:

{
  "transform": {
    "^.+\\.(ts|tsx)$": "<rootDir>/config/jest/preprocessor.js"
  }
}

[Jessidhia]

This also might be necessary for supporting https://github.com/facebook/jest/blob/master/packages/babel-plugin-jest-hoist/src/index.js

As ES6 imports are defined to execute before any code, it's not possible to run the jest mock config before your tests actually import something... unless you use that plugin to hoist them even higher than the resulting commonjs requires.

[kulshekhar]

It doesn't look like Jest allows us to easily hook into the part related to coverage. It would be nice to be able to have more robust coverage support but I'm not sure how that can be achieved. If anyone has any ideas, PRs are welcome :)

[Lodin]

@Kovensky Thanks for advice, I'll try it.
@kulshekhar Saying robust, are you talking about current implementation? As I understood ts-jest saves processed code with sourcemap in cache and remaps istanbul results using it. Does this approach have any problems? And is it possible to remap es5 to es6 and then es6 to typescript? Or to combine es6 -> es5 sourcemaps with typescript -> es6 and remap result?

[Jessidhia]

I have tried doing something like:

'use strict'

const babelJest = require('babel-jest').createTransformer({
  compact: false,
  // if inline generation is enabled, babel tries to parse inline source maps too
  sourceMaps: 'inline'
})
const tsJest = require('ts-jest/preprocessor')

module.exports = function process (src, path, config) {
  const isTypeScript = path.endsWith('.ts') || path.endsWith('.tsx')
  src = isTypeScript ? tsJest.process(src, path, config) : src
  // babelJest needs to receive the full path of the file in order to locate
  // the correct .babelrc, but babel will refuse to process files it thinks
  // have a .ts / .tsx extension. It does not actually try to open the file,
  // though, only using it for .babelrc resolution, so just string replace
  // the extension with .js.
  src = babelJest.process(src, isTypeScript ? path.replace(/tsx?$/, 'js') : path, config)

  // Update ts-jest's coverage cache if it's enabled
  if ('__ts_coverage__cache__' in global) {
    if (path in global.__ts_coverage__cache__.sourceCache) {
      global.__ts_coverage__cache__.sourceCache[path] = src
    }
  }
  return src
}

However, this does not produce accurate results after remapping, with or without poking the __ts_coverage__cache__. This happens even if babel runs an empty config that does nothing at all -- it just converts the input to AST and then re-generates the same thing as JS, which might change formatting / semicolons / source maps.

The main difference I can think of between babel's and typescript's source maps is that ts-jest requests that typescript embeds the source code itself in the source maps, instead of just pointers.

[kulshekhar]

@Lodin by robust, I meant that it would be nice if there were hooks/apis that we could tap into to modify the functionality of either jest or the coverage processor to get ts-jest to work seamlessly with them

@Kovensky I haven't used Babel but I think I read that it can preserve line numbers when transpiling. Would that help in this case?

[Bnaya]

adding babel will also help with #90

[Bnaya]

i've made this standalone repo to investigate it
https://github.com/Bnaya/jest-ts-test
It contains a modified version of the preprocessor from babel-jest + typescript compilation
there's no custom coverage processor, the code sending the ts-transpiled code to babel + inline source maps and babel-jest using its internal coverage processor and mapping it to the source ts files.
its works with one problem that i can see,
the line numbers aren't aligned.
i'm gonna try to understand why, any help would be appreciated :)

const babel = require('babel-core');
const tsc = require('typescript');
const crypto = require('crypto');
const fs = require('fs');
const jestPreset = require('babel-preset-jest');
const es2015Preset = require('babel-preset-es2015');
const path = require('path');

const BABELRC_FILENAME = '.babelrc';

const cache = Object.create(null);
const tsconfig = require('./tsconfig.json');


const getBabelRC = (filename, {useCache}) => {
  const paths = [];
  let directory = filename;
  while (directory !== (directory = path.dirname(directory))) {
    if (useCache && cache[directory]) {
      break;
    }

    paths.push(directory);
    const configFilePath = path.join(directory, BABELRC_FILENAME);
    if (fs.existsSync(configFilePath)) {
      cache[directory] = fs.readFileSync(configFilePath, 'utf8');
      break;
    }
  }
  paths.forEach(directoryPath => {
    cache[directoryPath] = cache[directory];
  });

  return cache[directory] || '';
};

const createTransformer = (options) => {
  options = Object.assign({}, options, {
    // auxiliaryCommentBefore: ' istanbul ignore next ',
    presets: ((options && options.presets) || []).concat([jestPreset]),
    retainLines: true,
  });
  delete options.cacheDirectory;

  options.presets = options.presets.concat([es2015Preset]);

  return {
    canInstrument: true,
    getCacheKey(
      fileData,
      filename,
      configString,
      {instrument, watch}
    ) {
      return crypto.createHash('md5')
        .update(fileData)
        .update(configString)
        // Don't use the in-memory cache in watch mode because the .babelrc
        // file may be modified.
        .update(getBabelRC(filename, {useCache: !watch}))
        .update(instrument ? 'instrument' : '')
        .digest('hex');
    },
    process(
      src,
      filename,
      config,
      transformOptions
    ) {
      let plugins = options.plugins || [];

      if (transformOptions && transformOptions.instrument) {
        // Copied from jest-runtime transform.js
        plugins = plugins.concat([
          [
            require('babel-plugin-istanbul').default,
            {
              // files outside `cwd` will not be instrumented
              cwd: config.rootDir,
              exclude: [],
            },
          ],
        ]);
      }

    //   console.log(transformOptions);
    //   console.log(JSON.stringify(options));
    // console.log('src', src);

      // ts compile
      const diag = [];
      const tsOutput = tsc.transpileModule(src, {diagnostics: diag, filename, compilerOptions: tsconfig.compilerOptions, reportDiagnostics: true});

    //   console.log(tsOutput.outputText)

      if (babel.util.canCompile(filename) || true) {
        const babelOutput = babel.transform(
          tsOutput.outputText,
          Object.assign({}, options, {filename, plugins})
        );

        // console.log('babelOutput', babelOutput.code);

        return babelOutput.code; 
      }

      // return src;
    },
  };
};

module.exports = createTransformer();

[mohsen1]

Once TypeScript supports external AST transforms we can avoid adding Babel dependency.

microsoft/TypeScript#10786

[filipsuk]

@Lodin @Bnaya I have encountered the same problem while using babel for the babel-plugin-rewire plugin. Sourcemaps of untested files are broken (I'm collecting coverage from untested files too). Sourcemaps of tested files seem fine.

What was your solution? Thanks alot!