Instrument JS code to detect types at runtime

[bartosz-k]

The idea is to instrument JavaScript code and collect types information from the runtime. For example:

Consider JavaScript code (in ES2015):

File render.js:

export function renderContact(contact, container) {
  let span = document.createElement('span');
  span.textContent = `${contact.firstName} ${contact.secondName}`;
  container.appendChild(span);
}

Then we use it in file main.js:

import {renderContact} from "./render";

renderContact({firstName: "Steve", lastName: "Works"}, document.body);

Running this code as-is gives no help, but if it's instrumented then we may callect runtime information. main.js could be instrumented to the following form using esprima/escodegen or TypeScript's parser API:

import {renderContact} from "./render";

CallExpression({line:3, col:0},
               null, 
               renderContact, 
               [{firstName: "Steve", lastName: "Works"}, document.body]) 

CallExpression body could roughly look like:

function CallExpression(position:Position, callee:any, fn:Function, args:any[) {
    // collect types information part
    args.forEach(arg=>CollectArgumentTypesInformation(position, arg));

    // execution part to conform JavaScript semantics and do not break the running application code
    return fn.apply(callee, args);   
}


function CollectArgumentTypesInformation(position:Position, arg:any) {
    let type={};
    Object.keys(arg).forEach(key=>{
        type[key]=typeof arg[key];
    });   
    return type;
}

CollectArgumentTypesInformation for contact parameter would yield something like:

{
    firstName:string,
    lastName:string
}

that could be later transformed into interface contact (name of the interface is based on renderContact 1st parameter name):

interface contact {
    firstName:string,
    lastName:string
}

Later on this TypeScript type could be injected into the file like render.ts (next to the render.js) and then be used to generate d.ts file using TypeScript's tsc --declaration. Before running tsc --declaration author could refactor names for better naming and the instrumentation and types collection tool would respect that changes and wouldn't override them.

position would be used to source-map .js and .ts files. Or it could be something similar, maybe using sourceMaps project itself.

[mhegazy]

duplicate of #7546?

[billti]

This is not a duplicate. This is actually a pretty cool idea that I was considering as a possibility a while back also. Effectively you have unannotated code, then you run it (either launch under the debugger, run the tests, whatever), and it has been instrumented (by our emitter) to take note of all the function param types that were observed at run-time. You could then have a simple UI gesture to go and add the types observed at runtime to the code annotations, rather than having to figure them out and type them in yourself.

(This is also a potentially cool feature for if you already have type annotations, we could flag or assert at runtime if a different type is observed for debugging).

[bartosz-k]

@billti cool. So how would you take on that? My idea is to transpile js/ts to this instrumented form using tsc and then collect types information by the nodejs service and incrementally add types information to the file.

[aluanhaddad]

I think would really help with TypeScript adoption. I recently wanted to generate some .d.ts files from third party .js files. I renamed every file to .ts and then generated declarations. It was unwieldy, but it worked.
Ideally, there would be an option, which could be conveniently exposed via a context menu, to generate a basic .d.ts file from a JavaScript source file. It would reduce friction and encourage package authors not using TypeScript to type their APIs.

Package managers could even potentially hook into the process and generate these files where appropriate.

[ghost]

Noticed that https://github.com/Microsoft/dts-gen hadn't been mentioned yet, although it probably isn't as advanced as is being talked about here.

[aluanhaddad]

@andy-ms thanks, I didn't know that existed! Perhaps @RyanCavanaugh needs to promote it more instead of spending so much time on typescript 😃

[urish]

@Bartq @billti you may be interested in checking out TypeWiz, which does exactly that - instrumenting TypeScript (JavaScript) code and collecting type information from the runtime.

I also wrote a blog post explaining about the motivation for the project, how to use it, etc.

[ghost]

#26588 covers basic functionality. Leaving this issue open to cover instrumenting JS code for even better types.

[bartosz-k]

Assembled a quick demo of MetaES running small example: http://metaes.org/playground.html.

Pushing original idea a bit forward:

• assuming MetaES supports ECMAScript at some specification level - let's say ES2018, we wouldn't have to instrument code (as in TypeWiz from @urish), we'd just run it with MetaES. Currently MetaES specification compliance is very limited. Interpreter is more powerful than instrumentation, but much more complex in implementation and slower,
• AST could be produced by TypeScript's parser and as long it's compatible with ECMAScript tree, MetaES could skip TS type related nodes and collect actual types from a run,
• or, MetaES could use TS produced type nodes and enrich them from runtime data, @urish already created part that produces updated TypeScript code with types,
• runtime data also would be collected from actual running of application, there should be a TS server part listening to interceptor messages. TypeWiz proves that it already works,
• in what context code could be run? It depends, it can be browser, node, maybe TS could run single blocks, single functions and infer additional types on demand.

I don't know if this should be a part of TypeScript server though, maybe a command line tool or IDE plugin. Or optional TS extension under a flag. That's something to discuss.