Roadmap for Node Intellisense?

[ghost]

I was comparing WebStorm 10 with VS 2015 earlier this week and when I had some issues with WebStorm's autocomplete, I told JetBrains support how unfavorably their feature compared to Visual Studio's with regard to third party NPMs (my particular problem was with lodash). Their support staff responded:

"For WebStorm 10, the possible workaround is [manually setting up the] typescript stubs ...

As for Visual Studio... Well, indeed it provides more accurate completion - as it's based on JSLS service that works by executing the JavaScript code to dynamically provide syntactically correct completions. But, as you can imagine, need to execute code dynamically in background causes a huge memory and CPU overdraft, especially when working with Node.js projects that are much larger and more complicated than average client-side apps. That's why MS is planning to abandon this service and start using typescript stubs for completion when working with Node.js"

I told them that I don't think you'd replace your current system with something that's less automatic.
But, now I'm wondering where this is coming from (maybe here) and I want to confirm - are we going to be forced to find and setup Typescript stubs for all third-party libs at some point?

Thanks for your work!

[mousetraps]

Yeah, it sounds like they got that from here and from the comment you referenced. Though it's a little weird that their answer focuses on JSLS and completely disregards the static analysis we do in NTVS to provide completions.

The new ES6 preview feature is completely experimental at the moment, and also happened to be the quickest way for us to get ES6 support.

Our primary goals with our efforts here are to:

• reduce duplication of effort (thus freeing up our time to iterate more quickly on other elements of the experience)
• reduce memory/CPU overhead
• provide a more consistent experience throughout Visual Studio.

Throughout this process, we're investigating all of our current JavaScript-based language services (JSLS, TypeScript, and NodeLS), and trying to understand how we can leverage the best features of them all.

That said, these efforts are highly guided by user feedback. So try downloading the latest release, flipping the ES6 Preview flag on, and letting us know what you love / hate so we can work to improve in the next iteration.
https://aka.ms/ntvslatest

Hope that helps, and lmk if you have any more questions!

[ghost]

@mousetraps Thanks for your detailed response. My chief concern is that I'd rather not have to go find a definition file for every external library that I want Intellisense for. So my question is - do you think that is a direction we're heading in for ES5 code? Or, will what we have for ES5 just continue to get better and faster and more reliable? (I understand that you said things depends on user feedback, so I'll take your personal opinion if you care to provide it.)

The ES6 announcement you referenced was unclear to me if Typescript definition files were required for every single external lib. It says "[the ES6 preview] can take advantage of TypeScript definition files", but I'm not sure if it requires them. Does it?

I would try out the ES6 mode, but I'm afraid for a few reasons:

• I don't want to install Typescript 1.6 today for various reasons.
• I don't want it to show me ES6 things that I can't use. I'm not using ES6 and I won't until it has been widely available in most browsers for some time.

Here is my general feedback for this feature though - I want things to be as accurate and automatic as possible. That's the main reason why I prefer VS and Microsoft tech in general. Javascript libs move too fast and it's not worth the amount of maintenance time spent to make sure things are up to date.

Thanks again!

[mousetraps]

My personal opinion is that we're taking a step back in the interest of taking two steps forward. I'm also excited because there are also a lot of very smart people with diverse opinions on the matter working towards this common goal, and that tends to be a good recipe for building something awesome 😃.

Regarding automaticity, I agree that we should strive towards a "just works" experience. What remains to be seen is just how close we can realistically get. With tsd files in particular, we could totally take advantage of our existing npm integration to automatically attempt to grab the tsd file when a new package is installed.

That doesn't fully address the issue, though. My primary concern with a tsd-only approach is the state of DefinitelyTyped. Specifically...

• # of packages:
  npm registry has hockey-stick growth, DefinitelyTyped does not (and only 1,235 entries compared to 188,201 npm pkgs)... I've often compared our Node.js Tools static analysis support of particular library/framework to the support a browser has for a particular website. Essentially, we try and understand the patterns people are using, and we'll do well so long as there weren't any unforeseen patterns.

  Taking that analogy a bit further... one of the biggest lessons we've learned with Internet Explorer is that even though the top 9000 websites gets 88% of traffic, addressing just those top 9000 of sites isn't sufficient - in other words, the "long tail" matters. And based on some of the numbers and stats about npm and various user-reported issues with our own analysis engine, my hunch is that this same tail end phenomenon exists with npm, so we may run up against the same challenges if DefinitelyTyped only addresses the top 1% of packages.

• versioning of packages:
  New npm versions are released all the time, so what happens to IntelliSense if a particular version of a package is required? DefinitelyTyped doesn't have a good story for this yet afaik

• general quality concerns:
  The nice thing is that there's incentive for library authors to create and share good tsd files so that they'll provide a better experience for their users. However, that hasn't been a strong enough incentive for everyone to write great docs, so I question whether people's behaviors will change here.

My hope is that both the DefinitelyTyped community will continue to grow and we can eventually land on a hybrid approach that enables a simultaneously lightweight and seamless experience. Some ideas regarding how we might be able to do that include:

• automatic tsd download as I mentioned above
• an additional level of static analysis
• using tsd files where we have them, and otherwise falling back to static analysis.
• maybe we can repurpose our existing static analysis engine to help people generate tsd files automatically and decrease the barrier of writing them from scratch.
• providing a good extensibility model that will enable ppl to customize the experience further.
• Perhaps a hybrid dynamic / static approach is possible.
• one complaint about our NTVS intellisense is that it's sort of a black box from a user-perspective because it's not entirely clear which patterns work and which ones don't. So finding a way to communicate those expectations better would be valuable.

That said, the above was just a personal braindump, and there certainly other ideas floating about. Long story short, this whole effort is going to take a lot of work, and one of our biggest challenges is trying to understand what aspects of our current IntelliSense experience people care most about so we can prioritize pulling those best-parts into our newly combined experience (which is especially difficult thanks to the last bullet point above 😃).

So any feedback you have to offer would be amazing. I understand you don't want to mess up your existing environment. Maybe one of our VMs would make it easier? http://blogs.msdn.com/b/visualstudio/archive/2015/04/24/node-js-tools-1-0-on-github-and-vms-available.aspx

[mousetraps]

And regarding general direction... the "ES6 IntelliSense Preview" may as well have been labeled "IntelliSense Preview", but

• we wanted a user-friendly name for it that made the advantages clear (the biggest value add at this point is ES6)
• we know it's not up to snuff to our current IntelliSense experience in some ways (in others - e.g. consistent browser/node.js intellisense experience - it's better) and we don't want to make it the default until we can improve on it enough and bridge that gap. That's not to say that there won't be sacrifices, but we're aiming for a better overarching experience.

[ghost]

One concern that I have about using tsd for Javascript Intellisense is that I don't want those files to become part of my project source code in any way. I don't want non-Visual Studio users to have to deal with them. If Visual Studio needs them to do stuff internally, that's fine with me as long as it's mostly automatic. Something like the way NuGet works sounds reasonable to me at first - the only thing that would have to be stored in my project repo would be a list of tsd packages. Of course that involves running a package server or depending on third parties like GitHub and that's probably a deal-breaker. I'm really hoping for something that just runs the Javascript, creates and caches some tsds for internal use and doesn't make me do anything.

I bit the bullet and installed a bunch of updates for VS2015 today, including Typescript 1.6. I was hesitant because I had to burn an MSDN support incident a couple months ago to fix a problem after an update. So, I'll certainly try it out when I can!

[code-Review-2015]

@mousetraps I'd be happy to send reproductions of any issues, even my VHD, but I'd probably need to use what, Azure(?) to send you 40GB+ of a .VHD in a reasonable amount of time. Have you guys already looked at the Lexical Analysers that Microsoft uses for other languages? V8 compiles .js files into native code, right?

[ghost]

@code-Review-2015 V8 compiles js directly to machine code and I don't think there are any lexical analyzers that can reverse that into an abstract syntax tree that resembles the original js code.

[EdMaurer]

Bill outlines the current thinking for future JavaScript/TypeScript editing plans in Visual Studio and beyond here. @waynebloss your feedback on the provisioning and managing of assets like TypeScript typings in a JavaScript project is exactly what we are looking for.

[bowdenk7]

Hey @waynebloss I work directly on the VS 2015 IntelliSense (auto complete) experience and would love to hear more about your perspective on code completion. Any chance you would be interested in a quick skype/phone call?

[ghost]

Hi @bowdenk7 - no problem. Please email me at waynebloss at gmail.com and I'll send you my contact details and availability.

[SteveALee]

I'm brining my comments over from #613 as I didn't see this thread when responding to the call for feedback ( @mousetraps - perhaps link directly here?)

I'm +1 on @waynebloss points.

@waynebloss said:

I don't want it to show me ES6 things that I can't use. I'm not using ES6 and I won't until it has been widely available in most browsers for some time.

Well nodejstool is primarily for node (server side) and since 4.x.x node has included ES6 features. Generally we are in control of versions on the server ;)

That said, as far as I know the same intellisense is applied to all files, whether they run on server or client. Perhaps the ability to mark folders for the browser should also affect intellisense?

In any case your workflow should really include a lint stage which should be able to restrict to ES5 features on browser files (eg ESLint has plugins and environments). That does mean intellisense highlight will be off in some cases.

[SteveALee]

I also have problems with DT's quality as @mousetraps mentioned.

I'm using Hapi and DT is at version 8.x.x, while the latest release is 11.x.x. That is is seriously out of date!

I also not that the GitHub file listing has been significantly truncated making it hard to find the files you want there.

[SteveALee]

Am I correct in understanding that every source file that requires() a lib module will need the stub reference as well?

That is a lot of overhead and a source of errors in addition to finding,download and updating the correct stubs themselves.

My vote is this just has to be automated or I will not use it.

[SteveALee]

One final point: I love it hat NTVS adds almost zero dependencies on the project source. You can just import existing code or checkin the VS solution & project files. That allows cross platform development as anyone can use the commandline workflow or other GUI/IDE tools of their choice. In an opensource project you want to minimise restrictions placed on developers.

I feel adding a requirement to add stub references in the source breaks this. Admitedly it is code comments and so makes no new tooling requirements unless you want to use VS intellisense. But that is changing source files to support one tool on one dev platform. That's quite backwards in my view.

[billti]

The goal is that acquiring the library typings would be automatic and leave no trace in your project (unless you wanted to specifically configure something regarding it). The tools can determine what libraries are referenced via the package.json.

For your own JavaScript modules (or modules there aren't library typings for), we're adding better inference for Node style modules (see microsoft/TypeScript#5266 ), and the code can better describe it's API using JsDoc comments if desired (see microsoft/TypeScript#4790)

[SteveALee]

@billti that is ideal and jsDoc is on my roadmap.

Do you have a ballpark for when that functionality will land?

[SteveALee]

FYI @ericelliott has a new project for typings called rfx

[SteveALee]

And this looks germane - https://medium.com/@mweststrate/how-to-create-strongly-typed-npm-modules-1e1bda23a7f4#.l8njhfgaa

[mjbvz]

Thanks for all the discussion and feedback here.

In order to offer users a more modern and stable IntelliSense today, NTVS 1.2 moves to using TypeScript's analysis engine to provide IntelliSense. This brings a number of improvements, including ES6 support. The TypeScript analysis engine should also far more stable and should receive fixes and updates for new JavaScript language features much more quickly.

In VS 2015, to support this experience, we have to download typings files automatically and have to include these files in your project. The typings files can be .gitignored for any users not on VS and do not need to be checked in, but they must be part of the Visual Studio project. We know this is not an ideal experience, but these workarounds are required in VS2015. With VS15, we will be able to provide a less obtrusive experience that does not require changing your project files.

Please try out NTVS 1.2 Beta and let us know what you think.