Windows Support #1017

daviddias · 2017-09-13T13:43:24Z

More and more users have been trying to use js-ipfs on Windows and seeing their experiments fail. This happens because, in fact, js-ipfs doesn't have Windows support at the moment.

Reported issues:

We welcome a captain to develop Windows support for js-ipfs. It should be really close and mostly be an issue with pathing and native deps.

Beanow · 2017-09-20T09:43:47Z

*cough* what about this? https://msdn.microsoft.com/en-us/commandline/wsl/install_guide

victorb · 2017-09-20T11:43:27Z

Did a look into this as I have a windows machine close by.

First discover was that fs-ext does not like windows, and gives a "binding.fcntl is not a function" error (as in #1016)

The only module that uses fs-ext is lock-me (https://github.com/dignifiedquire/lock-me) which is supposed to work on Windows but does not for some reason, it crashes instead of using javascript version of fs-ext.

However, fs-ext is a optional dependency, and removing it from lock-me makes both portable and unix tests on lock-me to pass.

Also, removing js-ext from node_modules in js-ipfs, makes me able to run the daemon successfully at least. Have not yet tried to add/get files though.

Code I tested with:

const ipfs = require('./src/code/index.js')
const node = new ipfs()
node.once('ready', () => console.log('ready'))
node.on('error', (err) => console.log(err))

As I usually don't develop on windows, I didn't have a dev env setup. But I found that windows-build-tools is a very useful module for setting up everything required for it to work. So after installing nodejs, running npm install --global --production windows-build-tools installs python and any .NET Framework stuff needed to develop.

JonasDev17 · 2017-10-11T16:00:09Z

C:\Program Files\nodejs\node.exe" "C:\Program Files\nodejs\node_modules\npm\bin\npm-cli.js" install ipfs

Standard error:
npm WARN deprecated [email protected]: deprecated in favor of js-ipld-ethereum https://github.com/ipld/js-ipld-ethereum
npm WARN deprecated [email protected]: 🙌 Thanks for using Babel: we recommend using babel-preset-env now: please read babeljs.io/env to update!
npm WARN lifecycle npm is using C:\Program Files\nodejs\node.exe but there is no node binary in the current PATH. Use the --scripts-prepend-node-path option to include the path for the node binary npm was executed with.
'node' is not recognized as an internal or external command,
operable program or batch file.
'node' is not recognized as an internal or external command,
operable program or batch file.
'node' is not recognized as an internal or external command,
operable program or batch file.
'node' is not recognized as an internal or external command,
operable program or batch file.
'node' is not recognized as an internal or external command,
operable program or batch file.
'node' is not recognized as an internal or external command,
operable program or batch file.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules\fs-ext):
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] install: node-gyp configure build
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: Exit status 1
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules\gc-stats):
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] install: node-pre-gyp install --fallback-to-build
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: Exit status 1

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: prebuild-install || node-gyp rebuild
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users...\AppData\Roaming\npm-cache_logs\2017-10-11T15_58_12_138Z-debug.log

Standard output:

[email protected] install C:\Users\Jonas\WebstormProjects\untitled\node_modules\fs-ext
node-gyp configure build

C:\Users...\WebstormProjects\untitled\node_modules\fs-ext>if not defined npm_config_node_gyp (node "C:\Program Files\nodejs\node_modules\npm\bin\node-gyp-bin\....\node_modules\node-gyp\bin\node-gyp.js" configure build ) else (node "" configure build )

[email protected] install C:\Users...\WebstormProjects\untitled\node_modules\gc-stats
node-pre-gyp install --fallback-to-build

[email protected] install C:\Users\Jonas\WebstormProjects\untitled\node_modules\keccak
prebuild-install || node-gyp rebuild || echo "Keccak bindings compilation fail. Pure JS implementation will be used."

C:\Users...\WebstormProjects\untitled\node_modules\keccak>if not defined npm_config_node_gyp (node "C:\Program Files\nodejs\node_modules\npm\bin\node-gyp-bin\....\node_modules\node-gyp\bin\node-gyp.js" rebuild ) else (node "" rebuild )
"Keccak bindings compilation fail. Pure JS implementation will be used."

[email protected] install C:\Users...\WebstormProjects\untitled\node_modules\leveldown
prebuild-install || node-gyp rebuild

C:\Users...\WebstormProjects\untitled\node_modules\leveldown>if not defined npm_config_node_gyp (node "C:\Program Files\nodejs\node_modules\npm\bin\node-gyp-bin\....\node_modules\node-gyp\bin\node-gyp.js" rebuild ) else (node "" rebuild )

Process finished with exit code 1

victorb · 2017-10-11T16:36:37Z

 npm WARN lifecycle npm is using C:\Program Files\nodejs\node.exe but there is no node binary in the current PATH. Use the --scripts-prepend-node-path option to include the path for the node binary npm was executed with.

and

'node' is not recognized as an internal or external command,
operable program or batch file.

makes me believe that you have not installed node correctly. You should make sure the node binary is in your $PATH variable. You can find how to do that here: https://stackoverflow.com/a/27864253/360186

mkg20001 · 2017-10-18T14:57:17Z

I finally got a windows VM setup and did some tests what works:
The gateway works after doing "jsipfs init" which crashes with Error: Invalid key: /CIQGKXXGEEETAXRWRSFJYDYW5VSDR5YLCWWGVI6N4KJ2Y6MYMKNDG6I (but "jsipfs daemon" still works fine).
After a peer connects and initializes the /ipfs/bitswap/ protocol the daemon crashes with the same error.

I suspect this error is occuring because \ or / is used in paths or path splitting where it shouldn't be used on windows. (Most likely it tries to split some string that has / with \)

I would really like help getting jsipfs to run on windows, but I don't think that I could work on that so much.

richardschneider · 2017-10-31T00:10:15Z

@dryajov I'm willing to spend some time on getting this to work on Windows. What would you like me to look at.

richardschneider · 2017-10-31T01:26:20Z

I tried installing, npm install js-ipfs, and got errors on fs-ext. It turns out that my node version (5.7.0) is the culprit. Updating it to version 8.8.1 resolved the issues and js-ipfs now installs without an error.

You can download the latest node for windows at https://nodejs.org/en/download/current/

Unfortunately, as @mkg20001 reported, I now get an Invalid Key error.

dryajov · 2017-10-31T01:50:19Z

Hi @richardschneider! Thanks so much for looking into this, I believe the next steps would be figuring out the error that both @mkg20001 and you are seeing as well as any other that pop up. We also need CI setup with this, which I'll look into next.

Keep up the awesome work @mkg20001 and @richardschneider! ❤️

richardschneider · 2017-10-31T01:55:15Z

@dryajov I recommend using AppVeyor for a CI. I've used it for both JS and C# projects to test on windows and the people are really responsive.

dryajov · 2017-10-31T02:04:58Z

👍 I believe that's what we're using already for some of our projects - check github.com/ipfs/aegir for example (that's our built/release system).

pvh · 2017-11-02T20:05:16Z

Hi guys, I've been doing some hacking on IPFS/libp2p on Windows. The ipfs-pubsub-room-demo doesn't send direct messages due to something going wrong with the direct dial. Same code works fine on my linux box. @dryajov and I did some diagnostics and isolated it to just node on Windows (example code works in the browser and on my linux machine.)

dryajov · 2017-11-02T20:13:27Z

@pvh yeah, windows support is precarious at best at this moment, we're just starting this work and so far it's quite possible that most things are broken. @richardschneider has been doing great work in getting the datastores to behave in windows, but there are quite possible more things broken.

It would help if you could provide a full trace of your runs in windows, we use the debug module so enabling the DEBUG=* env variable should yield some some pretty verbose output, if you could create a gist with that output, that would go a long way helping us troubleshoot this.

Thanks for helping us getting IPFS working on windows!

daviddias · 2017-11-03T09:02:52Z

Fixing Datastore Paths

A number of PRs have been opened to fix the lack of OS agnostic'ity of datastore. These are:

I'm going to start reviewing these. To make sure we don't break anything, I would really appreciate if someone could test it all out to js-ipfs (npm link all the things) and confirm it works on windows and that the repo is still interoperable with Go)

richardschneider · 2017-11-03T12:36:07Z

My current understanding is that repo/blocks is interoperable with Go; even with sharding!

However, repo/datastore with levelUp is not.

the-townsend · 2017-11-03T14:07:33Z

I'd be happy to help here as it would be good to get js-ipfs on windows. Happy to take on direction on where to put my efforts.

richardschneider · 2017-11-03T20:56:13Z

@the-townsend Thanks for the offer. I'm in the middle of getting the datastores to work on windows. Hopefully will be done today.

Then we can take a look at any outstanding issues.

daviddias · 2017-11-04T10:02:40Z

We will need some help on adding all the badges to all the repos with appveyor enabled. I'll be going repo by repo and enabling and listing here where there is a interop bug

daviddias · 2017-11-04T10:09:51Z

the-townsend · 2017-11-04T15:49:57Z

I have now managed to install. Details attached/linked below (*). I was using Node 9, which may have been the issue. The README for js-ipfs has 'node@6 or above'. However it could also be the change that @diasdavid made that made it work. [Note I uninstalled/deleted ''.window-build-tools'' which changed the set up issues previously, so this isn't a dependency as far as I can see.]

Unfortunately, as you can see from the attached I am getting the same issue as above coming from 'datastore' (specifically key.js); am going to try and take a look at those now.

(*)
IPFS_Window_Install.txt

richardschneider · 2017-11-04T22:15:21Z

@diasdavid We need ipfs/js-datastore-level#4 and then update js-ipfs-repo before checking the other modules.

richardschneider · 2017-11-04T22:29:39Z

@diasdavid @dignifiedquire The other modules should be run on appveyor. I'm happy to create the appveyor.yaml and do the testing but maybe one of you should create the appveyor account.

richardschneider · 2017-11-04T22:36:42Z

the-townsend · 2017-11-07T10:39:30Z

Not sure what the protocol here is.
https://github.com/kumavis/ethereumjs-ipld-dump/blob/master/src/index.js
has '/.jsipfs' lines 23. Do we edit that code and give a pullrequest to 'kumavis'?

richardschneider · 2017-11-07T10:46:38Z

If its not part of IPFS we can ignore it.

the-townsend · 2017-11-08T12:36:35Z

Can't see that anything needs doing on

ipld-cli

richardschneider · 2017-11-08T14:46:58Z

@the-townsend Thanks I'll tick it off the list

the-townsend · 2017-11-08T18:01:57Z

js-libp2p-pstn-node is not part of ipfs

richardschneider · 2017-11-08T21:17:27Z

@the-townsend js-ipfsd-ctl is not working on windows. Its blocking js-ipfs-api from being tested.

Would you mind looking at it.

the-townsend · 2017-11-09T09:10:44Z

I’ll try to later today. Any more details available? Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

…

________________________________ From: Richard Schneider <[email protected]> Sent: Wednesday, November 8, 2017 9:17:33 PM To: ipfs/js-ipfs Cc: Christopher Townsend; Mention Subject: Re: [ipfs/js-ipfs] Windows Support (#1017) @the-townsend<https://github.com/the-townsend> js-ipfs<https://github.com/ipfs/js-ipfsd-ctl> is not working on windows. Its blocking js-ipfs-api being tested. Would you mind looking at it. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#1017 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AaNA4Yg6fKuM0pqntIvRfWj6t4LveERHks5s0hpsgaJpZM4PWIhG>.

richardschneider · 2017-11-09T09:20:09Z

It's mainly a package to spinup an IPFS server. Could you get the tests running on a windows machine

the-townsend · 2017-11-09T11:09:25Z

Results attached.

27 passing (55s)
3 pending
1 failing

Can't immediately see whether this is a simple problem or not.
js-ipfsd-ctl-test-results.txt

the-townsend · 2017-11-09T20:35:15Z

It's the same error that @diasdavid got here

https://ci.appveyor.com/project/diasdavid/js-ipfsd-ctl-a9ywu/build/job/sl48bopxa3hfs9eo

richardschneider · 2017-11-10T03:23:03Z

It looks like js-ipfsd-ctl is receiving some TLC this comment

daviddias · 2017-11-10T08:55:13Z

The next big piece should be getting https://github.com/ipfs/js-ipfs-api to work on Windows too.

daviddias · 2017-11-10T19:46:28Z

Windows support is here!!! 🌟🚀🎆

With #1065

BIG THANK YOU to @richardschneider who almost single-handedly defeated all the Windows dragons 🐉 and brought Windows support to js-ipfs and its ipfs dependencies.

Let's celebrate in the next all hands! @richardschneider will you be able to join and give us a demo?

daviddias · 2017-11-10T19:53:00Z

Closing this issue. Let's track the last js dragon on js-ipfs-api at ipfs-inactive/js-ipfs-http-client#408 (not required for js-ipfs to work on Windows)

the-townsend · 2017-11-11T11:56:50Z

@diasdavid @richardschneider
FYI getting below when trying to replicate locally:
...
22 passing (2m)
1 pending
7 failing
...

[details attached]

I can't access the test results above easily to try and see what's going on.

js-ipfs-test-results.txt

barnendu · 2018-01-27T08:41:49Z

I had same issue, jsipfs was giving me error while installing. Its has been solved after using
npm install --global --production windows-build-tools in windows 10 run power shell has admin. Then
run npm install ipfs --global

This was referenced Sep 13, 2017

node js ipfs on windows #861

Closed

Win10 64b - binding.fcntl is not a function #1016

Closed

daviddias added the env:Windows label Sep 13, 2017

daviddias added the status/ready Ready to be worked label Oct 13, 2017

This was referenced Oct 18, 2017

Unable to install js-ipfs --> Windows, Webstorm #1037

Closed

Broken on Windows since interface-datastore use of os-specific path separators ipfs/js-ipfs-repo#144

Closed

daviddias added the P2 Medium: Good to have, but can wait until someone steps up label Oct 18, 2017

dryajov self-assigned this Oct 18, 2017

Beanow mentioned this issue Oct 31, 2017

feat: adding appveyor support #1054

Merged

This was referenced Oct 31, 2017

os agnostic ipfs/interface-datastore#12

Merged

Status of the js-datastore-fs ipfs/js-datastore-fs#3

Closed

richardschneider mentioned this issue Nov 3, 2017

Windows libp2p/js-libp2p-kad-dht#18

Closed

daviddias mentioned this issue Nov 3, 2017

Running on windows libp2p/js-libp2p-kad-dht#19

Closed

richardschneider mentioned this issue Nov 8, 2017

Fix error with key generation on Windows ipfs/js-ipfs-repo#148

Closed

daviddias unassigned dryajov Nov 10, 2017

daviddias closed this as completed Nov 10, 2017

ghost removed the status/ready Ready to be worked label Nov 10, 2017

daviddias mentioned this issue Nov 20, 2017

⚡️ v0.27.0 RELEASE 🚀 #1095

Closed

24 tasks

Windows Support #1017

Windows Support #1017

Comments

daviddias commented Sep 13, 2017 • edited Loading

Beanow commented Sep 20, 2017

victorb commented Sep 20, 2017 • edited Loading

JonasDev17 commented Oct 11, 2017

victorb commented Oct 11, 2017

mkg20001 commented Oct 18, 2017

richardschneider commented Oct 31, 2017

richardschneider commented Oct 31, 2017

dryajov commented Oct 31, 2017

richardschneider commented Oct 31, 2017

dryajov commented Oct 31, 2017

pvh commented Nov 2, 2017

dryajov commented Nov 2, 2017

daviddias commented Nov 3, 2017 • edited by richardschneider Loading

Fixing Datastore Paths

richardschneider commented Nov 3, 2017 • edited Loading

the-townsend commented Nov 3, 2017

richardschneider commented Nov 3, 2017

daviddias commented Nov 4, 2017

daviddias commented Nov 4, 2017 • edited Loading

the-townsend commented Nov 4, 2017

richardschneider commented Nov 4, 2017

richardschneider commented Nov 4, 2017

richardschneider commented Nov 4, 2017 • edited Loading

the-townsend commented Nov 7, 2017

richardschneider commented Nov 7, 2017

the-townsend commented Nov 8, 2017

richardschneider commented Nov 8, 2017

the-townsend commented Nov 8, 2017

richardschneider commented Nov 8, 2017 • edited Loading

the-townsend commented Nov 9, 2017 via email

richardschneider commented Nov 9, 2017

the-townsend commented Nov 9, 2017

the-townsend commented Nov 9, 2017

richardschneider commented Nov 10, 2017

daviddias commented Nov 10, 2017

daviddias commented Nov 10, 2017

Windows support is here!!! 🌟🚀🎆

BIG THANK YOU to @richardschneider who almost single-handedly defeated all the Windows dragons 🐉 and brought Windows support to js-ipfs and its ipfs dependencies.

Let's celebrate in the next all hands! @richardschneider will you be able to join and give us a demo?

daviddias commented Nov 10, 2017 • edited Loading

the-townsend commented Nov 11, 2017

barnendu commented Jan 27, 2018

daviddias commented Sep 13, 2017 •

edited

Loading

victorb commented Sep 20, 2017 •

edited

Loading

daviddias commented Nov 3, 2017 •

edited by richardschneider

Loading

richardschneider commented Nov 3, 2017 •

edited

Loading

daviddias commented Nov 4, 2017 •

edited

Loading

richardschneider commented Nov 4, 2017 •

edited

Loading

richardschneider commented Nov 8, 2017 •

edited

Loading

daviddias commented Nov 10, 2017 •

edited

Loading