Building the mumax3 from source code on windows

[Lee-89757]

Hi,
Has anyone tried building the mumax3 source code on the windows? Github only provides the way on the Linux. I tried similar way but finally failed. Has anyone succeeded?

[JonathanMaes]

Hi,

Have a look at the README.md in the 3.11 branch. I have recently updated it to include information on building on Windows, as I too needed to compile on that OS. If anything is unclear, or does not work on your machine, please let me know so I can improve the README.md further.

For additional information, see this mailing list message.

[Lee-89757]

Hi Jonathan,

Thanks for your reply. I'm sorry I didn't find the 3.11 branch. I followed the steps listed in the link you shared to me. However, when I tried the step "go get -u -v github.com/mumax/3/...", the errors appear as shown in the picture. I attached the my GOPATH and GOROOT direction in the picture.

Hi,

Have a look at the README.md in the 3.11 branch. I have recently updated it to include information on building on Windows, as I too needed to compile on that OS. If anything is unclear, or does not work on your machine, please let me know so I can improve the README.md further.

For additional information, see this mailing list message.

[JonathanMaes]

You can find the updated 3.11 README here.

go get should indeed no longer be used as your error message indicates.
The new README uses a git clone command to fetch the source code instead.

Give the README I linked to a try, and do not hesitate to reply if you encounter any other issues.

[jplauzie]

Hi,

The Readme looks good, similar to what I used. A few small things:

I believe "which" is a linux (or GitBash/Msys/Cygwin) command by default. On windows it's "where" or in powershell where.exe (or Get-command). One other quirk is that which/where will only search on your PATH, so can fail if VS isn't added to the PATH (I'm not sure if it adds itself during installation. My version does not seem to be on PATH by default, but it is a bit old at this point). So you might need to give a new user a bit more direction there. I don't know a better way than what you did, since Microsoft sometimes moves where it defaults installing VS between different versions.

The readme mentions issues with whitespaces in the Makefile. You can probably get them to work if you escape them properly (usually "\ " or "\ " if a double escape is needed). It is is probably not worth trying to fix, as it's easier to just use the powershell script. But in principle it should be doable.

(Also, welcome Jonathan! It is nice to see a fresh face and see the repo get some maintenance)

Best regards,
Josh L.

[Lee-89757]

Hi Jonathan and jplauzie,

Thank you so much for your reply and help! I followed the Readme step by step and successfully build the Mumax3 from the source code. But I still have a stupid question that might need your help. How can I change the environment variables "CGO_CFLAGS", "CGO_LDFLAGS" and "CUDA_CC"?
By the way, it is true that which will only search on my PATH and my VS didn't add itself to the PATH (my version is 2022). So before the which command, we should add the VS to the PATH.

[Lee-89757]

Hi Jonathan and jplauzie,

Sorry to bother you again. when I run the "./deploy_windows.ps1" in the powershell, the erros apears:



It seems it fails in the end.

[JonathanMaes]

Thanks for your comments both, I'll update the README with this information.

@Lee-89757 : you can edit the environment variables on Windows by following this guide, but I'm not sure if CGO_...FLAGS can easily be added as an environment variable because it contains quotes. If you use the updated deploy_windows.ps1 script on the 3.11 branch, the CGO_...FLAGS will already be set inside the script so you don't need to add them as a separate environment variable (unless I'm forgetting something here).

As for the error you're encountering now, I haven't seen that before. It seems like Go is looking for standard packages in the wrong place: I think it should look in C:/Go/src, not C:/Go/bin/src. Maybe some wrong value in PATH?

[jplauzie]

You shouldn't need to edit CGO_CFLAGS, or CGO_LDFLAGS, as they are set within the powershell script itself on line 48/49. If you wanted to edit these, you could edit them within the script directly. Or you could copy/paste those lines into powershell, they are valid powershell commands (or set them in windows, as well).

I now notice Jonathan added some extra parentheses to these lines compared to the old script, I didn't need to change those lines at all (probably a difference of where CUDA was installed. I used C:\cuda so it had no spaces, but maybe if he used the normal CUDA install location which is usually something like C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2, it has spaces so probably needs extra the parentheses). The new version should still work fine and is probably more robust.

But again, I would recommend not changing those flags, it shouldn't be necessary. Especially because other areas of the mumax source code also use these particular flags, it might mess with those if you set it globally outside the script.

CUDA_CC should be changed within the powershell script itself "deploy_windows.ps1", on line 34:
"11.0" { $CUDA_CC = 30,32,35,37,50,52,53,60,61,62,70,72,75,80 }
This should be updated to your CUDA version, and compute capability. So something like: "12.4" { $CUDA_CC = 89 }

As for the error, I think it's probably related to GOROOT. It used to default to C:\Go\ in older versions of Go, but checking my current go installation it is now pointed to C:\Go\bin for some reason (which it won't like, since standard packages will be in C:\Go\src. It won't be able to find them if it starts in C:\Go\bin. Setting GOROOT to C:\Go\ means it can find both C:\Go\bin\go.exe which is the compiler and also C:\Go\src for packages). So I would guess that's the new default GOROOT. I think this is because stuff like GOROOT was deprecated around Go 1.17 and mumax hasn't yet been updated for modules. You can set GOROOT using (in powershell before calling the script) $env:GOROOT="C:\\go" (I'm not 100% sure if you need the double \ or not. It at least did not hurt my installation ).

If/when mumax gets updated to use modules natively, this issue with GOROOT probably won't be an issue in the future.

Looking to my past installation, I did also set CUDA_PATH_V12_2 to C:\cuda explicitly (this is assuming you installed CUDA to C:\cuda, or copy/pasted it there). This should already automatically be set from installing CUDA if you told it to install at C:\cuda, but if not, you can add it in environment variables in Windows (or in powershell, by doing $env:CUDA_PATH_V12_2="C:\cuda" prior to calling the powershell script.). This just changes it from the default location (which is usually something like C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2) to C:\cuda. This might be unnecessary.

[Lee-89757]

Hi Jonathan and jplauzie,

Thank you so much for your help and patient explanation! I changed my GOROOT PATH to C:\Go and the former errors seem to be fixed. However, unfortunately, new errors appear. The following picture shows the errors. Does that mean it didn't find the compiler? But I did set the location of cl.exe in the $CCBIN. The second pucture shows the setting of my CCBIN.

[jplauzie]

I haven't seen that cuda2go error before. My guess is it's because you're building in C:\go-work\src, instead of C:\Go\src . If your Go installation is in C:\Go, and your GOROOT is set there, it is easier if you try to build there as well. (In principle, Go is designed so you should be able to build anywhere, but it requires setting various PATH/variables correctly. I've found it's always a pain if I try to use a custom directory like that).

This all assumes your Go installation is in C:\Go, though. If you installed it in a custom place like C:\go-work\ instead, you might have to tweak your path variables (so GOROOT might need to be C:\go-work instead of C:\Go, etc). This might also include setting GOPATH to be consistent as well. It's hard to tell, since your go env variables are mixed between both places.

The issue does not look related to CCBIN, it's with line to build cuda2go (line 55). It seems like it can't find the Go compile tool to build cuda2go (Which is different from the cl.exe C compiler needed for NVCC, the CUDA compiler. CCBIN is for cl.exe). When I google the error "go: no such tool as "compile"", people with that similar issue seems to be related to GOPATH/GOROOT being set improperly as well.

So I think you need to tweak your GOROOT/GOPATH, but I'm not 100% sure.

[Lee-89757]

Hi Jonathan and jplauzie,

I seemed to fix the problem. It is because I made a very stupid mistake. I download two versions of GO! It seems I use the old version of go.exe to search for the new version of compile file. Then it meets the problem. After I deleting the old version of GO, these problems are fixed.
However, when I thought things were getting better, new errors apear. It seems the compiling process is working. But there's a trouble in the final step, and then it fails, as shown in the picture.

[jplauzie]

Hi,

If you made it that far without other errors, then the executable has been made. You just need to find it. I would check your different folders manually. It is probably in either C:\go-work\bin or C:\go\bin (or it could also be in C:\Users\username\go\bin) , but it should be somewhere. My guess it is is in C:\Go\bin because your GOBIN variable is misconfigured due to your 2 conflicting installations, but the deploy script is looking in C:\go-work\bin (because that is your new GOPATH/bin, which does not match GOBIN), similar to how GOROOT was misconfigured. Normally one doesn't need to set GOBIN, it's set automatically.

But these last steps are just copying the already made executable to the current folder. They're not important.

If you still can't find it manually, probably something in your go env is still misconfigured. Things get weird when you have multiple conflicting go installations, and you'll have to go triple-check GOPATH,GOROOT, etc.

[Lee-89757]

Hi jplauzie,

Thank you very much for your reply! I searched in my C:\ and it seems mumax3.exe does not exist.

And for the line in the last comment picture, mabe it is an error? But I don't know how does this happen.

Hi,

If you made it that far without other errors, then the executable has been made. You just need to find it. I would check your different folders manually. It is probably in either C:\go-work\bin or C:\go\bin (or it could also be in C:\Users\username\go\bin) , but it should be somewhere. My guess it is is in C:\Go\bin because your GOBIN variable is misconfigured due to your 2 conflicting installations, but the deploy script is looking in C:\go-work\bin (because that is your new GOPATH/bin, which does not match GOBIN), similar to how GOROOT was misconfigured. Normally one doesn't need to set GOBIN, it's set automatically.

But these last steps are just copying the already made executable to the current folder. They're not important.

If you still can't find it manually, probably something in your go env is still misconfigured. Things get weird when you have multiple conflicting go installations, and you'll have to go triple-check GOPATH,GOROOT, etc.

[jplauzie]

Hi,

I haven't seen that error before. Googling suggests it's either an issue with your go.mod file generated when you do go mod install/ go mod tidy (important: You should delete this file and regenerate it after tweaking your go installation. If there is an existing go.mod file, go will simply reuse the existing one when you re-run the 2 go mod commands listed in the readme steps. This is particularly important in your case, as your go.mod file might point towards your previous incorrect go install). Or it happens if you improperly changed something like the CGOFLAGS or LDFLAGS.

Basically, it's telling you that you set something so that it shouldn't compile the go files in that folder, even if they're there. If deleting and remaking the go.mod file doesn't work, I would try to do a fresh installation, and set everything from scratch as much as possible. I can't tell what exactly is causing the error, but it is an error that is caused by something being misconfigured.

[Lee-89757]

Hi jplauzie and Jonathan,

I finally succeeded in building mumax3!!! Thank you so much for your patient explanation and help!

The errors appear due to the wrong setting of the enviromental variables "CGO_ENABLED" . The default setting is 0 but we should set it to 1 to enable the c compiling. I think this information might should be included in README file.

Another thing is that it seems gcc is also needed in the compiling besides cl.exe. I've downloaded the VS and put the cl.exe in the PATH but the system still says gcc is needed. Then I downloaded gcc and finally succeeded.

Best wishes
Lee

[jplauzie]

Hi,

Glad to hear you got it! "CGO_Enabled" should be 1 by default, so I'm not sure why it was turned off.

Go itself does use gcc, but it also comes bundled with Go. Since it seems you had some nonstandard Go settings, perhaps something like GCCGO or CC was not set. CL is just used for the Cuda drivers, NVCC specifically prefers it.

Cheers,
Josh L.

[Lee-89757]

I'm not sure. I reinstalled the go before and kept everything as the default setting. Maybe it happens occasionally

Hi,

Glad to hear you got it! "CGO_Enabled" should be 1 by default, so I'm not sure why it was turned off.

Go itself does use gcc, but it also comes bundled with Go. Since it seems you had some nonstandard Go settings, perhaps something like GCCGO or CC was not set. CL is just used for the Cuda drivers, NVCC specifically prefers it.

Cheers, Josh L.