[help] build healpy on Windows

[zonca]

should we make available an exe packages for Windows users?
is anybody willing to help?

[mreineck]

You mean, should we support building Windows DLLs?
As far as the C++ part is concerned, this should work pretty smoothly if the Cygwin package is installed..
I don't have any experience with Intel and Microsoft compilers on Windows.

[zonca]

ok, I don't have experience with Windows compilers either, let's see if somebody that does is willing to collaborate on this. We'd be happy to support the effort.

[zonca]

we might consider using conda

[lpsinger]

I have temporary access to a Windows box at work now. I got as far as @ARepp. However, I think that it is going to be very hard to build the bundled healpix-cxx and cfitsio sources because there is no easy way to use Anaconda with Cygwin or MSYS.

I think that if we want to support Windows, then we must go back to using setuptools itself to build the libraries instead of running ./configure && make && make install. We could still use pkg-config/pykg-config to attempt to build and link against pre-build libraries.

A few questions:

1. Do we need to support Windows?
2. Do we need to support building and linking against pre-built copies of healpix-cxx and cfitsio on Windows?
3. Do we need to support OpenMP acceleration when building the bundled healpix-cxx sources?

[lpsinger]

Oh---another question. Does healpy really have to link against cfitsio, or can we get all of the FITS functionality we need through pyfits/astropy at the Python level?

[crosset]

Hi,

Only healpix_cxx needs to link against cfitsio.

Healpy uses healpix_data_io.h for read_pixwin() and read_weight_ring() functions (which read the window and weights data files), so if healpix_cxx is linked against cfitsio, it should be enough. The include file healpix_data_io.h also doesn’t includes fitsio.h itself.

read_map() and family are only using pyfits/astropy.

Cyrille.

Le 27 avr. 2015 à 17:58, Leo Singer [email protected] a écrit :

Oh---another question. Does healpy really have to link against cfitsio, or can we get all of the FITS functionality we need through pyfits/astropy at the Python level?

—
Reply to this email directly or view it on GitHub #25 (comment).

[mikesperry]

I'd be really pleased to see Healpy running on Windows. Many planetariums run on Windows, and an API for python is becoming more common. If needed, I have a number of Windows XP or Windows 8 machines I can test on.
--Mike

[astroOcean]

I wanna ask that does the bug (environment can only contain strings) in installing healpy on windows be fixed? I try install healpy1.9.1 on my windows pc, but it has error that environment can only contain strings. The version of my python is 2.7.11
Any help is appreciative.

[ARepp]

Hi there—I was never able to get it to work, so I ended up using another computer. Thus I don’t know if it’s been fixed or not.

Thanks,

Andrew

From: astroOcean [mailto:[email protected]]
Sent: Friday, April 29, 2016 5:06 PM
To: healpy/healpy [email protected]
Cc: Andrew Repp [email protected]; Mention [email protected]
Subject: Re: [healpy/healpy] [help] build healpy on Windows (#25)

I wanna ask that does the bug (environment can only contain strings) in installing healpy on windows be fixed? I try install healpy1.9.1 on my windows pc, but it has error that environment can only contain strings. The version of my python is 2.7.11
Any help is appreciative.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub #25 (comment)

[astroOcean]

Thanks for your reply. Anyway, I can use it on my Linux system.

------------------ Original ------------------
From: "Andrew Repp";
Date: 2016年5月1日(星期天) 凌晨5:26
To: "healpy/healpy";
Cc: "astroOcean"; "Comment";
Subject: Re: [healpy/healpy] [help] build healpy on Windows (#25)

Hi there—I was never able to get it to work, so I ended up using another computer. Thus I don’t know if it’s been fixed or not.

Thanks,

Andrew

From: astroOcean [mailto:[email protected]]
Sent: Friday, April 29, 2016 5:06 PM
To: healpy/healpy [email protected]
Cc: Andrew Repp [email protected]; Mention [email protected]
Subject: Re: [healpy/healpy] [help] build healpy on Windows (#25)

I wanna ask that does the bug (environment can only contain strings) in installing healpy on windows be fixed? I try install healpy1.9.1 on my windows pc, but it has error that environment can only contain strings. The version of my python is 2.7.11
Any help is appreciative.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub <" target="_blank">https://github.com/healpy/healpy/issues/25#issuecomment-215928213>

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub

[Grillard]

Hello,

So I manage to build healpy in Windows 10 using msys2. I have not tested if it is fully functional, but as far as I am concerned it is working great for me, so I thought I should share how I did it.

I had to hack a few things, but perhaps I am over-complicating things and I would not be surprised if I am doing something awful or unnecessary in the middle. Some of the steps I did may be fixed from the setup.py, I think. But anyway, this is what I did:

1. Install Msys2. After the initial update as instructed in their guidelines, I installed gcc, make, cfitsio and other essentials packages with the following command:
   pacman -S --needed mingw-w64-x86_64-toolchain git subversion mercurial mingw-w64-x86_64-cmake mingw-w64-x86_64-cfitsio
2. Add Msys2 mingw64 folder to the path.
3. Download the latest version of Healpix and put it somewhere in the mingw64 folder of Msys2.
4. From the mingw64 console provided with Msys2, install healpix c++ version following the linux instructions. Note that you will need to link to the previously installed cfitsio library, which should be in /mingw64/include/cfitsio. When prompted for compiler, pick generic gcc
5. Create the healpix_cxx.pc pkg-config file for healpix and place it in mingw64/lib/pkgconfig/ within the msys2 main folder (for some reason this is not created in the installed process even when the documentation said it should)
   Anyway, I am no expert in creating these files, and I created this one by hand, it works, but it might include some unnecessary things. The file looks like this:

prefix=/mingw64
libdir=${prefix}/Healpix_3.31/src/cxx/generic_gcc/lib
bindir=${prefix}/Healpix_3.31/src/cxx/generic_gcc/bin
includedir=${prefix}/Healpix_3.31/src/cxx/generic_gcc/include

Name: healpix
Description: HealPix
Version: 3.31
Libs: -L${libdir} -lhealpix_cxx -lcxxsupport -lsharp -lfftpack -lc_utils -lcfitsio -fopenmp
Libs.private: -lm -lz
Cflags: -I${includedir}

6. Tell python to use the mingw compiler by creating the disutils.cfg file in Lib/disutils in the python main directory, and fill it with

[build]
compiler=mingw32
[build_ext]
compiler=mingw32

This is where it gets non ideal, since I modified essential files in python, so if you know of a better alternative please let me know!

7. There is a problem with using ::hypot in the cmath package when called from python. I fixed this by including math.h in inlcude/pyconfig.h in the third line. I think it is possible to solve this by adding the math package before calling python.h in healpy/src/_healpy_pixel_lib.cc, but I couldn't test this.

8. The next issue arises from sysconfig.get_config_var("CC"), which returns None. In this case it should return mingw32. Since I do not know how to fix it, I added a catch in unixcompiler.py to set the compiler to mingw32 when it fails to get it from sysconfig (search for the line that contain the sysconfig.get_config_var("CC")). Aditionally, mingw32 needs to be recongnized as a gcc compiler, so one also needs to add or compiler_name=='mingw32 in the _is_gcc function in the same file. This is super ugly and I do not like it...

9. Next, there is an issue arising from python not recognizing the g++ call as a 64 bit one with compiling. The specific error is undefined reference to __imp_Py_InitModule4 . To solve this, we need to tell g++ to use the Msys64 dependence when calling gcc. I did this by adding -D MS_WIN64 in the set_sexecutables variable from the Mingw32CCompiler in the /lib/disutils/cygwincompiler.py python file, specifically in the compiler, compiler_so and compiler_cxx variables. I am fine with this modificiation, since I will only have x64 compiler in my system, but I think this very very far from ideal... I believe this ugly hack can be avoided by setting the -D MS_WIN64 in the healpy setup.py, but again I am not sure...

Well, that is it! next you can just run pip install healpy and it should work.

Disclaimer: I am no expert and I probably do not know what I am doing, so please let me know if something is wrong here! Just wanted to comment that one of the things that bugs me the most is that I link a x64 compiler to the mingw32 compiler variable in python. Although this is apparently the way it should be done according to the wiki

Hope this helps!

[zonca]

thanks @Grillard , it would be a great contribution to the project if you can package this into a conda recipe and make a Pull Request to our repository for creating conda packages at https://github.com/conda-forge/healpy-feedstock

This way a conda package for windows could be automatically built for healpy.

[cmbant]

I had some spare budget that could be used to pay someone a $300 incentive to get healpy conda-forge version working for Windows (but needs to be in the next two months).

[mreineck]

Not sure how relevant this is for the task at hand, but I'm currently assembling a package that contains most of my C++ codes (support library, pocketfft, libsharp, healpix_cxx and more) which has the following characteristics:

• C++ only (using features up to C++17)
• no external dependencies at all (except pybind11 for the Python interface, which is a source-only dependency)
• compilable with g++, clang++, icpc and (as long as someone tests it for me) Microsoft compilers. (@ziotom78 has reported successful compilation on Windows 10.)
• does only computation (i.e. no I/O, all I/O needs to be done on the calling side)
• no OpenMP (has been replaced by standard C++ threading)
• new classes for array handling which are more flexible about data ownership (i.e. it is now much easier to work on data that is owned by the caller, no more ugly hacks like the one we are currently using in the arr class).
• optional Python interface based on pybind11

Current state is available at https://gitlab.mpcdf.mpg.de/mtr/cxxbase.

This may be a good back end for healpy to use once it is finished, but of course it's still far from complete and healpy will need quite a lot of internal adjustments. In the longer term I think it's the way to go, however.

[Edit: remove icpc from the list, it doesn't have sufficient C++17 support yet.]

[zonca]

Yes, this looks like a good option!

[joezuntz]

Due to the pandemic many of us have project students working from home on Windows machines on projects needing healpy. I was able to get it installed via the Windows Subsystem for Linux, and available in a jupyter notebook.

In case useful to others I've put instructions below.

1. Restart your computer, and follow the instructions (which appear before windows starts) to enter BIOS.
   Usually this means pressing DEL or F2 just after powering on.
   Find the option to enable virtualization (exact name will depend on your system, can google your machine brand name + "enable virtualization" for instructions)

2. Follow these instructions to install Windows Subsystem for Linux:
   https://docs.microsoft.com/en-us/windows/wsl/install-win10
   Following the instructions for WSL version 2, and choosing Ubuntu from the store.

3. Restart machine

4. Open the newly installed Ubuntu application from the Start menu and follow the setup instructions.

5. When they are complete, run these commands:

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install python3 python3-pip

6. Quit ubuntu, restart it, and run:

pip3 install numpy jupyter matplotlib healpy ipython jupyter

7. Quit ubuntu again, restart it, and run:
   ipython notebook --no-browser

8. Copy and paste the line starting
   http://localhost:8888/?token=
   into your normal windows web browser.

[zonca]

thanks @joezuntz! this is very helpful, I added instructions to https://github.com/healpy/healpy/blob/master/INSTALL.rst

[KingwithQueen]

I would appreciate windows version of this. I can get it working in virtual environment, but I am struggling to use other components on virtual environment. :)

Hello, I want to ask you what you did after seeing Healpy in your browser? I did not find the Healpy folder in the browser according to the above steps, but I found it in other ways. I copy it directly to the corresponding directory of the python installed in Windows. But when import healpy, the _ int_.py function will report an error.

[ASainio]

Still hoping healpy that supports windows. There are so many astro-related packages that rely on healpy and I can't use them.

[DaftPict]

Also looking for a windows version - I tried the WSL instructions but they didn't work for me.
I'll try downloading the source code and have a look.

[cmbant]

DUCC by @mreineck does work well on Windows, also (using that) lenspyx including lensed sky simulation. This works fine for transforms (and very fast), but visualizationa and some other functions still require healpy. WSL should certainly work fine though.

[DaftPict]

Thanks for the info - I do need healpy though as it is a dependency of the package I need for a collaborative pipeline.

[Angrite]

I also managed to build healpy on Windows 10 using latest MSYS2 in UCRT64 environment:

after I got MSYS2 up and running on Windows 10 (following MSYS2 docs), I installed requirements in UCRT64 environment:

• build tools: pacman -S --needed base-devel mingw-w64-ucrt-x86_64-gcc mingw-w64-ucrt-x86_64-toolchain git subversion mercurial mingw-w64-ucrt-x86_64-cmake
• cfitsio: pacman -S mingw-w64-ucrt-x86_64-cfitsio
• scipy: pacman -S mingw-w64-ucrt-x86_64-python-scipy
• astropy: build from source git clone ...
• other dependent packages were installed via pip install ... or SETUPTOOLS_USE_DISTUTILS=stdlib pip3 install ....

Then, I installed healpix 3.82 C++ version following docs, copied lib/ and include/ contents to /ucrt64/lib/ and /ucrt64/include/
And finally, I got healpy 1.16.6 installed: SETUPTOOLS_USE_DISTUTILS=stdlib pip3 install healpy.

Non-trivial task, but manageable.

[lpsinger]

https://github.com/msys2/setup-msys2 might help to accomplish this in GitHub Actions.

[lpsinger]

vcpkg supports building autoconf/automake libraries. @dakota002, would you try that please?

[tieneupin]

I also managed to build healpy on Windows 10 using latest MSYS2 in UCRT64 environment:

after I got MSYS2 up and running on Windows 10 (following MSYS2 docs), I installed requirements in UCRT64 environment:

• build tools: pacman -S --needed base-devel mingw-w64-ucrt-x86_64-gcc mingw-w64-ucrt-x86_64-toolchain git subversion mercurial mingw-w64-ucrt-x86_64-cmake
• cfitsio: pacman -S mingw-w64-ucrt-x86_64-cfitsio
• scipy: pacman -S mingw-w64-ucrt-x86_64-python-scipy
• astropy: build from source git clone ...
• other dependent packages were installed via pip install ... or SETUPTOOLS_USE_DISTUTILS=stdlib pip3 install ....

Then, I installed healpix 3.82 C++ version following docs, copied lib/ and include/ contents to /ucrt64/lib/ and /ucrt64/include/ And finally, I got healpy 1.16.6 installed: SETUPTOOLS_USE_DISTUTILS=stdlib pip3 install healpy.

Non-trivial task, but manageable.

@Angrite , I saw your post, and was interested in trying to reproduce your build of healpy for Windows 10. I've been able to follow everything up until building astropy from source, where I started encountering issues. What version of astropy did you use, and what commands did you run in MSYS2 in order to build it?

At present, I've tried using python -m pip install -ve . --no-build-isolation --config-settings editable-verbose=true, which the astropy docs recommended, but found that there were key errors and missing files while going through the build process.