Windows Build/Test Data #392
Comments
|
Hi John,
|
|
Here is the link to the monstree images: |
|
Hi Fabien, Thanks for the info. I'll try that shortly. Here are the issues that came up while building for windows. As a primer, I'm using the alicevision branch of vcpkg. Also, I'm trying to build the x64-windows-static triplet (I believe the default is x86-windows). Personally, I try to statically link everything and I think that caused a few issues. Here is my cmake command line for reference: cmake -DCMAKE_TOOLCHAIN_FILE=D:/Projects/AliceVision_v3/vcpkg-alicevision_master/scripts/buildsystems/vcpkg.cmake -DVCPKG_TARGET_TRIPLET=x64-windows-static -DALICEVISION_USE_CUDA=ON -DALICEVISION_USE_OPENCV=ON -DBOOST_NO_CXX11=1 -DCMAKE_INSTALL_PREFIX=../install ../AliceVision -G "Visual Studio 15 2017 Win64" -T v140 The first issue I ran into was link library paths. All the vcpkg libraries were in this directory for release mode: And a debug\ subfolder should have all the other debug versions.
For some reason, the compiler can't figure out what an EdgeMapAlias is, so to fix it just replace with the line: This one seems like a compiler bug, but fortunately there is a simple workaround. That was for linking. A few more issues came up relating to the install script.
#include "dependencies/vectorGraphics/svgDrawer.hpp" When doing a make install and including this file, an error occurs because the "dependencies" directory is actually called "aliceVision_dependencies" in the install location.
#include "ClpSimplex.hpp" But those files are not in the same directory. To work around it you have to add the following include search paths, even though I was only using the aliceVision classes. It's a lower priority, but it would be much cleaner if the user didn't have to add include search directories for internal files inside the library. Thanks for the help, and let me know if you need some extra testing. John |
|
Hi, I'm having issues trying to build on Windows :
No errors during .sln generation. Errors during io.cpp compilation, and seems to be related to OpenImageIO simd.h : Any idea what can cause the problem ? Thanks. |
|
Hi Paul, Can you go to one of your projects (such as aliceVision_dataio), and show the preprocessor includes? Right click the project, Properties, then Configuration Properties->C/C++/Preprocessor. Then set configuration to Release and platform to x64 (or x86) if you're building that. The copy and paste the Preprocessor Definitions? Mine look like this, but I hacked them a bit so they won't be exact. WIN32;_WINDOWS;NDEBUG;OIIO_STATIC_BUILD;NOMINMAX;_USE_MATH_DEFINES;SSE2;SSE3;SSSE3;SSE4_1;SSE4_2;AVX;FMA;BMI2;AVX2;BOOST_ALL_NO_LIB;BOOST_ALL_DYN_LINK;BOOST_NO_CXX11_SCOPED_ENUMS;EIGEN_DONT_ALIGN_STATICALLY=1;EIGEN_DONT_VECTORIZE=1;GLOG_NO_ABBREVIATED_SEVERITIES;CMAKE_INTDIR="Release";%(PreprocessorDefinitions) In my code in simd.h, it looks like this, and the #if is false. So one way that your configuration is different from mine is that somehow you are getting OIIO_SIMD_AVX >= 512 to be true. #if OIIO_SIMD_AVX >= 512 Do you have AVX512F defined somewhere? And if you hover over the code in visual studio, does it tell you what the value of OIIO_SIMD_AVX is? I have OIIO_SIMD_AVX=2. John |
|
Oops, markdown commenting changed the underscores in my post. There should be two underscores before and after AVX512F. As in |
|
Hi @johnhable thanks for your help, My Preprocessor definitions look like this (x64 Release): I'll try to remove the AVX512 definitions. |
|
Update : removing Most of the sln projects compiled, I still have some errors about cuda, like this one, seems like a version conflict with the cuda toolkit I installed : i'm also getting an undefined variable, maybe hardcoded path to test images ? : Thanks. |
|
It's surprising because there is a |
|
Hi Paul, Cool, that's good to see. For CUDA, I've got version 9.0. It looks like you are linking to the wrong version of the cuda libraries. Also, make sure that you are linking to the Visual Studio 15 version, which is v140. IIRC a value of 1600 means you are trying to link with VS 2012 cuda libs. John |
|
Hi, Don't know why MSVC 2017 -v140 is not linking to the correct Cuda libs, will investigate, and try to build popSIFT too. Thanks. |
|
Finally managed to build on Windows, quick list of the problems I encountered :
Will now test in meshroom, and try to build with openCV/popSift. |
|
Hey paul, once you have a working build with all hardware acceleration features can you publish the binaries somewhere? I would highly recommend that if the team wants their software to succeed, publishing a nightly x64 windows and linux binary should be a goal, much like other software kits like VLC and Mozilla. There are dozens of people I know personally looking to test this software that simply cannot expend the resources to compile on their own. Publishing a nightly as well insures people aren't locked on an older version - it's extremely common for someone to latch onto the first binary available and simply use that as none other exists - even if significant improvements to the product have been made in source. There are hundreds of thousands of impressive and groundbreaking computer vision software packages that languish in obscurity due to simply there being no binary available. I know of no less than 5 real-time dense mesh reconstruction algorithms from video that have been published and demonstrated that some would consider the "holy grail" of consumer 3d scanning that simply faded away into vaporware despite the code being freely available. |
|
Hi Lhun, No problem, If I manage to build everything, I'll try to package it for windows x64.
|
|
FWIW I need to second Lhun's statement. While it would be great to have a nightly build, I would definitely settle for an occasional binary release, even if it's only once per year. Going through this build process is a non-starter for most people. This is true for many open source projects. I.e. I don't know many people on windows that compile OpenCV from source. I've done it, which is why I'm able to figure out some of these problems, but most people don't. Also, for windows, there needs to be 4 versions, for each of the MSVC runtime libraries. It's a pain, but it's something we all have to deal with. IMO, I'd estimate that 90% of users would be happy with 64bit Visual Studio 2015 (v140) binaries. A full build for that configuration would massively help AliceVision adoption. Even if we don't have a completely solid build process, duct-taping a single build together and putting it online would let most users get started. John |
|
Hi, @Lhun and @johnhable, we fully agree with you, providing binaries for each platform is an important step for the project but it takes time to setup.
Recommandations or help on these aspects would be much appreciated. |
|
Hi Fabien, Here is what I would recommend:
Having a proper automated build platform would be a great long term solution, but it's going to be a long time before any solution is in place. So a "good enough" solution for the vast majority of users is for one person to do a manual build and post it. A manual build could be uploaded tomorrow, as opposed to the automatic build which is weeks/months/years away from being ready. John |
|
Yeah, I was trying to build AliceVision for our art team to evaluate it in our pipeline, even having a version of the binaries months old would be better than nothing. I can't really justify spending more than a day trying to get it to build so will just have to wait for the binaries to come out. Not hate'n , just giving feedback :) |
|
OpenMVG gives some binaries for windows on the release page (https://github.com/openMVG/openMVG/releases/tag/v1.3). |
|
PS: I work for a team at Microsoft and if we could get some hand made binaries to eval the system and decide it's worth further investigation I'm sure we could wrangle some dev time to help out with the auto build process. |
|
Thanks for the message, that would be great! We are still working on it and made good progress. Windows and Linux binaries will be released in 2 weeks. |
|
Hi, just checking in. How has progress been going on the binaries? Asking for a friend. Thanks! John |
|
To make life simpler for the team, I suggest the following:
|
|
Hi Zelijko, Have to disagree. In order for a library to be useful to the developer community, the full set of MT/MD/MTd/MDd needs to be supported on windows. It's a pain, but it's something we all have to deal with. John |
|
It's a pain, but, IMO, it's also something that you / the "developer community" chooses to deal with. Compiling with /MD and disabled optimizations produces code that is no harder to debug than the code compiled with /MDd. I've been developing for Windows only for years now and found that what I suggested is the most sane approach. I don't care about debuggability of 3rd-party dependencies (after all, a 3rd-party dependency is a black box assumed to work correctly), so I compile them with full optimizations with the added benefit that they run at full speed. Then, the only way I can use them is to use only /MD for my code... with optimizations turned on or off. The project set up got way simpler (no release/debug variants to link with), it's possible to set up unix-like bin, include, lib hierarchy, check it in in a version control and have a build environment ready, on a fresh developer machine, in a matter of minutes. (Something is lost when not using /MDd… like debuggability of iterators and heap pointers. That alone makes code heavily based on iterators run 2x slower.. or more if you're dealing with But to each his own. |
|
PS: so what I think AliceVision team should do is to
It's a lot of work for the team, but it's a matter of minutes for anyone else to set up the build environment. (Yes, I did once such an endeavor for OpenMVG. Took me 2-3 days. The worst dependency to build was HDF5 as it didn't compile cleanly, but now the project has switched Alembic storage to default to Ogawa format, so HDF5 is not required anymore unless you need to read "legacy" files.) Oh, and a tip for the team: be sure to DISABLE whole-program optimizations. It can generate invalid code even with rather new MSVC (15.7.x). |
|
As for why support only DLLs: because a module (DLL) is an isolation boundary and aids in managing dependencies. If component A.DLL needs LIBv1.DLL and B.DLL needs LIBv2.DLL you can load all of them into the same address space without conflicts, even if there are duplicate symbols in LIBv1.DLL and LIBv2.DLL (because A imports symbols only from LIBv1, etc.). But if you have static libraries A.LIB, B.LIB, LIBv1.LIB and LIBv2.LIB, you'll get linker errors due to symbol clashes between LIBv1 and LIBv2 when trying to link them into the same program. So, IMO, it's pointless to build anything meant for public consumption as a static library. EDIT: only Windows DLLs have the benefit of acting as an isolation boundary. Linux/ELF with its global symbol table search is broken in this respect, see the 1st answer here and concrete examples here https://holtstrom.com/michael/blog/post/437/Shared-Library-Symbol-Conflicts.html |
"setup appveyor to release windows binaries from the develop branch" #392 (comment) Save auto builds to https://ci.appveyor.com/project/AliceVision/alicevision/build/artifacts for download to test new features. https://www.appveyor.com/docs/packaging-artifacts/
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
Hi!
Do you have any test data for the pipeline steps? After quite a bit of tweaking I've been able to build the main AliceVision (not Meshroom) project on windows. I'm currently in the process of trying to run the whole pipeline but I'm unable to get the alicevision_incrementalSfM or alicevision_globalSfM commands to work. I'm a bit baffled because it could be something wrong with my sample data, my params, or the actual build might be broken on my end.
Since all the pipeline steps can be run from the command line, would it be possible to include some known good test data? The ideal would be to post some images (such as the tree from the Meshroom video) as well as the command line for each of the nodes. Then we can just run the script and that would give us a simple way to verify that all the pipeline steps work.
Also, I was able to build Windows, but I had to make a lot of manual fixes, especially to make it work as a static library (MT/MTd). Has anyone actually gone through this process yet? If not, would it help to post the issues I ran into?
Thanks!
John
The text was updated successfully, but these errors were encountered: