Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generating Windows installer #1

Closed
Archomeda opened this issue Jul 25, 2015 · 6 comments
Closed

Generating Windows installer #1

Archomeda opened this issue Jul 25, 2015 · 6 comments

Comments

@Archomeda
Copy link

Hey there!

I noticed that you based your installers on the changes in my fork. Looks great! In that fork, I only changed some stuff myself to see if I could let my semi-private Jenkins automatically build the installer before Windows 10 will be released (the desktop duplication feature looks very promising, as I'm currently still on Windows 7), but I didn't expect you to take it and improve it.

If I understand correctly, you're building the project for Windows on Linux? Or are you just preparing and packaging stuff on Linux, but building on Windows instead? Either way, I made it working to let it fully build on Windows, although it was a pain. In case you want to try it out, here are my scripts that I'm using on my Jenkins:

  1. Preparing build-vars.prf in a suboptimal way (PowerShell) (I'm not even sure why this needs to be created, it seems that some variables are not even used with qmake):

    cd Software
    
    'win32: {' | Out-File 'build-vars.prf' -Encoding Default
    '    QTDIR = c:/Qt/5.5/msvc2013' | Out-File 'build-vars.prf' -Append -Encoding Default
    '    MINGW_RUNTIME_DIR = c:/MinGW/msys/1.0/bin' | Out-File 'build-vars.prf' -Append -Encoding Default
    '    WIN81_SDK_DIR = c:/Program Files (x86)/Windows Kits/8.1' | Out-File 'build-vars.prf' -Append -Encoding Default
    '}' | Out-File 'build-vars.prf' -Append -Encoding Default
    
    qmake -tp vc -r .
  2. Build Software\Lightpack.sln with VS2013.

  3. Copy the output to bin, windeployqt it, package it and create an installer as well (also Powershell):

    mkdir bin
    Copy-Item Software/src/bin/*.exe bin
    Copy-Item Software/src/bin/*.dll bin
    windeployqt bin/Prismatik.exe
    7za a "artifacts/$env:JOB_NAME $env:BUILD_DISPLAY_NAME.zip" "./bin/*"
    
    Copy-Item bin/* Software/dist_windows/content -recurse
    ISCC Software/dist_windows/script.iss
    7za a "artifacts/$env:JOB_NAME $env:BUILD_DISPLAY_NAME Setup.zip" "./Software/dist_windows/Output/*"

Everything that's in the scripts above can definitely be improved, but I just want to show you how I've made it working. Especially windeployqt, it can copy the Qt dependencies very easily without you having to copy the Qt DLLs every time manually or with a custom script. Sadly, this also adds some other DLLs that I have no clue about what they do (like D3Dcompiler_47.dll, libEGL.dll, libGLESV2.dll and opengl32sw.dll), but they seemed to be needed on my installation. I have very little knowledge about Qt, but it seems to be dependent on which OS or environment you're building Prismatik. That said, I'm currently successfully running my version of the installer on my Windows 7 installation from my fork, not sure about yours though, I haven't tried it.

On a similar note, seeing that you pushed a couple of "trigger CI" commits, how are you handling CI? If you're interested, seeing that the official version is not really maintained anymore, what do you think of enabling some public CI services like Travis (Linux) and AppVeyor (Windows)? I have some experience with AppVeyor myself, and you can automatically create and upload artifacts during builds.

@psieg
Copy link
Owner

psieg commented Jul 25, 2015

Hi!

I stumbled upon your changes while trying to setup (indeed) CI for the project before releasing any installers. Unfortunately, it seems we did some duplicate work. I too run a Jenkins which is why i created all the scripts in the Software/scripts directory. Due to limitations in qmake, some additional steps are necessary to get everything to build correctly. Using one of the free-for-opensource CI systems would be an option I'd consider if there were more active development = more need of CI performance. If you want to contribute I can create an account on my instance. Feel free to tell me what you think. (Setting GitHub status on the commits doesn't seem to work correctly)

For windows build, I do have a Windows VM as slave which runs the .sh scripts through MinGW and build using VS2013. I'm not entirely sure about inclusion of the libraries you named either, but on the systems I had available for testing they were not required (Prismatik does not run any graphics rendering [*gl*.dll] that would need shaders [D3Dcompiler]).

About build-vars.prf: I know QTDIR and QMAKE_TARGET.arch are important, not sure about the rest.

@Archomeda
Copy link
Author

Your scripts go way beyond my knowledge (currently at least). As I've said earlier, I've very little knowledge about Qt. I haven't tested anything related to DirectX injection. It didn't seem to work on the official application when I tried using it. So instead I'm using different profiles depending on what I'm doing, all based on the Windows API grabber or whatever it's called (different FPS settings because Aero). But if statically linking is needed for that part, then I guess you do have to do some .sln patching.

Have you tried using windeployqt? That command seems to check the binary and copies every dependency to the same directory. This command also copies the DLLs I mentioned in my case. But it also might be some screw up on my end, I don't know. At least it copies various files that I otherwise had to hunt for.

Hmm, I try to run as many things as possible on external CI services, even if I'm the only contributor to my own project. But each their own, I respect that. I don't mind contributing and such, but do I need an account on your Jenkins? Currently I have no idea what I can contribute, except for making a better structure in the project (and a better Lightpack.pro file, etc.). I hate having to work with messy and outdated documentation, scripts, etc. This is definitely not your fault though.

A Windows slave is what I'm using as well. It's actually a VM that's managed with a modified plugin that boots it up, and shuts it down and reverts it to a previous state afterwards. I really can't afford to have a Windows VM or physical pc running all the time.

It's weird that your status is stuck for PRs. I can't help you with that, I tried to link a project to my Jenkins, and that seems to work. But yeah, it's Jenkins. In my opinion it's one messy software that needs to be built up from the ground again.

I'm actually quite sad that Woodenshark has no helpful compilation documentation or just generic info about anything. When you release something to the public, in my opinion you should at least get your building steps, etc. right. It took me a day or two to get everything working as I wanted it to be.

Sorry for the wall of text, but there was a lot of different points I wanted to respond to.

@Archomeda
Copy link
Author

To add to windeployqt, this is my current output:

*********\Lightpack\Software\src\bin\Prismatik.exe 32 bit, release executable
Adding Qt5Svg for qsvgicon.dll
Direct dependencies: Qt5Core Qt5Gui Qt5Network Qt5SerialPort Qt5Widgets
All dependencies   : Qt5Core Qt5Gui Qt5Network Qt5SerialPort Qt5Widgets
To be deployed     : Qt5Core Qt5Gui Qt5Network Qt5SerialPort Qt5Svg Qt5Widgets
Warning: Cannot find Visual Studio installation directory, VCINSTALLDIR is not set.
Qt5Core.dll is up to date.
Qt5Gui.dll is up to date.
Qt5Network.dll is up to date.
Qt5SerialPort.dll is up to date.
Updating Qt5Svg.dll.
Qt5Widgets.dll is up to date.
Updating libGLESV2.dll.
Updating libEGL.dll.
Updating D3Dcompiler_47.dll.
Creating directory bearer.
Updating qgenericbearer.dll.
Updating qgenericbearerd.dll.
Updating qnativewifibearer.dll.
Updating qnativewifibearerd.dll.
Creating directory iconengines.
Updating qsvgicon.dll.
Updating qsvgicond.dll.
Creating directory imageformats.
Updating qdds.dll.
Updating qddsd.dll.
Updating qgif.dll.
Updating qgifd.dll.
Updating qicns.dll.
Updating qicnsd.dll.
Updating qico.dll.
Updating qicod.dll.
Updating qjp2.dll.
Updating qjp2d.dll.
Updating qjpeg.dll.
Updating qjpegd.dll.
Updating qmng.dll.
Updating qmngd.dll.
Updating qsvg.dll.
Updating qsvgd.dll.
Updating qtga.dll.
Updating qtgad.dll.
Updating qtiff.dll.
Updating qtiffd.dll.
Updating qwbmp.dll.
Updating qwbmpd.dll.
Updating qwebp.dll.
Updating qwebpd.dll.
Creating directory platforms.
Updating qwindows.dll.
Creating *********\Lightpack\Software\src\bin\translations...
Creating qt_ca.qm...
Creating qt_cs.qm...
Creating qt_de.qm...
Creating qt_fi.qm...
Creating qt_fr.qm...
Creating qt_hu.qm...
Creating qt_it.qm...
Creating qt_ja.qm...
Creating qt_ko.qm...
Creating qt_lv.qm...
Creating qt_ru.qm...
Creating qt_sk.qm...
Creating qt_uk.qm...

After looking up why it was adding those odd DLLs, I found this:

If ANGLE (the default) is used, you additionally need to include both libEGL.dll and libGLESv2.dll from Qt's 'lib' directory as well as the HLSL compiler from DirectX. The HLSL compiler library, d3dcompiler_XX.dll, where XX is the version number that ANGLE (libGLESv2) was linked against.

From http://doc.qt.io/qt-5/windows-deployment.html#creating-the-application-package

Regarding opengl32sw.dll, somehow I can't reproduce the creation of that file locally.

It would also seem that the translations in res/translations are somehow automatically compiled and don't have to be included in the installer as separate files. My install doesn't have these files, but Prismatik seems to pick them up just fine.

EDIT: Actually, it shouldn't copy the debug DLLs, I'm wondering why it does that now. It didn't do that before.
EDIT2: Heh. Running it in MinGW provided by Qt seems to add the debug versions, but running it manually in PowerShell doesn't.
EDIT3: Running it in PowerShell prevents the debug DLLs from being copied, but it does add opengl32sw.dll however

@psieg
Copy link
Owner

psieg commented Jul 26, 2015

I'll try to split it a bit:
static linking and Win32 in x64 magic
is needed only for projects related to DX grabbing. If you disable DX grabbing in configure-grabbers.prf, you don't need the rewrites - but the scripts should support the full (release) build.

windeployqt
is indeed an option to consider. I just adopted what I found in place and tested in on a machine without a Qt installation and was happy if it worked. Having translations handled by it as well would be nice. I have to read on the Qt workflow of translations some time, afaik you compile ts files to qm files and ship them with your binary.

additional libraries
The description of the ANGLE project is "... to seamlessly run WebGL and other OpenGL..." which Prismatik doesn't as far as I can judge. The libraries needed for DX injection are different and can be expected on a system of a gamer who needs DX injection.

CI
You'd need an account to see and change the job config, so.... shall be done once necessary. Using established externals is probably easier but I don't mind learing a bit how jenkins and the GitHub API work. The status issue is partially resolved now and based on a bug in the plugin. It's Jenkins.

project setup
Well yeah... no matter how complex the project is, if I have the all dependencies ready, there should be a script that builds everything for you.

@Archomeda
Copy link
Author

static linking and Win32 in x64 magic
I figured as much. For a universal build it's better to rewrite it yeah.

windeployqt
It's always great to see that something works 😄. I'm still unsure why the translations work without the *.qm files from res/translations. It might be possible that compiling with VS adds it as a generated source file? I'm not sure. I did see that these *.ts and *.qm files are included in the generated project file. Qt also seems to add its own translations, but I'm unsure what those are (probably generic ones).

On a similar note, I just noticed that in https://github.com/psieg/Lightpack/blob/master/Software/src/src.pro#L107-118 it already copies some of the Qt dependencies. I think it would be confusing to copy dependencies at two locations, one from the generated VS project file and one afterwards in a shell script.

additional libraries
Yeah. After some more investigation, I came across some threads that mentioned that the ANGLE detection was broken for some projects. You can disable it with a command line option, which is helpful. Doesn't help the fact that I seem to get different results from windeployqt depending in which environment I'm running it. I'll play around with it to see if I can get stable results from it. If I get it working, it might remove some copy redundancy in the scripts, etc.

CI
That's true I guess. But if it's set up correctly, the necessary things can be changed in the scripts. And Jenkins being complex compared to external CI services is also exactly the reason why I dislike using Jenkins on my public projects. But it's a fun (and frustrating) learning experience. You can do a lot with Jenkins though, which is nice, but the interface is... yeah...

project setup
Indeed, having that eventually is very much needed. There seems to be a lot of extra files of which I'm unsure if they are needed or just leftovers. I'll see how things play out.

@Archomeda
Copy link
Author

Since this discussion was mostly about improving the compilation of Prismatik on Windows that's now being discussed in #8, I'll close this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants