Skip to content

Running the worker on Windows

Jump to bottom Edit New page
ppigazzini edited this page Mar 29, 2020 · 84 revisions

Propedeutical steps

Read Running the worker: overview before installing the worker.

Windows Subsystem for Linux

If you are a Linux user and you have a supported Windows platform, you can run the worker:

Stockfish got a little speedup (2%) when running as Linux application in WSL with respect to when running as Windows application.

If you are not a Linux user, better to use the following instructions to do a manual installation or use A portable fishtest worker for Windows 64 bit.

MSYS2 & MinGW-w64

MSYS2 is a software distribution and a building platform for Windows. It provides an Unix-like environment, a command-line interface and a software repository making it easy to install software on Windows or build software on Windows with the GCC compiler. On the MSYS2 wiki read the installation instruction and the short introduction to understand MSYS2 subsystems and shelles.

TL;DR: MSYS2 installation and worker setup

MSYS2 consists of three subsystems and their corresponding package repositories, msys2, mingw32, and mingw64:

  • the mingw subsystems provide native Windows programs that use Win32 API.
  • the msys2 subsystem provides an emulated mostly-POSIX-compliant environment based on Cygwin.

Every subsystem has an associated "shell", which is essentially a set of environment variables that allow the subsystems to co-operate properly:

  • MSYS2 MinGW 64-bit, used to build Windows native 64 bit applications.
  • MSYS2 MinGW 32-bit, used to build Windows native 32 bit applications.
  • MSYS2 MSYS, used to build POSIX application.
  1. Download and install MSYS2, use the 64 bit or 32 bit installer according your operating system, and update the MSYS2 packages:
  • update the core packages executing pacman -Syuu, when requested close the windows pushing the top right X button
  • open a MSYS2 MSYS shell and update the others packages executing pacman -Syuu

The default setting installs MSYS2 in to C:\msys64 folder, the user home is the folder C:\msys64\home<your_username> (C:\msys32 for 32 bit Windows).

  1. write the script setup_worker.sh with a text editor, inserting your fishtest username/password and the number of cores to be contributed. Copy the file in C:\msys64\home<your_username> 
#!/bin/bash
# setup_worker.sh
# install and configure fishtest worker

# installation device (e.g. C), fishtest credentials, cores to be contributed
fish_device=C
usr_name="YOUR_USERNAME"
usr_pwd="YOUR_PASSWORD"
n_cores="NUMBER_OF_CORES"

# install packages if not already installed
unzip -v &> /dev/null || pacman -S --noconfirm unzip
make -v &> /dev/null || pacman -S --noconfirm make
g++ -v &> /dev/null || pacman -S --noconfirm mingw-w64-x86_64-gcc
python3 --version &> /dev/null || pacman -S --noconfirm mingw-w64-x86_64-python3

tmp_dir=___${RANDOM}
mkdir ${tmp_dir} && cd ${tmp_dir}

# download fishtest
wget https://github.com/glinscott/fishtest/archive/master.zip

unzip master.zip
fish_path=/${fish_device}/fishtest-worker

mkdir ${fish_path}
mv fishtest-master/worker ${fish_path}

# find the CPU architecture
# CPU without popcnt and bmi2 instructions (e.g. older than Intel Sandy Bridge)
arch_cpu=x86-64
# CPU with bmi2 instruction (e.g. Intel Haswell or newer)
if [ "$(g++ -Q -march=native --help=target | grep mbmi2 | grep enabled)" ] ; then
  # CPU AMD zen
  if [ "$(g++ -Q -march=native --help=target | grep march | grep 'znver[12]')" ] ; then
    arch_cpu=x86-64-modern
  else
    arch_cpu=x86-64-bmi2
# CPU with popcnt instruction (e.g. Intel Sandy Bridge)
elif [ "$(g++ -Q -march=native --help=target | grep mpopcnt | grep enabled)" ] ; then
  arch_cpu=x86-64-modern
fi
 
echo "make -j profile-build ARCH=${arch_cpu} COMP=mingw" > ${fish_path}/worker/custom_make.txt

cat << EOF >> ${fish_path}/worker/start_worker.bat
@echo off
SET PATH=C:\msys64\mingw64\bin;C:\msys64\usr\bin;%PATH%
set CXXFLAGS=-march=native
cmd /k python3 -i worker.py --concurrency ${n_cores} ${usr_name} ${usr_pwd}
EOF

cd ..
rm -rf ${tmp_dir}
  1. execute the script in the MSYS2 shell with bash setup_worker.sh, this install and configure the worker in C:\fishtest-worker\worker
  2. start the worker executing start_worker.bat (e.g. double click on start_worker.bat)

MinGW-w64 & MSYS (deprecated, use only with Windows XP and Windows Server 2003)

In alternative to MSYS2 it's possible to use the older and not maintained MSYS.

Setup Python on Windows

Download Python 3.8.x, choose 32 or 64 bit according your Windows version:

Python suggested installation:

  1. at first screen (Install Python 3.8.0) chose Customize installation and push Next
  2. at second screen (Optional Features) keep the defaults and push Next
  3. at third screen (Advanced Options):
  • select Install for all users
  • write C:\Python38 in Customize install location
  • and push Install

Be sure to have the file C:\Python38\python.exe

Setup fishtest

Download fishtest directly from Github and unzip the archive in C: drive:

https://github.com/glinscott/fishtest/archive/master.zip

Be sure to have the file C:\fishtest-master\worker\worker.py

  1. download and install MinGW (32-bit), use the default settings. When the installation is completed select Continue to start the MinGW Installation Manager:

  2. in the packages section, right click on the package msys-base (the last package) and select Mark for Installation

  3. in the menu Installation (upper left corner) select the voice Apply changes

  4. select Apply in the following window

  5. when the installation is finished, close the MinGW Installation Manager

  6. check to have the directory C:\MinGW\msys\1.0\bin

  7. install MinGW-w64 (64 bit or 32 bit according your operating system):

  • 64 bit Windows: download MinGW-w64 (64-bit) GCC 8.1.0 and extract the mingw64 folder to C:\MinGW, check to have the directory C:\MinGW\mingw64\bin
  • 32 bit Windows: download MinGW-w64 (32-bit) GCC 8.1.0 and extract the mingw32 folder to C:\MinGW, check to have the directory C:\MinGW\mingw32\bin
  1. in C:\fishtest-master\worker directory, create a text file named fishtest.bat and copy in it the text below (write the <n_cores> that you want dedicate to fishtest, and your fishtest's <username> and <password>)
  • 64 bit Windows:
@echo off
SET PATH=C:\MinGW\mingw64\bin;C:\MinGW\msys\1.0\bin;C:\Python38;%PATH%
set CXXFLAGS=-march=native
cmd /k python.exe -i C:\fishtest-master\worker\worker.py --concurrency <n_cores> <username> <password>
  • 32 bit Windows:
@echo off
SET PATH=C:\MinGW\mingw32\bin;C:\MinGW\msys\1.0\bin;C:\Python38;%PATH%
set CXXFLAGS=-march=native
cmd /k python.exe -i C:\fishtest-master\worker\worker.py --concurrency <n_cores> <username> <password>
  1. start the worker executing fishtest.bat (e.g. double click on fishtest.bat)

  • the worker's script uses the compiling directive mingw32-make ARCH=x86-64-modern .... This can be a problem because:

    • mingw32-makedoes not work with Windows XP and Windows Server 2003, you need to use make
    • ARCH=x86-64-modern does not work with CPU older than Sandy Bridge or with 32 bit Windows

    In these cases you MUST override the default settings writing a custom_make.txt file in C:\fishtest-master\worker directory, according the architecture of your CPU and/or your OS.

Add a custom footer
Add a custom sidebar
Clone this wiki locally