Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Completely static Windows build on CircleCI #9811

Merged
merged 4 commits into from
Sep 16, 2020
Merged

Completely static Windows build on CircleCI #9811

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
860226e
Allow statically linked windows build in CMake.
ekpyron Sep 15, 2020
d66b6f1
CircleCI Windows config and install_deps.ps1 script.
ekpyron Sep 15, 2020
d0fa9a5
Update AppVeyor config to set Boost_USE_STATIC_RUNTIME.
ekpyron Sep 16, 2020
b97ce76
Upload binaries individually as artifacts.
ekpyron Sep 16, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .circleci/build_win.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
cd "$PSScriptRoot\.."
mkdir build
cd build
$boost_dir=(Resolve-Path $PSScriptRoot\..\deps\boost\lib\cmake\Boost-*)
..\deps\cmake\bin\cmake -G "Visual Studio 16 2019" -DBoost_DIR="$boost_dir\" -DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded -DCMAKE_INSTALL_PREFIX="$PSScriptRoot\..\upload" ..
msbuild solidity.sln /p:Configuration=Release /m:5 /v:minimal
..\deps\cmake\bin\cmake --build . -j 5 --target install --config Release
37 changes: 37 additions & 0 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,9 @@ parameters:
type: string
default: "solbuildpackpusher/solidity-buildpack-deps@sha256:23dad3b34deae8107c8551804ef299f6a89c23ed506e8118fac151e2bdc9018c"

orbs:
win: circleci/[email protected]

defaults:

# --------------------------------------------------------------------------
Expand Down Expand Up @@ -64,6 +67,11 @@ defaults:
path: build/solc/solc
destination: solc

# windows binary archive
- artifact_solc_windows: &artifact_solc_windows
path: upload/solidity-windows.zip
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it actually still need to be a zip?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, currently it builds all binaries, so the compiler, the phaser and the upgrade tool...
If we only want to distribute solc.exe, then it doesn't need to be a zip file anymore :-). But it'd probably also break the new binary distribution scheme of @cameel's, wouldn't it :-)?

Weirdly enough the zip files of our previous releases did not include the phaser or the upgrade tool, but only solc.exe, the required DLLs and then soltest.exe for whatever reason...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm... well... what to release generally is a different question from what to put up as artifacts here, though.
I could change it to upload the three binaries separately as artifacts? I don't mind either way.

Copy link
Member

@cameel cameel Sep 16, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think replacing the zip with an exe would be a big problem as long as it's still a single file. We have a file list so external tools will be able to find it. They just need to be adjusted to handle both possibilities and not try to unpack the exe. It would be nice to give some advance notice to tool authors but other than that the whole system is still fresh so adjustments should be understandable. The change does not break anything for the old releases.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason for the zip file was the requirement to ship the dlls. If that is not an issue anymore, I think we should ship the .exe.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. And we can still decide and think about if and where and how we want to distribute the phaser and upgrade tool.
I'll change this PR to upload all three executables separately as artifacts in any case.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When is the next release? If it's this week, then I'd keep it as a zip until then, not to introduce this change too suddenly... We'll need some minor changes in the update script for it to pick up the exe.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we still have the zip from appveyor. I would not cut the appveyor run until after the next release.

destination: solidity-windows.zip

# compiled tool executable target
- artifacts_tools: &artifacts_tools
path: build/tools/solidity-upgrade
Expand Down Expand Up @@ -876,6 +884,32 @@ jobs:
- run: *gitter_notify_failure
- run: *gitter_notify_success

b_win:
executor:
name: win/default
shell: powershell.exe
steps:
- checkout
- restore_cache:
keys:
- dependencies-win-{{ checksum "scripts/install_deps.ps1" }}
- run:
name: "Installing dependencies"
command: if ( -not (Test-Path .\deps\boost) ) { .\scripts\install_deps.ps1 }
- save_cache:
key: dependencies-win-{{ checksum "scripts/install_deps.ps1" }}
paths:
- .\deps\boost
- .\deps\cmake
- run:
name: "Building solidity"
command: .circleci/build_win.ps1
- run:
name: "Package solidity"
command: Compress-Archive -Path .\upload\bin\* -DestinationPath .\upload\solidity-windows.zip
- store_artifacts: *artifact_solc_windows
- persist_to_workspace: *artifacts_build_dir

workflows:
version: 2

Expand Down Expand Up @@ -927,6 +961,9 @@ workflows:
- t_ems_compile_ext_gnosis: *workflow_emscripten
- t_ems_compile_ext_zeppelin: *workflow_emscripten

# Windows build and tests
- b_win: *workflow_trigger_on_tags

nightly:

triggers:
Expand Down
2 changes: 1 addition & 1 deletion appveyor.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ install:
before_build:
- if not exist build mkdir build
- cd build
- cmake -G "Visual Studio 15 2017 Win64" .. -DTESTS=On
- cmake -G "Visual Studio 15 2017 Win64" .. -DTESTS=On -DBoost_USE_STATIC_RUNTIME=OFF
build_script:
- msbuild solidity.sln /p:Configuration=%CONFIGURATION% /m:%NUMBER_OF_PROCESSORS% /v:minimal
- cd %APPVEYOR_BUILD_FOLDER%
Expand Down
3 changes: 3 additions & 0 deletions cmake/EthDependencies.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,9 @@ set(ETH_SCRIPTS_DIR ${ETH_CMAKE_DIR}/scripts)
## use multithreaded boost libraries, with -mt suffix
set(Boost_USE_MULTITHREADED ON)
option(Boost_USE_STATIC_LIBS "Link Boost statically" ON)
if(WIN32)
option(Boost_USE_STATIC_RUNTIME "Link Boost against static C++ runtime libraries" ON)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm actually not sure if this should default to OFF and only be set to ON in CI...

Copy link
Member Author

@ekpyron ekpyron Sep 16, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then again the boost binary distribution also doesn't have static libraries in general, so the same holds for Boost_USE_STATIC_LIBS above...

The problem is that linking entirely statically will only happen if one additionally uses -DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded - but that again only works with CMake 3.16+ :-)...

I think otherwise building will also work (if a static boost library linked against a static runtime is found, like the one built by the new scripts/install_deps.ps1), but then boost will be linked against a static C++ runtime while solidity will link against a dynamic one, which is asking for trouble, unless both are compiled together using the exact same libraries... although it may be that MSVC has a stable ABI in the C++ library within each version anyways, so maybe it's fine...

Not sure how large the actual danger in all this is, though - and we can just take care to document all of this properly... So I guess we can merge this PR regardless, but I thought better to mention this - and we should at least update the docs soon after and before the next release.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But defaulting to ON here will always break the existing AppVeyor builds, if we don't change the config for those :-)...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just pushed another commit setting that for AppVeyor to at least give it a chance :-).

endif()

set(BOOST_COMPONENTS "filesystem;unit_test_framework;program_options;system")

Expand Down
6 changes: 5 additions & 1 deletion cmake/EthPolicy.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,9 @@ macro (eth_policy)
# do not interpret if() arguments as variables!
cmake_policy(SET CMP0054 NEW)
endif()
endmacro()

if (POLICY CMP0091)
# Allow selecting MSVC runtime library using CMAKE_MSVC_RUNTIME_LIBRARY.
cmake_policy(SET CMP0091 NEW)
endif()
endmacro()
7 changes: 7 additions & 0 deletions cmake/jsoncpp.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,12 @@ if(CMAKE_VERSION VERSION_GREATER 3.1)
set(byproducts BUILD_BYPRODUCTS "${JSONCPP_LIBRARY}")
endif()

# Propagate CMAKE_MSVC_RUNTIME_LIBRARY on Windows builds, if set.
if (WIN32 AND POLICY CMP0091 AND CMAKE_MSVC_RUNTIME_LIBRARY)
list(APPEND JSONCPP_CMAKE_ARGS "-DCMAKE_POLICY_DEFAULT_CMP0091:STRING=NEW")
list(APPEND JSONCPP_CMAKE_ARGS "-DCMAKE_MSVC_RUNTIME_LIBRARY=${CMAKE_MSVC_RUNTIME_LIBRARY}")
endif()

ExternalProject_Add(jsoncpp-project
PREFIX "${prefix}"
DOWNLOAD_DIR "${CMAKE_SOURCE_DIR}/deps/downloads"
Expand All @@ -51,6 +57,7 @@ ExternalProject_Add(jsoncpp-project
-DJSONCPP_WITH_PKGCONFIG_SUPPORT=OFF
-DCMAKE_CXX_FLAGS=${JSONCPP_CXX_FLAGS}
-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}
${JSONCPP_CMAKE_ARGS}
${byproducts}
)

Expand Down
15 changes: 15 additions & 0 deletions scripts/install_deps.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Needed for Invoke-WebRequest to work via CI.
$progressPreference = "silentlyContinue"

New-Item -ItemType Directory -Force -Path "$PSScriptRoot\..\deps"

Invoke-WebRequest -URI "https://github.com/Kitware/CMake/releases/download/v3.18.2/cmake-3.18.2-win64-x64.zip" -OutFile cmake.zip
tar -xf cmake.zip
mv cmake-3.18.2-win64-x64 "$PSScriptRoot\..\deps\cmake"

Invoke-WebRequest -URI "https://dl.bintray.com/boostorg/release/1.74.0/source/boost_1_74_0.zip" -OutFile boost.zip
tar -xf boost.zip
cd boost_1_74_0
.\bootstrap.bat
.\b2 -j4 -d0 link=static runtime-link=static variant=release threading=multi address-model=64 --with-filesystem --with-system --with-program_options --with-test --prefix="$PSScriptRoot\..\deps\boost" install
cd ..