Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Completely static Windows build on CircleCI #9811

Merged
merged 4 commits into from
Sep 16, 2020
Merged

Conversation

ekpyron
Copy link
Member

@ekpyron ekpyron commented Sep 15, 2020

Once this works I'll move some more logic to scripts - just checking how CI behaves so far.

@ekpyron ekpyron force-pushed the windowsStaticCircleCI branch 19 times, most recently from 7ee3b98 to 02dbade Compare September 16, 2020 09:23
@ekpyron
Copy link
Member Author

ekpyron commented Sep 16, 2020

For the record: so far I get 3.5 minutes to build and install boost (and cmake) on cache mismatch, and <30 seconds for cache restore - but for some reason the actual solidity build takes a bit more than half an hour so far (which is more than in #9476 - and I'm pretty sure that cannot only be due to the completely static linking), so I'm still trying to figure out why that is and if it can be improved.
The newer boost version could affect build times, but I'd still not expect that much of a difference...

@chriseth
Copy link
Contributor

The bulid times are still better than appveyor, aren't they?

@ekpyron
Copy link
Member Author

ekpyron commented Sep 16, 2020

Actually with the msbuild build command I just tried instead of the cmake --build command it's down to 20 minutes, which is in line with the other PR - weird that that makes that much of a difference, but well. And yes, that's better than appveyor (which is also at ~35 minutes I think, once it actually started).

So I think I'll extend this by the release logic and tests now.

@ekpyron ekpyron changed the title [WIP] Windows static circle ci test PR Windows static circle ci test PR Sep 16, 2020
@@ -25,6 +25,9 @@ set(ETH_SCRIPTS_DIR ${ETH_CMAKE_DIR}/scripts)
## use multithreaded boost libraries, with -mt suffix
set(Boost_USE_MULTITHREADED ON)
option(Boost_USE_STATIC_LIBS "Link Boost statically" ON)
if(WIN32)
option(Boost_USE_STATIC_RUNTIME "Link Boost against static C++ runtime libraries" ON)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm actually not sure if this should default to OFF and only be set to ON in CI...

Copy link
Member Author

@ekpyron ekpyron Sep 16, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then again the boost binary distribution also doesn't have static libraries in general, so the same holds for Boost_USE_STATIC_LIBS above...

The problem is that linking entirely statically will only happen if one additionally uses -DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded - but that again only works with CMake 3.16+ :-)...

I think otherwise building will also work (if a static boost library linked against a static runtime is found, like the one built by the new scripts/install_deps.ps1), but then boost will be linked against a static C++ runtime while solidity will link against a dynamic one, which is asking for trouble, unless both are compiled together using the exact same libraries... although it may be that MSVC has a stable ABI in the C++ library within each version anyways, so maybe it's fine...

Not sure how large the actual danger in all this is, though - and we can just take care to document all of this properly... So I guess we can merge this PR regardless, but I thought better to mention this - and we should at least update the docs soon after and before the next release.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But defaulting to ON here will always break the existing AppVeyor builds, if we don't change the config for those :-)...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just pushed another commit setting that for AppVeyor to at least give it a chance :-).

@ekpyron ekpyron marked this pull request as ready for review September 16, 2020 11:24
@ekpyron ekpyron changed the title Windows static circle ci test PR Completely static Windows build on CircleCI Sep 16, 2020
@@ -64,6 +67,11 @@ defaults:
path: build/solc/solc
destination: solc

# windows binary archive
- artifact_solc_windows: &artifact_solc_windows
path: upload/solidity-windows.zip
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it actually still need to be a zip?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, currently it builds all binaries, so the compiler, the phaser and the upgrade tool...
If we only want to distribute solc.exe, then it doesn't need to be a zip file anymore :-). But it'd probably also break the new binary distribution scheme of @cameel's, wouldn't it :-)?

Weirdly enough the zip files of our previous releases did not include the phaser or the upgrade tool, but only solc.exe, the required DLLs and then soltest.exe for whatever reason...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm... well... what to release generally is a different question from what to put up as artifacts here, though.
I could change it to upload the three binaries separately as artifacts? I don't mind either way.

Copy link
Member

@cameel cameel Sep 16, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think replacing the zip with an exe would be a big problem as long as it's still a single file. We have a file list so external tools will be able to find it. They just need to be adjusted to handle both possibilities and not try to unpack the exe. It would be nice to give some advance notice to tool authors but other than that the whole system is still fresh so adjustments should be understandable. The change does not break anything for the old releases.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason for the zip file was the requirement to ship the dlls. If that is not an issue anymore, I think we should ship the .exe.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. And we can still decide and think about if and where and how we want to distribute the phaser and upgrade tool.
I'll change this PR to upload all three executables separately as artifacts in any case.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When is the next release? If it's this week, then I'd keep it as a zip until then, not to introduce this change too suddenly... We'll need some minor changes in the update script for it to pick up the exe.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we still have the zip from appveyor. I would not cut the appveyor run until after the next release.

@chriseth
Copy link
Contributor

Fine apart from the question whether we want the artifact to be a zip - but I think we can just merge as is now.

@ekpyron
Copy link
Member Author

ekpyron commented Sep 16, 2020

Please note that this doesn't properly deal with the distinction between develop and release builds yet (i.e. it doesn't create prerelease and commit hash files yet) - I'll take care of that in a separate PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants