Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmake: add compiler + remove compiler.version to reduce number of builds #1783

Merged
merged 10 commits into from
Jul 2, 2020
Merged

cmake: add compiler + remove compiler.version to reduce number of builds #1783

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
4a209d5
cmake: add compiler + remove compiler.version to reduce number of builds
madebr May 29, 2020
570f312
cmake: don't import Version
madebr May 30, 2020
213a27b
cmake: build cmake from the root directory
madebr May 30, 2020
92b1b37
cmake: add openssl to requires
madebr May 30, 2020
2931d4d
cmake: add openssl dependency optionally
madebr May 30, 2020
aaf55f8
cmake: patch cmakelists to include conanbuildinfo.cmake
madebr May 30, 2020
ecbb847
cmake: limit to Release build_type + only allow static non-debug MSVC…
madebr May 31, 2020
6af0f89
cmake: remove compiler and build_type from package_id
madebr Jun 2, 2020
e6074ae
cmake: remove `-lz` exe linker flag
madebr Jun 12, 2020
d8e61fc
cmake: allow all os/build_type/compiler
madebr Jun 15, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 0 additions & 11 deletions recipes/cmake/3.x.x/CMakeLists.txt
View file
Open in desktop

This file was deleted.

43 changes: 37 additions & 6 deletions recipes/cmake/3.x.x/conanfile.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
import os
from conans import tools, ConanFile, CMake
from conans.tools import Version
from conans.errors import ConanInvalidConfiguration, ConanException


Expand All @@ -11,19 +10,40 @@ class CMakeConan(ConanFile):
url = "https://github.com/conan-io/conan-center-index"
homepage = "https://github.com/Kitware/CMake"
license = "BSD-3-Clause"
exports_sources = ["CMakeLists.txt"]
generators = "cmake"
settings = "os", "arch", "build_type"
settings = "os", "arch", "compiler", "build_type"

options = {
"with_openssl": [True, False, "auto"],
}
default_options = {
"with_openssl": "auto",
}

_source_subfolder = "source_subfolder"
_cmake = None

def _minor_version(self):
return ".".join(str(self.version).split(".")[:2])

@property
def _with_openssl(self):
if self.options.with_openssl == "auto":
return self.settings.os != "Windows"
return self.options.with_openssl

def configure(self):
if self.settings.os == "Macos" and self.settings.arch == "x86":
raise ConanInvalidConfiguration("CMake does not support x86 for macOS")
if self.settings.build_type != "Release":
raise ConanInvalidConfiguration("only Release build_type is supported")
if self.settings.compiler == "Visual Studio":
if self.settings.compiler.runtime != "MT":
madebr marked this conversation as resolved.
Show resolved Hide resolved
raise ConanInvalidConfiguration("Only MT MSVC runtime is supported")
madebr marked this conversation as resolved.
Show resolved Hide resolved

def requirements(self):
if self._with_openssl:
self.requires("openssl/1.1.1g")

def source(self):
tools.get(**self.conan_data["sources"][self.version])
Expand All @@ -35,12 +55,17 @@ def _configure_cmake(self):
self._cmake = CMake(self)
self._cmake.definitions["CMAKE_BOOTSTRAP"] = False
if self.settings.os == "Linux":
self._cmake.definitions["OPENSSL_USE_STATIC_LIBS"] = True
self._cmake.definitions["CMAKE_EXE_LINKER_FLAGS"] = "-lz"
self._cmake.configure(source_dir=self._source_subfolder)
self._cmake.definitions["CMAKE_USE_OPENSSL"] = self._with_openssl
if self._with_openssl:
self._cmake.definitions["OPENSSL_USE_STATIC_LIBS"] = not self.options["openssl"].shared
self._cmake.configure(source_folder=self._source_subfolder)
return self._cmake

def build(self):
tools.replace_in_file(os.path.join(self._source_subfolder, "CMakeLists.txt"),
"project(CMake)",
"project(CMake)\ninclude(\"{}/conanbuildinfo.cmake\")\nconan_basic_setup()".format(
self.build_folder.replace("\\", "/")))
if self.settings.os == "Linux":
tools.replace_in_file(os.path.join(self._source_subfolder, "Utilities", "cmcurl", "CMakeLists.txt"),
"list(APPEND CURL_LIBS ${OPENSSL_LIBRARIES})",
Expand All @@ -54,6 +79,12 @@ def package(self):
cmake.install()
tools.rmdir(os.path.join(self.package_folder, "doc"))

def package_id(self):
self.info.options.with_openssl = self._with_openssl
uilianries marked this conversation as resolved.
Show resolved Hide resolved

del self.info.settings.compiler
del self.info.settings.build_type
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jgsogo @SSE4
By not removing compiler and build_type, at least 50 packages of cmake will be built for each version.

Suggested change
del self.info.settings.compiler
del self.info.settings.build_type

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. I understand that the desired scenario would be to remove these settings but to be able to tell C3i that we want to build the Release build.... that way, when someone retrieves the package from CCI it will be the good one.

This is not possible right now, the only alternative is to build them all 😞

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll apply both suggestions then.

Is there already an issue open on github about this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, none of the two issues here, but both are really hard to tackle.

From the CCI perspective we don't want to run a different pipeline based on custom attributes and Conan right now doesn't have a feature to classify packages (and IMO we don't need it yet). The other issue: being able to assign different build profiles to different branches of the graph, requires a new graph model, something we are working for Conan 2.0, but it will require time to implement.

So, for the time being, we are building ~100 packages for each CMake version even though we will probably use only a dozen of them

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't you just add an entry in the top-level yml file for telling cci which permutations to build? If 'compiler' and 'build_type' are removed from the exported 'package_id', no consumer should ever need to rebuild this package, no matter what profile they use. Or am I missing something?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can think about different alternatives to tell CCI which configurations to build and probably it is about something in the config.yml file as you have suggested. OTH, if we remove build_type from the package ID we cannot guarantee (at this moment) that CCI will choose Release build (this was the issue originating this PR, I think).

Even though these things might look almost trivial, they are not as straightforward as they look like, and there are other things in the backlog with higher priority. We would really want to address all these issues, but we need to go step by step.


def package_info(self):
minor = self._minor_version()

Expand Down