Merge pull request #12 from lazar-ivanov/lazari2

updating makefiles and .md files

CONTRIBUTING.md

@@ -9,6 +9,89 @@ If you are currently J.P. Morgan Chase employee and interested in contributing y
 
 The swblocks-baselib library comes with a comprehensive unit tests suite and virtually all parts of it are very extensively tested. It also comes currently with a couple of application binaries - 'bl-tool' and 'bl-messaging-broker'.
 
-In order to be able to make changes to the library code and contribute these back you will need to be able to modify and add to the tests code suite and also of course be able to build and run the unit tests suite, etc. The library comes also with its own flexible make files and build system which allows you to build these binaries and unit tests suite.
+In order to be able to make changes to the library code and contribute these back you will need to be able to modify and add to the tests code suite and also of course be able to build and run the unit tests suite, etc. The way you can do this is by using the swblocks-baselib development environment and its own build system. The library comes also with its own flexible make files and build + test system which allows you to build these binaries and unit tests and to run the unit tests. To learn how to use the swblocks-baselib library development environment and build system look at the next section.
 
-More details on how to be able to build and execute the unit tests suite will be coming soon
+## Development environment and build system
+
+The library development environment is based on GNU make, but it has somewhat sofisticated structure and implementation that is [declarative and build-by-convention](https://docs.gradle.org/current/userguide/overview.html) and of course also avoids the [pitfalls of recursive make](https://www.google.com/#q=recursive+make+considered+harmful). The development environment and the build system assumes certain structure of dependencies and how they are built. This structure and its assumptions are embedded in the make files and the steps to build and prepare the external dependencies are described in the various notes files (under notes folder in the root). As mentioned in the [README.md](README.md) file the swblocks-baselib library and its development environment and build system has very limited dependencies, namely C++11 compliant compiler and toolchain (e.g. recent versions of Clang, GCC or VC++), Boost, OpenSSL and lastly a small JSON parser library which is header files only and does not need to be built (JSON Spirit 4.08), so preparing the dependencies for the development environment is relatively easy, but it is not automated (you have to follow the instructions in the notes files).
+
+The development environment and build system has a minimal notion of versioning to facilitate evolution without having to break the world, but it generally requires "forced upgrade" model where older versions are quickly depreceiated and the users / clients of the library are expected to upgrade and move forward. That applies to both the development environment and the code itsef. The code of course is changed carefully with backward compatibility in mind and older versions of the developemt environment or dependencies are not explicitly broken or not supported, but they might be, so if you are a user of the library and you can't upgrade easily at some point you might not be able to pick up the latest version of swblocks-baselib library and might have to stay on older version - btw, the model is very similar to the way Boost is versioned and how backward compatibility is maintained there.
+
+A development environment version is a notion of collection of specific versions of compiler toolchains + a collection of compatible versions of the 3rd party dependencies. Currently swblocks-baselib library officially supports two developmnent environment versions - devenv3 (latest) and devenv2 (old) with the devenv2 environment of course to be eventually depreciated in the somwhat near future (probably within a year or so). Here are the collections of the compiler toolchains versions and the 3rd party dependencies versions for each development environment:
+
+* **devenv2**
+  * Operating Systems
+    * Linux (RedHat rhel5, rhel6, rhel7, Ubuntu 12.04, 14.04, 16.04)
+    * Windows (7, 8.x, 10 - both x86 and x64)
+  * Compilers
+    * GCC 4.9
+    * Clang 3.5
+    * vc12 (Visual C++ 2013)
+  * C++11 standard library implementations
+    * libstdc++ for GCC and Clang
+    * msvcrt for vc12
+  * Boost 1.58
+  * OpenSSL 1.0.2d
+  * JSON Spirit 4.08
+* **devenv3**
+  * Operating Systems
+    * Darwin (15.x, 16.x - i.e. El Capitan and macOS Sierra)
+    * Linux (RedHat rhel5, rhel6, rhel7, Ubuntu 12.04, 14.04, 16.04)
+    * Windows (7, 8.x, 10 - both x86 and x64)
+  * Compilers
+    * GCC 6.3
+    * Clang 3.9
+    * Clang 7.3 (for Darwin)
+    * vc14 (Visual C++ 2015)
+  * C++11 standard library implementations
+    * libc++ (for Darwin)
+    * libstdc++ for GCC and Clang
+    * msvcrt for vc14
+  * Boost 1.63
+  * OpenSSL 1.1.0d
+  * JSON Spirit 4.08
+
+## Development environment distributions and links
+
+In addition to the code dependencies of the library itself (compiler toolchain, boost, openssl, JSON spirit) the development environment also has few aditional dependencies such as GNU make (e.g. via msys on Windows), git, python, etc, plus some optional such as Eclipse CDT (to use C++ indexer and IDE). The "binary blob" that contains the pre-built versions of the development environment with all code dependencies plus the additional and optional such is called "devenv distribution". Some poluplar combinations of devenv distributions were prebuilt and uploaded on Google drive, so they can be downloaded directly and facilitate the development and ease of use of the swblocks-baselib library. Here is the list of the currently supported devenv distributions with the link of where to download them from (more will be coming in the future):
+
+* **devenv3** for Windows (Win7 - Win10, including latest Win10 creator's edition, SDK 8.1 and 10), vc14 (VS 2015) can be downloaded from [here](https://drive.google.com/open?id=0B7R8Vjb6OGTHcmNOUjJOTHRWLWs)
+* **devenv3** for Darwin (d156 - 15.x - 16.x, El Capitan and macOS Sierra), clang 7.3, libc++ can be downloaded [here](https://drive.google.com/open?id=0B7R8Vjb6OGTHTzB3WG1RTjNfWE0)
+* **devenv3** for Ubuntu 16.04 LTS with GCC 6.3 and Clang 3.9 can be downloaded [here](https://drive.google.com/open?id=0B7R8Vjb6OGTHVWpXRnR3LVZYbUk)
+
+All links above are either .zip file (for Windows) or a .gz tar file for Darwin and Linux. Once they are downloaded they can be extracted into some location (e.g. **c:\\swblocks** for Windows or **/Users/userid/swblocks** for Darwin and Linux) and then once you clone the swblocks-baselib library from github in order to use it you simply need to create a small .mk file in **projects/make** folder called **ci-init-env.mk** and point the 3 _DIST_ roots to the place where you have extracted the development environment blob.
+
+E.g. if you have extracted the development environment into **c:\\swblocks** then the **projects/make/ci-init-env.mk** can look the following way for Windows:
+
+```make
+# initialize the important env roots
+
+DIST_ROOT_DEPS1 = /c/swblocks/dist-devenv3-windows
+DIST_ROOT_DEPS2 = /c/swblocks/dist-devenv3-windows
+DIST_ROOT_DEPS3 = /c/swblocks/dist-devenv3-windows
+```
+
+Note that on Windows you should use UNIX style paths since we use GNU make (via msys)
+
+E.g. **projects/make/ci-init-env.mk** can look the following way for Darwin (or you can adjust for Linux):
+
+```make
+# initialize the important env roots
+
+DIST_ROOT_DEPS1 = /Users/userid/swblocks/dist-devenv3-darwin-15.6
+DIST_ROOT_DEPS2 = /Users/userid/swblocks/dist-devenv3-darwin-15.6
+DIST_ROOT_DEPS3 = /Users/userid/swblocks/dist-devenv3-darwin-15.6
+```
+
+Once you create unpack the development environment distribution and create the **projects/make/ci-init-env.mk** as per above you can now build the code run tests by specifying the targets, etc. The unit test targets start with **utf_baselib** prefix (e.g. utf_baselib, utf_baselib_data) and the respective test targets start with **test_utf_baselib**. The other targets are the respective binary names and there are special targets too, just type make help for more information.
+
+Here are some examples e.g.:
+
+```make
+make utf_baselib
+make test_utf_baselib_data
+make -k -j6
+make -k -j6 && make -k -j8 test
+make help
+make -k -j6 && make -k -j8 test && make install
+```

README.md

@@ -35,14 +35,16 @@ As mentioned above in the design choices and principles section the library orga
 
 In order to use the library all you need is the following:
 
-1. A C++11 compliant compiler toolchain. It has been extensively tested with GCC 4.9, Clang 3.5 and Visual C++ vc12, but it should generally work or made easily to work with any subsequent versions.
-1. Builds of Boost and OpenSSL. It has been extensively tested with Boost version 1.58 and OpenSSL version 1.0.2d, but it should generally work or easily made to work with any subsequent versions of these libraries. You need to make sure that both Boost and OpenSSL libraries are on the include path in your projects / build environment and that you are also linking to them.
-1. [json-spirit library](https://www.codeproject.com/Articles/20027/JSON-Spirit-A-C-JSON-Parser-Generator-Implemented). This is a header files only JSON parser implementation base on Boost.Spirit. It has been tested with version 4.08, but it should probably work fine with any subsequent versions. You need to make sure that json-spirit library is on the include path in your projects / build environment.
-1. An existing build environment, IDE and / or project structure in general - whatever you currently use for your C++ projects in general, could be Visual Studio, Eclipse, Code Blocks, etc, for IDE and / or make, cmake, gradle or something else for command line build system, etc
+1. A C++11 compliant compiler toolchain. It has been extensively tested with the following cross section of complier toolchains GCC 4.9, GCC 6.3, Clang 3.5, Clang 3.9, Visual C++ vc12 and vc14, but it should generally work or made easily to work with any subsequent versions.
+1. Builds of Boost and OpenSSL. It has been extensively tested with Boost version 1.58 and 1.63 and OpenSSL version 1.0.2d and 1.1.0d, but it should generally work or easily made to work with any subsequent versions of these libraries. You need to make sure that both Boost and OpenSSL libraries are on the include path in your projects / build environment and that you are also linking to them.
+1. [json-spirit library](https://www.codeproject.com/Articles/20027/JSON-Spirit-A-C-JSON-Parser-Generator-Implemented). This is a header files only JSON parser implementation base on Boost.Spirit. It has been tested with version 4.08, but it should probably work fine with any subsequent versions. You need to make sure that json-spirit library is on the include path in your projects / build environment. Note that in order to down,oad the JSON Spirit library from he code project link above you need to register and login (free), but if you don't want to do that you can get it from one of the GitHub forks - e.g. one possible such fork is  [here](https://github.com/png85/json_spirit)
+1. An existing build environment, IDE and / or project structure in general - whatever you currently use for your C++ projects in general, could be Visual Studio, Eclipse, XCode, Code Blocks, CLion, etc, for IDE and / or make, cmake, gradle or something else for command line build system, etc
 1. Just add src/include path to your header includes in your projects / build environment.
 
 That's it! You can now just include the code that you need and use it.
 
+If you don't have your own C++ build / development environment and / or you want to use the one that is been used for the development of swblocks-baselib itself then check out the see the [CONTRIBUTING.md](CONTRIBUTING.md) file for more information.
+
 ## How to modify the library code, validate your changes and contribute back
 
 See the [CONTRIBUTING.md](CONTRIBUTING.md) file for more information on how to modify the code and contribute back

notes/devenv_dist_links.txt

@@ -0,0 +1,5 @@
+********************************* devenv3 dist links ********************************
+
+Windows (win7 - win10 creator's edition, SDK 8.1 and 10) : https://drive.google.com/open?id=0B7R8Vjb6OGTHcmNOUjJOTHRWLWs
+Darwin (d156 - 15.x - 16.x, El Capitan and macOS Sierra) : https://drive.google.com/open?id=0B7R8Vjb6OGTHTzB3WG1RTjNfWE0
+Ubuntu (ub16, 16.04 LTS) : https://drive.google.com/open?id=0B7R8Vjb6OGTHVWpXRnR3LVZYbUk

projects/make/common.mk

@@ -2,12 +2,10 @@
 # Initialize and bootstrap the CI environment before we do anything else
 #
 
-ifeq (, $(CI_ENV_ROOT))
-ifeq ("$(wildcard $(MKDIR)/ci-init-env.mk)","")
-$(error Either ci-init-env.mk or CI_ENV_ROOT environment variable must be defined and point to the CI env root)
-else
+ifneq ("$(wildcard $(MKDIR)/ci-init-env.mk)","")
 CI_ENV_MKDIR := $(MKDIR)
-endif
+else ifeq (, $(CI_ENV_ROOT))
+$(error Either ci-init-env.mk or CI_ENV_ROOT environment variable must be defined and point to the CI env root)
 else ifeq ("$(wildcard $(CI_ENV_ROOT)/projects/make/ci-init-env.mk)","")
 $(error The CI env file $(CI_ENV_ROOT)/projects/make/ci-init-env.mk pointed by CI_ENV_ROOT does not exist)
 else

projects/make/toolchain/gcc-default.mk

@@ -111,14 +111,15 @@ ifeq ($(BL_PLAT_IS_DARWIN),1)
 # to also use -fvisibility=hidden
 #
 CXXFLAGS += -fvisibility=default
+# Produces debugging information for use by LLDB on Darwin
+CXXFLAGS += -g
 else
 CXXFLAGS += -fvisibility=hidden
-endif
-
 # Produces debugging information for use by GDB.
 # This means to use the most expressive format available,
 # including GDB extensions if at all possible.
 CXXFLAGS += -ggdb
+endif
 
 #
 # We need a way to externally disable strict warnings level, so we can build

projects/make/toolchain/msvc-default.mk

@@ -31,7 +31,7 @@ INCLUDE  += $(WINSDK10)/Include/10.0.10240.0/ucrt
 endif
 
 ifeq ($(TOOLCHAIN),vc14)
-PATH     := $(MSVC)/Common7/ide:$(WINSDK)/bin/$(ARCH):$(WINSDK10)/bin/$(ARCH)/ucrt:$(WINSDK)/Debuggers/$(ARCH):$(PATH)
+PATH     := $(MSVC)/Common7/ide:$(WINSDK)/bin/$(ARCH):$(WINSDK10)/bin/$(ARCH)/ucrt:$(WINSDK)/Debuggers/$(ARCH):$(WINSDK10)/Redist/ucrt/DLLs/$(ARCH):$(PATH)
 else
 PATH     := $(MSVC)/Common7/ide:$(WINSDK)/bin/$(ARCH):$(WINSDK)/Debuggers/$(ARCH):$(PATH)
 endif