-
Notifications
You must be signed in to change notification settings - Fork 25
Building a Microkernel ISO
The Microkernel used by Razor is actually a variant of the Tiny Core Linux (TCL) kernel. The Microkernel is built using the 'Core' release of TCL, which can be found here. In order to run the tools that are needed by the Microkernel (Facter for discovery and MCollective for management), a number of TCL Extensions need to be added to our Microkernel. It is these extensions that differentiate our Microkernel instance from the basic TCL Core release. Previously, we relied on a procedure that involved booting a VM using the 'standard' TCL Core ISO, installing (and configuring) the extensions that we needed from the command line, and then creating an overlay (basically a snapshot, in the form of a gzipped tarfile) of key parts of the filesystem from that running TCL instance that we could use to 'automatically' build a Microkernel instance from that same TCL ISO using a couple of scripts (and a few other dependencies that were extracted 'by hand' from the Razor-Microkernel project).
This manual process was fairly tedious and somewhat error prone. Not only that, but the process was difficult (if not impossible) to automate. As such, we decided to take a different approach to building the Razor Microkernel ISO. This new approach uses a bash script (which is part of the Razor-Microcore project itself) to create a single gzipped tarfile (which we'll refer to as the 'bundle file' in this document) that contains an 'overlay file' (another gzipped tarfile), as well as the scripts and dependencies are needed to merge that overlay file with the 'standard' TCL Core ISO (which is also included in the bundle file) in order to build a new version of the Razor Microkernel ISO. The components for the overlay file (and the dependencies that it needs) are all either copied over from the Razor-Microkernel project or downloaded from online sources to local directories (if they are not part of the Razor-Microkernel project) before being included in the bundle file. Below, we will outline the procedure that followed by this script, and in a follow-on page we will describe how the gzipped tarfile (the 'bundle') that is built by this new bash script can be used to easily build your own Razor Microkernel ISO.
The script that is used for this process (the 'build-bundle-file.sh' script) can be found in the top-level directory of the Razor-Microkernel project itself. The usage for this script is readily available by adding a '-h' (or '--help') flag to the command line when invoking the script:
test@server:~/Razor-Microkernel$ ./build-bundle-file.sh -h
Usage: ./build-bundle-file.sh OPTIONS
This script builds a gzipped tarfile containing all of the files necessary to
build an instance of the Razor Microkernel ISO.
OPTIONS:
-h, --help print usage for this command
-r, --reuse-prev-dl reuse the downloads rather than downloading again
-b, --builtin-list FILE file containing extensions to install as builtin
-m, --mirror-list FILE file containing extensions to add to TCE mirror
-p, --build-prod-image build a production ISO (no openssh, no passwd)
-d, --build-debug-image build a debug ISO (enable automatic console login)
-t, --tc-passwd specify a password for the tc user
Note; currently, the default is to build a development ISO (which includes the
openssh.tcz extension along with the openssh/openssl configuration file changes
and the passwd changes needed to access the Microkernel image from the command
line or via the console). Also note that only one of the '-p' and '-d' flags
may be specified and the '-t' option may not be used when building a production
ISO (using the '-p' flag).
test@server:~/Razor-Microkernel$
When invoking this script, the '-b' and '-m' flags (or their more verbose equivalents) MUST be provided (if they are not, then an error will be thrown and the command usage will be printed), and the files that are included as arguments to these two flags must be readable. These files should contain a list of builtin extensions and local TCE mirror extensions (respectively) that should be downloaded and included in the overlay file we are creating. The builtin extensions are those that should be installed as part of the boot process for the Microkernel (making them available when tasks like network initialization take place, for example), while the TCE mirror extensions are simply placed into a local TCE mirror in the Microkernel's filesystem (where they can be used as part of the Microkernel Controller initialization process, post-boot). Examples for these two files are included in the 'additional-build-files' subdirectory in the Razor Microkernel project (the files are called 'builtin-extensions.lst' and 'mirror-extensions.lst', respectively). The other arguments to the script are optional, and default values are used if they are not specified (by default, the previous downloads are not re-used and the system builds a development Microkernel, not a production one). A simple example of the usage for this shell script that uses the default lists of extensions provided as a part of the Razor-Microkernel project is as follows:
./build-bundle-file.sh -r -b additional-build-files/builtin-extensions.lst \
-m additional-build-files/mirror-extensions.lst -d -t test1234 2>&1 | tee t.t
The 'build-bundle-file.sh' script must be run from the top-level directory of the Razor-Microkernel project. When it is invoked, this script will first create a directory structure that looks something like the following:
tmp-build-dir/
└── build_dir
Here, 'tmp-build-dir' is the working directory created by the script and the 'tmp-build-dir/build_dir' subdirectory is the directory we will be using to create the final bundle. This 'bundle' file will contain the overlay file itself (in the 'razor-microkernel-files.tar.gz' file that is placed into the 'tmp-build-dir/build_dir/dependencies' directory), along with all of the files and scripts that are needed in order to turn that to turn that overlay file into a working Razor Microkernel instance.
It should be noted that the sub-directories of the 'tmp-build-dir' directory (with the exception of the 'build_dir' subdirectory) will eventually be mapped into the root-level directories in the overlay file that we are building here (eg. the 'tmp-build-dir/usr/local/bin' directory in the local filesystem will correspond to the 'usr/local/bin' directory in our overlay). We should also note that the final directory structure that results from this build process will be left in place at the completion of the bundle build process (so that the downloads that the directory structure contains can be re-used, if necessary, to build another bundle (with additional extensions?).
After creating the initial 'build_dir' subdirectory (and the 'build_dir/dependencies' subdirectory), the 'build-bundle-file.sh' script copies over the scripts that will be needed to build the Razor Microkernel ISO directly from the 'iso-build-files' subdirectory of the Razor-Microkernel project. These scripts are placed directly into the 'tmp-build-dir/build_dir' subdirectory. If we are building a production ISO, the copy of the 'rebuild_iso.sh' script that is made via this process will also be modified (on the fly using 'sed') so that it outputs an ISO file with the string 'prod' in its name (rather than the string 'dev') when it is invoked. The next step in the process is to copy over the modified DHCP client scripts from the Razor-Microkernel project to the corresponding locations in the 'tmp-build-dir'. After that, the script copies over the Razor Microkernel Controller script (and its dependencies) into the appropriate locations in the 'tmp-build-dir/usr/local/bin' and 'tmp-build-dir/usr/local/lib/razor_microcontroller' subdirectories. Finally, the script copies over the list of gems from the 'opt/gems/gem.list' file of the Razor-Microkernel project into the 'tmp-build-dir/opt/gems' subdirectory. With that step, the process of copying files from the Razor Microkernel project into the directories that we will be creating our overlay from is complete and we are ready to start downloading some dependencies.
The download process starts by using the list of gems that was copied over from the Razor-Microkernel project (above) to determine what gems should be downloaded (so that they will be included in the overlay file). The gems in that list are downloaded to the 'tmp-build-dir/opt/gems' directory using the 'gem fetch...' command (Note; this assumes that the local system has RubyGems already installed on it and that the 'gem' command is accessible by the user running this shell script). With the gems in place, the script then downloads two sets of extensions from the TCL Extension Repository (or TCE repository). The first set of extensions downloaded are used to construct a local TCE mirror, and the second set are placed into a subdirectory from which they will automatically be installed during the boot process. The actual extensions that are installed are read from flat files containing the list of extensions to include in the local mirror and a list of extensions to set up as 'built-in extensions' (which appear in the 'additional-build-files/mirror-extensions.lst' and 'additional-build-files/builtin-extensions.lst' files in the Razor-Microkernel project, respectively). Files from the first list will be placed into a mirror subdirectory under the 'tmp-build-dir/tmp/tinycorelinux' subdirectory, while those from the second will be placed into the 'tmp-build-dir/tmp/builtin' subdirectory. In the second case, the list of extensions that is loaded is also used to construct a 'tmp-build-dir/tmp/builtin/onboot.lst' file that will be used by the Microkernel to determine which of the 'built-in' extensions should be loaded on boot.
After the 'bundle initialization' process is complete, the directory structure will actually look a bit more complicated (currently, this is what it looks like after initializing the temporary build directory but before building the bundle):
build-files
└── razor-microkernel-bundle-dev.tar.gz
tmp-build-dir
├── build_dir
│ ├── add_version_to_mk_fs.rb
│ ├── build_initial_directories.sh
│ ├── build_iso_yaml.rb
│ ├── Core-current.iso
│ ├── dependencies
│ │ ├── mcollective-setup-files.tar.gz
│ │ ├── mk-open-vm-tools.tar.gz
│ │ ├── razor-microkernel-overlay.tar.gz
│ │ └── ssh-setup-files.tar.gz
│ └── rebuild_iso.sh
├── etc
│ ├── init.d
│ │ └── dhcp.sh
│ ├── inittab
│ ├── passwd
│ └── shadow
├── mcollective-2.0.0.tgz
├── opt
│ ├── bootsync.sh
│ ├── gems
│ │ └── ...
│ └── rubygems-1.8.24.tgz
├── tmp
│ ├── builtin
│ │ ├── onboot.lst
│ │ └── optional
│ │ └── ...
│ ├── first_checkin.yaml
│ ├── mk_conf.yaml
│ └── tinycorelinux
│ ├── 4.x
│ │ └── x86
│ │ └── tcz
│ │ └── ...
│ └── ...
├── usr
│ ├── local
│ │ ├── bin
│ │ │ ├── lscpu
│ │ │ └── ...
│ │ ├── lib
│ │ │ └── ruby
│ │ │ └── 1.8
│ │ │ └── razor_microkernel
│ │ │ └── ...
│ │ ├── mcollective -> /usr/local/tce.installed/mcollective-2.0.0
│ │ ├── sbin
│ │ │ └── sfdisk
│ │ └── tce.installed
│ │ └── mcollective-2.0.0
│ │ └── ...
│ ├── sbin
│ │ └── dmidecode -> /usr/local/sbin/dmidecode
│ └── share
│ └── udhcpc
│ └── dhcp_mk_config.script
└── util-linux.tcz
With this directory structure in place, building the actual bundle is a very simple process (consisting of one command in the 'build-bundle-file.sh' script). When this process is complete, a single bundle file will be added to the 'build-files' subdirectory of the Razor-Microkernel project (named 'razor-microkernel-bundle-dev.tar.gz' for a development bundle and 'razor-microkernel-bundle-prod.tar.gz' for a production bundle). That bundle file will contain all of the scripts and files that are needed to build a new Microkernel ISO from the current Razor-Microkernel project (and the latest versions of its dependencies). This is what that current contents of a development bundle look like:
-rw-rw-r-- tjmcs/tjmcs 8290304 2012-06-04 22:24 Core-current.iso
-rwxrwxr-x tjmcs/tjmcs 1385 2012-05-02 13:55 add_version_to_mk_fs.rb
-rwxrwxr-x tjmcs/tjmcs 1263 2012-06-19 16:50 build_initial_directories.sh
-rwxrwxr-x tjmcs/tjmcs 2501 2012-05-02 13:25 build_iso_yaml.rb
drwxrwxr-x tjmcs/tjmcs 0 2012-06-21 16:27 dependencies/
-rw-rw-r-- tjmcs/tjmcs 30176283 2012-06-21 16:27 dependencies/razor-microkernel-overlay.tar.gz
-rw-rw-r-- tjmcs/tjmcs 813074 2012-06-08 14:19 dependencies/mk-open-vm-tools.tar.gz
-rw-r--r-- tjmcs/tjmcs 5723 2012-06-07 19:45 dependencies/ssh-setup-files.tar.gz
-rw-r--r-- tjmcs/tjmcs 374 2012-06-06 15:30 dependencies/mcollective-setup-files.tar.gz
-rwxr-xr-x tjmcs/tjmcs 1414 2012-05-02 13:46 rebuild_iso.sh
To use this bundle file, simply copy it over to another directory on this (or another) machine and unpack it
# tar zxvf razor-microkernel-bundle-dev.tar.gz
From that same directory, the sequence of commands to build a new ISO is actually quite simple, first, build the directory structure needed to for the script that constructs the new ISO:
# ./build_initial_directories.sh
and then invoke the script that actually builds the ISO
# ./rebuild_iso.sh 0.9.0.1
Note; the single argument to this second script will create an Razor Microkernel ISO with a version number of 0.9.0.1. You can replace this argument with any version number you would like to use (although we tend to use 4 digit version numbers like the one shown above in current releases of the Razor Microkernel).
Once the second shell script command finishes, you should have a new Razor Microkernel ISO build in the current working directory; an ISO that is suitable for use with Razor and that contains all of the dependencies and extensions included in the bundle build process (above). If, for some reason, you find an issue with the ISO you have built, rebuilding a new ISO is as simple as running the same three shell scripts that are shown, above; perhaps with a new set of built-in and/or additional extensions declared during the initial bundle build process.
There are a few dependencies that must be met for this build process to run successfully. The bundle file build process (described above) relies on the 'unsquashfs' command to extract the 'lscpu' and 'sfdisk' commands from the 'util-linux.tcz' extension that it downloads from the standard Tiny Core Linux Extension Mirror. That same script also makes extensive use of the 'wget' command to download the dependencies that it needs and uses the 'gem' command to 'fetch' the gems that are needed from the 'Ruby Gems' gem repository. Finally, the dynamic downloading of the dependencies and extensions that need to be included in the overlay file that is being constructed require that an active network connection exists (with access to the internet) and that the local Razor-Microkernel project directory be writeable by the current user. If any of these dependencies are not met, the bundle file build process will fail.
The script that builds the Microkernel ISO also has a number of dependencies on system level commands that may or may not exist on all platforms (and that may or may not be accessible for all users on any given platform). It uses the 'chroot' and 'ldconfig' commands to set up the contents of the 'tmp' subdirectory that it uses when building the 'core.gz' file that will be placed into the Microkernel ISO. If these commands are not accessible (or are not usable by the current user), then the resulting ISO may or may not be usable as a boot image by Razor. This script also uses the 'cpio' command to construct the 'core.gz' file and uses the 'advdef' command to further compress that file before placing it into the Microkernel ISO. We have seen issues on some platforms with the flags for these commands not being consistent (or with these commands not being available at all). The build process has been successfully tested and used under recent releases of Ubuntu (the 64-bit server release, but that's probably not important), but failed for some users who were trying to use the same scripts under OS X. The Ubuntu Server instance we have used most extensively uses the GNU cpio command and advancecomp v1.15 (for the 'advdef' command). This build process hasn't much testing on other platforms to date, so your mileage may vary in terms when it comes to using this build process (depending on the platform that you use).
By following the procedure outlined, above, we were able to create 'bundle file' containing an 'overlay file' that can be used to add a number of extensions to the default TCL Core distribution. We were then able to use that 'bundle file' (the 'overlay file', additional dependencies, and scripts that it contains) to build a new Microkernel ISO from the standard TCL Core distribution ISO.
The current 'build-bundle-file.sh' script can be found here. For convenience, we've also provided a nearly complete 'tmp-build-dir' directory structure for reference (showing what this directory structure looks like after the 'build-bundle-file.sh' script has been run). We say nearly complete because we're only showing the 'tcz' files for the extensions, not the associated md5, list, info, and dep files, and we've cut out all of the files and subdirectories under the 'tmp-build-dir/usr/local/tce.installed/mcollective-2.0.0' directory, but the rest of the structure is intact).
Followed by the post-build directory structure:
tmp-build-dir/
├── build_dir
│ ├── add_version_to_mk_fs.rb
│ ├── build_initial_directories.sh
│ ├── build_iso_yaml.rb
│ ├── Core-current.iso
│ ├── dependencies
│ │ ├── mcollective-setup-files.tar.gz
│ │ ├── mk-open-vm-tools.tar.gz
│ │ ├── razor-microkernel-overlay.tar.gz
│ │ └── ssh-setup-files.tar.gz
│ └── rebuild_iso.sh
├── etc
│ ├── init.d
│ │ └── dhcp.sh
│ ├── inittab
│ ├── passwd
│ └── shadow
├── mcollective-2.0.0.tgz
├── opt
│ ├── bootsync.sh
│ ├── gems
│ │ ├── daemons-1.1.8.gem
│ │ ├── facter-1.6.9.gem
│ │ ├── gem.list
│ │ ├── json_pure-1.7.3.gem
│ │ └── stomp-1.2.2.gem
│ └── rubygems-1.8.24.tgz
├── tmp
│ ├── builtin
│ │ ├── onboot.lst
│ │ └── optional
│ │ ├── bash.tcz
│ │ ├── dmidecode.tcz
│ │ ├── firmware-bnx2.tcz
│ │ ├── gcc_libs.tcz
│ │ ├── libssl-0.9.8.tcz
│ │ ├── lshw.tcz
│ │ ├── openssh.tcz
│ │ ├── openssl-1.0.0.tcz
│ │ ├── ruby.tcz
│ │ └── scsi-3.0.21-tinycore.tcz
│ ├── first_checkin.yaml
│ ├── mk_conf.yaml
│ └── tinycorelinux
│ ├── 4.x
│ │ └── x86
│ │ └── tcz
│ │ ├── fuse.tcz
│ │ ├── glib2-dev.tcz
│ │ ├── glib2.tcz
│ │ ├── libdnet.tcz
│ │ ├── libffi-dev.tcz
│ │ ├── libffi.tcz
│ │ ├── ncurses-common.tcz
│ │ ├── ncurses.tcz
│ │ ├── perl5.tcz
│ │ └── procps.tcz
│ ├── kmod-install-list.yaml
│ └── tce-install-list.yaml
├── usr
│ ├── local
│ │ ├── bin
│ │ │ ├── lscpu
│ │ │ ├── mcollectived -> /usr/local/mcollective/bin/mcollectived
│ │ │ ├── rz_mk_controller.rb
│ │ │ ├── rz_mk_control_server.rb
│ │ │ ├── rz_mk_init.rb
│ │ │ ├── rz_mk_tce_mirror.rb
│ │ │ └── rz_mk_web_server.rb
│ │ ├── lib
│ │ │ └── ruby
│ │ │ └── 1.8
│ │ │ └── razor_microkernel
│ │ │ ├── logging.rb
│ │ │ ├── rz_host_utils.rb
│ │ │ ├── rz_mk_configuration_manager.rb
│ │ │ ├── rz_mk_fact_manager.rb
│ │ │ ├── rz_mk_gem_controller.rb
│ │ │ ├── rz_mk_hardware_facter.rb
│ │ │ ├── rz_mk_kernel_module_manager.rb
│ │ │ ├── rz_mk_registration_manager.rb
│ │ │ └── rz_network_utils.rb
│ │ ├── mcollective -> /usr/local/tce.installed/mcollective-2.0.0
│ │ ├── sbin
│ │ │ └── sfdisk
│ │ └── tce.installed
│ │ └── mcollective-2.0.0
│ │ └── ...
│ ├── sbin
│ │ └── dmidecode -> /usr/local/sbin/dmidecode
│ └── share
│ └── udhcpc
│ └── dhcp_mk_config.script
└── util-linux.tcz