Skip to content

Commit

Permalink
Update main docs
Browse files Browse the repository at this point in the history
  • Loading branch information
actions-user committed Sep 27, 2024
1 parent 4a867d1 commit e8d4ac4
Show file tree
Hide file tree
Showing 30 changed files with 258 additions and 191 deletions.
26 changes: 2 additions & 24 deletions static/docs/main/_sources/contents/configuration.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ explained as follows:
When ``true``, ``wwclient`` uses TCP port 987.

Changing this option requires rebuilding node overlays and rebooting
compute nodes, to configure them to use a privileged port.
compute nodes to configure them to use a privileged port.

* ``warewulf:update interval``: This defines the frequency (in
seconds) with which the Warewulf client on the compute node fetches
Expand Down Expand Up @@ -205,29 +205,7 @@ a "runtime overlay" specified, the respective value from
``defaultnode`` is used. If a network device does not specify a
"device," the device value of the ``dummy`` device is used.

If ``defaults.conf`` does not exist, the following values are used as
compiled into Warewulf at build-time:

.. code-block:: yaml
--
defaultnode:
runtime overlay:
- generic
system overlay:
- wwinit
kernel:
args: quiet crashkernel=no vga=791 net.naming-scheme=v238
init: /sbin/init
root: initramfs
ipxe template: default
profiles:
- default
network devices:
dummy:
device: eth0
type: ethernet
netmask: 255.255.255.0
If ``defaults.conf`` does not exist, compiled-in defaults are used.

There should never be a need to change this file: all site-local
parameters should be specified using either nodes or profiles.
Expand Down
2 changes: 1 addition & 1 deletion static/docs/main/_sources/contents/disks.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ Ignition Implementation
=======================

The ignition implementation uses systemd services, as the underlying ``sgdisk`` command relies on dbus notifications.
All necessary services are distributed by the ``wwinit`` overlay and depends on the existence of the file ``/warewulf/ignition.json``.
All necessary services are distributed by the ``ignition`` overlay and depends on the existence of the file ``/warewulf/ignition.json``.
This file is created by the template function ``{{ createIgnitionJson }}`` only if the configuration contains necessary specifications for disks, partitions, and file systems.
If the file ``/warewulf/ignition.json`` exists, the service ``ignition-disks-ww4.service`` calls the ignition binary which takes creates partitions and file systems.
A systemd ``.mount`` unit is created for each configured file system, which also creates the necessary mount points in the root file system.
Expand Down
8 changes: 4 additions & 4 deletions static/docs/main/_sources/contents/initialization.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,10 @@ properly. Here are the things it will do:
the system service.
* **hostfile**: Update the system's /etc/hosts file based on the
**host** template ``/etc/hosts.ww``.
* **nfs**: Configure the NFS server on the control node as well as the
``/etc/fstab`` in the system overlay based on the configuration in
``/etc/warewulf/warewulf.conf`` and enable the system service. Also
the file ``/etc/exports.ww`` from the **host** template is installed.
* **nfs**: Configure the NFS server on the control node based on the
configuration in ``/etc/warewulf/warewulf.conf`` and enable the NFS
service. Also the file ``/etc/exports.ww`` from the **host** template is
installed.
* **ssh**: Create the appropriate host keys (stored in
``/etc/warewulf/keys/``) and user keys for passwordless ``ssh`` into
the nodes. Addionally the shell profiles
Expand Down
6 changes: 3 additions & 3 deletions static/docs/main/_sources/contents/ipmi.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ individual node would be the IP address.

The settings are only written to the IPMI interface if ``--ipmiwrite``
is set to `true`. The write process happens at every boot of the node
through the script ``/warewulf/init.d/50-ipmi`` in the **system**
through the script ``/warewulf/init.d/50-ipmi`` in the wwinit
overlay.

If an individual node has different settings, you can set the IPMI
Expand Down Expand Up @@ -99,8 +99,8 @@ Node View
n001 cluster -- --
n001 container default sle-micro-5.3
n001 ipxe -- (default)
n001 runtime -- (generic)
n001 wwinit -- (wwinit)
n001 runtime -- (hosts,ssh.authorized_keys,syncuser)
n001 wwinit -- (wwinit,wwclient,fstab,hostname,ssh.host_keys,issue,resolv,udev.netname,systemd.netname,ifcfg,NetworkManager,debian.interfaces,wicked,ignition)
n001 root -- (initramfs)
n001 discoverable -- --
n001 init -- (/sbin/init)
Expand Down
8 changes: 4 additions & 4 deletions static/docs/main/_sources/contents/nodeconfig.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -87,8 +87,8 @@ You can also see the node's full attribute list by specifying the
n001 cluster -- --
n001 container default sle-micro-5.3
n001 ipxe -- (default)
n001 runtime -- (generic)
n001 wwinit -- (wwinit)
n001 runtime -- (hosts,ssh.authorized_keys,syncuser)
n001 wwinit -- (wwinit,wwclient,fstab,hostname,ssh.host_keys,issue,resolv,udev.netname,systemd.netname,ifcfg,NetworkManager,debian.interfaces,wicked,ignition)
n001 root -- (initramfs)
n001 discoverable -- --
n001 init -- (/sbin/init)
Expand Down Expand Up @@ -193,8 +193,8 @@ container, kernel, and network:
n001 cluster -- --
n001 container default sle-micro-5.3
n001 ipxe -- (default)
n001 runtime -- (generic)
n001 wwinit -- (wwinit)
n001 runtime -- (hosts,ssh.authorized_keys,syncuser)
n001 wwinit -- (wwinit,wwclient,fstab,hostname,ssh.host_keys,issue,resolv,udev.netname,systemd.netname,ifcfg,NetworkManager,debian.interfaces,wicked,ignition)
n001 root -- (initramfs)
n001 discoverable -- --
n001 init -- (/sbin/init)
Expand Down
149 changes: 103 additions & 46 deletions static/docs/main/_sources/contents/overlays.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
Warewulf Overlays
=================

So at this point, we have discussed how Warewulf is designed to
At this point we have discussed how Warewulf is designed to
scalably provision and manage thousands of cluster nodes by utilizing
identical stateless boot images. And there-in lies a problem to
solve. If these boot images are completely identical, then how do we
Expand All @@ -19,57 +19,114 @@ through the provisioning process. A node or profile can configure an
overlay in two different ways:

* An overlay can be configured to run during boot as part of the
``wwinit`` process. These overlays are called **system overlay** or
**wwinit overlays**.
``wwinit`` process. These overlays are called **system overlays**.
* An overlay can be configured to run periodically while the system is
running. These overlays are called **runtime overlays** or **generic
overlays**.
running. These overlays are called **runtime overlays**.

The default profile has both a **wwinit** and a **runtime** overlay
configured.

Overlays are compiled for each compute node individually.

Defined Overlays
================
Provided overlays
=================

These overlays are provided as part of Warewulf.

wwinit
------

The **wwinit** overlay performs initial configuration of the Warewulf node.
Its `wwinit` script runs before ``systemd`` or other init is called and
contains all configurations which are needed to boot.

In particular:

- Configure the loopback interface
- Configure the BMC based on the node's configuration
- Update PAM configuration to allow missing shadow entries
- Relabel the file system for SELinux

Other overlays may place additional scripts in ``/warewulf/init.d/`` to affect
node configuration in this pre-boot environment.

wwclient
--------

All configured overlays are provisioned initially along with the node image
itself; but **wwclient** periodically fetches and applies the runtime overlay
to allow configuration of some settings without a reboot.

Network interfaces
------------------

Warewulf ships with support for many different network interface configuration
systems. All of these are applied by default; but the list may be trimmed to
the desired system.

- ifcfg
- NetworkManager
- debian.interfaces
- wicked

Warewulf also configures both systemd and udev with the intended names of
configured network interfaces, typically based on a known MAC address.

System or wwinit overlay
------------------------
- systemd.netname
- udev.netname

This overlay contains all the necessary scripts to provision a
Warewulf node. It is available before the ``systemd`` or other init is
called and contains all configurations which are needed to bring up
the compute node. It is not updated during run time. Besides the
network configurations for
Basics
------

The **hostname** overlay sets the hostname based on the configured Warewulf
node name.

The **hosts** overlay configures ``/etc/hosts`` to include all Warewulf nodes.

The **issue** overlay configures a standard Warewulf status message for display
during login.

The **resolv** overlay configures ``/etc/resolv.conf`` based on the value of
"DNS" nettags. (In most situations this should be unnecessary, as the network
interface configuration should handle this dynamically.)

fstab
-----

* wicked
* NetworkManager
* EL legacy network scripts
The **fstab** overlay configures ``/etc/fstab`` to mount NFS shares defined in
``/etc/warewulf.conf`` and file systems created by Ignition.

it also contains udev rules, which will set the interface names
based on the hardware addresses configured for the node.
Before the ``systemd`` init is called, the overlay loops through the scripts in
``/wwinit/warwulf/init.d/*`` which will setup
ssh
---

* Ipmi
* wwclient
* selinux
Two SSH overlays configure host keys (one set for all node in the cluster) and
``authorized_keys`` for the root account.

Runtime Overlay or generic Overlay
----------------------------------
- ssh.authorized_keys
- ssh.host_keys

The runtime overlay is updated by the ``wwclient`` service on a
regular basis (by default, once per minute). In the standard
configuration it includes updates for ``/etc/passwd``, ``/etc/group``
and ``/etc/hosts``. Additionally the ``authorized_keys`` file of the
root user is updated. It is recommended to use this overlay for
dynamic configuration files like ``slurm.conf``. Once the system is
provisioned and booted, the ``wwclient`` program (which is provisioned
as part of the ``wwinit`` system overlay) will continuously update the
node with updates in the runtime overlay.
syncuser
--------

Host Overlay
------------
The **syncuser** overlay updates ``/etc/passwd`` and ``/etc/group`` to include
all users on both the Warewulf server and from the container image. To function
properly, ``wwctl container syncuser`` must have also been run on the container
image to synchronize its user and group IDs with those of the server.

ignition
--------

The **ignition** overlay defines partitions and file systems on local disks.

debug
-----

The **debug** overlay is not intended to be used in configuration, but can be
used as an example and applied with ``wwctl overlay show --render <nodename>
debug /warewulf/template-variables.md.ww``.

host
----

Configuration files used for the configuration of the Warewulf host /
server are stored in the **host** overlay. Unlike other overlays, it
Expand All @@ -91,25 +148,25 @@ The following services get configuration files via the host overlay:
Combining Overlays
==================

When changing the overlays, it is recommended not to change them, but
to add the changed files to a new overlay and combine them in the
We recommended not changing the provided overlays. Rather,
add the changed files to a new overlay and combine them in the
configuration. This is possible as the configuration fields for the
**wwinit** and **runtime** overlays are lists and can contain several
system and runtime overlays are lists and can contain several
overlays. As an example for this, we will overwrite the
``/etc/issue`` file from the **wwinit** overlay. For this we will
create a new overlay called welcome and import the file ``/etc/issue``
``/etc/issue`` file from the "issue" overlay. For this we will
create a new overlay called "welcome" and import the file ``/etc/issue``
from the host to it. This overlay is then combined with the existing
**wwinit** overlay.
overlays.

.. code-block:: console
# wwctl overlay create welcome
# wwctl overlay mkdir welcome /etc
# wwctl overlay import welcome /etc/issue
# wwctl profile set default --wwinit=wwinit,welcome
# wwctl profile set default --wwinit=wwinit,wwclient,welcome
? Are you sure you want to modify 1 profile(s)? [y/N] y
# wwctl profile list default -a |grep welcome
default SystemOverlay wwinit,welcome
default SystemOverlay wwinit,wwclient,welcome
Templates
=========
Expand Down Expand Up @@ -143,7 +200,7 @@ permissions, etc.

..
note::
There is no possibility to delete files with an overlay!
It is not possible to delete files with an overlay.
Build
-----
Expand Down
4 changes: 2 additions & 2 deletions static/docs/main/_sources/contents/provisioning.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ occur:
#. The Warewulf server will generate the iPXE configuration which
will include directions of what else is necessary to download
and how to boot.
#. The kernel, container image, kernel modules, and system overlay
#. The kernel, container image, kernel modules, and overlays
are all downloaded over REST HTTP from the Warewulf Server
#. iPXE executes the kernel and processes the overlays to provide
a unified root file system
Expand All @@ -81,7 +81,7 @@ through the following command :
# wwctl node status
NODENAME STAGE SENT LASTSEEN (s)
================================================================================
c001 RUNTIME_OVERLAY generic.img.gz 16
c001 RUNTIME_OVERLAY __RUNTIME__.img.gz 16
For each node, there is 4 different stages :
Expand Down
4 changes: 2 additions & 2 deletions static/docs/main/_sources/contents/setup.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -118,8 +118,8 @@ You can check the result with ``wwctl node list``.
deliverynet Comment default This profile is automatically included for each node
deliverynet ContainerName default leap15.5
deliverynet Ipxe -- (default)
deliverynet RuntimeOverlay -- (generic)
deliverynet SystemOverlay -- (wwinit)
deliverynet RuntimeOverlay -- (hosts,ssh.authorized_keys,syncuser)
deliverynet SystemOverlay -- (wwinit,wwclient,fstab,hostname,ssh.host_keys,issue,resolv,udev.netname,systemd.netname,ifcfg,NetworkManager,debian.interfaces,wicked,ignition)
deliverynet Root -- (initramfs)
deliverynet Init -- (/sbin/init)
deliverynet Kernel.Args -- (quiet crashkernel=no vga=791 net.naming-scheme=v238)
Expand Down
2 changes: 1 addition & 1 deletion static/docs/main/_sources/contents/templating.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ on demand.

In Warewulf, you can find templates both for the provisioning services
(e.g. ``/etc/warewulf/ipxe/``, ``/etc/warewulf/dhcp/``, and
``/etc/warewulf/hosts.tmpl``) as well as within the runtime and system
``/etc/warewulf/hosts.tmpl``) as well as within many of the provided
overlays.

(more documentation coming soon)
Expand Down
2 changes: 1 addition & 1 deletion static/docs/main/_sources/contents/wwctl.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ component of Warewulf's functionality.
* ``node`` manages nodes in the cluster
* ``profiles`` defines configuration which can be applied to multiple
nodes
* ``overlays`` manages nodes
* ``overlays`` manages overlays
* ``power`` turns nodes on and off
* ``ssh`` provides basic parallel ssh functionality

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -98,8 +98,8 @@ Build and install Warewulf on wwdev
# Configure system service automatically
sudo wwctl configure dhcp # Create the default dhcpd.conf file and start/enable service
sudo wwctl configure tftp # Install the base tftp/PXE boot files and start/enable service
sudo wwctl configure nfs # Configure the exports and create an fstab in the default system overlay
sudo wwctl configure ssh # Build the basic ssh keys to be included by the default system overlay
sudo wwctl configure nfs # Configure NFS exports
sudo wwctl configure ssh # Build the basic ssh keys to be included by the ssh overlays
# Pull and build the VNFS container and kernel
sudo wwctl container import docker://ghcr.io/warewulf/warewulf-centos:7 centos-7 --setdefault
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -113,8 +113,8 @@ I have VirtualBox running on my desktop.
# Configure system service automatically
sudo wwctl configure dhcp --persist # Create the default dhcpd.conf file and start/enable service
sudo wwctl configure tftp --persist # Install the base tftp/PXE boot files and start/enable service
sudo wwctl configure nfs --persist # Configure the exports and create an fstab in the default system overlay
sudo wwctl configure ssh --persist # Build the basic ssh keys to be included by the default system overlay
sudo wwctl configure nfs --persist # Configure the NFS exports
sudo wwctl configure ssh --persist # Build the basic ssh keys to be included by the SSH overlay
# Pull and build the VNFS container and kernel
sudo wwctl container import docker://ghcr.io/warewulf/warewulf-centos:7 centos-7 --setdefault
Expand Down
2 changes: 1 addition & 1 deletion static/docs/main/_sources/quickstart/debian12.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -201,7 +201,7 @@ typing the following:
sudo wwctl node list -a n0000.cluster
To make node changes effective, it is a good practice to update warewulf
To make node changes effective, it is a good practice to update Warewulf
overlays with the following command:

.. code-block:: bash
Expand Down
2 changes: 1 addition & 1 deletion static/docs/main/_sources/quickstart/suse15.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,7 @@ called. This enables us to prepopulate node configurations with
content that is node specific like networking and service
configurations.

Runtime overlays are provisioned after the node has booted and
Runtime overlays are re-applied
periodically during the normal runtime of the node. Because these
overlays are provisioned at periodic intervals, they are very useful
for content that changes, like users and groups.
Expand Down
Loading

0 comments on commit e8d4ac4

Please sign in to comment.