From 1735af7596d1c39fcff1e4f335062d6fa62d86be Mon Sep 17 00:00:00 2001
From: mikee47 <mike@sillyhouse.net>
Date: Thu, 27 Jun 2024 11:09:56 +0100
Subject: [PATCH] Tidy Host Emulator documentation

Put everything into Host/README, arch/host/index is just a landing page.
Clarify availability of ARCH_xx and SOC_xx macros.
---
 Sming/Arch/Host/README.rst      | 172 +++++++++++++++++++++++++++--
 Sming/building.rst              |  11 ++
 docs/source/arch/host/index.rst | 189 +-------------------------------
 3 files changed, 177 insertions(+), 195 deletions(-)

diff --git a/Sming/Arch/Host/README.rst b/Sming/Arch/Host/README.rst
index 7519c2063f..2bbb67f175 100644
--- a/Sming/Arch/Host/README.rst
+++ b/Sming/Arch/Host/README.rst
@@ -15,40 +15,192 @@ code prior to flashing a real device.
 This is not a machine emulator; if you need something operating at a
 lower level take a look at `QEMU <https://www.qemu.org/>`__.
 
+The design goals for Host builds are:
+
+- Simplify development of complex applications
+- Enable use of advanced testing and code quality tools (valgrind, code sanitizers)
+- Support CI testing by actually executing code, not just building it
+
+
 Requirements
 ------------
 
-``CMake`` is required to build LWIP
+``CMake`` and ``Ninja`` are required to build some libraries (e.g. lwip).
+
+Tested compilers:
+
+- GCC version 8 or later
+- Clang version 15 or later
+- Apple-clang version 14 or later
+
+.. _linux_requirements:
+
+Linux
+~~~~~
+
+Host executables can be built in 32-bit emulation mode, which is the default for Linux.
+This requires installation of 32-bit runtime libraries.
+For debian/ubuntu::
+
+   sudo apt install g++-multilib
+
+For Fedora::
+
+   sudo dnf install glibc-devel.i686 libstdc++.i686
 
-Ensure you are using relatively recent compilers, with 32-bit libraries available.
-For Linux, you may require the ``gcc-multilib`` and ``g++-multilib``
-packages to build 32-bit executables on a 64-bit OS.
 Alternatively set :envvar:`BUILD64` to 1 to build in native 64-bit mode.
 
+MacOS
+~~~~~
+
 MacOS comes pre-installed with ``Apple Clang`` as the standard toolchain.
 This should be sufficient to build Sming in Host mode.
 Note that MacOS does not support 32-bit applications so the emulator will build in 64-bit mode.
 
+Windows
+~~~~~~~
+
 For Windows, make sure your ``MinGW`` distro is up to date.
 See :doc:`/getting-started/windows/index` for further details.
 
+
 Building
 --------
 
-Build the framework and application as usual, specifying :envvar:`SMING_ARCH` =Host. For example::
+Environment variables
+~~~~~~~~~~~~~~~~~~~~~
+
+:envvar:`SMING_ARCH` must be set to use ``Host`` as the desired architecture::
 
-   cd $SMING_HOME/../samples/Basic_Serial
-   make SMING_ARCH=Host
+   export SMING_ARCH=Host
 
-This builds the application as an executable in, for example,
-``out/Host/firmware/app.exe``. Various command-line options are
-supported, use ``--help`` for details.
+Debug Build
+~~~~~~~~~~~
+
+If you plan to use a debugger make sure to set :envvar:`ENABLE_GDB` and (optionally)
+:envvar:`ENABLE_LWIPDEBUG` before compiling the code::
+
+   export ENABLE_GDB=1
+   export ENABLE_LWIPDEBUG=1  # <!-- this will compile also LWIP with debug symbols
+
+Initial Clean-up
+~~~~~~~~~~~~~~~~
+
+Make sure that you are working on a clean source code tree. The following commands may help::
+
+   cd $SMING_HOME
+   make dist-clean
+
+Sample compilation
+~~~~~~~~~~~~~~~~~~
+
+Compile a sample as usual::
+
+   cd $SMING_HOME/../samples/Basic_Blink
+   make
 
 The easiest way to run the emulator is via ``make run``. Variables are
 used to pass the appropriate options to the command line.
 
 To find out what options are in force, use ``make list-config``.
 
+Note: The application is built as an executable in, for example, ``out/Host/firmware/app``,
+which may be run directly. Various command-line options are supported, use ``--help`` for details.
+
+Adapting existing code
+~~~~~~~~~~~~~~~~~~~~~~
+
+You may need to modify your existing applications to work with the emulator.
+
+Architecture-specific code should be kept in separate code modules.
+These can be build and linked as required using makefile logic.
+For example, source code required only for Host builds can be placed in ``app/Host``
+with these lines in the project's ``component.mk`` file::
+
+   ifeq ($(SMING_ARCH),Host)
+   COMPONENT_SRCDIRS += app/Host
+   endif
+
+There are plenty of examples of this in the framework and associated libraries,
+such as :sample:`Basic_Tasks`.
+
+Pre-processor macros are also defined for code use, such as :c:macro:`ARCH_ESP8266`.
+So code that is specific to the ESP8266, for example, may be conditionally compiled using ``#ifdef ARCH_ESP8266``.
+Similarly, you can check if the code is compiled for the emulator using ``#ifdef ARCH_HOST``.
+See :envvar:`SMING_ARCH` and :envvar:`SMING_SOC`.
+
+
+Running on the host
+-------------------
+
+Once the sample is compiled, initialise the :component-host:`vflash`::
+
+   make flash
+
+This writes all the prepared partitions (including any filing systems such as SPIFFS)
+to the virtual flash backing file.
+
+To run the application, do this::
+
+   make run
+
+
+Networking
+~~~~~~~~~~
+
+Support is provided via TAP network interface (a virtual network layer
+operating at the ethernet frame level). A TAP interface must be created
+first, and requires root privilege.
+
+For Linux, you can use ``Sming/Arch/Host/Tools/setup-network-linux.sh``. Here is the manual approach::
+
+   sudo ip tuntap add dev tap0 mode tap user `whoami`
+   sudo ip a a dev tap0 192.168.13.1/24
+   sudo ip link set tap0 up
+
+   # The following lines are needed if you plan to access Internet
+   sudo sysctl net.ipv4.ip_forward=1
+   sudo sysctl net.ipv6.conf.default.forwarding=1
+   sudo sysctl net.ipv6.conf.all.forwarding=1
+
+   export INTERNET_IF=wlan0 # <!--- Make sure to replace wlan0 with the network interface connected to Internet
+
+   sudo iptables -t nat -A POSTROUTING -o $INTERNET_IF -j MASQUERADE
+   sudo iptables -A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
+   sudo iptables -A FORWARD -i tap0 -o $INTERNET_IF -j ACCEPT
+
+This creates the ``tap0`` interface. The emulator will automatically
+select the first ``tap`` interface found. To override this, use the
+``--ifname`` option.
+
+You can list available network interfaces thus::
+
+   ip link
+
+
+For MacOS, networking support first requires installation of a kernel extension
+using ``Sming/Arch/Host/Tools/setup-network-macos.sh``.
+On first run, you will need to confirm security then reboot the system.
+On rebooting, run the script again.
+
+The ``tap0`` network interface is created dynamically when ``/dev/tap0`` is opened.
+At present, you must run the application with elevated permissions::
+
+   sudo out/Host/debug/firmware/app
+
+
+
+Troubleshooting
+---------------
+
+The following errors may be encountered when building:
+
+- fatal error: sys/cdefs.h: No such file or directory
+- fatal error: bits/c++config.h: No such file or directory
+
+These indicate that the 32-bit support libraries are not installed.
+See :ref:`Linux Requirements<linux_requirements>`.
+
 
 Configuration
 -------------
diff --git a/Sming/building.rst b/Sming/building.rst
index 39a3f4b44b..7bd6550a22 100644
--- a/Sming/building.rst
+++ b/Sming/building.rst
@@ -46,6 +46,13 @@ These are the main variables you need to be aware of:
 
    -  **Rp2040** Supports Raspberry Pi RP2040-based boards.
 
+   Exactly **one** of the following macros is defined for code use:
+
+      .. c:macro:: ARCH_ESP8266
+      .. c:macro:: ARCH_RP2040
+      .. c:macro:: ARCH_ESP32
+      .. c:macro:: ARCH_HOST
+
 
 .. envvar:: SMING_SOC
 
@@ -54,6 +61,10 @@ These are the main variables you need to be aware of:
 
    Will automatically set SMING_ARCH to the appropriate value.
 
+   A corresponding macro is defined for code use, such as:
+
+      .. c:macro:: SOC_ESP32S2
+
 
 .. envvar:: SMING_CPP_STD
 
diff --git a/docs/source/arch/host/index.rst b/docs/source/arch/host/index.rst
index 2c75d60fda..85a372a44e 100644
--- a/docs/source/arch/host/index.rst
+++ b/docs/source/arch/host/index.rst
@@ -3,194 +3,13 @@ Host Emulator
 
 .. highlight:: bash
 
-The latest versions of Sming allow most of components of the library and
-the sample applications to be compiled on a Linux/Windows host system
-and be tested before uploading them to the microcontroller.
+Sming allows most libraries and sample applications to be compiled on a Linux/MacOs/Windows
+development system and be tested before uploading them to the microcontroller.
 
 If you want to try it we have an
 `interactive tutorial <https://killercoda.com/slaff/scenario/sming-host-emulator>`__
 that can be run directly from your browser.
 
-See :doc:`/getting-started/index` for installation details.
-
-The following sections provide more background detail.
-
-
-Requirements (Linux)
---------------------
-
-Modern Linux distribution
-~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Most popular modern distributions should be fine. At the moment we
-develop only with Ubuntu 16.04 and 18.04.
-
-C/C++ 32 bit compiler and libraries
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The latest versions of Sming require GCC version 8 or newer.
-If your OS is 64 bit then 32 bit C/C++ compiler and libraries are also required.
-Here's a summary of how to do this::
-
-   sudo apt update
-   sudo apt install gcc-9 g++-9
-
-   # For 64-bit systems
-   sudo apt install gcc-9-multilib g++-9-multilib
-
-   # Set GCC 9 as default
-   sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-9 60 --slave /usr/bin/g++ g++ /usr/bin/g++-9
-
-The current version is `GCC 9.3.0 <https://www.gnu.org/software/gcc/gcc-9>`__.
-
-CMake 3.8 or newer
-~~~~~~~~~~~~~~~~~~
-
-For the compilation of LWIP ``CMake`` version 3.8 or newer is required.
-In order to get newer CMake version under Ubuntu 16.04, do this::
-
-   cd /tmp
-   wget https://github.com/Kitware/CMake/releases/download/v3.14.4/cmake-3.14.4-Linux-x86_64.sh
-   sudo mkdir /opt/cmake
-   sudo sh cmake-3.14.4-Linux-x86_64.sh --prefix=/opt/cmake
-   sudo mv /usr/bin/cmake /usr/bin/cmake.orig
-   sudo ln -s /opt/cmake/bin/cmake /usr/bin/cmake
-
-Requirements (Windows)
-----------------------
-
-For Windows, make sure your ``MinGW`` distro is up to date.
-The current version is `GCC 8.2.0 <https://gcc.gnu.org/gcc-8>`__.
-
-.. note::
-
-   If you don't already have MinGW installed, see :doc:`/getting-started/windows/index`.
-
-
-
-Compilation
------------
-
-Environment variables
-~~~~~~~~~~~~~~~~~~~~~
-
-:envvar:`SMING_ARCH` must be set to use ``Host`` as the desired architecture::
-
-   export SMING_ARCH=Host
-
-Debug Build
-~~~~~~~~~~~
-
-If you plan to use a debugger make sure to set :envvar:`ENABLE_GDB` and (optionally)
-:envvar:`ENABLE_LWIPDEBUG` before compiling the code::
-
-   export ENABLE_GDB=1
-   export ENABLE_LWIPDEBUG=1  # <!-- this will compile also LWIP with debug symbols
-
-Initial Clean-up
-~~~~~~~~~~~~~~~~
-
-Make sure that you are working on a clean source code tree. The following commands may help::
-
-   cd $SMING_HOME
-   make dist-clean
-
-Sample compilation
-~~~~~~~~~~~~~~~~~~
-
-Compile a sample as usual::
-
-   cd $SMING_HOME/../samples/Basic_Blink
-   make
-
-Adapting existing code
-~~~~~~~~~~~~~~~~~~~~~~
-
-You may need to modify your existing applications to work with the
-emulator. Architecture-specific code should be moved into separate code
-modules. Code that is specific to the ESP8266 may be conditionally
-compiled using ``#ifdef ARCH_ESP8266``. Similarly, you can check if the
-code is compiled for the emulator using ``#ifdef ARCH_HOST``.
-
-Running on the host
--------------------
-
-Once the sample is compiled, initialise the :component-host:`vflash`::
-
-   make flash
-
-If enabled, this will also create a SPIFFS file system and write it to flash.
-
-To run the application, do this::
-
-   make run
-
-Networking
-~~~~~~~~~~
-
-Support is provided via TAP network interface (a virtual network layer
-operating at the ethernet frame level). A TAP interface must be created
-first, and requires root privilege.
-
-For Linux, you can use ``Sming/Arch/Host/Tools/setup-network-linux.sh``. Here is the manual approach::
-
-   sudo ip tuntap add dev tap0 mode tap user `whoami`
-   sudo ip a a dev tap0 192.168.13.1/24
-   sudo ip link set tap0 up
-
-   # The following lines are needed if you plan to access Internet
-   sudo sysctl net.ipv4.ip_forward=1
-   sudo sysctl net.ipv6.conf.default.forwarding=1
-   sudo sysctl net.ipv6.conf.all.forwarding=1
-
-   export INTERNET_IF=wlan0 # <!--- Make sure to replace wlan0 with the network interface connected to Internet
-
-   sudo iptables -t nat -A POSTROUTING -o $INTERNET_IF -j MASQUERADE
-   sudo iptables -A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-   sudo iptables -A FORWARD -i tap0 -o $INTERNET_IF -j ACCEPT
-
-This creates the ``tap0`` interface. The emulator will automatically
-select the first ``tap`` interface found. To override this, use the
-``--ifname`` option.
-
-You can list available network interfaces thus::
-
-   ip link
-
-
-For MacOS, networking support first requires installation of a kernel extension
-using ``Sming/Arch/Host/Tools/setup-network-macos.sh``.
-On first run, you will need to confirm security then reboot the system.
-On rebooting, run the script again.
-
-The ``tap0`` network interface is created dynamically when ``/dev/tap0`` is opened.
-At present, you must run the application with elevated permissions::
-
-   sudo out/Host/debug/firmware/app
-
-
-
-Troubleshooting
----------------
-
-Issue 1: fatal error: sys/cdefs.h: No such file or directory
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Make sure to install the 32bit version of the GNU C and C++ compiler,
-development package and libraries.
-
-Issue 2: fatal error: bits/c++config.h: No such file or directory
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Make sure to install the 32bit version of the GNU C and C++ compiler,
-development package and libraries.
-
-
-Further reading
----------------
-
-.. toctree::
-   :titlesonly:
-
-   /_inc/Sming/Arch/Host/README
+For installation details see :doc:`/getting-started/index`.
 
+For full details see :doc:`/_inc/Sming/Arch/Host/README`.