Merge branch 'master' of github.com:bakerstu/openmrn into bracz-examp…

…le-cs-logon

* 'master' of github.com:bakerstu/openmrn: (62 commits)
  Simplifies traction node OpenLCB addresses for DCC locomotives. (#615)
  Explicit handling of NOT_RESPONDING aliases. (#614)
  Adds support for RS485 mode to the tiva UART driver. (#613)
  Optimizes the computation speed for crc16-ibm. (#612)
  Fixes bug in FlashFile O_TRUNC handling. (#611)
  Fixes interpretation of O_CREAT in the CC32xxDeviceFile. (#610)
  Python3 fixes in revision.py (#609)
  Fixes the flash command for TM4C129.  (#608)
  Adds a hello message to the DCC decoder. (#607)
  Removes duplicate -MD argument. The prog.mk already contains the necessary -MMD instead, and clang complains if both are set.
  Adds a script for computing optimal CAN-bus timing configurations. (#606)
  Updates the CAN bit timing parameters of the TivaCAN driver. (#605)
  Fixes Tiva GPIO and build (#604)
  Switched GcPacketPrinter to use stderr. (#603)
  JS.emscripten: adds connect callbacks to the ports. (#602)
  Fixes sequence problem when more than one CDI file gets rendered. (#601)
  Minor fixes to SNIP client and SNIP handler: (#600)
  Removes dependency on usleep from header.
  Avoids adding the system directory files to the dependency outputs.
  Updates distcc documentation.
  ...

DISTCC.md

@@ -0,0 +1,180 @@
+# Using distcc with OpenMRN compilation framework
+
+## Overview
+
+### What is this?
+
+This feature allows using a powerful remote computer to perform most of the
+compilation steps when compiling OpenMRN-based projects.
+
+### Why?
+
+It is much faster.
+
+OpenMRN compiles a large number of c++ files that can take a lot of CPU to
+compile. If your workstation has constrained CPU (e.g. a laptop or a virtual
+machine with limited number of CPUs), you can use a powerful remote machine to
+offload most of the compilation steps.
+
+### How does it work?
+
+We use the opensource distcc package (`[email protected]:distcc/distcc.git`). We
+create an SSH link to the remote machine, and start the distcc daemon. We wrap
+the local compilation command in distcc. Distcc will preprocess the source file
+locally, then transmit the preprocessed source to the remote machine over the
+SSH link. The daemon running on the powerful machine will then invoke the
+compiler, create the .o file, and transmit the .o file back over the SSH
+link. The linking steps then happen locally.
+
+## Prerequisites
+
+- You have to have the same compiler that OpenMRN uses available on the remote
+  machine. We have a [specific step](#configuring-the-compilers) for telling
+  distcc and OpenMRN what the matching compilers are.
+- You need to start the SSH link before trying to compile things. If you forget
+  this, local compilation will be performed instead.
+- A small change needs to be made to the distcc sources, because it does not
+  support a compiler flag that we routinely pass to armgcc. For this reason you
+  need to compile it from source.
+
+### Installing
+
+#### Installing distcc
+
+1. `sudo apt-get install distcc` on both the local and remote machine.
+2. Clone the distcc sources from `[email protected]:distcc/distcc.git`. Check the
+   INSTALL file for an apt-get commandline near the very top for compile-time
+   dependencies to be installed.
+3. edit `src/arg.c`. Find the place where it checks for `"-specs="`
+   argument. Comment out the early return. (Not doing anything in that if is
+   what you want.)
+4. edit `src/serve.c`. Find where it checks for `"-specs="`, and disable that
+   check.
+4. edit `src/filename.c`. Find where it checks for `"cxx"`, and add another
+   line so that it supports `"cxxtest"` as well.
+5. run through the compile steps of distcc, see the INSTALL file. These are
+   normally:
+   ```
+   ./autogen.sh
+   ./configure
+   make
+   ```
+6. Copy `distcc` to `~/bin`, and copy `distccd` to the remote machine at `~/bin`
+
+#### Configuring the compilers
+
+When setting up compilers, we need to ensure that we can determine what the
+remote compiler will be called for any given local compiler that we have. In
+addition to this, we need to be able to call the same compiler underthe same
+command on both machines, because the local machine will be calling the
+compiler for preprocessing and the remote machine will be calling it for
+compilation.
+
+For any given compiler, you need to make a symlink on both the local machine
+and the remote machine:
+
+```bash
+cd ~/bin
+ln -sf $(realpath /opt/armgcc/gcc-arm-none-eabi-8-2018-q4-major/bin/arm-none-eabi-gcc) armgcc-2018-q4-gcc
+ln -sf $(realpath /opt/armgcc/gcc-arm-none-eabi-8-2018-q4-major/bin/arm-none-eabi-g++) armgcc-2018-q4-g++
+```
+
+Do this on BOTH the local and remote machine. The real path may differ but the
+name must be exactly the same.
+
+Only on the remote machine, the compiler also needs to be added to the
+masquerade directory, otherwise distccd server will refuse to execute it.
+
+```bash
+cd /usr/lib/distccd
+sudo ln -sf ../../bin/distcc armgcc-2018-q4-g++
+sudo ln -sf ../../bin/distcc armgcc-2018-q4-gcc
+```
+
+#### Setting parameters
+
+For the OpenMRN compilation scripts to find distcc and the remote machine, we
+store a few parameters in files under `~/.distcc`.
+
+```bash
+mkdir -p ~/.distcc
+echo 3434 > ~/.distcc/port
+echo my-fast-remote-machine.myplace.com > ~/.distcc/ssh_host
+echo '127.0.0.1:3434/20,lzo' > ~/.distcc/hosts
+```
+
+The port number will be used with SSH port forwarding. The remote machine is
+used by the ssh tunnel setup script.
+
+The "/20" part of the hosts line is the capacity of the remote machine. This
+one says it's good for 20 parallel jobs, which I picked for a 6-core (12
+hyperthread) machine with lots of RAM.
+
+You can configure a lot of other things in the hosts file. `man distcc` for the
+full documentation. It is also possible to configure the local CPU to run some
+number of jobs while the remote CPU runs other builds. It is possible to use
+multiple remote machines as well.
+
+## Using
+
+After every restart of your machine you need to start the ssh tunnel:
+
+```bash
+~/openmrn/bin/start-distcc.sh
+```
+
+This will set up the SSH tunnel to the remote host and start the distccd server
+on the far end of the tunnel.
+
+Then compile OpenMRN stuff with very high parallelism:
+
+```bash
+~/openmrn/applications/io_board/target/freertos.armv7m.ek-tm4c123gxl$ make -j21
+```
+
+### How do I know if it worked?
+
+OpenMRN build system will automatically detect that you have the SSH tunnel
+running to distcc and use distcc.
+
+If it uses distcc, then you will see the compilation done remotely. Check the
+beginning of most lines:
+
+```
+distcc armgcc-2018-q4-gcc -c -Os -fno-strict-aliasing [...] STM32L4xx_HAL_Driver/Src/stm32l4xx_hal_usart.c -o stm32l4xx_hal_usart.o
+```
+
+If it is done locally, then you don't see distcc but see the execution of the
+compiler directly:
+
+```
+/opt/armgcc/gcc-arm-none-eabi-8-2018-q4-major/bin/arm-none-eabi-gcc -c -Os -fno-strict-aliasing [...] STM32L4xx_HAL_Driver/Src/stm32l4xx_hal_usart.c -o stm32l4xx_hal_usart.o
+```
+
+### How much does it help?
+
+With distcc (parallelism of 21, all remotely; this is on a W-2135 Xeon CPU @
+3.70GHz):
+
+```bash
+   text	   data	    bss	    dec	    hex	filename
+ 113808	    312	   3768	 117888	  1cc80	io_board.elf
+/opt/armgcc/default/bin/arm-none-eabi-objdump -C -d -h io_board.elf > io_board.lst
+
+real    0m44.704s
+user    2m41.551s
+sys     0m29.235s
+```
+
+Without distcc (parallelism of 9, all locally; this is on an intel i7-8665U CPU
+@ 1.90GHz -- this is a 15W TDP mobile CPU with 4 cores, 8 hyperthreads):
+
+```bash
+   text	   data	    bss	    dec	    hex	filename
+ 113808	    312	   3768	 117888	  1cc80	io_board.elf
+/opt/armgcc/default/bin/arm-none-eabi-objdump -C -d -h io_board.elf > io_board.lst
+
+real    2m8.602s
+user    14m21.471s
+sys     0m42.488s
+```

Makefile

@@ -38,3 +38,37 @@ js-tests:
 	$(MAKE) -C targets/js.emscripten run-tests
 
 alltests: tests llvm-tests
+
+release-clean:
+	$(MAKE) -C targets/linux.x86 clean
+
+RELNAME=$(shell uname -sm | tr ' A-Z' '.a-z')
+RELDIR=$(OPENMRNPATH)/bin/release/staging-$(RELNAME)
+JSRELDIR=$(OPENMRNPATH)/bin/release/staging-js
+
+include $(OPENMRNPATH)/etc/release.mk
+
+# These are the applications that are packaged into the binary release.
+$(call RELEASE_BIN_template,hub,applications/hub/targets/linux.x86)
+$(call RELEASE_BIN_template,memconfig_utils,applications/memconfig_utils/targets/linux.x86)
+$(call RELEASE_BIN_template,bootloader_client,applications/bootloader_client/targets/linux.x86)
+$(call RELEASE_BIN_template,send_datagram,applications/send_datagram/targets/linux.x86)
+
+$(call RELEASE_JS_template,openmrn-bootloader-client,applications/bootloader_client/targets/js.emscripten)
+$(call RELEASE_JS_template,openmrn-hub,applications/js_hub/targets/js.emscripten)
+
+release-bin:
+	rm -rf $(RELDIR)/*
+	mkdir -p $(RELDIR)
+	+$(MAKE) -C . release-bin-all
+	cp LICENSE.md $(RELDIR)/LICENSE.txt
+	cd $(RELDIR); zip -9r ../applications.$(RELNAME).zip . 
+
+release-js:
+	rm -rf $(JSRELDIR)/*
+	mkdir -p $(JSRELDIR)
+	+$(MAKE) -C . release-js-all
+	cp LICENSE.md $(JSRELDIR)/win/LICENSE.txt
+	cp LICENSE.md $(JSRELDIR)/macos/LICENSE.txt
+	cd $(JSRELDIR)/win; zip -9r ../../applications.win.zip .
+	cd $(JSRELDIR)/macos; zip -9r ../../applications.macos.zip .

RELEASE.md

@@ -0,0 +1,85 @@
+# Release process
+
+This document details how releases need to be built and published for OpenMRN.
+
+## Purpose and content of releases
+
+Releases fulfill two purposes:
+
+- Precompiled binaries are made available for download. This is important for
+  those that do not have facilities or the experience to compile OpenMRN
+  applications from source.
+
+- (At a later point) a packaged library is made available with include headers
+  and `*.a` files that enable importing OpenMRN into a different compilation
+  environment, such as various IDEs for embedded development.
+
+The following is out of scope of this documentation:
+
+- Releases for OpenMRNLite (Arduino compatible library) are documented in the
+  [arduino/RELEASE.md](arduino/RELEASE.md) file.
+
+## Requirements for building releases
+
+- You need to be able to compile OpenMRN binaries. You need a linux host (or
+  VM). The necessary packages have to be installed:
+
+  - beyond standard `g++` you will need `sudo apt-get install
+    libavahi-client-dev`
+
+- You need both a linux.x86_64 and a raspberry pi host.
+
+- On the linux host you should install node.js, npm, emscripten, and the
+  packager:
+
+  `sudo npm install -g pkg`
+
+## Release numbering
+
+Release numbers are marked as major.minor.patch, such as `v2.10.1`.
+
+- Major release numbers are incremented once per year (2020 is 0, 2021 is 1,
+  2022 is 2, ...).
+
+- Minor release numbers are 10, 20, 30, 40 for given quarters, then incremented
+  by one if there are multiple releases built within a quarter.
+
+- Patch release numbers start at 1, and are only incremented if the same
+  release needs to be re-built with a patch.
+
+
+## How to build
+
+All of the make commands here need to be run in the openmrn directory
+(toplevel).
+
+1. Start with a clean checkout. Run `make release-clean`.
+
+2. On the linux.x86_64 host, run
+
+   `make -j5 release-bin`
+
+3. Copy `openmrn/bin/release/applications.linux.x86_64.zip` as one of the
+   artifacts.
+
+3. On the raspberry pi host, do the same:
+
+   `make release-clean`
+
+   `make -j4 release-bin`
+
+4. Copy `openmrn/bin/release/applications.linux.armv7l.zip` as one of the
+   artifacts. Rename it to `applications.linux.armv7l-raspberry-pi.zip`
+
+5. On the linux.x86_64 host, build the javascript binaries. Run
+
+   `make -j5 release-js`
+
+6. Copy `openmrn/bin/release/applications.win.zip` and
+   `openmrn/bin/release/applications.macos.zip`.
+
+7. On the OpenMRN GitHub project, select Releases, create a new release, select
+   create a new tag in the form of `release-v2.10.1`. Upload the release
+   artifacts that you collected.
+
+8. Publish the release.

applications/blink_raw/targets/freertos.armv7m.ti-launchpad-tm4c129/Makefile

@@ -48,7 +48,7 @@ ifeq ($(call find_missing_deps,OPENOCDSCRIPTSPATH OPENOCDPATH),)
 flash: $(EXECUTABLE)$(EXTENTION) $(EXECUTABLE).lst
 	@if ps ax -o comm | grep -q openocd ; then echo openocd already running. quit existing first. ; exit 1 ; fi
 	cp $< last-flashed-$<
-	$(GDB) $< -ex "target remote | $(OPENOCDPATH)/openocd -c \"gdb_port pipe\" --search $(OPENOCDSCRIPTSPATH) $(OPENOCDARGS)" -ex "monitor reset init" -ex "monitor reset run" -ex "load" -ex "monitor reset init" -ex "monitor reset run"  -ex "detach" -ex "quit"
+	$(GDB) $< -ex "target remote | $(OPENOCDPATH)/openocd -c \"gdb_port pipe\" --search $(OPENOCDSCRIPTSPATH) $(OPENOCDARGS)" -ex "monitor reset init"  -ex "load" -ex "monitor reset init" -ex "monitor reset run"  -ex "detach" -ex "quit"
 
 gdb:
 	@if ps ax -o comm | grep -q openocd ; then echo openocd already running. quit existing first. ; exit 1 ; fi

applications/bootloader/targets/bare.armv6m.st-stm32f091rc-nucleo-dev-board/BootloaderHal.cxx

@@ -0,0 +1 @@
+../../../../boards/st-stm32f091rc-nucleo/BootloaderHal.cxx

applications/bootloader/targets/bare.armv6m.st-stm32f091rc-nucleo-dev-board/BootloaderHal.hxx

@@ -0,0 +1 @@
+../../../../boards/st-stm32f0x1_x2_x8-generic/BootloaderHal.hxx

applications/bootloader/targets/bare.armv6m.st-stm32f091rc-nucleo-dev-board/Makefile

@@ -0,0 +1,64 @@
+APP_PATH ?= $(realpath ../..)
+-include $(APP_PATH)/config.mk
+-include local_config.mk
+
+OPENMRNPATH ?= $(shell \
+sh -c "if [ \"X`printenv OPENMRNPATH`\" != \"X\" ]; then printenv OPENMRNPATH; \
+     elif [ -d /opt/openmrn/src ]; then echo /opt/openmrn; \
+     elif [ -d ~/openmrn/src ]; then echo ~/openmrn; \
+     elif [ -d ../../../src ]; then echo ../../..; \
+     else echo OPENMRNPATH not found; fi" \
+)
+
+-include $(OPENMRNPATH)/etc/config.mk
+LINKCORELIBS = -lopenlcb -lutils
+
+include $(OPENMRNPATH)/etc/stm32cubef0.mk
+
+CFLAGSEXTRA += -DSTM32F091xC
+CXXFLAGSEXTRA += -DSTM32F091xC
+SYSLIBRARIES += -lfreertos_drivers_stm32cubef091xc
+OPENOCDARGS = -f board/st_nucleo_f0.cfg
+
+export TARGET := bare.armv6m
+
+include $(OPENMRNPATH)/etc/prog.mk
+
+ifndef DEFAULT_ADDRESS
+DEFAULT_ADDRESS=0xff
+endif
+
+include $(OPENMRNPATH)/etc/node_id.mk
+
+SYSLIB_SUBDIRS=
+OBJEXTRA=
+LDFLAGSEXTRA += -nostartfiles 
+
+all: $(EXECUTABLE).bin
+
+# How to use: make multibin ADDRESS=0x20 ADDRHIGH=0x45 NUM=3
+# starting address, high bits (user range), count
+multibin:
+	for i in $$(seq 1 $(NUM)) ; do $(MAKE) $(EXECUTABLE).bin ADDRESS=$$(printf 0x%02x $$(($(ADDRESS)+$$i))) ; cp $(EXECUTABLE).bin $(EXECUTABLE).$(MCU_SHORT).$$(printf %02x%02x $(ADDRHIGH) $$(($(ADDRESS)+$$i-1))).bin ; done
+
+ifeq ($(call find_missing_deps,OPENOCDPATH OPENOCDSCRIPTSPATH),)
+
+flash: $(EXECUTABLE)$(EXTENTION) $(EXECUTABLE).bin $(EXECUTABLE).lst
+	@if ps ax -o comm | grep -q openocd ; then echo openocd already running. quit existing first. ; exit 1 ; fi
+	$(GDB) $< -ex "target remote | $(OPENOCDPATH)/openocd -c \"gdb_port pipe\" --search $(OPENOCDSCRIPTSPATH) $(OPENOCDARGS)" -ex "monitor reset halt" -ex "load" -ex "monitor reset init" -ex "monitor reset run"  -ex "detach" -ex "quit"
+
+lflash: $(EXECUTABLE).bin $(EXECUTABLE).lst
+	@if ps ax -o comm | grep -q openocd ; then echo openocd already running. quit existing first. ; exit 1 ; fi
+	$(GDB) $< -ex "target remote | $(OPENOCDPATH)/openocd -c \"gdb_port pipe\" --search $(OPENOCDSCRIPTSPATH) $(OPENOCDARGS)" -ex "monitor reset halt" -ex "monitor program $< 0x08000000 verify reset exit" -ex "monitor reset run"  -ex "detach" -ex "quit"
+
+gdb:
+	@if ps ax -o comm | grep -q openocd ; then echo openocd already running. quit existing first. ; exit 1 ; fi
+	$(GDB) $(EXECUTABLE)$(EXTENTION) -ex "target remote | $(OPENOCDPATH)/openocd -c \"gdb_port pipe\" --search $(OPENOCDSCRIPTSPATH) $(OPENOCDARGS)" -ex "continue" # -ex "monitor reset halt"
+
+else
+
+flash gdb:
+	echo OPENOCD not found ; exit 1
+
+endif
+