Added deployer tool.

Sming/Libraries/OtaUpgradeMqtt/README.rst

@@ -1,14 +1,63 @@
 OTA Firmware Upgrade via MQTT
 =============================
 
-.. highlight:: bash
+.. highlight:: c++
 
 Introduction
 ------------
 
 This library allows Sming applications to upgrade their firmware Over-The-Air (OTA) using the MQTT protocol.
 MTQTT has less overhead compared to HTTP and can be used for faster delivery of application updates.
 
+Using
+-----
+
+1. Add ``COMPONENT_DEPENDS += OtaUpgradeMqtt`` to your application componenent.mk file.
+2. Add these lines to your application::
+
+      #include <OtaUpgrade/Mqtt/RbootPayloadParser.h>
+
+      #if ENABLE_OTA_ADVANCED
+      #include <OtaUpgrade/Mqtt/AdvancedPayloadParser.h>
+      #endif
+
+      MqttClient mqtt;
+
+      // Call when IP address has been obtained
+      void onIp(IpAddress ip, IpAddress mask, IpAddress gateway)
+      {
+         // ...
+
+         mqtt.connect(Url(MQTT_URL), "sming");
+
+      #if ENABLE_OTA_ADVANCED
+          /*
+           * The advanced parser suppors all firmware upgrades supported by the `OtaUpgrade` library.
+           * `OtaUpgrade` library provides firmware signing, firmware encryption and so on.
+           */
+          auto parser = new OtaUpgrade::Mqtt::AdvancedPayloadParser(APP_VERSION_PATCH);
+      #else
+          /*
+           * The command below uses class that stores the firmware directly
+           * using RbootOutputStream on a location provided by us
+           */
+          auto parser = new OtaUpgrade::Mqtt::RbootPayloadParser(part, APP_VERSION_PATCH);
+       #endif
+
+            mqtt.setPayloadParser([parser]
+            (MqttPayloadParserState& state, mqtt_message_t* message, const char* buffer, int length) -> int
+            {
+               return parser->parse(state, message, buffer, length);
+            });
+
+            String updateTopic = "/app/test/u/4.3";
+            mqtt.subscribe(updateTopic);
+
+         // ...
+      }
+
+See the :sample:`Upgrade` sample application.
+
 Versioning Principles
 ---------------------
 To simplify the OTA process we strongly recommend the following versioning principles for your application:
@@ -29,11 +78,11 @@ Theory Of Operation
    Depending on the size of the new firmware and the speed of the connection an update can take 10 to 20 seconds.
 
 2. The application connects via MQTT to a remote server and subscribes to a special topic. The topic is based on the
-   application id and its current version. If the current application version is 4.3.1 then the topic that will be used for OTA is ``/a/test/u/4.3``.
+   application id and its current version. If the current application id is ``test`` and version is ``4.3.1`` then the topic that will be used for OTA is ``a/test/u/4.3``.
 
 3. If there is a need to support both stable and unstable/nightly builds then the topic name can have `s` or `u` suffix. For example
-   all stable versions should be published and downloaded from the topic ``/a/test/u/4.3/s``. For the unstable ones we can use the topic ``/a/test/u/4.3/u``.
-   If an application is interested in both then it can subscribe using the following pattern ``/a/test/u/4.3/+``.
+   all stable versions should be published and downloaded from the topic ``a/test/u/4.3/s``. For the unstable ones we can use the topic ``a/test/u/4.3/u``.
+   If an application is interested in both stable and unstable versions then it can subscribe using the following pattern ``a/test/u/4.3/+``.
 
 4. The application is waiting for new firmware. When the application is on battery than it makes sense to wait for a limited time and if there is no
    message coming back to disconnect.
@@ -51,6 +100,27 @@ One MQTT message contains:
 Based on the :envvar:`ENABLE_OTA_VARINT_VERSION` the patch version can be encoded either using one byte or a `varint <https://developers.google.com/protocol-buffers/docs/encoding#varints>`_.
 Based on :envvar:`ENABLE_OTA_ADVANCED` the firmware data can be either without any encoding or be signed and encrypted.
 
+To simplify the packaging this library comes with a tool called ``deployer``. To create a package type the following from your application::
+
+   make ota-pack OTA_PATCH_VERSION=127
+
+Replace 127 with the desired patch version.
+If the option ``OTA_PATCH_VERSION`` is omitted from the command line then the patch version will be generated automatically and it will contain the current unix timestamp.
+
+Once a package is created it can be deployed to the firmware MQTT server using the command below::
+
+   make ota-deploy MQTT_FIRMWARE_URL=mqtt://relser:[email protected]/a/test/4.3
+
+The ``MQTT_FIRMWARE_URL`` above specifies that
+
+- protocol is: mqtt without SSL.
+- user is: relser
+- password is: relpassword
+- host is: attachix.com
+- path is: /a/test/4.3. The topic name is based on the path with removed leading and ending slashes.
+
+Make sure to replace the MQTT_FIRMWARE_URL value with your MQTT server credentials, host and topic.
+
 Security
 --------
 For additional security a standard SSL/TLS can be used

Sming/Libraries/OtaUpgradeMqtt/api.rst

@@ -5,3 +5,4 @@ OTA Upgrade over MQTT classes
 
 .. doxygenclass:: OtaUpgrade::Mqtt::PayloadParser
 .. doxygenclass:: OtaUpgrade::Mqtt::RbootPayloadParser
+.. doxygenclass:: OtaUpgrade::Mqtt::AdvancedPayloadParser

Sming/Libraries/OtaUpgradeMqtt/component.mk

@@ -16,4 +16,34 @@ COMPONENT_VARS += ENABLE_OTA_VARINT_VERSION
 ENABLE_OTA_VARINT_VERSION ?= 1
 
 COMPONENT_CXXFLAGS := -DENABLE_OTA_ADVANCED=$(ENABLE_OTA_ADVANCED) \
-					  -DENABLE_OTA_VARINT_VERSION=$(ENABLE_OTA_VARINT_VERSION)
+					  -DENABLE_OTA_VARINT_VERSION=$(ENABLE_OTA_VARINT_VERSION)
+
+##@Firmware Upgrade
+
+OTA_TOOLS := $(COMPONENT_PATH)/tools
+OTA_DEPLOYMENT_TOOL = $(OTA_TOOLS)/deployer/out/Host/debug/firmware/deployer$(TOOL_EXT)
+
+$(OTA_DEPLOYMENT_TOOL):
+	$(Q) $(MAKE) -C $(OTA_TOOLS)/deployer SMING_ARCH=Host ENABLE_CUSTOM_LWIP=2
+
+
+# SDP = Sming Deployment Pakage
+OTA_PACKAGE_EXT =.sdp
+
+OTA_PATCH_VERSION ?= $(shell date +%s)
+
+PACKAGE_IN = $(RBOOT_ROM_0_BIN)
+ifneq ($(ENABLE_OTA_ADVANCED), 0)
+	PACKAGE_IN = $(OTA_UPGRADE_FILE)
+endif
+PACKAGE_OUT = $(PACKAGE_IN)$(OTA_PACKAGE_EXT)
+
+.PHONY: ota-pack
+ota-pack: $(PACKAGE_OUT) ##Creates a deployment package from the current application (use OTA_PATCH_VERSION to specify the version number)
+
+$(PACKAGE_OUT): $(PACKAGE_IN) $(OTA_DEPLOYMENT_TOOL) 
+	$(Q) $(OTA_DEPLOYMENT_TOOL) pack -- $(PACKAGE_IN) $(PACKAGE_OUT) $(OTA_PATCH_VERSION) $(ENABLE_OTA_VARINT_VERSION) 2>/dev/null
+
+.PHONY: ota-deploy
+ota-deploy: $(PACKAGE_OUT) ##Uploads new firmware version of the current application (use MQTT_FIRMWARE_URL to specify the MQTT URL)
+	$(Q) $(OTA_DEPLOYMENT_TOOL) deploy -- $(PACKAGE_OUT) $(MQTT_FIRMWARE_URL) 2>/dev/null

Sming/Libraries/OtaUpgradeMqtt/samples/Upgrade/README.rst

@@ -11,6 +11,12 @@ This application uses :component:`OtaUpgradeMqtt` and follows the recommended ve
 
 Based on :envvar:`ENABLE_OTA_ADVANCED` the firmware data can be either without any encoding or be signed and encrypted.
 
+Tools
+-----
+There are two tools that facilitate the packiging and deployment of a new firmware.
+
+For more information read ``Firmware packaging`` in the documentation of the :component:`OtaUpgradeMqtt` component.
+
 Security
 --------
 Depending on :envvar:`ENABLE_SSL` a standard SSL/TLS can be enabled. This way

Sming/Libraries/OtaUpgradeMqtt/samples/Upgrade/app/application.cpp

@@ -98,7 +98,7 @@ void otaUpdate()
 	mqtt.setPayloadParser([parser](MqttPayloadParserState& state, mqtt_message_t* message, const char* buffer,
 								   int length) -> int { return parser->parse(state, message, buffer, length); });
 
-	String updateTopic = "/a/";
+	String updateTopic = "a/";
 	updateTopic += APP_ID;
 	updateTopic += "/u/";
 	updateTopic += APP_VERSION;

Sming/Libraries/OtaUpgradeMqtt/tools/deployer/.cproject

@@ -0,0 +1,97 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<?fileVersion 4.0.0?><cproject storage_type_id="org.eclipse.cdt.core.XmlProjectDescriptionStorage">
+	<storageModule moduleId="org.eclipse.cdt.core.settings">
+		<cconfiguration id="0.131700304">
+			<storageModule buildSystemId="org.eclipse.cdt.managedbuilder.core.configurationDataProvider" id="0.131700304" moduleId="org.eclipse.cdt.core.settings" name="Default">
+				<externalSettings/>
+				<extensions>
+					<extension id="org.eclipse.cdt.core.VCErrorParser" point="org.eclipse.cdt.core.ErrorParser"/>
+					<extension id="org.eclipse.cdt.core.GmakeErrorParser" point="org.eclipse.cdt.core.ErrorParser"/>
+					<extension id="org.eclipse.cdt.core.CWDLocator" point="org.eclipse.cdt.core.ErrorParser"/>
+					<extension id="org.eclipse.cdt.core.GCCErrorParser" point="org.eclipse.cdt.core.ErrorParser"/>
+					<extension id="org.eclipse.cdt.core.GASErrorParser" point="org.eclipse.cdt.core.ErrorParser"/>
+					<extension id="org.eclipse.cdt.core.GLDErrorParser" point="org.eclipse.cdt.core.ErrorParser"/>
+				</extensions>
+			</storageModule>
+			<storageModule moduleId="cdtBuildSystem" version="4.0.0">
+				<configuration buildProperties="" description="" id="0.131700304" name="Default" optionalBuildProperties="" parent="org.eclipse.cdt.build.core.prefbase.cfg">
+					<folderInfo id="0.131700304." name="/" resourcePath="">
+						<toolChain id="org.eclipse.cdt.build.core.prefbase.toolchain.481709424" name="No ToolChain" resourceTypeBasedDiscovery="false" superClass="org.eclipse.cdt.build.core.prefbase.toolchain">
+							<targetPlatform id="org.eclipse.cdt.build.core.prefbase.toolchain.481709424.1637922483" name=""/>
+							<builder id="org.eclipse.cdt.build.core.settings.default.builder.1861624051" keepEnvironmentInBuildfile="false" managedBuildOn="false" name="Gnu Make Builder" superClass="org.eclipse.cdt.build.core.settings.default.builder"/>
+							<tool id="org.eclipse.cdt.build.core.settings.holder.libs.927548352" name="holder for library settings" superClass="org.eclipse.cdt.build.core.settings.holder.libs"/>
+							<tool id="org.eclipse.cdt.build.core.settings.holder.1318259878" name="Assembly" superClass="org.eclipse.cdt.build.core.settings.holder">
+								<inputType id="org.eclipse.cdt.build.core.settings.holder.inType.49095807" languageId="org.eclipse.cdt.core.assembly" languageName="Assembly" sourceContentType="org.eclipse.cdt.core.asmSource" superClass="org.eclipse.cdt.build.core.settings.holder.inType"/>
+							</tool>
+							<tool id="org.eclipse.cdt.build.core.settings.holder.451866905" name="GNU C++" superClass="org.eclipse.cdt.build.core.settings.holder">
+								<inputType id="org.eclipse.cdt.build.core.settings.holder.inType.952230018" languageId="org.eclipse.cdt.core.g++" languageName="GNU C++" sourceContentType="org.eclipse.cdt.core.cxxSource,org.eclipse.cdt.core.cxxHeader" superClass="org.eclipse.cdt.build.core.settings.holder.inType"/>
+							</tool>
+							<tool id="org.eclipse.cdt.build.core.settings.holder.639761705" name="GNU C" superClass="org.eclipse.cdt.build.core.settings.holder">
+								<inputType id="org.eclipse.cdt.build.core.settings.holder.inType.2127747414" languageId="org.eclipse.cdt.core.gcc" languageName="GNU C" sourceContentType="org.eclipse.cdt.core.cSource,org.eclipse.cdt.core.cHeader" superClass="org.eclipse.cdt.build.core.settings.holder.inType"/>
+							</tool>
+						</toolChain>
+					</folderInfo>
+				</configuration>
+			</storageModule>
+			<storageModule moduleId="org.eclipse.cdt.core.externalSettings">
+				<externalSettings containerId="Sming;" factoryId="org.eclipse.cdt.core.cfg.export.settings.sipplier"/>
+			</storageModule>
+		</cconfiguration>
+	</storageModule>
+	<storageModule moduleId="cdtBuildSystem" version="4.0.0">
+		<project id="Basic_ControlPoint.null.1347473968" name="Basic_ControlPoint"/>
+	</storageModule>
+	<storageModule moduleId="scannerConfiguration">
+		<autodiscovery enabled="true" problemReportingEnabled="true" selectedProfileId=""/>
+		<scannerConfigBuildInfo instanceId="0.2058741737">
+			<autodiscovery enabled="true" problemReportingEnabled="true" selectedProfileId=""/>
+		</scannerConfigBuildInfo>
+		<scannerConfigBuildInfo instanceId="cdt.managedbuild.toolchain.gnu.mingw.base.1135534147;cdt.managedbuild.toolchain.gnu.mingw.base.1135534147.86962463;cdt.managedbuild.tool.gnu.c.compiler.mingw.base.2032390008;cdt.managedbuild.tool.gnu.c.compiler.input.1700912488">
+			<autodiscovery enabled="true" problemReportingEnabled="true" selectedProfileId=""/>
+		</scannerConfigBuildInfo>
+		<scannerConfigBuildInfo instanceId="0.147001341">
+			<autodiscovery enabled="true" problemReportingEnabled="true" selectedProfileId=""/>
+		</scannerConfigBuildInfo>
+		<scannerConfigBuildInfo instanceId="0.1962726862">
+			<autodiscovery enabled="true" problemReportingEnabled="true" selectedProfileId=""/>
+		</scannerConfigBuildInfo>
+		<scannerConfigBuildInfo instanceId="0.358052692">
+			<autodiscovery enabled="true" problemReportingEnabled="true" selectedProfileId=""/>
+		</scannerConfigBuildInfo>
+		<scannerConfigBuildInfo instanceId="0.131700304">
+			<autodiscovery enabled="true" problemReportingEnabled="true" selectedProfileId=""/>
+		</scannerConfigBuildInfo>
+		<scannerConfigBuildInfo instanceId="cdt.managedbuild.toolchain.gnu.mingw.base.1135534147;cdt.managedbuild.toolchain.gnu.mingw.base.1135534147.86962463;cdt.managedbuild.tool.gnu.cpp.compiler.mingw.base.454898447;cdt.managedbuild.tool.gnu.cpp.compiler.input.501261625">
+			<autodiscovery enabled="true" problemReportingEnabled="true" selectedProfileId=""/>
+		</scannerConfigBuildInfo>
+	</storageModule>
+	<storageModule moduleId="org.eclipse.cdt.core.LanguageSettingsProviders"/>
+	<storageModule moduleId="org.eclipse.cdt.make.core.buildtargets">
+		<buildTargets>
+			<target name="all" path="" targetID="org.eclipse.cdt.build.MakeTargetBuilder">
+				<buildCommand>make</buildCommand>
+				<buildArguments/>
+				<buildTarget>all</buildTarget>
+				<stopOnError>true</stopOnError>
+				<useDefaultCommand>true</useDefaultCommand>
+				<runAllBuilders>true</runAllBuilders>
+			</target>
+			<target name="rebuild" path="" targetID="org.eclipse.cdt.build.MakeTargetBuilder">
+				<buildCommand>make</buildCommand>
+				<buildArguments/>
+				<buildTarget>rebuild</buildTarget>
+				<stopOnError>true</stopOnError>
+				<useDefaultCommand>true</useDefaultCommand>
+				<runAllBuilders>true</runAllBuilders>
+			</target>
+			<target name="flash" path="" targetID="org.eclipse.cdt.build.MakeTargetBuilder">
+				<buildCommand>make</buildCommand>
+				<buildArguments/>
+				<buildTarget>flash</buildTarget>
+				<stopOnError>true</stopOnError>
+				<useDefaultCommand>true</useDefaultCommand>
+				<runAllBuilders>true</runAllBuilders>
+			</target>
+		</buildTargets>
+	</storageModule>
+</cproject>

Sming/Libraries/OtaUpgradeMqtt/tools/deployer/.project

@@ -0,0 +1,30 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<projectDescription>
+	<name>FirmwareDeployer</name>
+	<comment></comment>
+	<projects>
+		<project>SmingFramework</project>
+		<project>Libraries</project>
+		<project>Sming</project>
+	</projects>
+	<buildSpec>
+		<buildCommand>
+			<name>org.eclipse.cdt.managedbuilder.core.genmakebuilder</name>
+			<triggers>clean,full,incremental,</triggers>
+			<arguments>
+			</arguments>
+		</buildCommand>
+		<buildCommand>
+			<name>org.eclipse.cdt.managedbuilder.core.ScannerConfigBuilder</name>
+			<triggers>full,incremental,</triggers>
+			<arguments>
+			</arguments>
+		</buildCommand>
+	</buildSpec>
+	<natures>
+		<nature>org.eclipse.cdt.core.cnature</nature>
+		<nature>org.eclipse.cdt.core.ccnature</nature>
+		<nature>org.eclipse.cdt.managedbuilder.core.managedBuildNature</nature>
+		<nature>org.eclipse.cdt.managedbuilder.core.ScannerConfigNature</nature>
+	</natures>
+</projectDescription>

Sming/Libraries/OtaUpgradeMqtt/tools/deployer/Makefile

@@ -0,0 +1,9 @@
+#####################################################################
+#### Please don't change this file. Use component.mk instead ####
+#####################################################################
+
+ifndef SMING_HOME
+$(error SMING_HOME is not set: please configure it as an environment variable)
+endif
+
+include $(SMING_HOME)/project.mk

Sming/Libraries/OtaUpgradeMqtt/tools/deployer/README.rst

@@ -0,0 +1,62 @@
+Device_Scanner
+==============
+
+.. highlight:: bash
+
+Utility to package and deploy firmware files for upgrade over MQTT
+See :library:`OtaUpgradeMqtt` for details.
+
+
+Packaging
+---------
+
+To scan your local network, do this::
+
+   make pack HOST_PARAMETERS=scan
+
+
+This will start by scanning for all root devices ``:upnp:rootdevice``.
+If you want to use a specific search type, just add it to the command line::
+
+   make run HOST_PARAMETERS='scan urn:schemas-upnp-org:service:ContentDirectory:1'
+
+This will take some time to complete. If there are any failures, you can retry
+by running the command again: any descriptions which have already been fetched will
+be skipped.
+
+You find output in the following directories:
+
+out/upnp/devices
+   Contains a hierarchical map of all found devices on your local UPnP network.
+
+
+The schema directories are populated with modified versions of the description files.
+These reflect the C++ class structures which we will later create:
+
+out/upnp/schema/services
+   All services are extracted into a directory by ``{domain}``. Each service has a separate .xml file.
+
+   Service schema are re-usable and new ones should be added to the UPnP library so they are available for
+   everyone to use.
+
+
+out/upnp/schema/devices
+   All devices (both root and embedded) are extracted to a separate directory ``{manufacturer}/{friendlyName}``
+   with a single .xml file for each device.
+
+   Device schema are specific to each device implementation so will generally be kept in a separate library
+   or application.
+
+
+Processing existing descriptions
+--------------------------------
+
+You can also process device descriptions files pulled in via other means. For example:
+
+   make run HOST_PARAMETERS='parse config/panasonic/viera dmr/ddd.xml dms/ddd.xml nrc/ddd.xml'
+
+The first parameter ``parse`` is the command.
+The second parameter ``config/panasonic/viera`` gives the root directory for the device.
+The remaining parameters are the relative locations from this directory of a device description file.
+References to service files are pulled in: if they are missing, this may fail.
+