Moving the code to a library with some samples.

Sming/Libraries/OtaUpgradeMqtt/README.rst

@@ -0,0 +1,85 @@
+OTA Firmware Upgrade via MQTT
+=============================
+
+.. highlight:: bash
+
+Introduction
+------------
+
+This library allows Sming applications to upgrade their firmware Over-The-Air (OTA) using the MQTT protocol.
+MTQTT has less overhead compared to HTTP and can be used for faster delivery of application updates.
+
+Versioning Principles
+---------------------
+To simplify the OTA process we strongly recommend the following versioning principles for your application:
+
+1. Use `semantic versioning <https://semver.org/>`_.
+   If your current application version is 4.3.1 then 4 is the major, 3 is the minor and 1 is the patch version number.
+
+2. Every application firmware knows its version.
+
+3. An application with the same major and minor version should be compatible for update no matter what the patch number is.
+   If the new firmware is not compatible then a new minor or major version should be used.
+
+Theory Of Operation
+-------------------
+1. On a period of time the application connects to check if there is a new version of the firmware.
+   In your application this period has to be carefully selected so that OTA updates occur when the device has
+   enough resources: memory, space on flash, power and time to complete such an update. Also there should be no critical task running at the moment.
+   Depending on the size of the new firmware and the speed of the connection an update can take 10 to 20 seconds.
+
+2. The application connects via MQTT to a remote server and subscribes to a special topic. The topic is based on the
+   application id and its current version. If the current application version is 4.3.1 then the topic that will be used for OTA is ``/a/test/u/4.3``.
+
+3. If there is a need to support both stable and unstable/nightly builds then the topic name can have `s` or `u` suffix. For example
+   all stable versions should be published and downloaded from the topic ``/a/test/u/4.3/s``. For the unstable ones we can use the topic ``/a/test/u/4.3/u``.
+   If an application is interested in both then it can subscribe using the following pattern ``/a/test/u/4.3/+``.
+
+4. The application is waiting for new firmware. When the application is on battery than it makes sense to wait for a limited time and if there is no
+   message coming back to disconnect.
+
+Firmware packaging
+------------------
+The firmware update must come as one MQTT message. The MQTT protocol allows messages with a maximum size of 268435455 bytes approx 260MB.
+This should be perfectly enough for a device that has maximum 1MB available for an application ROM.
+
+One MQTT message contains:
+
+- patch version of the firmware
+- followed by the firmware data itself
+
+Based on the :envvar:`ENABLE_OTA_VARINT_VERSION` the patch version can be encoded either using one byte or a `varint <https://developers.google.com/protocol-buffers/docs/encoding#varints>`_.
+Based on :envvar:`ENABLE_OTA_ADVANCED` the firmware data can be either without any encoding or be signed and encrypted.
+
+Security
+--------
+For additional security a standard SSL/TLS can be used
+
+1. The communication should be secured using standard SSL.
+
+2. To prove that the server is the correct one: The MQTT clients should pin the public key fingerprint on the server.
+   OR have a list of public key fingerprints that are allowed.
+
+3. To prove that the clients are allowed to connect: Every MQTT client should also have a client certificate that is signed by the server.
+
+Configuration
+-------------
+
+.. envvar:: ENABLE_OTA_VARINT_VERSION
+
+   Default: 1 (enabled)
+
+   If set to 1 the OTA upgrade mechanism and application will use a `varint <https://developers.google.com/protocol-buffers/docs/encoding#varints>`_
+   encoding for the patch version. Thus allowing unlimited number of patch versions. Useful for enumerating unstable/nightly releases.
+   A bit more difficult to read and write but allows for unlimited versions.
+
+   If set to 0 the OTA upgrade mechanism and application will use one byte for the patch version which will limit it to 256 possible patch versions.
+   Useful for enumarating stable releases. Easier to write and read but limited to 256 versions only.
+
+.. envvar:: ENABLE_OTA_ADVANCED
+
+   Default: 0 (disabled)
+
+   If set to 1 the library will work with OtaUpgradeStream which supports signature and encryption of the firmware data itself.
+   See :component:`OtaUpgrade` for details. In the application the AdvancedPayloadParser can be used to do the MQTT message handling.
+

Sming/Libraries/OtaUpgradeMqtt/api.rst

@@ -0,0 +1,7 @@
+OTA Upgrade over MQTT classes
+=============================
+
+.. doxygentypedef:: OtaUpgrade::Mqtt
+
+.. doxygenclass:: OtaUpgrade::Mqtt::PayloadParser
+.. doxygenclass:: OtaUpgrade::Mqtt::RbootPayloadParser

Sming/Libraries/OtaUpgradeMqtt/component.mk

@@ -0,0 +1,14 @@
+COMPONENT_SRCDIRS := src/
+COMPONENT_INCDIRS := src/include
+
+# If enabled (set to 1) then we can use all sofisticated mechanisms to upgrade the firmware using the ``OtaUpgrade`` library.
+ENABLE_OTA_ADVANCED ?= 0
+
+# If enabled (set to 1) then we can use unlimited number of patch versions 
+ENABLE_OTA_VARINT_VERSION ?= 1
+
+ifneq ($(ENABLE_OTA_ADVANCED),0)
+	COMPONENT_DEPENDS := OtaUpgrade	
+endif
+
+COMPONENT_VARS := ENABLE_OTA_ADVANCED ENABLE_OTA_VARINT_VERSION 

Sming/Libraries/OtaUpgradeMqtt/samples/Upgrade/README.rst

@@ -0,0 +1,82 @@
+OTA over MQTT
+=============
+
+.. highlight:: bash
+
+Introduction
+------------
+
+This example demonstrates how you can create an application that updates its firmware Over The Air (OTA) using the MQTT protocol.
+This application uses :component:`OtaUpgradeMqtt` and follows the recommended versioning principles.
+
+Based on :envvar:`ENABLE_OTA_ADVANCED` the firmware data can be either without any encoding or be signed and encrypted.
+
+Security
+--------
+Depending on :envvar:`ENABLE_SSL` a standard SSL/TLS can be enabled. This way
+
+1. The communication between the application and the server will be encrypted using standard SSL.
+
+2. To prove that the server is the correct one: The MQTT clients should pin the public key fingerprint on the server.
+   OR have a list of public key fingerprints that are allowed.
+
+3. Depending on :envvar:`ENABLE_CLIENT_CERTIFICATE` the application can send a client certificate that is signed by the server.
+
+Configuration
+-------------
+
+.. envvar:: APP_ID
+
+   Default: "test"
+
+   This variable contains the unique application name.
+
+.. envvar:: APP_VERSION
+
+   Default: not set
+
+   Contains the application major and minor versions separated by comma. Example "4.2".
+   If not set will use the current major and minor version from Sming.
+
+.. envvar::APP_VERSION_PATCH
+
+   Default: not set
+
+   Contains the application patch version as integer. For stable versions you can use 0 until 255.
+   For unstable versions the current timestamp can be used as a patch version.
+
+.. envvar:: ENABLE_OTA_VARINT_VERSION
+
+   Default: 1 (enabled)
+
+   If set to 1 the OTA upgrade mechanism and application will use a `varint <https://developers.google.com/protocol-buffers/docs/encoding#varints>`_
+   encoding for the patch version. Thus allowing unlimited number of patch versions. Useful for enumerating unstable/nightly releases.
+   A bit more difficult to read and write but allows for unlimited versions.
+
+   If set to 0 the OTA upgrade mechanism and application will use one byte for the patch version which will limit it to 256 possible patch versions.
+   Useful for enumarating stable releases. Easier to write and read but limited to 256 versions only.
+
+.. envvar:: ENABLE_OTA_ADVANCED
+
+   Default: 0 (disabled)
+
+   If set to 1 the library will work with OtaUpgradeStream which supports signature and encryption of the firmware data itself.
+   See :component:`OtaUpgrade` for details.
+
+.. envvar:: ENABLE_SSL
+
+   Default: unset (disable)
+
+   If set to 1 (highly recommended), OTA upgrade files will be trasnferred securely over TLS/SSL.
+
+.. envvar:: ENABLE_CLIENT_CERTIFICATE
+
+   Default: 0 (disabled)
+
+   Used in combination with ``ENABLE_SSL``. Set to 1 if the remote server requires the application to authenticate via client certficate.
+
+.. envvar:: MQTT_URL
+
+   Default: depends on  ``ENABLE_SSL`` and ``ENABLE_CLIENT_CERTIFICATE`` values
+
+   Url containing the location of the firmware update MQTT server.

samples/Ota_Mqtt/app/application.cpp → Sming/Libraries/OtaUpgradeMqtt/samples/Upgrade/app/application.cpp

@@ -1,5 +1,11 @@
 #include <SmingCore.h>
-#include <Data/Stream/RbootOutputStream.h>
+#include <rboot-api.h>
+#include <Storage/Partition.h>
+#include <OtaUpgrade/Mqtt/RbootPayloadParser.h>
+
+#if ENABLE_OTA_ADVANCED
+#include <OtaUpgrade/Mqtt/AdvancedPayloadParser.h>
+#endif
 
 // If you want, you can define WiFi settings globally in Eclipse Environment Variables
 #ifndef WIFI_SSID
@@ -22,16 +28,6 @@ IMPORT_FSTR(privateKeyData, PROJECT_DIR "/files/private.pem.key.der");
 IMPORT_FSTR(certificateData, PROJECT_DIR "/files/certificate.pem.crt.der");
 #endif
 
-struct UpdateState {
-	RbootOutputStream* stream{nullptr};
-	bool started{false};
-	size_t offset{0}; // The bytes used for encoding the version.
-	size_t version{0};
-};
-
-constexpr const uint8_t VERSION_NOT_READY = -1;
-constexpr const uint8_t VERSION_MAX_BYTES_ALLOWED = 24;
-
 Storage::Partition findRomPartition(uint8_t slot)
 {
 	String name = F("rom");
@@ -43,47 +39,6 @@ Storage::Partition findRomPartition(uint8_t slot)
 	return part;
 }
 
-void switchRom()
-{
-	uint8 before, after;
-	before = rboot_get_current_rom();
-	if(before == 0) {
-		after = 1;
-	} else {
-		after = 0;
-	}
-	Serial.printf("Swapping from rom %d to rom %d.\r\n", before, after);
-	rboot_set_current_rom(after);
-	Serial.println("Restarting...\r\n");
-	System.restart();
-}
-
-#if ENABLE_VARINT_PATCH_VERSION
-int getPatchVersion(const char* buffer, int length, size_t& offset, size_t versionStart = 0)
-{
-	size_t version = versionStart;
-	offset = 0;
-	int useNextByte = 0;
-	do {
-		version += (buffer[offset] & 0x7f);
-		useNextByte = (buffer[offset++] & 0x80);
-	} while(useNextByte && (offset < length));
-
-	if(useNextByte) {
-		// all the data is consumed and we still don't have a version number?!
-		return VERSION_NOT_READY;
-	}
-
-	return version;
-}
-#else
-int getPatchVersion(const char* buffer, int length, size_t& offset, size_t versionStart = 0)
-{
-	offset = 1;
-	return buffer[0];
-}
-#endif
-
 void otaUpdate()
 {
 	if(mqtt.isProcessing()) {
@@ -133,79 +88,23 @@ void otaUpdate()
 #endif
 
 	mqtt.connect(Url(MQTT_URL), "sming");
-	mqtt.setPayloadParser(
-		[part](MqttPayloadParserState& state, mqtt_message_t* message, const char* buffer, int length) -> int {
-			if(message == nullptr) {
-				debug_e("Invalid MQTT message");
-				return 1;
-			}
-
-			if(length == MQTT_PAYLOAD_PARSER_START) {
-				UpdateState* updateState = new UpdateState();
-				updateState->stream = nullptr;
-				updateState->started = false;
-
-				state.offset = 0;
-				state.userData = updateState;
-				return 0;
-			}
-
-			auto updateState = static_cast<UpdateState*>(state.userData);
-			if(updateState == nullptr) {
-				debug_e("Update failed for unknown reason!");
-				return -1;
-			}
-
-			if(length == MQTT_PAYLOAD_PARSER_END) {
-				bool skip = (updateState->stream == nullptr);
-				if(!skip) {
-					delete updateState->stream;
-					switchRom();
-				}
-
-				return 0;
-			}
 
-			if(!updateState->started) {
-				size_t offset = 0;
-				int patchVersion = getPatchVersion(buffer, length, offset, updateState->version);
-				updateState->offset += offset;
-#if ENABLE_VARINT_PATCH_VERSION
-				if(patchVersion == VERSION_NOT_READY) {
-
-					if(updateState->offset > VERSION_MAX_BYTES_ALLOWED) {
-						debug_e("Invalid patch version.");
-						return -3; //
-					}
-					return 0;
-				}
+#if ENABLE_OTA_ADVANCED
+	/**
+	 * The advanced parser suppors all firmware upgrades supported by the `OtaUpgrade` library.
+	 * It comes with firmware signing, firmware encryption and so on.
+	 */
+	auto parser = new OtaUpgrade::Mqtt::AdvancedPayloadParser(APP_VERSION_PATCH);
+#else
+	/**
+	 * The command below uses class that stores the firmware directly
+	 * using RbootOutputStream on a location provided by us
+	 */
+	auto parser = new OtaUpgrade::Mqtt::RbootPayloadParser(part, APP_VERSION_PATCH);
 #endif
 
-				updateState->started = true;
-				if(patchVersion < APP_VERSION_PATCH) {
-					// The update is not newer than our current patch version
-					return 0;
-				}
-
-				if(message->common.length - updateState->offset > part.size()) {
-					debug_e("The new rom is too big to fit!");
-					return -2;
-				}
-
-				length -= offset;
-				buffer += offset;
-
-				updateState->stream = new RbootOutputStream(part.address(), part.size());
-			}
-
-			auto rbootStream = static_cast<RbootOutputStream*>(updateState->stream);
-			if(rbootStream == nullptr) {
-				return 0;
-			}
-
-			auto written = rbootStream->write(reinterpret_cast<const uint8_t*>(buffer), length);
-			return (written - length);
-		});
+	mqtt.setPayloadParser([parser](MqttPayloadParserState& state, mqtt_message_t* message, const char* buffer,
+								   int length) -> int { return parser->parse(state, message, buffer, length); });
 
 	String updateTopic = "/a/";
 	updateTopic += APP_ID;