New modular OTA structure (#101)

* components: Remove ota_for_aws_iot_embedded_sdk Remove the ota_for_aws_iot_embedded_sdk component, as we will be replacing it with the new modular OTA. Signed-off-by: Chuyue Luo <[email protected]> * applications: Add C Runtime Helpers Add a crt-helpers/ directory within applicatons/helpers. Within this directory, add a custom implementation of `strnlen` (based on TF-M's `tfm_strnlen` implementation). This is required because the Arm Compiler for Embedded (v6.21) does not support `strnlen`. Signed-off-by: Chuyue Luo <[email protected]> * components: Add Jobs-for-AWS-IoT-embedded-sdk component Add the Jobs-for-AWS-IoT-embedded-sdk repository as a submodule. This library is used to interact with AWS IoT Jobs (remote operations that are sent to and executed on devices connected to AWS IoT). It is one of the two libraries that must be integrated to allow the new modular OTA to be used - second library (aws-iot-core-mqtt-file-streams-embedded-c) is integrated in a later commit. In addition, the required integration CMake files to build the component are added. Signed-off-by: Chuyue Luo <[email protected]> * components: Add patches for Jobs-for-AWS-IoT-embedded-sdk library Two patches are added for the Jobs-for-AWS-IoT-embedded-sdk library: - The Jobs library assumes the OTA job is signed using ECDSA. However, we currently use RSA. Therefore, add a patch to change the check for an ECDSA signature to a check for an RSA signature. - The Jobs library contains calls to the `strnlen` function. However, this function is not supported by the Arm Compiler for Embedded (v6.21). Therefore, add a patch which replaces these calls with calls to our custom implementation `app_strnlen`. Signed-off-by: Chuyue Luo <[email protected]> * components: Add aws-iot-core-mqtt-file-streams-embedded-c component Add the aws-iot-core-mqtt-file-streams-embedded-c repository as a submodule. This library allows files from a stream (an abstraction for a list of files) to be transferred to an IoT device. It is the second of the two libraries that must be integrated to allow the new modular OTA to be used. In addition, the required integration CMake files to build the component are added. Signed-off-by: Chuyue Luo <[email protected]> * components: Add patch for MQTT File Streams library The aws-iot-core-mqtt-file-streams-embedded-c library uses the `strnlen` function, which is not supported by the Arm Compiler for Embedded (v6.21). Therefore, add a patch which replaces the call to `strnlen` with a call to our custom implementation `app_strnlen`. Signed-off-by: Chuyue Luo <[email protected]> * applications: Add MQTT File Downloader config for keyword detection THe MQTT File Streams library allows a MQTTFileDownloader_config.h file to be provided, which defines custom values for build configuration macros. This commit adds a MQTTFileDownloader_config.h file for the keyword detection example. This file defines the block size that should be used when downloading the firmware image. Signed-off-by: Chuyue Luo <[email protected]> * components: Add patch for FreeRTOS OTA PAL PSA Add a patch for FreeRTOS OTA PAL PSA to allow it to work with the new modular OTA structure. This patch does the following: - Update header includes to remove headers from the old ota-for-aws-iot-embedded-sdk library, replace these with headers from the new Jobs-for-AWS-IoT-embedded-sdk library - Remove usage of data structures from ota-for-aws-iot-embedded-sdk library, replace these with data structures from Jobs-for-AWS-IoT-embedded-sdk library. Signed-off-by: Chuyue Luo <[email protected]> * ota: Add OTA orchestrator Add an OTA orchestrator as a helper within the applications/ directory. The OTA orchestrator uses functionality from the Jobs and MQTT File Streaming libraries to enable OTA updates. In addition, update the keyword detection CMakeLists.txt to allow this example to use the new modular OTA. Signed-off-by: Chuyue Luo <[email protected]> * applications: New modular OTA for speech recognition example Update speech recognition CMakeLists.txt and add MQTTFileDownloader_config.h file to allow this example to use the new modular OTA. Signed-off-by: Chuyue Luo <[email protected]> * applications: New modular OTA for object detection example Update object detection CMakeLists.txt and add MQTTFileDownloader_config.h file to allow this example to use the new modular OTA. Signed-off-by: Chuyue Luo <[email protected]> * applications: New modular OTA for FreeRTOS IoT Libraries Tests Update FreeRTOS IoT Libraries Tests CMakeLists.txt and add MQTTFileDownloader_config.h file to allow the tests to use the new modular OTA. Signed-off-by: Chuyue Luo <[email protected]> --------- Signed-off-by: Chuyue Luo <[email protected]>
FreeRTOS · Dec 5, 2024 · 0048e5f · 0048e5f
1 parent 3780112
commit 0048e5f
Show file tree

Hide file tree

Showing 65 changed files with 3,424 additions and 3,353 deletions.
diff --git a/.github/.cSpellWords.txt b/.github/.cSpellWords.txt
@@ -39,6 +39,7 @@ coef
 compatibil
 coremqtt
 COSE
+coverity
 CLRF
 Cqqpk
 CSRS
@@ -147,6 +148,7 @@ Merkle
 Mfcc
 MFCC
 MFLN
+MISRA
 mlek
 mqtt
 mqttconfig

diff --git a/.gitmodules b/.gitmodules
@@ -43,9 +43,6 @@
 [submodule "open_iot_sdk_toolchain"]
 	path = components/tools/open_iot_sdk_toolchain/library
 	url = https://git.gitlab.arm.com/iot/open-iot-sdk/toolchain.git
-[submodule "ota_for_aws_iot_embedded_sdk"]
-	path = components/aws_iot/ota_for_aws_iot_embedded_sdk/library
-	url = https://github.com/aws/ota-for-aws-iot-embedded-sdk.git
 [submodule "tinycbor"]
 	path = components/aws_iot/tinycbor/library
 	url = https://github.com/intel/tinycbor.git
@@ -76,3 +73,9 @@
 [submodule "googletest"]
 	path = components/tools/googletest/library
 	url = https://github.com/google/googletest.git
+[submodule "components/aws_iot/jobs_for_aws_iot_embedded_sdk/library"]
+	path = components/aws_iot/jobs_for_aws_iot_embedded_sdk/library
+	url = https://github.com/aws/Jobs-for-AWS-IoT-embedded-sdk.git
+[submodule "components/aws_iot/aws_iot_core_mqtt_file_streams_embedded_c/library"]
+	path = components/aws_iot/aws_iot_core_mqtt_file_streams_embedded_c/library
+	url = https://github.com/aws/aws-iot-core-mqtt-file-streams-embedded-c/
diff --git a/applications/freertos_iot_libraries_tests/CMakeLists.txt b/applications/freertos_iot_libraries_tests/CMakeLists.txt
@@ -108,7 +108,7 @@ target_link_libraries(${CMAKE_PROJECT_NAME}
         freertos-ota-pal-psa
         fri-bsp
         helpers-events
-        ota-for-aws-iot-embedded-sdk
+        ota-update
         provisioning-lib
         mbedtls
         tfm-ns-interface

diff --git a/applications/freertos_iot_libraries_tests/configs/aws_configs/CMakeLists.txt b/applications/freertos_iot_libraries_tests/configs/aws_configs/CMakeLists.txt
@@ -31,8 +31,3 @@ target_include_directories(freertos-pkcs11-psa-config
     INTERFACE
         .
 )
-
-target_include_directories(ota-for-aws-iot-embedded-sdk-config
-    INTERFACE
-        .
-)
diff --git a/applications/freertos_iot_libraries_tests/configs/aws_configs/MQTTFileDownloader_config.h b/applications/freertos_iot_libraries_tests/configs/aws_configs/MQTTFileDownloader_config.h
@@ -0,0 +1,31 @@
+/*
+ * Copyright Amazon.com, Inc. and its affiliates. All Rights Reserved.
+ * Copyright 2024 Arm Limited and/or its affiliates
+ * <[email protected]>
+ *
+ * SPDX-License-Identifier: MIT
+ * Licensed under the MIT License. See the LICENSE accompanying this file
+ * for the specific language governing permissions and limitations under
+ * the License.
+ */
+
+/**
+ * @file MQTTFileDownloader_config.h
+ * @brief MQTT File Streams options.
+ */
+
+#ifndef MQTT_FILE_DOWNLOADER_CONFIG_H
+#define MQTT_FILE_DOWNLOADER_CONFIG_H
+
+/* Standard includes */
+#include <stdint.h>
+
+/**
+ * @ingroup mqtt_file_downloader_const_types
+ * @brief Configure the Maximum size of the data payload. The
+ * smallest value is 256 bytes, maximum is 128KB. For more see
+ * https://docs.aws.amazon.com/general/latest/gr/iot-core.html
+ */
+#define mqttFileDownloader_CONFIG_BLOCK_SIZE    4096U
+
+#endif /* MQTT_FILE_DOWNLOADER_CONFIG_H */
diff --git a/applications/helpers/CMakeLists.txt b/applications/helpers/CMakeLists.txt
@@ -2,10 +2,12 @@
 # <[email protected]>
 # SPDX-License-Identifier: MIT
 
+add_subdirectory(crt_helpers)
 add_subdirectory(device_advisor)
 add_subdirectory(events)
 add_subdirectory(hdlcd)
 add_subdirectory(logging)
+add_subdirectory(ota_orchestrator)
 add_subdirectory(provisioning)
 # sntp helper library depends on FreeRTOS-Plus-TCP connectivity stack as it
 # includes `FreeRTOS_IP.h` header file in one of its source files (sntp_client_task.c),

diff --git a/applications/helpers/crt_helpers/CMakeLists.txt b/applications/helpers/crt_helpers/CMakeLists.txt
@@ -0,0 +1,18 @@
+# Copyright 2024 Arm Limited and/or its affiliates
+# <[email protected]>
+# SPDX-License-Identifier: MIT
+
+if(BUILD_TESTING AND NOT CMAKE_CROSSCOMPILING)
+    # Left empty for future mocks.
+else()
+    add_library(crt-helpers)
+
+    target_include_directories(crt-helpers
+        PUBLIC
+            inc/)
+
+    target_sources(crt-helpers
+        PUBLIC
+            src/app_strnlen.c
+    )
+endif()
diff --git a/applications/helpers/crt_helpers/inc/app_strnlen.h b/applications/helpers/crt_helpers/inc/app_strnlen.h
@@ -0,0 +1,25 @@
+/*
+ * Copyright (c) 2021-2024, Arm Limited. All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ *
+ */
+
+#ifndef APP_STRNLEN_H
+
+#include <stddef.h>
+
+/**
+ * @brief Determine the length of a fixed-size string, excluding the terminating
+ * null byte ('\0'), and at most `maxlen`.
+ *
+ * @param[in] s The string to determine the length of.
+ * @param[in] maxlen The maximum number of characters of the string `s` that
+ * should be checked.
+ *
+ * @return The length of the string, up to `maxlen`.
+ */
+size_t app_strnlen( const char * s,
+                    size_t maxlen );
+
+#endif
diff --git a/applications/helpers/crt_helpers/src/app_strnlen.c b/applications/helpers/crt_helpers/src/app_strnlen.c
@@ -0,0 +1,34 @@
+/*
+ * Copyright (c) 2021-2024, Arm Limited. All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ *
+ */
+
+/*
+ * This file is based on
+ * https://git.trustedfirmware.org/plugins/gitiles/TF-M/trusted-firmware-m.git/+/c9352b59f2a501b5af3f648b3fc91065993c002f/secure_fw/partitions/lib/runtime/crt_strnlen.c
+ */
+
+#include <stddef.h>
+
+size_t app_strnlen( const char * s,
+                    size_t maxlen )
+{
+    size_t idx;
+
+    if( s == NULL )
+    {
+        return 0;
+    }
+
+    for( idx = 0; idx < maxlen; idx++ )
+    {
+        if( s[ idx ] == '\0' )
+        {
+            return idx;
+        }
+    }
+
+    return idx;
+}
diff --git a/applications/helpers/ota_orchestrator/CMakeLists.txt b/applications/helpers/ota_orchestrator/CMakeLists.txt
@@ -0,0 +1,33 @@
+# Copyright 2024 Arm Limited and/or its affiliates
+# <[email protected]>
+# SPDX-License-Identifier: MIT
+
+if(BUILD_TESTING AND NOT CMAKE_CROSSCOMPILING)
+    # Left empty for future mocks.
+else()
+    add_library(ota-update
+        src/mqtt_helpers.c
+        src/ota_orchestrator_helpers.c
+        src/ota_os_freertos.c
+        src/ota_orchestrator.c
+    )
+
+    target_include_directories(ota-update
+        PUBLIC
+            inc/
+    )
+
+    target_link_libraries(ota-update
+        jobs-for-aws-iot-embedded-sdk
+        aws-iot-core-mqtt-file-streams-embedded-c
+        freertos_kernel
+        corejson
+        coremqtt
+        coremqtt-agent
+        tinycbor
+        freertos-ota-pal-psa
+        helpers-events
+        backoff-algorithm
+        crt-helpers
+    )
+endif()
diff --git a/applications/helpers/ota_orchestrator/inc/mqtt_helpers.h b/applications/helpers/ota_orchestrator/inc/mqtt_helpers.h
@@ -0,0 +1,78 @@
+/* Copyright 2023-2024 Arm Limited and/or its affiliates
+ * <[email protected]>
+ * SPDX-License-Identifier: MIT
+ */
+
+#ifndef MQTT_HELPERS_H
+#define MQTT_HELPERS_H
+
+/* Standard includes. */
+#include <stdlib.h>
+#include <stdint.h>
+
+/**
+ * @ingroup ota_enum_types
+ * @brief The OTA MQTT interface return status.
+ */
+typedef enum OtaMqttStatus
+{
+    OtaMqttSuccess = 0,          /*!< @brief OTA MQTT interface success. */
+    OtaMqttPublishFailed = 0xa0, /*!< @brief Attempt to publish a MQTT message failed. */
+    OtaMqttSubscribeFailed,      /*!< @brief Failed to subscribe to a topic. */
+    OtaMqttUnsubscribeFailed     /*!< @brief Failed to unsubscribe from a topic. */
+} OtaMqttStatus_t;
+
+/**
+ * @brief Subscribe to a specified topic.
+ *
+ * Precondition: pTopicFilter is not null.
+ *
+ * @param[in] pTopicFilter The topic filter to subscribe to.
+ * @param[in] topicFilterLength Length of the topic filter.
+ * @param[in] ucQoS Quality of Service (QoS), the level of reliability for
+ * message delivery. Can be 0, 1 or 2.
+ *
+ * @return OtaMqttSuccess if the subscribe completed successfully, otherwise
+ * OtaMqttSubscribeFailed.
+ */
+OtaMqttStatus_t prvMQTTSubscribe( const char * pTopicFilter,
+                                  uint16_t topicFilterLength,
+                                  uint8_t ucQoS );
+
+/**
+ * @brief Publish a message to a specified topic.
+ * *
+ * @param[in] pacTopic The topic that the message should be published to.
+ * @param[in] topicLen Length of the topic.
+ * @param[in] pMsg The message to be published.
+ * @param[in] msgSize Size of the message.
+ * @param[in] ucQoS Quality of Service (QoS), the level of reliability for
+ * message delivery. Can be 0, 1 or 2.
+ *
+ * @return OtaMqttSuccess if the subscribe completed successfully, otherwise
+ * OtaMqttSubscribeFailed.
+ */
+OtaMqttStatus_t prvMQTTPublish( const char * const pacTopic,
+                                uint16_t topicLen,
+                                const char * pMsg,
+                                uint32_t msgSize,
+                                uint8_t ucQoS );
+
+/**
+ * @brief Unsubscribe from a specified topic.
+ *
+ * Precondition: pTopicFilter is not null.
+ *
+ * @param[in] pTopicFilter The topic filter to unsubscribe from.
+ * @param[in] topicFilterLength Length of the topic filter.
+ * @param[in] ucQoS Quality of Service (QoS), the level of reliability for
+ * message delivery. Can be 0, 1 or 2.
+ *
+ * @return OtaMqttSuccess if the unsubscribe completed successfully, otherwise
+ * OtaMqttSubscribeFailed.
+ */
+OtaMqttStatus_t prvMQTTUnsubscribe( const char * pTopicFilter,
+                                    uint16_t topicFilterLength,
+                                    uint8_t ucQoS );
+
+#endif /* MQTT_HELPERS_H */
diff --git a/applications/helpers/ota_orchestrator/inc/ota_appversion32.h b/applications/helpers/ota_orchestrator/inc/ota_appversion32.h
@@ -0,0 +1,88 @@
+/*
+ * AWS IoT Over-the-air Update v3.4.0
+ * Copyright (C) 2020 Amazon.com, Inc. or its affiliates.  All Rights Reserved.
+ * Copyright 2024 Arm Limited and/or its affiliates
+ * <[email protected]>
+ *
+ * SPDX-License-Identifier: MIT
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy of
+ * this software and associated documentation files (the "Software"), to deal in
+ * the Software without restriction, including without limitation the rights to
+ * use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
+ * the Software, and to permit persons to whom the Software is furnished to do so,
+ * subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in all
+ * copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
+ * FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
+ * COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+ * IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+ */
+
+/**
+ * @file ota_appversion32.h
+ * @brief Structure to represent the application build version.
+ */
+
+#ifndef IOT_APPVERSION32_H
+#define IOT_APPVERSION32_H
+
+/* *INDENT-OFF* */
+#ifdef __cplusplus
+    extern "C" {
+#endif
+/* *INDENT-ON* */
+
+/* Standard includes. */
+#include <stdint.h>
+
+/**
+ * @ingroup ota_struct_types
+ * @brief Application version structure.
+ *
+ */
+typedef struct
+{
+    /* MISRA Ref 19.2.1 [Unions] */
+    /* More details at: https://github.com/aws/ota-for-aws-iot-embedded-sdk/blob/main/MISRA.md#rule-192 */
+    /* coverity[misra_c_2012_rule_19_2_violation] */
+    union
+    {
+        #if ( defined( __BYTE_ORDER__ ) && defined( __ORDER_LITTLE_ENDIAN__ ) && ( __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ ) ) || ( __little_endian__ == 1 ) || WIN32 || ( __BYTE_ORDER == __LITTLE_ENDIAN )
+            struct version
+            {
+                uint16_t build; /*!< @brief Build of the firmware (Z in firmware version Z.Y.X). */
+                uint8_t minor;  /*!< @brief Minor version number of the firmware (Y in firmware version Z.Y.X). */
+
+                uint8_t major;  /*!< @brief Major version number of the firmware (X in firmware version Z.Y.X). */
+            } x;                /*!< @brief Version number of the firmware. */
+        #elif ( defined( __BYTE_ORDER__ ) && defined( __ORDER_BIG_ENDIAN__ ) && __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__ ) || ( __big_endian__ == 1 ) || ( __BYTE_ORDER == __BIG_ENDIAN )
+            struct version
+            {
+                uint8_t major;  /*!< @brief Major version number of the firmware (X in firmware version X.Y.Z). */
+                uint8_t minor;  /*!< @brief Minor version number of the firmware (Y in firmware version X.Y.Z). */
+
+                uint16_t build; /*!< @brief Build of the firmware (Z in firmware version X.Y.Z). */
+            } x;                /*!< @brief Version number of the firmware. */
+        #else /* if ( defined( __BYTE_ORDER__ ) && defined( __ORDER_LITTLE_ENDIAN__ ) && __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ ) || ( __little_endian__ == 1 ) || WIN32 || ( __BYTE_ORDER == __LITTLE_ENDIAN ) */
+        #error "Unable to determine byte order!"
+        #endif /* if ( defined( __BYTE_ORDER__ ) && defined( __ORDER_LITTLE_ENDIAN__ ) && __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ ) || ( __little_endian__ == 1 ) || WIN32 || ( __BYTE_ORDER == __LITTLE_ENDIAN ) */
+        uint32_t unsignedVersion32;
+        int32_t signedVersion32;
+    } u; /*!< @brief Version based on configuration in big endian or little endian. */
+} AppVersion32_t;
+
+extern AppVersion32_t appFirmwareVersion; /*!< @brief Making the version number available globally through external linkage. */
+
+/* *INDENT-OFF* */
+#ifdef __cplusplus
+    }
+#endif
+/* *INDENT-ON* */
+
+#endif /* ifndef IOT_APPVERSION32_H */
-Original file line number
+Diff line change
@@ Expand Up / @@ -39,6 +39,7 @@ coef @@
     compatibil
     coremqtt
     COSE
+    coverity
     CLRF
     Cqqpk
     CSRS
@@ Expand Down Expand Up / @@ -147,6 +148,7 @@ Merkle @@
     Mfcc
     MFCC
     MFLN
+    MISRA
     mlek
     mqtt
     mqttconfig
@@ Expand Down @@