Storing email in NVS (IDFGH-3522) #5475

AbdeenM · 2020-06-21T07:55:38Z

Environment

Development Kit: ESP32-DevKitC
Kit version (for WroverKit/PicoKit/DevKitC): v1
Module or chip used: ESP32-WROOM-32
IDF version: 201912.00-396-g314275376
Build System: CMake
Compiler version: 1.22.0-80-g6c4433a5
Operating System: Windows
(Windows only) environment type: PowerShell
Using an IDE?: No
Power Supply: USB

Problem Description

So am trying to save an email “[email protected]” to the devices memory using the function nvs_set_str() with a key “user_email” but the string fails to be written to memory, meaning after I store the email and fetch it from memory and log it I receive weird symbols mostly question marks as in “??/ ????” Or something similar. Even if I restart the device and fetch the key value I get the same thing.

Am positive the way am storing or my code is correct as I am storing other details along with the email and they are stored and fetched successfully.

My question is, is there a different or specific way to handle storing emails?
I suspect my issue could be because the email in question has a “.” or “@“ if that’s the cause is there a specific way to handle these characters? Because I need to store any given user email that may be supplied.

I have tried posting the issue in aws-freertos github repo but they suggested i revert to you.

Expected Behavior

Email storied in NVS.

Actual Behavior

Email not storing in NVS.

Code to reproduce this issue

#include "esp_log.h"

// Decoded email comes from different part of the code, arrives as
char decodedEmail = "[email protected]";

// Opening Non-Volatile Storage (NVS) handle
ESP_LOGI(TAG, "Opening Non-Volatile Storage (NVS) handle... \n");
esp_err_t err = nvs_flash_init();
nvs_handle wifi_handle;
err = nvs_open("storage", NVS_READWRITE, &wifi_handle);
if (err != ESP_OK) {
    ESP_LOGI(TAG, "Error (%s) opening NVS handle! \n", esp_err_to_name(err));
    ESP_LOGI(TAG, "Restarting... \n");
    esp_restart();
} else {
    ESP_LOGI(TAG, "Done, opening storage! \n");

    // // Write Email value to Memory
    ESP_LOGI(TAG, "The user email that is writing to nvs right now = %s \n", decodedEmail);
    err = nvs_set_str(wifi_handle, "user_email", decodedEmail);
    printf((err != ESP_OK) ? "Failed! \n" : "Done \n");

    /* Other values wirtten to memory, which works fine
     * ...
     */

    // Commit writings to Memory
    ESP_LOGI(TAG, "Committing updates in NVS ... \n");
    err = nvs_commit(wifi_handle);
    printf((err != ESP_OK) ? "Failed! \n" : "Done \n");
}

// Close NVS
nvs_close(wifi_handle);

0xjakob · 2020-06-24T01:12:39Z

Hi @AbdeenM, the code to store the email address looks OK. Dots "." shouldn't be an issue, not sure about "@" right now. We'll have a look at this. Thanks for posting this here.

AbdeenM · 2020-06-24T14:02:19Z

Hi @0xjakob thanks for the comment, you are right i forgot to mention i did several tests, for example storing "abdeen" alone, then "abdeen.mohd" then "abdeen.mohamed" as values and it worked fine so "." is not the issue.
however the moment "@" is introduced the issue arises. i havent investigated further (i.e. testing something like "abdeen@" without a ".") but i think our issue is from the @ char.
I will be working on this tomorrow so i will update with my findings as well.

0xjakob · 2020-06-28T02:29:42Z

@AbdeenM
I justed modified one of the IDF nvs examples:

#include <stdio.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "esp_system.h"
#include "nvs_flash.h"
#include "nvs.h"

void app_main(void)
{
    // Initialize NVS
    esp_err_t err = nvs_flash_init();
    if (err == ESP_ERR_NVS_NO_FREE_PAGES || err == ESP_ERR_NVS_NEW_VERSION_FOUND) {
        // NVS partition was truncated and needs to be erased
        // Retry nvs_flash_init
        ESP_ERROR_CHECK(nvs_flash_erase());
        err = nvs_flash_init();
    }
    ESP_ERROR_CHECK( err );

    // Open
    printf("\n");
    printf("Opening Non-Volatile Storage (NVS) handle... ");
    nvs_handle_t my_handle;
    err = nvs_open("storage", NVS_READWRITE, &my_handle);
    if (err != ESP_OK) {
        printf("Error (%s) opening NVS handle!\n", esp_err_to_name(err));
    } else {
        printf("Done\n");

        // Read
        printf("Reading restart counter from NVS ... ");
        int32_t restart_counter = 0; // value will default to 0, if not set yet in NVS
        char str [128] = "[email protected]";
        size_t length;
        err = nvs_get_str(my_handle, "email", str, &length);
        switch (err) {
            case ESP_OK:
                printf("Done\n");
                printf("email address: = %s\n", str);
                break;
            case ESP_ERR_NVS_NOT_FOUND:
                printf("The email addr is not initialized yet!\n");
                break;
            default :
                printf("Error (%s) reading!\n", esp_err_to_name(err));
        }

        // Write
        printf("Updating email address NVS ... ");
        restart_counter++;
        err = nvs_set_str(my_handle, "email", str);
        printf((err != ESP_OK) ? "Failed!\n" : "Done\n");

        // Commit written value.
        // After setting any values, nvs_commit() must be called to ensure changes are written
        // to flash storage. Implementations may write to storage at other times,
        // but this is not guaranteed.
        printf("Committing updates in NVS ... ");
        err = nvs_commit(my_handle);
        printf((err != ESP_OK) ? "Failed!\n" : "Done\n");

        // Close
        nvs_close(my_handle);
    }

    printf("\n");

    // Restart module
    for (int i = 10; i >= 0; i--) {
        printf("Restarting in %d seconds...\n", i);
        vTaskDelay(1000 / portTICK_PERIOD_MS);
    }
    printf("Restarting now.\n");
    fflush(stdout);
    esp_restart();
}

This code works, the email address is printed after the reboot. I wonder whether it's an encoding issue...
Could you try to print out the email address in hex before you save it to nvs?

Actually, '@' also exists in the normal ascii table. So if the issue persists, one way to solve this would be to convert the string to ascii before saving it.

AbdeenM · 2020-06-28T09:25:42Z

HI @0xjakob thank you for the response!
Okay so ran your code and it does work fine, i even modified it alittle and it works with no issues (see below);

/**
 * @brief Run tests for saving to NVS.
 */
void test_nvs() {
    // Initialize NVS
    esp_err_t err = nvs_flash_init();
    if (err == ESP_ERR_NVS_NO_FREE_PAGES || err == ESP_ERR_NVS_NEW_VERSION_FOUND) {
        // NVS partition was truncated and needs to be erased
        // Retry nvs_flash_init
        ESP_ERROR_CHECK(nvs_flash_erase());
        err = nvs_flash_init();
    }
    ESP_ERROR_CHECK( err );

    // Open
    printf("\n");
    printf("Opening Non-Volatile Storage (NVS) handle... ");
    nvs_handle my_handle;
    err = nvs_open("storage", NVS_READWRITE, &my_handle);
    if (err != ESP_OK) {
        printf("Error (%s) opening NVS handle!\n", esp_err_to_name(err));
    } else {
        printf("Done\n");

        // Read
        printf("Reading restart counter from NVS ... ");
        int32_t restart_counter = 0; // value will default to 0, if not set yet in NVS
        char data [128] = "[email protected]";
        char str [128];
        size_t length;
        err = nvs_get_str(my_handle, "user_email", str, &length);
        switch (err) {
            case ESP_OK:
                printf("Done\n");
                printf("email address: = %s\n", str);
                break;
            case ESP_ERR_NVS_NOT_FOUND:
                printf("The email addr is not initialized yet!\n");
                
                // Write
                printf("Updating email address NVS ... ");
                restart_counter++;
                err = nvs_set_str(my_handle, "user_email", data);
                printf((err != ESP_OK) ? "Failed!\n" : "Done\n");

                // Commit written value.
                // After setting any values, nvs_commit() must be called to ensure changes are written
                // to flash storage. Implementations may write to storage at other times,
                // but this is not guaranteed.
                printf("Committing updates in NVS ... ");
                err = nvs_commit(my_handle);
                printf((err != ESP_OK) ? "Failed!\n" : "Done\n");

                break;
            default :
                printf("Error (%s) reading!\n", esp_err_to_name(err));
        }

        // Close
        nvs_close(my_handle);
    }

    printf("\n");

    // Restart module
    for (int i = 10; i >= 0; i--) {
        printf("Restarting in %d seconds...\n", i);
        vTaskDelay(1000 / portTICK_PERIOD_MS);
    }
    printf("Restarting now.\n");
    fflush(stdout);
    esp_restart();
}

what you are doing is pretty much what am doing as well, i really don't get what could be wrong on my main code. i have attached a gist, please have a look and let me know what you think.
The flow is as follows:

once the device launches, it checks if wifi ssid and password are available. (if not it runs wifi provisioning)
after confirming wifi is provisioned it checks if aws is provisioned. (if not it runs aws provisioning)
after confirming aws is provisioned it connects to aws and handles other functions.

Thanks again

igrr · 2020-06-28T09:41:38Z

It looks like you are calculating string_size based on device_id value, but then using it to get user_email.

…

On Sun, Jun 28, 2020, 11:26 Abdeen Mohamed ***@***.***> wrote: HI @0xjakob <https://github.com/0xjakob> thank you for the response! Okay so ran your code and it does work fine, i even modified it alittle and it works with no issues (see below); /** * @brief Run tests for saving to NVS. */void test_nvs() { // Initialize NVS esp_err_t err = nvs_flash_init(); if (err == ESP_ERR_NVS_NO_FREE_PAGES || err == ESP_ERR_NVS_NEW_VERSION_FOUND) { // NVS partition was truncated and needs to be erased // Retry nvs_flash_init ESP_ERROR_CHECK(nvs_flash_erase()); err = nvs_flash_init(); } ESP_ERROR_CHECK( err ); // Open printf("\n"); printf("Opening Non-Volatile Storage (NVS) handle... "); nvs_handle my_handle; err = nvs_open("storage", NVS_READWRITE, &my_handle); if (err != ESP_OK) { printf("Error (%s) opening NVS handle!\n", esp_err_to_name(err)); } else { printf("Done\n"); // Read printf("Reading restart counter from NVS ... "); int32_t restart_counter = 0; // value will default to 0, if not set yet in NVS char data [128] = ***@***.***"; char str [128]; size_t length; err = nvs_get_str(my_handle, "user_email", str, &length); switch (err) { case ESP_OK: printf("Done\n"); printf("email address: = %s\n", str); break; case ESP_ERR_NVS_NOT_FOUND: printf("The email addr is not initialized yet!\n"); // Write printf("Updating email address NVS ... "); restart_counter++; err = nvs_set_str(my_handle, "user_email", data); printf((err != ESP_OK) ? "Failed!\n" : "Done\n"); // Commit written value. // After setting any values, nvs_commit() must be called to ensure changes are written // to flash storage. Implementations may write to storage at other times, // but this is not guaranteed. printf("Committing updates in NVS ... "); err = nvs_commit(my_handle); printf((err != ESP_OK) ? "Failed!\n" : "Done\n"); break; default : printf("Error (%s) reading!\n", esp_err_to_name(err)); } // Close nvs_close(my_handle); } printf("\n"); // Restart module for (int i = 10; i >= 0; i--) { printf("Restarting in %d seconds...\n", i); vTaskDelay(1000 / portTICK_PERIOD_MS); } printf("Restarting now.\n"); fflush(stdout); esp_restart(); } what you are doing is pretty much what am doing as well, i really don't get what could be wrong on my main code. i have attached a gist <https://gist.github.com/AbdeenM/6b6de7080952e59e2c1052f7e12bc5ea>, please have a look and let me know what you think. The flow is as follows: 1. once the device launches, it checks if wifi ssid and password are available. (if not it runs wifi provisioning) 2. after confirming wifi is provisioned it checks if aws is provisioned. (if not it runs aws provisioning) 3. after confirming aws is provisioned it connects to aws and handles other functions. Thanks again — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#5475 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABBFY6QBVJXHKY25GBDJ7XLRY4EDBANCNFSM4ODXXPEQ> .

AbdeenM · 2020-06-28T13:25:15Z

@igrr Of course! that was it. thank you so much for the catch, that fixed my problem.
i refactored my code as it was abit of a mess on how am handling nvs.
Thanks again i will close the issue now.

github-actions bot changed the title ~~Storing email in NVS~~ Storing email in NVS (IDFGH-3522) Jun 21, 2020

AbdeenM closed this as completed Jun 28, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Storing email in NVS (IDFGH-3522) #5475

Storing email in NVS (IDFGH-3522) #5475

AbdeenM commented Jun 21, 2020 •

edited

Loading

0xjakob commented Jun 24, 2020

AbdeenM commented Jun 24, 2020

0xjakob commented Jun 28, 2020 •

edited

Loading

AbdeenM commented Jun 28, 2020

igrr commented Jun 28, 2020 via email

AbdeenM commented Jun 28, 2020

Storing email in NVS (IDFGH-3522) #5475

Storing email in NVS (IDFGH-3522) #5475

Comments

AbdeenM commented Jun 21, 2020 • edited Loading

Environment

Problem Description

Expected Behavior

Actual Behavior

Code to reproduce this issue

0xjakob commented Jun 24, 2020

AbdeenM commented Jun 24, 2020

0xjakob commented Jun 28, 2020 • edited Loading

AbdeenM commented Jun 28, 2020

igrr commented Jun 28, 2020 via email

AbdeenM commented Jun 28, 2020

AbdeenM commented Jun 21, 2020 •

edited

Loading

0xjakob commented Jun 28, 2020 •

edited

Loading