Deep Sleep (Feature Request)

[nullstalgia]

Feel free to trim the title for a proper tag. Not sure how you personally label these kinds of things :)

One of the reasons I adore epaper displays is because how low power they are! (I own several e-ink ereaders. Maybe hoard is a better word)

And that was the basis of the project that you commented on that summoned the scourge that is me upon your codebase got me interested in revamping my project with this, to make a nicer looking, easier to edit version.

However, it runs on a battery. And batteries are small and don't last that long! But, epaper doesn't need power to keep the previous data for a very long time.

Luckily, the basic implementation seems quite easy. (Just 2 functions! esp_sleep_enable_timer_wakeup(microseconds_to_sleep) and esp_deep_sleep_start())

Source

But there's a slight issue when it comes to the Web GUI. Which is keeping it online so a user can get to the UI. I see a few ways of solving this, but it's ultimately up to you.

1. Don't care about the user unless they connect
   • Start a timer on program start (say, 10-30s) and see if a user connects via HTTP. If they do, suspend the timer until board reset/button on dashboard similar to "Reboot"
2. Only start sleeping when told to or when previous reset was caused by deep sleep timer
   • Via button on dashboard similar to "Reboot"
   • But how do you tell it to stop?
3. Always sleep unless GPIO0 is held after boot
   • This is one I'm leaning towards, as it's resilient against sudden power loss, and is easy enough for most to understand. Only issue is timing when to hold the button, as on Reset, it will go into download mode, and too late it will go to sleep. Maybe having this configurable like the DC/RST/BUSY pins would be best.

Another thing that would be nice, is an MQTT Birth message. Otherwise, it would have to rely on either:

1. Broker-Retained messages

2. Last value heard by the ESP

# 1 Isn't that bad, but I prefer any persistent data to be kept in a proper database, and MQTT db files don't seem mission critical to me.
# 2 Is a bad idea all around, as it's more likely than not for the ESP to miss the publisher's message when it wakes.

Having it wake up, send a message saying "I'm alive!" and listening for 5-10 seconds before going back to sleep sounds ideal to me.

(Heck, even having the birth message in general could have positive side effects outside of deep sleep. Such as knowing for sure the ESP is connected to MQTT, making sure it has proper data very soon after start up, etc)

A pre-deep sleep message could be nice as well. Could be used simply for testing/logging, but I'm sure someone has a use for it.

[sidoh]

Great suggestion!

(3) makes perfect sense to me. To basically regurgitate what you've written with a few more specifics filled in:

• Add a deep sleep mode. When enabled, the firmware after boot will:
  • Check if a (configurable) GPIO pin is set. If it is, deep sleep mode will be disabled until the next boot.
  • Wait for a configurable timeout for data updates, refreshing the screen normally as they come in.
  • Enter deep sleep for a configurable period

I like the idea of the break-out-of-sleep safety valve being physical.

An MQTT birth message makes sense in conjunction with this, but is a worthwhile feature by itself.

Neither deep sleep nor MQTT are terribly difficult features when broken down this way. :)

[sidoh]

(by the way, really appreciate the involvement! thanks for taking the time to improve the project :)

[nullstalgia]

Extra brownie points if the pin state check is configurable to a HIGH or LOW! 🍪

Maybe an extra checkbox to set the pin to an INPUT_PULLUP/PULLDOWN to save the user a resistor? Or maybe it should be default depending on the setting..

And no problem! I'm just glad you're not annoyed by the flood of issues and pulls.

[nullstalgia]

One other thing that should be added, is a

display->hibernate();

This turns the display off and sets the controller to a deep sleep. This requires the RST pin to be connected, but it should be anyway. It didn't seem to like me sending it new data, but after a ESP reset (which is what happens after a deep sleep anyway), it worked fine again.

[sidoh]

I have a draft of this in the deep_sleep branch. Took the approach outlined above. Should be a new tab in settings for deep sleep configs:

The override pin is set to INPUT_PULLUP or INPUT_PULLDOWN as discussed, based on the override value:

void initSleepSettings() {
  if (settings.power.sleep_mode == SleepMode::DEEP_SLEEP) {
    uint8_t overrideValue = settings.power.sleep_override_value;
    // Use the internal pull-up/pull-down resistors
    uint8_t overridePinMode =
        overrideValue == HIGH ? INPUT_PULLDOWN : INPUT_PULLUP;
    pinMode(settings.power.sleep_override_pin, overridePinMode);
  }
}

@nullstalgia, if you have a second, could you give this a try?

[sidoh]

Meant to mention -- I also added a setting for MQTT client status. Filling in this topic will cause both birth and LWT status messages to get published there.

[nullstalgia]

So I've been messing around with it, and it seems good so far! I just have a couple questions/complaints.

1. Is there a reason you're having the MQTT Birth & LWT messages retained? Especially since (for some reason), NODE says they aren't retained (but I'm pretty sure they are). It could possibly cause a confusion if for some reason, connected is retained but disconnected isn't.

(I do like the idea of adding the LWT, though. Good forward thinking!)

(Unless retain: bool is whether or not that was a retained message? Indeed, that is how it works. )

retain boolean
true indicates the message was retained and may be old

2. Examples don't show up for me in either Firefox or Chromium. Is there a point for them, if that's intended? :P
   
3. Don't forget the display->hibernate() ^.^
4. Having it properly disconnect from MQTT when going to sleep might be a little nicer instead of forcing the broker to grab the LWT, but that's my personal opinion.

[nullstalgia]

Oh, also @sidoh

You should have it do a full refresh before going to sleep/hibernating the display!

[sidoh]

thanks for looking taking a look so quickly :)

Is there a reason you're having the MQTT Birth & LWT messages retained? Especially since (for some reason), NODE says they aren't retained (but I'm pretty sure they are). It could possibly cause a confusion if for some reason, connected is retained but disconnected isn't.

Both the birth and LWT messages should be retained. If done this way, any subscribers to the status topic will always get the correct and current status of the client. Obviously it's bad if one is retained and the other isn't, but if neither are retained, a newly-connected subscriber won't know what the status is until it's updated next.

Examples don't show up for me in either Firefox or Chromium. Is there a point for them, if that's intended? :P

The only utility it has right now is roughly the same a code comment has :)

Definitely possible to get react-json-schema-form to do something with the field, but it doesn't by default.

Don't forget the display->hibernate() ^.^

It should be doing this already (relevant code link). Is it not for you?

The internal display driver wrapper also powers off the display immediately after every update.

You should have it do a full refresh before going to sleep/hibernating the display!

Hmm... this should happen as long as the system is awake long enough for a full refresh to take place. Is that not what you're seeing?

[sidoh]

I see the display drawing about 5µA during deep sleep mode, which seems to indicate that hibernate is working.

[nullstalgia]

Fair enough. I think it's just some weird annoyance I have with retained messages. If you feel it's for the best, then I won't interfere. :)

Oh, my bad with the hibernate. I only did a quick sweep, not a proper Search of the code.

And with the full refresh, if I send data to it right when it announces its birth, there is no ghosting (as I presume its sneaking right in before the variables are set for the renderer). But if I send data, say, 5 seconds after that, the ghosting is there.

I have tried to capture this in a video. Let me know if you need more testing done. :)

https://streamable.com/mas7b

[nullstalgia]

Here is the flow being used. Pretty basic :P

Settings.json

{
  "display.display_type": "GDEW042T2",
  "display.full_refresh_period": "3600000",
  "display.template_name": "/t/fdg.json",
  "display.windowed_updates": "false",
  "hardware.busy_pin": "33",
  "hardware.dc_pin": "25",
  "hardware.rst_pin": "26",
  "mqtt.client_status_topic": "epaper_templates/corona/",
  "mqtt.password": "lolnope",
  "mqtt.server": "maybe",
  "mqtt.username": "tony",
  "mqtt.variables_topic_pattern": "epaper_templates/corona/variables/:variable_name",
  "network.hostname": "epaper-display",
  "network.mdns_name": "epaper-display",
  "network.setup_ap_password": "waveshare",
  "network.wifi_password": "",
  "network.wifi_ssid": "",
  "power.awake_duration": "30",
  "power.sleep_duration": "60",
  "power.sleep_mode": "DEEP_SLEEP",
  "power.sleep_override_pin": "0",
  "power.sleep_override_value": "0",
  "system.timezone": "PT",
  "web.admin_password": "",
  "web.admin_username": "",
  "web.port": "80"
}

[sidoh]

Retained messages actually work super well in my experience. I get that it probably feels like an additional source of truth, and can lead to some funky situations if the producer dies, but it greatly simplifies the information flow. (just my $0.02, not really a recommendation)

Oh, gotcha. This is happening because it's doing a partial refresh when new data arrives. If you always want a full refresh (which would get rid of the ghosting, but take longer), you could set display.full_refresh_period to 0.

Let me know if the full refresh period suggestion has the desired effect. Otherwise think this is probably ready to go.

Nice Node-RED flow, btw!

[sidoh]

(i suppose i should document the feature before merging as well, lol.)

[nullstalgia]

(Would be wise. :P)

Also I was about to ask if you could enable ArduinoOTA or something similar... but you already have the bin upload on the home page.

Also, can I commend you on your editor? For a small project like this, I would not expect something so well put together and following how people actually want to use it (like being able to select multiple types of entities and change all of the options they have in common at the same time)

👏

---

Anyway, the 0ms does work just fine for my purpose. No issues there. :)

But I do have one request.

Could you possibly add a "Suspend Deep Sleep" button to the homepage, ala the Reboot button?

You might want to automatically suspend it when a firmware update is pushed/being uploaded, as well.

[sidoh]

Added docs here -- c456dcd

Glad you like the editor. :)

Makes sense -- both great suggestions. Will take a look.

[nullstalgia]

Fantastic, thank you!

I guess the only thing holding me back from deploying this right now is the lack of a proper SPI Bus selector.

I started a little bit yesterday, but I got chumped when it came to the Web interface once again 😅

The main bit that matters is this:

...
void initDisplay() {
  #if defined(ESP32)
    if(settings.hardware.spi_bus != HSPI){
        if (settings.hardware.spi_bus == VSPI){
            SPI.end();
            SPI.begin(18, 23, 19, 5);
        }
    }
  #endif
  if (display) {
    delete display;
    display = NULL;
  }
...

And in case you're not using an editor that shows the definitions of HSPI/VSPI easily, here are screenshots from VSCode

I'll post these to the related issue as well (#5), but I just wanted to bring the simplicity to your attention while I had you. :)

---

Thanks for working with me through all of the suggestions!

[sidoh]

Happy to take a look at adding SPI bus selection.

If you've already got the settings/firmware changes committed somewhere, I'd be happy to add the relevant web stuff.

[nullstalgia]

#21 has been created with the small changes I have done. I'm sure I'm missing the way the config is loaded into the variable somewhere, but I'm not familiar with this section of the code.

[sidoh]

Implemented the suggestions. "System" card on dashboard shows number of seconds left in current awake cycle and has a button to cancel it:

Kicking off a firmware update should automatically cancel deep sleep, although that might not currently be reflected in the UI.

Let me know if it's working for you.

[nullstalgia]

Both seem functional to me!

I think the only thing I would change (and this is something I would do in a personal build, not something I feel would suit a public release) is to not initialize the display until it is about to go to sleep to make sure it only updates the one time.

And maybe add a listener to the status for a sleep message to tell it to go to sleep early, but that borders on on the edge of feature bloat and also risks stuff not updating properly.

---

All in all, fantastic work! The Sleeping... message is a little overactive (as you've mentioned), but that's peanuts compared to the great work you've done on this.

---

That's pretty much most of the big changes I personally wanted to see. There's a couple other things that I might come back to do (

• Figuring out and explaining custom fonts
• Doing more tests with the bitmap editor since I think I came across a glitch but can't remember since that was when I first started contributing
• Figure out a proper way to have a starting binary for the users.
  • In the meantime, you could go full-wacko and release a full binary image with esptool.py read_flash ... with the full 4mb.

)

Thanks for your help and patience!

[sidoh]

re: double-updates on wakeup, this makes sense. A decent general solution might be to implement update debouncing so that a screen update isn't triggered until a configurable threshold duration since the last update is crossed. Probably not something I'll try to tackle right away, though.

All great suggestions :)

Thanks for the detailed and speedy feedback on this one! Much appreciated.

[nullstalgia]

Added and current implementation is satisfactory.