Long raw custom entities is crashing gateway

[tp1de]

Defining long raw custom entities on dev.30 version with gateway E32 V2 does not work and will make gateway to reboot.
Any telegram length > 53 does not work and will create regular reboots /crashes:

[MichaelDvP]

Thanks for opening the new issue, i'll test and add some checks.

[MichaelDvP]

I've added some range checks to my build https://github.com/MichaelDvP/EMS-ESP32/releases
Please try and if it still crashes try to test if it crashes on API (don't query the value), mqtt (disable), writing the value or reading from bus.

[tp1de]

Still crashing. I disabled mqtt and no api reads on custom.
Whenever I set bytes for raw custom entity > 53 then I get a restart.

General behavior has changed: (without mqtt / api)

• whenever I change custom entity (e.g. bytes from 53 to 50) the entity is deleted from dashboard - was not the case before
• setting bytes from 53 to 54 there is not a reboot/crash but all devices start new initialization after one minute for 1st polling cycle
• after 2 minutes - 2nd polling cycle - the gateway reboots
• nothing within log (all) the logserver just stops and restarts
• and with bytes > 53 WiFi connection is lost. Re connection after 1-2 minutes

[MichaelDvP]

whenever I change custom entity (e.g. bytes from 53 to 50) the entity is deleted from dashboard - was not the case before

yes was a bug to show old data after datasource was changed.
I think i've found the bug. please download new build and test.

[tp1de]

First look is fine. Reading and writing seem to work now. I will continue testing.
(haven't you said to implement read only? Writing seems to work as well ... very nice)

Let me ask some questions in respect to polling / updating:

• I understood that custom entities are polled with read requests every minute - is this right?
• you told some times ago that custom entities should be limited to max 10 entries - is this still valid?

In respect to long raw telegrams - e.g. switchTimes - I think that polling time should be higher and maybe there is no polling after 1st one needed anymore. Any changes by thermostat or by app via standard gateway will create telegrams with same type.
Same for changes by api / mqtt.

• Couldn't you just update custom entity while reading those telegrams and avoid regular read requests?

[MichaelDvP]

Yes writing is implemented, was easier than blocking.
The raw telegrams checks if it is already catched and read the catched telegram, no extra bus request.
For custom entities with type a read for these bytes is generated to have a 32bit value in one telegram and not split in different telegram parts. There is no technical limit to number of custom entities. But each one eates some ram and sends request to the bus.
The idea behind the custom telegrams is: Users can test their findings and report to us, Then we implement a entity and the custom can be removed. So normally users should only have a few custom entities.

For the implemented switchtimes of RC35 the telegram is set to fetched and after the complete receive the fetch flag is cleared. That works for implemented entities that are known to be updated rarly, but we don't know what kind of telegram is in the custom entity.

For the raw custom entities:

• the result is cleared on every change of custom entities and waits for new receive (the buffer is reallocated with new length).
• the result is published when the complete requested length is received on the bus.
• if you request a length that is longer than telegram data, there will no result publish.

For switchtimes there are a lot of requests, we should find a way to implement, but in an extra topic.

[proddy]

I'm not in favour of abusing the custom entities to solve the switchtime challenge (I opened another issue for this). Also I don't want EMS-ESP to be a middleware just to be used as a read/write telegram parser either. It's not what the application was designed for.

[tp1de]

Also I don't want EMS-ESP to be a middleware just to be used as a read/write telegram parser either. It's not what the application was designed for.

EMS+ switchTimes have 2 challenges:

1. is long raw telegrams read & write
2. is deconstructing content - 6 times per day ? 42 entities and 42 values for each switchTime (I have 10)

In a lot of cases EMS-ESP is today the middleware to home automation systems like HA, ioBroker or others.
Shouldn't the deconstruction from telegram to content and vice versa be done there? (or on a separate app)

1st step could be to create switchTimes within ems-esp entities either in raw (less memory) or as JSON.
This might be better then using custom entities.

Where should we discuss this? #1918 was not the right place as well.

... and similar for holidayModes (hm1 to hm5)

[MichaelDvP]

I think #1594. (and not #1593, #1601, #1930) #1918 is only DHW.

Close this?

[tp1de]

Let's continue then in #1594 - I just added the telegram types