Large negative spikes on Dallas sensor reading

[glitter-ball]

Bug description
The boiler return temperature values on HA, from a Dallas sensor, are showing occasional large negative spikes at a value of -26240 or similar value

Steps to reproduce
Connect Dallas sensor.
I think the problem may only have started since I changed the sensor MQTT publish interval set to '0' (publish on change). Sensor value is changing, typically, about every 5 seconds. Happy, at some point, to change back to 10 seconds and see if it stops.
NTP server set.
Spikes seem to occur at random intervals. Values seen include : -26304, -26240, -26208, -26272.
Will try and leave MQTT Explorer open long enough to catch it there - proving this is an EMS-ESP issue and not HA.

Expected behavior
No spikes. They weren't there in 1.9.5.

Screenshots

Device information

ems-esp:/$ show
EMS-ESP version 2.0.1

Boiler: Worcester Logamax Plus/GB192/Condens GC9000 (DeviceID:0x08 ProductID:208, Version:01.01)
  Hot tap water: off
  Central heating: off
  Warm Water activated: on
  Warm Water charging type: 3-way valve
  Warm Water circulation pump available: off
  Warm Water circulation pump freq: 0x3min
  Warm Water circulation active: off
  Warm Water comfort setting: Hot
  Warm Water disinfection temperature: 70°C
  Warm Water selected temperature: 60°C
  Warm Water set temperature: 60°C
  Warm Water current temperature (intern): 60.0°C
  Warm Water current temperature (extern): 60.0°C
  Warm water storage temperature (extern): 60.0°C
  Warm Water current tap water flow: 0.0l/min
  Warm Water # starts: 3463
  Warm Water active time: 25 days 5 hours 29 minutes
  Warm Water charging: on
  Warm Water disinfecting: off
  Selected flow temperature: 5°C
  Current flow temperature: 55.2°C
  Gas: off
  Boiler pump: off
  Fan: off
  Ignition: off
  Burner selected max power: 0%
  Burner current power: 0%
  Flame current: 0.0uA
  System service code:  (203)
  Heating temperature setting on the boiler: 75°C
  Boiler circuit pump modulation max power: 100%
  Boiler circuit pump modulation min power: 60%
  Boiler circuit pump delay time: 3min
  Boiler temp hysteresis on: -6°C
  Boiler temp hysteresis off: 4°C
  Boiler burner min period: 10min
  Boiler burner min power: 0%
  Boiler burner max power: 100%
  Outside temperature: 23.4°C
  Pump modulation: 0%
  Burner # starts: 12549
  Total burner operating time: 105 days 12 hours 29 minutes
  Total heat operating time: 80 days 7 hours 0 minutes
  Total UBA working time: 811 days 2 hours 51 minutes

Thermostat: RC300/RC310/Moduline 3000/CW400/Sense II (DeviceID:0x10, ProductID:158, Version:33.03)
  Clock: 17:47:50 13/09/2020
  Heating Circuit 1:
    Current room temperature: 21.9°C
    Setpoint room temperature: 18.5°C
    Mode: auto
    Mode Type: comfort
    Program is set to Summer mode
    Day temperature: 21.0°C
    Night temperature: 15.0°C
    Target flow temperature: 0°C

Dallas temperature sensors:
  ID: 28-BE2A-7791-0402-65, Temperature: 40.9°C

ems-esp:/
```$

[glitter-ball]

A bit more info.

HA definition:

platform: mqtt
  state_topic: 'ems-esp/sensor_28-BE2A-7791-0402-65'
  name: 'Return temp'
  unit_of_measurement: '°C'
  value_template: '{{ value_json.temp | round(1) }}'

MQTT plot over about 30 minutes, showing random spacing of updates

[MichaelDvP]

proving this is an EMS-ESP issue and not HA.

maybe a combination of both. If there is a sensor reading error (crc-check) the sensorvalue is set to undefined (NaN).
I think it's best to skip sensorupdate on reading errors and values out of range (dallas are defined -55 to 125 °C).

[MichaelDvP]

I've added a check in the dev build, @glitter-ball can you check for the spikes?

[glitter-ball]

Perversely, it didn't do it for over 24 hours when I upgraded to 2.0.1. But it's done it twice today at random times. I'll try and upgrade now and see what happens.

[glitter-ball]

Sorry, still a newbie navigating to find releases. How can I get a binary for the dev build with the sensor check in it, please? Thanks.

[proddy]

no problem, that's my fault. I'm always tweaking things. The dev builds have been moved to https://github.com/proddy/EMS-ESP/tree/firmware

[glitter-ball]

Thanks. Can I check I am upgrading in a sensible way? Drag and drop new .bin file into the upload box in the GUI? Seems to copy quickly and then nothing else? Does that just put the file in place pending a restart?

[proddy]

Drag & Drop should upload, takes about 15 seconds on fast line, and then automatically restarts the ESP. If the .bin file is corrupt it will report back an error. Make sure you're using the ESP8266 binary and not the gzipped version. If it still does not work I broke something (which is highly likely) and will need to test again.

[glitter-ball]

OK - will try and update this weekend. Have just replaced SD card with SSD which has been enough excitement for [more than] one day...

[glitter-ball]

I did the upload OK but still not sure about the process. I dragged the file in and waited a minute or so. Clicked another link and an orange box "Upload cancelled by user" appeared. Waited a bit longer and then it restarted OK on 2.0.2b3.

I think it's the no immediate action thing that's a bit disconcerting. If it popped up a dialog which said something like, "Upload completed. Copying firmware and rebooting - please wait", I wouldn't have kittens so quickly!

Anyway - done and I'll see if the negative spikes are squashed now. Thanks!

[proddy]

@glitter-ball I did a few tests. When you drag&drop the firmware it will start the upload (to 100%) and then show the green box with the message 'activating new firmware...'. Then it will de-compile the firmware and reboot which can easily take up to a minute. Would it help if I add some more text in that web page explaining the process and to wait?

[glitter-ball]

I don't think I've ever seen that stuff. The new firmware uploads in a second or two and then nothing appears to happen *8(.

I tried again tonight and left the system for 90 seconds - nothing visible happens at all. When I refresh the screen or click on a sidebar link, I get an orange "Upload cancelled by user" popup. Usually, after a bit of poking I can get back to the login page. When I log in the version's updated at the top of the sidebar - that's all the evidence I have that the firmware upgrade has happened.

Tried in Firefox and Edge and the result is the same.

BTW the sensor issue is fixed, so maybe the firmware upgrade thing should be a new thread?

[proddy]

yes, if you're still seeing issues with the web upload please create a new issue so other users can chime in.

[glitter-ball]

Seeing occasional negative spikes on my Dallas sensor again...

I'm on 2.1.0. Worth updating and seeing if it still happens before getting too excited?

[proddy]

@MichaelDvP can't remember, I think you made many improvements to the dallas code in 2.1.1?

[MichaelDvP]

It was before 2.1.0: check the value in range -55 to +125 °C to prevent readings of -26300, etc. Also sensors with wrong value or crc are not updated for 5 retries and than not published (VALUE_NOT_SET).
But here the readings are in range and the crc is correct, the dallas sending a wrong but valid value (EMC?).

[glitter-ball]

That's the difference this time - the values are within the valid range but not genuine! I've checked the HA Logbook to see if anything else happens coincident with a spike but nothing obvious. Also looked in HA History and, again, nothing odd.

Currently I have sensors set to MQTT publish on change so it updates at random intervals depending how long it takes for a 0.1ºC change - typically 4-5 seconds, more often if doing hot water.

Maybe I can validate and filter out duff values in HA? Return temp, in this case, should never be less than zero and not more than 85ºC.

[glitter-ball]

OK. Just had my Dallas cable fall apart in my hands. The +3.3V connection has fractured where it leaves the plug *8(. New cable on order, so best to hang fire on this issue until I replace it. Maybe it's been on the way out for a while and the failing supply has caused the Dallas errors?

[glitter-ball]

Have replaced connector and the negative spikes are still there. No obvious link to anything else when they occur. Wondering what to try next. I have two other unused sensors. Maybe I hook one of them up and see what happens.

[glitter-ball]

OK - a smattering of good news amidst all the other stuff that's going on. Since I upgraded from 2.1.0 to 2.2.0 a couple of days back, I've not had any spurious negative spikes from the Dallas sensor.

Goodness knows why this should be, though...

Happy for this to be closed unless I should try and do something (?) to work out why it was happening before.

[proddy]

wow, happy ending to 2020. Let's all have a drink at midnight!

[glitter-ball]

Yay! If I can stay awake that long. Family Zoom calls to survive first 😖...

Here’s to another cracking year for EMS-ESP!

[glitter-ball]

Spoke too soon...

Looked in detail at history at these times and there's nothing else going on which ties in. I'll have to try leaving MQTT Explorer running to catch one of these. That would confirm it's originating from EMS-ESP and not some weird hiccup in HA.