Devices still drop from network

[bochelork]

After 5 day without losing any device, last night five om my devices lost connection again.
I became very hopeful to have a stable network at last, but I'm back to earth now and desperate again.

The following network adresses are not working.

14035 -> 0x36D3   (0x00124b001c5bc596, GLEDOPTO GL-S-007Z)
56022 -> 0xDAD6   (0x00124b001c5bc69a, GLEDOPTO GL-S-007Z)
37931 -> 0x942B   (0x00124b001d434621, GLEDOPTO GL-S-007Z)
58663 -> 0xE527   (0x00124b001b50c657, GLEDOPTO GL-FL-004TZ)

If I remove them from the mains, and power on after 5 seconds, I can see device announcements in wireshark.
But the devices do not find a new network route.

I add some wireshark and debug=* data:

• devices get a power on/off command.
• mains off/5 seconds/mains on
• devices get a power on/off command.

My questions:

• Why do the devices get lost.
• Why don't they return to the network after mains off/on (2 other routers DID return)
• Which steps to follow to get the devices back to the network

(before I tried many different ways. Removing by mqtt command, removing from db, pairing in previous versions, and so on.)

Attached:
notagain.txt -> DEBUG=* npm start
start-on-off-mains-on-off.pcapng -> wireshark data
database.db
start-on-off-mains-on-off.zip
database.zip
notagain.zip

[Koenkk]

• You are running an outdated firmware, for your case I would recommend, https://github.com/Koenkk/Z-Stack-firmware/tree/master/coordinator/Z-Stack_Home_1.2/bin/source_routing (this firmware keeps track of the routes).
• Are only the gledopto devices lost?
• For gledopto I've added some fixes in the latest dev branch (please update).

[bochelork]

• You are right about the firmware. I thought I was running latest source route, but sometimes it's a bit confusion because the hex files have more or less the same name. Just updated my sniffer stick to latest master with source routing.
• Updated to latest dev.
• Had to remove the coordinator_backup.json file to get zigbee2mqtt started without errors
  zigbee2mqtt:error 2019-9-24 18:48:55 Error: Cannot restore backup, backup is for 'undefined', current is 'zStack'
• I had to repair my GledOpto GL-S-007Z devices. But they only partially paired (not successfully, Device with modelID 'undefined' is not supported) I copied lines from a working backup DB, but I used ID and nwkAddress from unsuccessful pairing. After restarting zigbee2mqtt the GledOpto devices work as expected (except for keeping white on when changing from white to colour, different issue).

Let me test if my network is stable for at least a week. I'll keep you updated here. From today I only have my regular PuTTY log, no debug=* or sniffer data. If there is anything you want to check about the GledOptos let me know.

`

[bochelork]

Hi Koen,

I just did some test.

• Tuinlicht2 was lost from the network.

part from db:
{"id":57,"type":"Router","ieeeAddr":"0x7cb03eaa00ae07cb","nwkAddr":5978,....
So network address in database is 5978

• Switched off/on the mains.

In zigbee2mqtt I receive a message:
`zigbee2mqtt:warn 2019-10-03T17:58:53: Received message from unsupported device with Zigbee model 'undefined'

• I sniffed the network and powered off/on the mains

I received messages:

3142 410.197921 0xb07c Broadcast ZigBee ZDP 57 Device Announcement, Nwk

So convert 0xb07c to 45180 and I manually changed the db line to:
{"id":57,"type":"Router","ieeeAddr":"0x7cb03eaa00ae07cb","nwkAddr":45180,....

Restarted Zigbee2mqtt and..... voila! Tuinlicht2 is back!

So, I think this can be integrated in zigbee2mqtt, as it receives a message from an unknown network address, but from a known device.. At the moment it does not handle changes of network addresses well, it explains the devices that get lost all the time.

[bochelork]

Tuinlicht2 are Osram Mini Garden Poles.

I did another test with a lost Ikea GU10 spot.

Mains off/on DID update the network address in database.db. It was online again.
Ikea GU10_6: https://pastebin.com/VCFuBb8i

I tested Tuinlicht2 again, changed address in database to a random address and power off/on, now it wasn't changed after receiving Device 'TUINLICHT2' announced itself.

Osram Tuinlicht2: https://pastebin.com/FYb5T1e6

[Koenkk]

This is a very interesting finding! Just for my knowledge; is this with zigbee2mqtt 1.6 or dev?

[bochelork]

zigbee2mqtt:info 2019-10-03T21:35:28: Starting zigbee2mqtt version 1.6.0 (commi t #caf92c5)
zigbee2mqtt:info 2019-10-03T21:35:29: Coordinator firmware version: '{"type":"zStack12","meta":{"transportrev":2,"product":0,"majorrel":2,"minorrel":6,"maintrel":3,"revision":20190619}}'

I hope I've the right firmware now....

[Koenkk]

Good, this will probably be an easy fix, we need to add the same thing of https://github.com/Koenkk/zigbee-herdsman/blob/master/src/controller/controller.ts#L295 to https://github.com/Koenkk/zigbee-herdsman/blob/master/src/controller/controller.ts#L250

Will try to provide it tomorrow.

[bochelork]

Thanks a lot Koen!

[Koenkk]

Fix released in the dev branch, let me know if this fixes the issue.

[bochelork]

@Koenkk, the network is much more stable now. And, if a device is lost, which rarely happens since de update, it immediately returns after a power cycle. It saves me a lot of frustrations. Thanks a lot!

[Koenkk]

Great! can this be closed?

[bochelork]

Sure, if something notable happens I'll be back!

[Koenkk]

Ok, let me know!