Issue connecting Xiaomi devices #43
Comments
|
I currently have a Philips Hue bulb that successfully connects. Comparing the logging for both the Xiaomi device and the Hue:
After this dataframe, both the Hue and the Xiaomi send/receive (not sure how to read the log, I think it's send) a ZDO request 0x0013:
I'm assuming the last number is something like a callback identifier.
Updating this post as I figure out more differences. |
|
The error that occurs during the join: Seems to be because the application receives a ZCL frame with a Structure type in it(?). This type has been uncommented in It seems that this happens on a packet that is sent after the Xiaomi assumes the controller knows about its endpoints, however. So I'm assuming this error is not actually related to the first issue, that of joining the network at all. But without serialize/unserialize support for a Structure field, the devices won't work even if I can get them to join. |
|
Any update on this one? Seeing the same issues in debug output as @lesteenman. |
|
It seems there are currently two issues with supporting Xiaomi devices in Bellows. The first is the unrecognized Structure type, which a colleague of mine has fixed already (but I don't think he's made a pull request; I'll ask him about that later). The second is that the endpoint discovery does not work for Xiaomi, as they don't explicitly send the endpoints initially. Instead, they just start sending data over the endpoints, and it seems one should just use them when they arrive. |
|
I did get a little further with this. I simply created the endpoint and output_clusters in the salute db. Now I am away to see the devices in HASS and I also receive updates. |
|
What devices have you tried it with? |
|
I only tested with the door sensor and the button. The SmartThings community had the same issue that they stop working after an hour of inactivity. Here is what I did to get it running for 1 hour: The sqlite DB looks as follows after the device has been joined. There is only an entry in the devices table since it does not respond to the endpoint discovery: Device table Now simply create the endpoint entry and entries for the output clusters: This should be enough to get the device visible as switch in Home Assistant. To also retrive updates I added the following function to the switch/zha.py component in the Switch class at the end of the file. I use the Bitron Zigbee Stick in Germany which has an EM 3587 chip. I did not have to flash any firmware since EZSP was working out of the box. |
|
same here however I managed to capture one of the devices drop form the network if its any use: 2017-11-21 16:16:54 DEBUG (MainThread) [bellows.zigbee.zcl] [0x95e5:1:0x0000] Attribute report received: 65281=b'\x04!\xa8\x13\n!\x00\x00' |
|
@lesteenman Did you find a pull-request for the unrecongnized structure? |
|
I would be happy to contribute to resolving this issue and get started as a contributor to bellows. While I am a programmer, I am new to bellows/home assistant however, so any pointers are appreciated. @twendt I tried your sql method and got something showing in the home assistant overview. However, I receive no updates. Could you please walk me trough the effect of the commands (and the "magic numbers" in the insert query)? I am currently working on getting the xiaomi door sensor to work, and I have hardware to work with. I have some IKEA trådfri as well so I hope to get that working as well to have something to compare with. RIght now, I am looking into why I am not receiving any updates. Without the sql "injections", I got the warnings previously mentioned in this thread, so I am looking for an entry point where the event actually gets registered.. There doesn't seem to be a lot of documentation on bellows yet though, so I am reading code and testing manually at the moment. |
|
I've been trying to get these newer Xiaomi door sensors to work for the past few days. It looks like all of these sensors were properly integrated into Smart Things, including the battery status. I wasn't able to get any of that code to work, however, because my door sensor doesn't actually implement all of the same clusters: When a new device joins or is loaded from bellows' database, Home Assistant's Here's a sample log of what happens: Oddly enough, the sensor successfully pairs with bellows and works for a little while: The last few lines show bellows successfully receiving updates when I tap the two pieces together. Home Assistant assumes that all binary sensors have the IAS Zone cluster ( Is this some profile issue with my ZigBee radio (Linear HUSBZB-1)? Or does this sensor just need some sort of special treatment? |
|
It need some special treatment. I invested some time the last week and got the door/windows sensor working.
I also modified code at many other places and added a lot of debug- logging statements during my try and error testing . Now as it's working, I want to start over from the newest HA code and do only the minimal changes to get it working and provide this patches to github or the maintainer of the code. We need for the ZHA code more configurable option for the configuration file. Like the option to set customized cluster profile to override the default settings for such non-standard devices. I just wanted to give a heads up and provide some hope:-) Many Thanks to the smartthings forum, the DH from a4refillpad and the discussions about the device were very helpful. |
|
I actually just pushed support for this (as well as TRÅDFRI lights to my own branch: jonatanolofsson/bellows . Have a look if you like. It does also contain some database updates though, so you may want to start a new database if you run my code. There's also a standalone command line option that you can use if you don't want to restart homeassistant all the time. @Yoda-x Instead of extending the HA zha module and further cementing bellows as a HA-only zigbee driver, I think a better option is to separate them and see bellows as its own project which HA uses for zigbee communication. The bellows abstractions should not leak into HA more than necessary. |
|
@jonatanolofsson Great stuff! Could you maybe push those upstream as seperate pull requests here? |
|
I'm intending to, I just finished it up this morning. However, I have been unable to get in contact with @rcloran so far, I hope he's not MIA.. |
|
Hello @jonatanolofsson The rest of the changes should be inside the ZHA HASS component and should be configurable inside the configuration yaml, or an include file. This is the goal for me. The ZHA HA component already supports configuration parameters, like the endpoint-id or device type. Thus, it should be doable to handover the clusters (or other paramters) based on the endpoint-id, that I already use define the door sensor as binary_sensor. At least for the xiaomi door sensor, bellows worked for me out of the box, for endpoint, cluster, manufacturer and model discovery, (after I did the paring the right way) The problem is more with the upstream to HA inside the ZHA component, making it a binary_sensor creating the entity and sending the event. It would appricate a new bellows version. seeing that many requests pending. BTW, good to see you were working on it. I was thinking this was dead and started my own way ;-) |
|
@Yoda-x excellent news! I implemented your changes and both of my sensors have stayed paired for the past two hours. @jonatanolofsson I'm not sure how much manufacturer-specific code you're suggesting bellows to handle, but here's a basic decoder for the heartbeat attribute ( For the latest door sensor, a heartbeat is sent when you click the button. The temperature sensor's heartbeat that is triggered by a button press doesn't contain anything useful, but I'll check my logs later and see if it might be periodically sending a large one like the door sensor. |
|
@puddly Awesome! I'm looking at getting automatic battery/heartbeat reporting working now for my device, but so far I can't get it to respond to the bind command at all. @Yoda-x did you get reporting working? Do you have an example code? From what I understand about your point 2 above, this only delays the problem, right? So instead of leaving after one hour, it leaves after a longer time. Point 3 is what really should fix the issue I suppose, if there's no way of disabling timeouts completely (?) |
|
@jonatanolofsson Here's the hacky code that I've been testing with. Attribute reporting has been working fine all day for me in Home Assistant with the increased timeout: I increased the timeout to about an hour and a half and both of my sensors have been working continuously for the past four hours. I think that as long as the device sends an attribute report before the timeout occurs, everything will work fine. Pressing the button or triggering the sensor during the timeout period doesn't seem to prevent it from being disconnected due to a timeout. |
|
FYI; I have posted a suggestion for Home Assistant to take owership of this library while @rcloran is M.I.A. @Yoda-x @jonatanolofsson @puddly and others, perhaps you care to comment regarding that there? |
|
Here is some sample code I used during my try and errors:, it's from the binary_sensor/zha and the async_setup_platform routine: It's very hacky.
For the xiaomi keep-alive it should be ok with the timeout increase, but other reports like battery every day and and an on/off status every 30min would be helpful. good to see we get on the track again for this sensors. |
|
For some reason I only get a timeout on the bind request.. Did you experience this as well at some point? I managed to get a response once, but mostly I get
It's like it's not listening to me.. :( |
|
@Yoda-x cie_addr is an attribute of the IasZone cluster, right? Did you add this to the OnOff cluster as well? |
|
@jonatanolofsson my temp/humidity/pressure sensor and the door sensor have been working for the past 5 days with just the binding to the OnOff input cluster and configuring reporting on attribute 0. When my Xiaomi gateway arrives I'll try and see what their actual binding process is. I have a feeling the sensors don't expect to be used outside of the gateway's network, so they just send attribute reports without you asking. |
|
I agree, I don't think reporting is needed as long as the global timeout is higher than the build in reporting from the xiaomi devices. |
|
Got the xiaomi HT sensor working, I modifed some of the zha and sensors code and added the option to configure reports from the configuration file |
|
@Yoda-x what do I have to add to HA to make humidity show up? (Ex temp and humidity = temp only, motion and lux = motion only, etc.) I have several multi function sensors and only the primary function gets a sensor in HA. I’d also like to get battery reports and sensors working as well. |
|
The Sensors/zha.py only has support for the temperature cluster. I extented it with the humitidy cluster |
|
It seems like the only thing left to do is make the My sensors have been working perfectly in home assistant for over a week straight with battery reporting and everything else. Interestingly, the undocumented built-in thermometer in the window and door sensor is accurate to within 1˚C, so it would be nice to expose that as a low-resolution temperature sensor usable from within Home Assistant. Is there any reason to keep this issue open now that we know that the only required change is just increasing the timeout? Or are there concrete plans to integrate manufacturer-specific extensions into bellows instead of the |
|
@puddly do you have a fork of HA that I can look at? |
|
@dmulcahey it's not pretty and probably won't work for you without tweaking, but you can try. I never tested pairing with Will there be any future problems if the longer timeout configuration is moved into bellows? Or should this be kept in |
|
Imho, the configuration timeout should be set on the application level and not in bellows. which for me is the underlaying api. We can discuss about the duration, but I think 6min is far to low, even if it's the default. @puddly : did you tried to create a xiaomi cluster inside the bellows/zigbee/zcl/clusters/manufacturer_specific.py file? This should identify and handle the heartbeats inside the bellows code and create attribute_reports. Which means less vendor specific code. BTW, I think we can close this issue. It's not really a bellows problem. But we need to open a homeassistant issue to get all the zha changes addressed for xiaomi |
|
@puddly I tried to mimick your code, but the reponse I get is: Just to confirm - all you did was
and that should be it to get periodical reports? |
|
To reply to myself: It seems that even though the response 130 is returned, it actually start reporting anyway.. Yay =) |
|
@jonatanolofsson It does seem like the attribute reporting command is extraneous. Is binding to the OnOff cluster even necessary then? @Yoda-x I have looked at the manufacturer-specific clusters but the Xiaomi sensors send a manufacturer-specific attribute ZigBee already defines battery reporting and device temperature capabilities, so I think it would be easiest to create some system that would proxy these manufacturer-specific attributes and clusters and transform them into attribute reports over standard ZigBee clusters (even if the end device doesn't actually support them). Home Assistant would then contain no specialized code for Xiaomi devices and they'd show up as normal sensors with battery reporting capabilities. For example, the door/window sensor currently reports supporting the following: The heartbeat provides data about its internal temperature and battery voltage, so two virtual clusters could be created that transform the heartbeat information into reports on the appropriate clusters: Any thoughts? |
|
Hi @Yoda-x I'm trying to give your version a go. I'm a bit confused with configuration. What's the difference between the zha.yaml and configuration.yaml in your git repo? Do I need both? I'm trying with an Aqara temp sensor which initially recorded as this (when using the 0.60 HA "release" version of bellows): This sort of lines up with your example but not quite. I have this in configuration.yaml Is zha.yaml needed in addition to this? I didn't see all the in_clusters you have listed and there's a few different ones. |
|
@ChristianHackAMS |
|
FYI; @jonatanolofsson created a Bellows dicussion group on Discord @ https://discord.gg/D5dRgXQ |
|
Please re-open in the new zigpy project (https://github.com/zigpy/zigpy). There appears to be no way to move issues between projects. FWIW, I've spent a bit of time playing with the Xiaomi devices. AFAICT, they implement some bits of the spec in some way. IMHO, the right approach here is to add support for custom device handlers to zigpy and let HA install those. A library of separate handlers could be inside of outside of HA. |
|
I have just created zigpy/zigpy#14 to track this issue. |
|
@twendt sorry if off-topic but did you manage to get your Bitron Video ZigBee USB Funkstick to work? http://www.bitronvideo.eu/index.php/produkte/smart-home-produkte/zb-funkstick/ Looks to be cheapest ZigBee stick in Europe based on Silicon Labs Telegesis ETRX357USB with CP2104: https://www.amazon.de/Telekom-Smart-ZigBee-Funkstick-USB-Hub/dp/B071YVVQD1/ also sold as "Deutsche Telekom Smart Home Kit Zigbee V.2 with USB Hub" in Germany: https://www.smarthome.de/geraete/bitron-video-smarthome-funkstick-fuer-zigbee-v-2 Read that a few people haven't got it to work with bellows, some tried to flash other firmware and failed. |
@rcloran I have done just this: https://github.com/dmulcahey/zha-device-handlers let me know your thoughts... |
|
@rcloran @AndreasBomholtz @jonatanolofsson @puddly @therve @lesteenman @Yoda-x @twendt I have just started a new Zigpy feature request to specifically discuss the ZHA Device Handler that @dmulcahey have written (and the quirks handlers for it) and whether or not it or something like it could be integrated into Zigpy : Hopefully, this type of modular ZHA Device Handler approach or something similar is something that the Zigpy project would like to adopt for dealing with the custom / unique exception handling of some devices. |
Note: This issue is somewhat of a scratchpad of my findings. I was barely familiar with the structure of the zigbee protocol before I started this. I'm hoping someone else will come across this and see something I'm missing, but until then, I'll continue this debugging session here, if that's alright.
I'm attempting to couple the following devices of Xiaomi using Bellows, to use them with Home Assistant: Xiaomi Smart Door and Window Sensor, Human Body sensor and Wireless switch. However, after connecting them using
bellows permit, I getMessage on unknown endpoint 1errors.As connecting seems a bit finnicky, it took me a while to get a somewhat reliable log. In this particular case, another exception was thrown by Bellows right after it started to look for endpoints. I think this might have been the case because the button was clicked right after (it is almost impossible to click the linking button without actually triggering the button).
It seems that the Active Endpoint Request never receives a response, due to which it never registers the endpoints.
I'm not well known with the whole Zigbee protocol stack, although I have been trying to read up on it. However, I have no device that does work to compare logs with, so I'm really not sure what I'm looking for in this case. I'm hoping for some input with this issue.
When the button is clicked without the endpoints being found, the following is logged:
Full log of
bellows -d /dev/ttyUSB0 -v DEBUG permit -t 150 -D zigbee.db:The text was updated successfully, but these errors were encountered: