Feature Request: Add multiple events detections #304
Comments
|
This integration reuses onvif from Home Assistant. Adding detectors for these would require to expand on it or handlers to be added into https://github.com/home-assistant/core/blob/dev/homeassistant/components/onvif/parsers.py . |
Please report them here. I tested on C200 with my Person Detection enabled and I have not received any events... |
|
I would like to help as I need my tapo cameras to trigger automations in HA when a person is detected. Until now the only way I could achieve that is to use Alexa between the cameras and HA. With this solution the cameras need to access internet which is not ideal. Using the video stream to detect people on another system is also not a solution because it uses the WiFi so much that it interferes with my Philips Hue remotes. Could you please tell me how to install https://github.com/JurajNyiri/HomeAssistant-Tapo-Control/tree/expand_onvif ? |
|
I downloaded the zip I am not sure what I should do? |
|
The code in the branch above doesnt have any new entities. However, it will catch any previously uncought events with an error message. If you are not seeing any other events then it looks like the camera is not issueing those events. I will do some more testing later with the subscription. |
This comment was marked as off-topic.
This comment was marked as off-topic.
|
@NEVdataDyne It looks like you are having the issue #303 (the last error in your comment). Is your motion sensor (classic one) working/triggered in Homeassistant? @JurajNyiri I'm not seeing any of the errors you mentioned. Could it be that the integration needs to subscribe explicitly for certain events? Something like "on event PERSON_DETECTED post to WEBHOOK". Also, because of #303 I'm having issues at the moment. I'm changing the file manually inside |
|
@AndreiArdelean1 No motion sensor doesn't seem to work. It doesn't seem to work even with my C110 camera so I am not really sure if I am doing somethng wrong. (I have C225, C320WS and one C110 camera) |
|
I've gotten the integration to work by modifying Also, the ONVIF integration declares I'll investigate this further and post updates if I find something else. |
|
I didn't manage to find a way of getting the special detection events, but the following will help with integrating the configuration of these events (enabling and sensitivity) into HA and pytapo. Here is the request the app makes on start: A few examples of setting detection configs:
|
|
Updated the code in https://github.com/JurajNyiri/HomeAssistant-Tapo-Control/tree/expand_onvif
You should now see:
|
|
I've already done something similar, but the only thing that changed was the |
|
I have added to main branch (not yet released to pypi):
I was not able to test: setBarkDetection @AndreiArdelean1 could you please test above? Edit: Released as pytapo 3.1.8. |
|
@AndreiArdelean1 now released in https://github.com/JurajNyiri/HomeAssistant-Tapo-Control/releases/tag/5.1.0.beta.1 Please test above, my devices do not support it. |
|
They all work except for |
|
I've been investigating this issue and here is what I've found so far: ONVIF:
UDP:
API:
What can be used, for event detection is the call: and returning: A few things to note here:
I've written this python script to get the events from the last 10 min to test it: Maybe this method of detection motion could be used to detect motion on #273.
Instead of making a call periodically, the ONVIF motion event could be used to trigger the call to Probably the best solution would be a combination of making a call periodically and watching for ONVIF motion events. Other possible methods not yet discovered/implemented on the camera:
|
|
Maybe in "Other possible methods not yet discovered/implemented" you can add "intercept the calls made by the camera to Alexa servers for notification" (for people detection). For some reason Alex seems to be the only service other than Tapo to get notified when a person is detected (and this is the only event Alexa can get). |
|
I've written this script (attached) to pull events from the camera. It works really well. I'd implement a proof of concept for |
|
Thank you @AndreiArdelean1 . I played with the script, it had a bug where it was calculating incorrect start time for cameras in different timezone to UTC-0. I adjusted a couple of things and released new function I am testing with following script: I tested on 2 C200s. One HW V1.0 and another one 3.0. The 3.0 also has tapo cloud subscription.
I tested also with your script only to verify I haven't made any mistakes when writing the new function. Could you try it and let me know if it works with your cameras? |
|
I get something like:
What I've noticed, if multiple sources (script and Tapo app) are requesting the events at once, one of them could return an empty list. |
|
Off-topic, but I'm not sure where I should post this: I'm not sure which work, which don't, or what they mean. I've extracted them from the Android app. |
|
@AndreiArdelean1 could you please test this script with the pytapo 3.1.13 to verify it still works? I think I managed to fix the timezone issue and it seems to be working fairly well now for me. |
|
It works! |
|
When I try to run the script above on the latest python docker image I get the following error: Do you have a tip? Maybe it has something todo with the IP not being a hostname? Is there a way to set the request to insecure? |
|
@AndreiArdelean1 Just to clarify, you were able to extract the AI detected events (person, pet, vehicle, baby crying, etc.)? I was playing around for a minute with the |
|
Hey, yes, it is working. It did work for motion, person, and pet, but it wasn't working for tamper detection (Or I didn't manage to get it in my testing as that type of detection was a bit hard to trigger). Sounds detection (meow, bark, baby, glass break) I couldn't trigger the actual event during testing so I can't say if it is working. But, if an event is shown with the correct icon in the playback list in the official app, then you should also be able to detect it with this function. You could:
I've only tested method 1. When investigating I had some issues with ONVIF so I couldn't test method 2. |
|
Thank you for letting me know! I'm mainly interested in the sounds detection, so might try to get this to work. (In case you're interested, I have had success in triggering the events resulting in notifications through the Tapo app by simply using youtube videos of baby crying, dog barking, etc.) @JurajNyiri Any plans to implement something like this in the Home Assistant integration? I will probably try to do something ad-hoc, but would be really cool to have this in the integration 😊 |
|
@basekson another way to get the xml dumps without having to use tshark/Wireshark is to use this script to connect to your camera: https://github.com/openvideolibs/python-onvif-zeep-async/blob/async/examples/events.py |
|
@DarwinData Im using jterrace latest version i PR to core: While troubleshooting ONVIF integration with my camera, I was asked to check the payloads being sent to my webhook for "Person Detected" events. Upon analyzing the logs, I discovered that the sequence of payloads being sent affects the behavior of the sensor. Findings First Payload (Empty Detection - 6.xml): This payload indicates no detection, as IsPeople has Value="false". Second Payload (Person Detected - 7.xml): This payload indicates detection, with IsPeople having Value="true". However, the issue arises because the first payload (6.xml) is sent immediately before the "Person Detected" payload (7.xml). As a result, the sensor stays in the "Detected" state because it transitions from empty detection to detection without being reset. Payloads 6.xml: Indicates no detection (Value="false"). 7.xml: Indicates detection (Value="true"). Conclusion The order of these payloads causes the sensor to remain in the "Detected" state. This is because the empty detection (Value="false") is immediately followed by the detection (Value="true") without any intermediate reset or transition. The event time in payloads are in correct order though, but this was the order in the post request load. |
|
I would still consider that in Tapo world, all of these detections count as events. Like, "oh, I've seen a person, whatever...", so it puts a person detected event on the timeline and records with that tag. (unless the programmers have flicked a boolean wrongly.) |
|
@basekson thank you, that is very helpful data. And it looks pretty broken to me. It almost looks like the camera thinks "oh, I forgot to send you the event that person detection stopped last time, so let me send it now, right before this new event". Just to confirm, you are using webhook here not pullpoint? As another data point, I wonder if you could turn webhook off and test to see if you see any different behavior with pullpoint? See #304 (comment) above. @GSzabados that's not how the onvif spec is defined and not how the other event types seems to work: they send a notification when an event is detected to "turn on" that type of detector and are supposed to send another when that detector has stopped. You can read more about it in section 9.4 of the onvif core spec: https://www.onvif.org/specs/core/ONVIF-Core-Specification.pdf - specifically, 9.4.2.1 has a good example:
|
|
Im kind of new to this kind of troubleshooting, how do I check pullpoint and how to disable webhooks? I can try if you guide me beacuse this was kind of fun :) I used tshark to get the logs from my earlier comment |
|
@basekson thanks! Instructions at #304 (comment) for enabling/disabling webhook |
@jterrace, you just caused me a serious deja vu feeling, as I was looking at some onvif specs. I am pretty sure that this discussion came up either here (in this repo) or another camera related topic. Wouldn't be the first time if Tapo has misinterpreted things. So, can be broken. Even seriously broken. Or doing weird stuff. And from the FAQ https://github.com/JurajNyiri/HomeAssistant-Tapo-Control#troubleshooting--faq
Otherwise, I don't know how the line crossing should work, if not an event, by looking at that example. |
|
@jterrace hmm, I did a complete reinstall again of everyting again and now all the requests are in the right order and binary_sensors is correct stage after a detections. |
|
@DarwinData Did that work for you? I tried the custom_component way but it doesn't give me the person detection. |
Nope, I tried various combinations of files copied from the dev branch but no luck. Disappointed but will patiently wait for next core release. |
|
Great news: tplink sent me a beta firmware to test on my C720:
I am receiving pet events now:
I will send a PR to ha core, hopefully tonight, to try to get it in before 2025.2 (not sure when the cutoff is). |
|
Awesome news! Really great to see Tp-link working on this with you. Thank you so very much for chasing them and getting them to actually fix stuff like this. I think I can speak for all of us when saying that we appreciate this a LOT :) Did tp-link mention anything about baby cry detection by any chance? |
|
Yes, on baby crying they said:
But I explained to them that playing a video of a baby crying wouldn't work for me either, since neither the C720 or C325WB have baby crying detection 😆 . I don't have a C210 or similar camera that actually has the crying detection feature. I told them we could probably find someone else from this thread to beta test it if they need. |
|
@jterrace, I have a C100 and C200, which both should support the baby cry. How is it possible to get enrolled into the beta firmwares? (Besides that app on Google Play...) |
|
@jterrace I have a C210 to test, if you could provide me the beta firmware. @GSzabados I think, you need to follow this process: #625 Please let me know, if there is an easier way for beta firmwares :) |
|
They don't have beta firmware yet for C100/C200/C210. The process is you give them the mac address of your camera and they add that camera to some allow list that makes your camera see a firmware update. If they say they have something ready, I'll update and ask if people want to test it. |
|
Does anyone have a C520WS that would be willing to use a beta firmware to see if baby crying events come in so we can get a sample? @GSzabados @CerebralBypass @DarwinData - send me an email at [email protected] if you can |
|
@jterrace, the C520WS does not support baby crying, as it is an outdoor camera.
But the C100/C200 does support that. Being both indoor cameras. Below from the C100.
|
|
Oh interesting, not sure why the TP-Link engineer said this then:
I will ask for some clarification. |
|
I would expect that the outdoor cameras would not have baby cry feature, but only the indoor cameras. |
Did you get a response yet? If not, I'm down to test that firmware for you. |
|
The response was that the indoor camera ONVIF is maintained by a different team and that he'd talk with the other team and get back to me. No news yet. |
|
Here's the update from TP-Link:
Sounds like we will get a firmware update for all cameras that fixes pet detection but that baby crying onvif is yet to be worked on. For folks who are looking for baby crying over onvif, I suggest starting a thread here: https://community.tp-link.com/en/smart-home/forum/777 and then linking it here for folks to upvote. And while you're at it, upvote my top feature request 😆 |
|
Sorry, i am a bit loss about all those events detections with tapo. I need to buy a surveillance camera for my living room to track if there is human inside (i can't use a sensor because of my cat 😅), is the person detection work in home assistant now ? Detection work without internet ? I am interest to buy a C222 that claim "person detection" |
|
I have two C210s and am VERY interested in getting baby crying alerts/notifications in HA. @jterrace thanks for the update from TP-Link - I understand this issue has not been resolved and is not ready to test yet, but if (or when) it is ready, I'm happy to help test with beta firmware or beta versions of this integration if needed. p.s., as you suggested, I opened a feature request thread on TP-Link's site here. |
|
Thank you so much for working on this! I have a D130 and would find person notifications in HA incredibly useful. What do you think the chances are they'd come to that device with near-term firmware upgrades? Like bluecrystaldex, I posted a feature request on the TP-Link forum. |
|
The D130 has ONVIF support. I'd expect it will work with person notifications once HA 2025.2 is released. |
|
Thank you! Knowing that, I'm pretty sure I misunderstood the assignment (I'm going to remove my TP-Link feature request). |
|
Tapo C310 - I just discovered this thread as I struggled with My C310 for days if not weeks to get Person Detection to work... As of today, I deleted the C310 from the integration, rebooted Camera and HA then readded the Cam to the Integration. One thing I did differently, I did not leave the Motion Detection and Person Detection sensitivity default.
Not sure if it was luck / persistence or something else happy it is working
Running on HA 2025.1.4 currently as of 5 days ago |
and others
Is your feature request related to a problem? Please describe.
I want to be able to detect multiple event types.
I'm not interested in actually highlighting/retrieving the rectangle where this was detected on the camera, just the detection event.
Describe the solution you'd like
I want to be able to detect and distinguish between:
These notifications are sent to the native app.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
These features are available on 3MP+ cameras for free, and on 1080p cameras using the paid subscription
Here is the response from
GetEventPropertiesof the ONVIF endpoint on Tapo C225 camera:Currently, motion detection doesn't work on Tapo C225 #303, but this feature would be nice to have once it is working.
The text was updated successfully, but these errors were encountered: