Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Yale YDF 40 - Payload with NULL infos for fingerprint action (it worked fine) #15665

Closed
gregolin opened this issue Dec 20, 2022 · 42 comments
Closed
Labels
problem Something isn't working stale Stale issues

Comments

@gregolin
Copy link

What happened?

After some update the door lock Yale YDF 40 is sending NULL when the unlocking is started by fingerprint.
The behavioral is different with passcode, which everything works fine.
Attached are two files:
Log 1 - Only Z2M log, where we can see line 1 for a passcode unlock action; line 90 for the unlock from internal side; line 94 a fingerprint action with null values (I've cleaned lines regarding other devices);
Log 2 - With some attempts related to fingerprint (still null values), including the herdsman log.
log_1.txt
log_herdsman.txt

What did you expect to happen?

When it is opened by fingerprint the payload must have the action_source value (4) and the action_user value (X). It worked for a long time, but it has been broken around 1-2 weeks.

How to reproduce it (minimal and precise)

No response

Zigbee2MQTT version

1.28.4-1

Adapter firmware version

20221213.1 - latest OR 0.14.76 (I'm not sure about the question)

Adapter

Sonoff Dongle

Debug log

log_herdsman.txt
log_1.txt

@gregolin gregolin added the problem Something isn't working label Dec 20, 2022
@Koenkk
Copy link
Owner

Koenkk commented Dec 20, 2022

Can you provide the debug log of this? (not the herdsman one)

See https://www.zigbee2mqtt.io/guide/usage/debug.html on how to enable debug logging.

@gregolin
Copy link
Author

I think you want the file log_1.txt, right?

Can you provide the debug log of this? (not the herdsman one)

See https://www.zigbee2mqtt.io/guide/usage/debug.html on how to enable debug logging.

@Koenkk
Copy link
Owner

Koenkk commented Dec 20, 2022

Yes, but then with log level set to debug.

@gregolin
Copy link
Author

gregolin commented Dec 21, 2022

Yes, but then with log level set to debug.

It's here.
Thanks.
(updated again because I sent the wrong file twice)

log_debug.txt

@Koenkk
Copy link
Owner

Koenkk commented Dec 21, 2022

Thanks, I assume that the following happened:

  • Locked not via fingerprint:
Zigbee2MQTT:debug 2022-12-20 21:14:42: Received Zigbee message from 'Sala_Fechadura', type 'commandOperationEventNotification', cluster 'closuresDoorLock', data '{"data":0,"opereventcode":1,"opereventsrc":1,"pin":{"data":[],"type":"Buffer"},"userid":0,"zigbeelocaltime":724886057}' from endpoint 1 with groupID 0
Zigbee2MQTT:info  2022-12-20 21:14:42: MQTT publish: topic 'zigbee2mqtt/Sala_Fechadura', payload '{"action":"lock","action_source":1,"action_source_name":"rf","action_user":0,"battery":255,"linkquality":78,"lock_state":"unlocked","state":"UNLOCK"}'
Zigbee2MQTT:info  2022-12-20 21:14:42: MQTT publish: topic 'zigbee2mqtt/Sala_Fechadura', payload '{"action":"","action_source_name":null,"action_user":null,"battery":255,"linkquality":78,"lock_state":"unlocked","state":"UNLOCK"}'
Zigbee2MQTT:info  2022-12-20 21:14:42: MQTT publish: topic 'zigbee2mqtt/Sala_Fechadura/action', payload 'lock'
Zigbee2MQTT:debug 2022-12-20 21:14:42: Received Zigbee message from 'Sala_Fechadura', type 'attributeReport', cluster 'closuresDoorLock', data '{"lockState":1}' from endpoint 1 with groupID 0
Zigbee2MQTT:info  2022-12-20 21:14:42: MQTT publish: topic 'zigbee2mqtt/Sala_Fechadura', payload '{"action":null,"action_source_name":null,"action_user":null,"battery":255,"linkquality":72,"lock_state":"locked","state":"LOCK"}'
Zigbee2MQTT:debug 2022-12-20 21:14:47: Received Zigbee message from 'Casal_Banheiro_Pia', type 'attributeReport', cluster 'genOnOff', data '{"onOff":0}' from endpoint 1 with groupID 0
Zigbee2MQTT:info  2022-12-20 21:14:47: MQTT publish: topic 'zigbee2mqtt/Casal_Banheiro_Pia', payload '{"linkquality":9,"state":"OFF"}'
Zigbee2MQTT:debug 2022-12-20 21:14:50: Received Zigbee message from 'Casal_TempHumi_Banheiro', type 'attributeReport', cluster 'msTemperatureMeasurement', data '{"measuredValue":2115}' from endpoint 1 with groupID 0
Zigbee2MQTT:info  2022-12-20 21:14:50: MQTT publish: topic 'zigbee2mqtt/Casal_TempHumi_Banheiro', payload '{"battery":100,"humidity":86.27,"linkquality":72,"temperature":21.15,"voltage":3100}'
  • Locked via fingerprint:
Zigbee2MQTT:debug 2022-12-20 21:14:51: Received Zigbee message from 'Sala_Fechadura', type 'attributeReport', cluster 'closuresDoorLock', data '{"lockState":2}' from endpoint 1 with groupID 0
Zigbee2MQTT:info  2022-12-20 21:14:51: MQTT publish: topic 'zigbee2mqtt/Sala_Fechadura', payload '{"action":null,"action_source_name":null,"action_user":null,"battery":255,"linkquality":81,"lock_state":"unlocked","state":"UNLOCK"}'

If this is correct, the issue is that the device doesn't send the commandOperationEventNotification on a fingerprint lock.

  • What was the last z2m version this worked on? (can you go back to that one and confirm it works then)
  • Can you try re-pairing the device?

@gregolin
Copy link
Author

On the block you consider not via fingerprint there is one with fingerprint, but the behavior is the same than the two lines you assume that is with fingerprint, so the understanding is correct.
I don't know the right version which it worked, but it worked by last Wed (I can't see the version because last Friday I reinstall my whole system, changing Debian from desktop to Debian server only, despite I return with a full HA back-up). But I'm sure it stopped before this migration because I have the last message on my cellphone.

Two questions please:
1 - Could you tell me how could I go back with older versions? Because I think I uninstall and install it again I'll loose the db, right?
2 - Is there any way to get the update history? Any log about updates/upgrades? It would be easier to understand the changes.

And about re-pairing, I did it on Monday and no success.

And the last info, there are other people with the same issue:
#14335

@Koenkk
Copy link
Owner

Koenkk commented Dec 21, 2022

1. you can make a backup of the data directory, this contains everything
2. the last changes regarding this device was in August

EDIT:
Could you check if the issue is fixed with the following external converter: https://gist.github.com/Koenkk/701ce7f60fe6b71879b3d445551179f9

  • save this as file next to configuration.yaml as ext_converter.js
  • add it to configuration.yaml:
external_converters:
  - ext_converter.js
  • start z2m, re-pair device, check if issue is fixed

@gregolin
Copy link
Author

Ok, I'll test it as soon as I arrive at home.
Just one more question... I have another Yale door lock which also needs an external converter (#8946).
After some Z2M change/upgrade, maybe it could result in a conflict?
Thanks.

@Koenkk
Copy link
Owner

Koenkk commented Dec 21, 2022

This shouldn't conflict, can you provide me the external converter you are currently using? Then I can add this device for out of the box support.

@gregolin
Copy link
Author

Hi Koenkk.
Converter added and repair done. It worked perfectly!!!
Regarding my other Yale (YMC 420D) and external converter, it is attached as well.
Do you think feasible on next release I remove these two converters?
Thanks a lot!

ymc420d.js.txt

@everson7
Copy link

everson7 commented Dec 21, 2022

I've been having this problem for a while and even with ext_converter.js it still says "null".

this is my log.
log.txt

@Koenkk
Copy link
Owner

Koenkk commented Dec 22, 2022

@gregolin

See https://www.zigbee2mqtt.io/guide/usage/debug.html on how to enable the herdsman debug logging. Note that this is only logged to STDOUT and not to log files.

@everson7 you probably need to re-pair the device

@gregolin
Copy link
Author

Koenkk, I'm working with this external converter since yesterday. I believe it fixed my issue.
The converter you sent me yesterday is the same you sent me today.
About YMC 420 D, I'm preparing the server and I'll send it to you soon.
Thanks.

@Koenkk
Copy link
Owner

Koenkk commented Dec 22, 2022

It is not the same, line 34 changed.

@gregolin
Copy link
Author

Ahahahaha, sorry... you sent me the same URL, I just have compared this :-)
Ok, soon I'm gonna send both results to you.
Tks.

@gregolin
Copy link
Author

@Koenkk , two important infos:
1 - I had another (3rd) door lock unused, just for tests, and it was paired as well. I've tested with these two external converters and the YDF 40 worked fine. I tried a test unpairing all 3 Yale locks and re-pairing again only the two that I use them. Both of them are working, with or without external converters... so I don't know if it is possible, but after I unpair the test one, the YDF 40 is working fine also without external converter (I repaired it before twice, so I don't think it is possible the issue was only related to re-pair the device). Apparently everything is working fine again without any external converter.
2 - Herdsman log attached, with some actions for YMC 420 D (opened by fingerprint and closed automatically).
Thanks a lot.

log_herdsman.txt

@relombardo
Copy link

@Koenkk, it is my first-time using converters and I already have another one. Should I save this converter with another name if I am already using a converter for other device in my Z2M configuration.yaml? I am wondering if I should rename your code to something like "_v2" and add like this:

external_converters:

  • ext_converter.js
  • ext_converter_v2.js

*Where the first one would be my existing converter

@everson7
Copy link

everson7 commented Dec 23, 2022

I did the re-pair and it continues with "null" status.

log.txt

@Koenkk
Copy link
Owner

Koenkk commented Dec 23, 2022

@gregolin

  1. Make sure to factory reset your device, it is strange indeed
  2. I don't see any message being published for your YMC 420 D (besides the cached state on startup). I searched for MQTT publish: topic 'zigbee2mqtt/Yale Testes and MQTT publish: topic 'zigbee2mqtt/Cozinha_Fechadura'. Regarding the YMC 420 D, let's continue in [New device support]: YALE YMC-420D #15721

@relombardo yes, you can give it a different file name (any you like, just make sure it ends with .js

@everson7 I see you are using an old version, 1.27.2, make sure to use 1.28.4 + external converter + re-pair

@relombardo
Copy link

relombardo commented Dec 26, 2022

@Koenkk ,
I changed line 34 and removed one 'extend' from your code, not sure if it is appropriate
Before:
extend: extend: lockExtend({battery: {dontDividePercentage: true}}, {max: 900}),
After
extend: lockExtend({battery: {dontDividePercentage: true}}, {max: 900}),

I got this result after adding converter with this change:
User 1 unlocking the door:
Zigbee2MQTT:debug 2022-12-26 20:25:33: Received Zigbee message from 'YDF40', type 'attributeReport', cluster 'closuresDoorLock', data '{"lockState":1}' from endpoint 1 with groupID null

User 2 unlocking the door:
Zigbee2MQTT:debug 2022-12-26 20:20:11: Received Zigbee message from 'YDF40', type 'attributeReport', cluster 'closuresDoorLock', data '{"lockState":2}' from endpoint 1 with groupID null

Is this the expected result? My automation for other unlock methods was based on payload messages and in this case, I did not find a way to translate this result as a flag on my Node-RED to identify this action.

@Koenkk
Copy link
Owner

Koenkk commented Dec 27, 2022

@relombardo this is good indeed (I've updated the converter with your changes). It seems no action is send (that's what this issue is about)

@relombardo
Copy link

relombardo commented Dec 27, 2022

@Koenkk, the model YMF40 sends payload.action_source = 4 for fingerprint unlock. I am wondering if we could replicate it on the YDF40 via converter, leveraging YMF40 logic. The weird thing is that their Z2M documentations look identical regarding action/source sections.

ydf40_ymf40_actions

@gregolin
Copy link
Author

Just to be clear here... my model is YDF40 and it is working fine with the present version after the tests I described here.

@relombardo
Copy link

Just to be clear here... my model is YDF40 and it is working fine with the present version after the tests I described here.

@gregolin , are you getting payload.action_source message for fingerprint unlock on your YDF40 without the converter?

@gregolin
Copy link
Author

Exactly. As I mentioned here (quoted below), I un-paired my two Yale locks (different models), re-paired them three times and everything is working fine so far. I've tested with and without the external converter and it doesn't change anything for me, I can see all messages again perfectly like before.
I suspect the cause was some conflict or anything else I don't know. I really suggest you to re-pair it once or twice before some other bigger change.
Thanks.

@Koenkk , two important infos: 1 - I had another (3rd) door lock unused, just for tests, and it was paired as well. I've tested with these two external converters and the YDF 40 worked fine. I tried a test unpairing all 3 Yale locks and re-pairing again only the two that I use them. Both of them are working, with or without external converters... so I don't know if it is possible, but after I unpair the test one, the YDF 40 is working fine also without external converter (I repaired it before twice, so I don't think it is possible the issue was only related to re-pair the device). Apparently everything is working fine again without any external converter. 2 - Herdsman log attached, with some actions for YMC 420 D (opened by fingerprint and closed automatically). Thanks a lot.

log_herdsman.txt

@github-actions
Copy link
Contributor

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days

@github-actions github-actions bot added the stale Stale issues label Jan 27, 2023
@relombardo
Copy link

@gregolin, thanks for the update. I removed the converte and then I tried removing and re-pairing my YDF40. Now it is being recognized as YMF40. I thought it could be promising, but I still cannot identify fingerprint unlock, as I am getting exactly same message as before.

@Koenkk, is the converter still necessary? Is there any reason for YDF40 be identified as YMF40 recently without the converter?

@github-actions github-actions bot removed the stale Stale issues label Jan 28, 2023
@Koenkk
Copy link
Owner

Koenkk commented Jan 29, 2023

@relombardo can you provide the data/database.db entry of your device?

@everson7
Copy link

here is exactly the same, my ydf40 being recognized as ymf40 and unable to identify fingerprint unlock.

@gregolin
Copy link
Author

gregolin commented Jan 29, 2023

I don't know if it helps for something, but mine also is recognized as YMF40, but it works perfectly.

image

@relombardo
Copy link

@relombardo can you provide the data/database.db entry of your device?

@Koenkk , here is the related line I see on data/database.db:

{"id":5,"type":"EndDevice","ieeeAddr":"0x000d6f0012bb3015","nwkAddr":2767,"manufId":4491,"manufName":"ASSA ABLOY iRevo","powerSource":"Battery","modelId":"iZBModule01","epList":[1],"endpoints":{"1":{"profId":260,"epId":1,"devId":10,"inClusterList":[0,1,3,4,5,9,10,32,257],"outClusterList":[10,25],"clusters":{"genBasic":{"attributes":{"modelId":"iZBModule01","manufacturerName":"ASSA ABLOY iRevo","powerSource":3,"zclVersion":1,"appVersion":11,"hwVersion":0}},"genPollCtrl":{"attributes":{"checkinInterval":360}},"genPowerCfg":{"attributes":{"batteryPercentageRemaining":90}},"closuresDoorLock":{"attributes":{"lockState":1}}},"binds":[{"cluster":32,"type":"endpoint","deviceIeeeAddress":"0x00212effff07e12c","endpointID":1},{"cluster":257,"type":"endpoint","deviceIeeeAddress":"0x00212effff07e12c","endpointID":1},{"cluster":1,"type":"endpoint","deviceIeeeAddress":"0x00212effff07e12c","endpointID":1}],"configuredReportings":[{"cluster":257,"attrId":0,"minRepIntval":0,"maxRepIntval":3600,"repChange":0},{"cluster":1,"attrId":33,"minRepIntval":3600,"maxRepIntval":62000,"repChange":0}],"meta":{}}},"appVersion":11,"hwVersion":0,"zclVersion":1,"interviewCompleted":true,"meta":{"configured":1439519238},"lastSeen":1675087190815,"defaultSendRequestWhen":"fastpoll"}

Koenkk added a commit to Koenkk/zigbee-herdsman-converters that referenced this issue Jan 30, 2023
@Koenkk
Copy link
Owner

Koenkk commented Jan 30, 2023

I think your device contains a generic module: iZBModule01, I've renamed the device to YMF40/YDM4109+/YDF40 (only a cosmetical change).

I don't know about the action, seems to be an issue with the module itself as re-pairing fixed it for @gregolin

@gregolin
Copy link
Author

gregolin commented Jan 30, 2023 via email

@relombardo
Copy link

relombardo commented Jan 30, 2023

Hi @gregolin , this is interesting, thanks for sharing. In my case it is identified as Unknown device, then it becomes the YMF40 after a while. Once I check that fingerprint payload message is not available, I force to remove the device on Z2M. Then I go to my locker (YMF40) and I use the Function 9 (create new connection) - remove connection by pressing 3, then I use the Function 9 again to create a new connection, by pressing option 1 while my Z2M is scanning to find new devices. At this point I restart the loop: The device is recognized as Unknown, then it becomes YMF40 once again.

Are you getting some similar behavior when you mention you need to retry?

@gregolin
Copy link
Author

Not exactly.
I need to try again because the Z2M takes a long time with the interview, and the time ends... so it is recognized but the interview isn't complete. I force the removal as well, go through the process you described (pressing 3, 9, 9 etc) and try to scan again. I repeat it until it is recognized and the interview is completed.
Usually it takes 3 or 4 attempts.
Thanks.

@relombardo
Copy link

I just tried again, I only noticed green pop-up messages coming in while interview was occurring and finally, I got a successfully completed message. I am assuming there were no errors. Then I tried fingerprint unlocking and got only this:

Zigbee2MQTT:debug 2023-01-30 17:37:15: Received Zigbee message from 'YMF40', type 'attributeReport', cluster 'closuresDoorLock', data '{"lockState":2}' from endpoint 1 with groupID null
Zigbee2MQTT:info 2023-01-30 17:37:15: MQTT publish: topic 'zigbee2mqtt/YMF40', payload '{"action":null,"action_source_name":null,"action_user":null,"battery":90,"linkquality":255,"lock_state":"unlocked","state":"UNLOCK"}'
Zigbee2MQTT:debug 2023-01-30 17:37:20: Received Zigbee message from 'YMF40', type 'commandCheckIn', cluster 'genPollCtrl', data '{}' from endpoint 1 with groupID null

@gregolin when you say it worked for you, are you getting a specific payload message? I believe it is action_source 4, but I never had the opportunity to succeed, so I am not sure.

@gregolin
Copy link
Author

I receive the correct message. See:

Zigbee2MQTT:info 2023-01-29 17:59:38: MQTT publish: topic 'zigbee2mqtt/Sala_Fechadura', payload '{"action":"unlock","action_source":4,"action_user":1,"battery":100,"linkquality":123,"lock_state":"locked","state":"LOCK"}'

The action_source 4 is fingerprint.

@relombardo
Copy link

relombardo commented Jan 31, 2023

Thanks @gregolin. I tried something different. I removed battery cells to force a shutdown of my locker, thank I turned it back on. It simply started working and I received payload message action_source":4. I tested several times with different users and it is consistently working now. Not sure if it was a coincidence, but I did not have to re-pair my device nor rebooting any system/application.

So, here are my thoughts on possible solution for users with similar issue:

  1. Ensure you are not using any converter for YDF40
  2. Force remove of YDF40 from Z2M
  3. Re-pair your YDF40 on Z2M. Ensure your device was completely interviewed and paired on Z2M without any error during the process.
  4. If you are still not receiving action_source 4 for fingerprint unlocking, remove batteries of your locker and re-insert them to power cycle your device.
  5. Re-test the fingerprint unlocking source

These were actions that worked so far, perhaps there are different insights with time of maybe a future update fix a potential root cause.

@everson7 , could you try step 4 once you are sure your device is fully interviewed with no errors. I am curious to check if it works for you and I really hope so.

  • @Koenkk for visibility
    Koenkk, thanks for the good support as usual.

@everson7
Copy link

everson7 commented Feb 8, 2023

Yes, it's working now!
Thank you very much!

@github-actions
Copy link
Contributor

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days

@github-actions github-actions bot added the stale Stale issues label Mar 11, 2023
@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Mar 19, 2023
@gregolin
Copy link
Author

gregolin commented Oct 3, 2024

Hi everyone.
Grabing this old topic again, I'm facing issue with this device and Node Red... does anyone use it with Node Red?
I can't do any action with it because the last Node Red version changed something and I just receive this message from Node Red log:

"InputError: Invalid action format: unlock"

And all actions following the node result in error. @Koenkk , is "action" a right label for this node? I'm thinking if ACTION could be some protected word, something similar that.
Any idea?
Thanks!

@gregolin
Copy link
Author

gregolin commented Oct 3, 2024

@Koenkk , it seems the word "action" is a reserved word nowadays, see:
(zachowj/node-red-contrib-home-assistant-websocket#1489)
Is there any way that I could change it by myself or it must be done by you, in the add-on level?
This is the object which is returned:

action: "unlock"
action_source: 2
action_source_name: "manual"
action_user: 0
auto_relock_time: null
battery: 100
battery_low: null
linkquality: 120
lock_state: "locked"
pin_code: null
sound_volume: null
state: "LOCK"

Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
problem Something isn't working stale Stale issues
Projects
None yet
Development

No branches or pull requests

4 participants