Support for 3.2 protocol

[pawel-szopinski]

Hi,

I have a couple of light switches that are based on protocol version 3.2. Like this one: https://www.aliexpress.com/item/32976986321.html

Using tinytuya I can turn them on/off (works when ver set to 3.3), but I cannot get status. Here's the debug log.

Is there any chance tinytuya could support 3.2?

DEBUG:status() entry (dev_type is default)
DEBUG:building command 10 payload=b'{"gwId":"bfce9bf5ec0e8ac2aboka5","devId":"bfce9bf5ec0e8ac2aboka5","uid":"bfce9bf5ec0e8ac2aboka5","t":"1660166885"}'
DEBUG:payload generated=b'000055aa000000000000000a000000885bc7f95f3c88bd83dbb763f880dd47ce9b931d026edcec1620dc32cb1877c4730e7beb3a39a3720a3bcba57876716162ea34f490f7a5ba8f2c946a4bd26ac7f57f40cd65b58b3fbd055fdf8bb6ecaa049b931d026edcec1620dc32cb1877c4737bee983e8504e94cc4961239cd76b0abbaf0dd14f091ba228fb16037aa6138b20686ae150000aa55'
DEBUG:received data=b'000055aa000000000000000a0000001d000000016461746120666f726d6174206572726f724ddf82ac0000aa55'
DEBUG:received message=TuyaMessage(seqno=0, cmd=10, retcode=(1,), payload=b'data format error', crc=1306493612)
DEBUG:raw unpacked message = TuyaMessage(seqno=0, cmd=10, retcode=(1,), payload=b'data format error', crc=1306493612)
DEBUG:decode payload=b'data format error'
DEBUG:decrypting=b'data format error'
DEBUG:incomplete payload=b'data format error' (len:17)
Traceback (most recent call last):
  File "/Users/paul/Library/Python/3.8/lib/python/site-packages/tinytuya/core.py", line 767, in _decode_payload
    payload = cipher.decrypt(payload, False)
  File "/Users/paul/Library/Python/3.8/lib/python/site-packages/tinytuya/core.py", line 246, in decrypt
    raw = cipher.decrypt(enc)
  File "/Users/paul/Library/Python/3.8/lib/python/site-packages/Crypto/Cipher/_mode_ecb.py", line 196, in decrypt
    raise ValueError("Data must be aligned to block boundary in ECB mode")
ValueError: Data must be aligned to block boundary in ECB mode
DEBUG:ERROR Unexpected Payload from Device - 904 - payload: null
DEBUG:status() received data={'Error': 'Unexpected Payload from Device', 'Err': '904', 'Payload': None}
set_status() result {'Error': 'Unexpected Payload from Device', 'Err': '904', 'Payload': None}

[jasonacox]

Hi @pawel-szopinski ! Thanks for opening this issue. This is the first 3.2 I have heard of! I'm not sure what the protocol difference would be since I haven't been able to find it in the Tuya code on their github repo. Thanks for the debug info. A few things:

• Can you show the scan output to see what it reports: python -m tinytuya scan
• Has anyone else heard/seen these 3.2 devices? Any links or tips appreciated if we are to determine the protocol. I have a good lead for 3.4 but nothing for 3.2

[pawel-szopinski]

Sure, here's one device (others produce the same output)

TinyTuya (Tuya device scanner) [1.6.5]

[Loaded devices.json - 6 devices]

Scanning on UDP ports 6666 and 6667 for devices (21 retries)...

Bedroom   Product ID = iapdgbaik9riugyn  [Valid payload]:
    Address = 192.168.0.116,  Device ID = xxx, Local Key = xxx,  Version = 3.2, MAC = 68:57:2d:08:8d:59
    Access rejected by 192.168.0.116: Unexpected Payload from Device

And the extract from snapshot.json

{
            "ip": "192.168.0.116",
            "gwId": "xxx",
            "active": 2,
            "ablilty": 0,
            "encrypt": true,
            "productKey": "iapdgbaik9riugyn",
            "version": "3.2",
            "err": "Unable to poll",
            "name": "Bedroom",
            "key": "xxx",
            "mac": "68:57:2d:08:8d:59",
            "id": "xxx",
            "ver": "3.2"
}

The only reference I found about 3.2 is in this github issue: codetheweb/tuyapi#234

[jasonacox]

Thanks for this!

From my initial research, it seems like this was spotted in 2019 but it doesn't seem like a handler for the 3.2 protocol has been discovered or added to the main tuya projects. That seems to indicate a very low population of 3.2 devices. Can you confirm that there is no firmware update available for the device in the SmartLife or Tuya smart app?

Also, based on the debug you provide, it seems to be more about decryption, but it could be that 3.2 is not using COMMAND 10. I wonder if it needs to use COMMAND 13 instead. You could try this:

import tinytuya

tinytuya.set_debug(True)  

d = tinytuya.OutletDevice(ID, IP, KEY, 'device22')
d.set_version(3.3)
d.set_dpsUsed({"1": None})  # This needs to be a datapoint available on the device
data =  d.status()
print(data)

[pawel-szopinski]

There is no firmware update in the SmartLife app. However, your code solution helped. I am now getting the correct status from all my switches. Thank you so much!

I am closing the issue, since it is resolved.

[jasonacox]

That's fantastic news! That means I can add logic to use device22 when 3.2 protocol is detected.

I'm going to re-open this issue to remind me to add that logic. Thanks for helping!

[jasonacox]

@pawel-szopinski I pushed updated code with 3.2 logic. Would you be able to pull it down and test it with your devices?

git clone https://github.com/jasonacox/tinytuya.git
cd tinytuya
python

import tinytuya

tinytuya.set_debug(True)  

d = tinytuya.OutletDevice(ID, IP, KEY)
d.set_version(3.2)
data =  d.status()
print(data)

And also test the scan feature (make sure you devices.json is in the same directory):

python -m tinytuya snapshot

[pawel-szopinski]

Sure, I can help with whatever you need.

The code snippet you provided works like a charm (correct JSON response), so this seems to be fixed now.

However, the scan and snapshot features still produce errors.

Scan:

TinyTuya (Tuya device scanner) [1.6.6]

[Loaded devices.json - 6 devices]

Scanning on UDP ports 6666 and 6667 for devices (21 retries)...

Bedroom   Product ID = iapdgbaik9riugyn  [Valid payload]:
    Address = 192.168.0.65,  Device ID = xxx, Local Key = xxx,  Version = 3.2, MAC = 68:57:2d:08:8d:59
    Access rejected by 192.168.0.65: Unexpected Payload from Device

Scan Complete!  Found 6 devices.

>> Saving device snapshot data to snapshot.json

Snapshot:

TinyTuya (Tuya device scanner) [1.6.6]

Loaded snapshot.json - 6 devices:

Name                      ID                       IP               Key               Version

Bedroom                   xxx                      192.168.0.65     xxx               3.2

Poll local devices? (Y/n):

Polling 6 local devices from last snapshot...
    [Bedroom] - 192.168.0.65 - No Response

Done.

EDIT: Also, I've noticed that when I want to get status of device's second switch (dps '2') I need to use the set_dpsUsed() function beforehand. Otherwise, I get KeyError: '2'. I think it would be more convenient to have the dps number as param in the status() function.

[jasonacox]

Thanks @pawel-szopinski ! This is pure gold! I found the issue in both scanning and wizard - those functions were set to force either 3.1 or 3.3 (no accommodation for 3.2).

                    if ver == "3.3":
                        d.set_version(3.3)
# changed to
                    d.set_version(float(ver))

If you don't mind, pull the latest from github (eg. git pull) and try the scan again.

I wonder if 3.2 actually needs the set_dpsUsed() accomodation (it is required for certain devices we classified as device22 since they all seem to have a 22 character ID). Does is work without it (need to force 3.3)?

import tinytuya

tinytuya.set_debug(True)  

d = tinytuya.OutletDevice(ID, IP, KEY, 'device22')
d.set_version(3.3)
# d.set_dpsUsed({"1": None})  # This needs to be a datapoint available on the device
data =  d.status()
print(data)

If that works, I will remove set_dpsUsed() from the default device 3.2 setup. Otherwise we might be able to use detect_available_dps() to set this value.

dps_avail = d.detect_available_dps()
d.set_dpsUsed(dps_avail)
data =  d.status()
print(data)

[pawel-szopinski]

It looks like the status is pulled correctly now in scan, snapshot, devices. It uses dps 1 by default. You could possibly change those functionalities to provide info on all dps'es by using detect_available_dps(), since it works well (your last code snippet).

The second code snippet produces None result. Here's the debug log:

DEBUG:TinyTuya [1.6.6]

DEBUG:status() entry (dev_type is device22)
DEBUG:building command 10 payload=b'{"devId":"xxx","uid":"xxx","t":"1660485136","dps":{}}'
DEBUG:payload generated=b'000055aa000000000000000d00000077332e330000000000000000000000005ef699fd78ea7221bfb758c119c9d24e6b00e60bd139b663bdcd8e2f29e396fc18c165d98468e8ffdca21a887b2fc6912f3db51aa93bf84b94f117718aa8cda346ba8a15ccab5e62b5cecf173e373620d7773a102bb4e2eb2fb52caaef9aed26ed66bf250000aa55'
DEBUG:received data=b'000055aa000000000000000d0000000c00000000edbdfa0f0000aa55'
DEBUG:received null payload (TuyaMessage(seqno=0, cmd=13, retcode=(0,), payload=b'', crc=3988650511)), fetch new one - retry 0 / 5
DEBUG:status() received data=None
None

Last but not least, I don't mind using set_dpsUsed() function to then be able to pull device's status, but there seems to be a small lack of consistency in the api. What I mean by that is, in order to get the status of a particular dps I need to use two functions, like so:

d = tinytuya.OutletDevice(ID, IP, KEY)
dps = '2'

d.set_dpsUsed({dps: None})
d.status()

Why not have status() take in a dps param, like in the case of set_status()

d = tinytuya.OutletDevice(ID, IP, KEY)
dps = '2'

d.set_status(True, dps)

Or, if that's not feasible, make set_status() also utilize (remember what was set in) set_dpsUsed(), so that the switch parameter can be optional (I know it is optional now, but if not provided, it defaults to 1, which is not always what one needs).
In the example below, dps number 2 would be turned on:

d = tinytuya.OutletDevice(ID, IP, KEY)
dps = '2'
d.set_dpsUsed({dps: None})
d.set_status(True)

If that's not feasible either, the easiest way not to make users confused about set_dpsUsed() is to expand the README file a bit, where the api description is provided, to include the info that set_dpsUsed() is required for some devices, and for all devices if one wants to get the ststus of anything else than dps '1', and maybe also that set_dpsUsed() does not have any effect on set_status().

[jasonacox]

Thanks @pawel-szopinski ! Your help on this has been awesome.

I agree with you on consistency. My intent it to make 3.2 devices behave the same as "non device22" 3.1 and 3.3 devices which don't need to use set_dpsUsed() at all. A call to status() gets you all DPS available.

I can have d.set_version(3.2) do the work for us. I am also going to see if the same could apply to device22 detection edge cases.

I just pushed an update. Can you pull the latest and try again with something like this to make sure you are seeing all DPS values now?:

import tinytuya

d = tinytuya.OutletDevice(ID, IP, KEY)
d.set_version(3.2)
data =  d.status()
print(data)

Also check the scan functions if you don't mind. Hopefully you see all DPS values show up there too.

Again, I really appreciate your help. I don't have 3.2 devices so your feedback and help to test this is gold.

[pawel-szopinski]

Yes! status() now provides info on all DPS values. All scan functionalities also pull correct data.

One small drawback of this approach, at least in my case, is that scripts requiring status (like, flipping a switch, where you need to pull status and then invert it) take about 10x longer - 450-500ms vs 45-50ms. I understand this is due to the way you have to handle device22 in detect_available_dps. I can work around it by reducing the ranges variable to suit my needs, and then only one roundtrip to the device is required, so that's not a big deal.

And finally, it is I who has to thank you for this awesome library, not the other way around! Thanks to it, I don't have to talk to tuya's cloud every time I want do something as simple as flipping a light switch in my living room :)

[jasonacox]

Haha! Thanks for the kind words, @pawel-szopinski. I love this community of home automation enthusiast. This project gets better because of the incredible contributors like you!

You have a great point about the latency for the detection. I see your Pull request #169 and will merge that in. Great idea!

Thanks for helping! I hope you don't mind if we ping you for 3.2 help in the future. 😉

[jasonacox]

@pawel-szopinski the update to support 3.2 and your PR are now live in v1.6.6.

Thanks again for your help!

[pawel-szopinski]

Thanks for merging my PR in and thanks for the kind words as well, @jasonacox!

I don't mind you pinging me at all in the future. Feel free to do so. I will try to help as best as I can :)

Since we're now live with v1.6.6 I am closing this issue now.