Tuya 2.0 Firmware

[tsightler]

Describe the bug
I have about 18 generic tuya wall switches/plugs that have always worked just fine with tuyapi, however, a couple of those switches recently received update to 2.0.0 firmware (all others are on 1.0.4 to 1.0.5). After this I can no longer seem to use tuyapi to communicate with these devices, I just get socket error.

I can still use tuya-cli with list-app to get device ID/key, but any attempt to communicate with the devices using this new firmware with the discovered id/key simply fails with "Error from socket" message.

Any chance this is a known compatibility issue or is there any data I can provide that might help isolate the issue further? I'm thinking I may need to try to block these things from Tuya so that they won't get future firmware updates. Either that, or just suck it up and switch them to custom firmware.

[kueblc]

Thanks for the report, I've been aware that an update was coming but the extent of the damage is unknown. This is very helpful for evaluating what has changed.

Was the update elective or did it occur silently?

Are you able to get a copy of the firmware bin? And/or a network capture of an update?

You can block Tuya from pushing further updates if you intercept DNS requests for a.gw.tuyaus.com a.tuyaus.com and a3.tuyaus.com (replace us with eu or cn if you're in EU or China). You can use dnsmasq on your router or home server and configuring your router to use this DNS first. Note this effectively removes any cloud capability.

[tsightler]

I'll admit in this case the update was elective. I was actually working with support to resolve an issue with these two devices due to the fact they were running firmware 1.0.0. However, I was anticipating them pushing 1.0.5, like all of my other devices, not 2.0.0, although the update did resolve my issue, so that's good, but broke my local control, so that's bad. That being said, I have received silent updates previously, from 1.0.2 to 1.0.5, on a few of the devices I've had a while.

I've been hesitant to block connection to Tuya due to the fact that I do make use of the cloud integration with Google Assistant/Alexa, so I'd have to plan an alternative. Not that I can't do that but everything works just fine now so it's becoming motivated enough to want to change all of that automation.

How do you get the firmware bin? I'm assuming there is a tool. Point me there and I'll be happy to grab it. I'll probe at this think over the weekend and see if I can figure out what it's doing.

[kueblc]

Out of curiosity, what was the original issue that 2.0.0 resolved for you?

If you want to keep cloud integration but block firmware updates, it's a little trickier but still possible. You might be able to set up nqinx to proxy requests and filter out via a regex matching upgrade strings in the URL. I also have a utility I wrote for testing that does something like this, I'll work on preparing this for release.

For retrieving the firmware bin, if you have physical access to the ESP microcontroller inside the device, you can connect over serial and use esptool.py to backup the firmware directly. There's plenty of guides to do this, such as on the Espurna and Sonoff-Tasmota wikis.

If you don't want to open your device, you can try capturing the network activity with tcpdump or Wireshark. You'll want to be capturing from somewhere inbetween Tuya and your device, ie your access point or with your network card in monitor mode.

It might be enough to remove the device and re-add it while recording. This should provide us with enough data to simulate an upgrade request without actually performing it. Or if you are planning/willing to actually upgrade another device, that capture would tell us where to find the firmware bin.

[tsightler]

The original issue was pretty minor, I'm regretting it now! With the old firmware the wifi indicator would go out any time the switch was turn on. It was completely inconsistent with the other 16 devices I have, where the wifi light stayed on regardless. There was also a weird presentation of these devices in the Smart Life app. Both issues are now corrected, but losing local control is a bummer for sure.

I'm not sure if I'll be able to grab the firmware if it requires physical access. I'm not unwilling to dig into the hardware, but these are installed home devices that are used on a daily basis.

Getting tcpdump would be more likely, but none of my other devices are showing the firmware upgrade as available. These two devices were "unique" in that they had different virtual IDs. Actually, I have 4 different sets of virtual ID numbering among the 18 devices I have, but these 2 were somehow weird from the start, with the old firmware. I think it was just old stock that I happened to receive from Amazon.

I can play with add/remove and see what I can capture, or maybe I'll order replacements for these two devices and be more willing to play with these two, that's probably the easiest thing to do.

[kueblc]

When you say virtual ID, are you referring to the gwIds?

I understand the difficultly in grabbing firmware from in use devices. Anything you manage to get is certainly appreciated.

[tsightler]

Yeah, just the ID I guess. It's called "Virtual ID" in the Device Information in the Smart Life app.

[tsightler]

So here's a little bit of data I've been able to dig up so far, I know it's not much, but I didn't have a ton of time today. All of the switches with the 1.0.x firmware show connections to Tuya Inc servers on port 1883, which is standard MQTT port. The switches with the 2.0.0 firmware show connections on 8886, which appears to be MQTT over TLS.

[tsightler]

Managed to do just a little more probing. The 2.0 firmware is sending out data on UDP 6667 instead of 6666. Also, it seems the data is encrypted vs the easily visible plaintext. Same seems to be true for the initial handshake on TCP 6668.

I had never really looked at the protocol but it seems to consist of some headers and then some json data portion, which is clear text in the 1.x firmware, but obviously encrypted in the 2.0 firmware, in both the UDP broadcast packets and the TCP handshake. Overall, just comparing the packets sent, and their sizes and headers, the handshake looks about the same so my hope is that maybe it's just encrypting the handshake data with the same cipher method previously used for command data, which should, at least in theory, be easy for me to hack up (it may be beyond my somewhat limited javascript skills, but I think I'm starting to make pretty good sense of the code).

[tsightler]

I've attached a couple of tcpdumps of the Smart Life app talking directly to port 6668 (offline mode) showing the exact same toggle of a switch, one with 1.0.5 firmware and the other with 2.0.0 firmware.

You can clearly see the difference. I tried to use tuyadump to look at the captures but it obviously only works with the 1.0.5 firmware dump. With the dump from the newer device it thinks some commands are clear text, but obviously they are not, but there is a 3.3 in the same place where 3.1 exist so I'm assuming it's a bump in protocol version. Hopefully somebody with more knowledge might be able to make better sense of it.

captures.zip

[kueblc]

Thank you so much for doing some digging on this and providing a capture. This is very helpful! I'll take a look and see what else I can find out.

[kueblc]

This is good data. Now just need to get our hands on a copy of the firmware. Were you able to capture the pairing process?

[tsightler]

Would a capture from my gateway be good enough?

[kueblc]

Yes, that should do it

[tsightler]

OK, I had some challenges capturing on my gateway, but here's a capture from my APs. My phone that I was running the pairing from was on a different AP so it ended up with two separate captures, one for the app and another for the switch. Really didn't see much interesting as it seemed to me that everything was encrypted or HTTPS. I'm wondering if I'll need mitmproxy to see what's going on.
pairing_captures.zip

[codetheweb]

Just saw this; hopefully we can work around the changes.

I'll upgrade one of my plugs and send over the firmware bin.

EDIT: it'll be a few days; I guess I don't have any spare plugs at home.

[codetheweb]

I thought the Tuya cloud endpoint responded with a link to the latest firmware when the app queries for an update; but I guess not.

I don't have the resources to manually extract the firmware so I can't help out there, sorry. It's on version 1.0.1 right now with no update available.

Also saw this in the sniffing session:

Would be exciting if Tuya is working on supporting HomeKit, as that's the main reason I started TuyAPI. 😛 Maybe that's also the reason for their sudden protocol changes.

[kueblc]

If you have a gwId and secKey (not localKey) pair, we can simulate the device making an upgrade request, which returns the link we're looking for.

Unfortunately, none of my devices have the update available yet. According to Tuya's statement they released the update January 28th, but it's up to the individual brands to push the update to the end user, so when it will hit any given device is unknown.

[tsightler]

For me Homekit wouldn't matter, as I could care less about it, I just want "optional" local control of my devices. Part of what I like about the Tuya devices is that they work via cloud, but also local, while otherwise it seems I have to choose either local control or cloud control, but not both. TuyaPI gives me both.

Regardless, is the secKey the same key used with TuyaPI or is that the localKey? I don't know the terms, but I can happily provide anything I can be instructed how to get.

[kueblc]

They're different; TuyAPI uses the localKey for LAN level encryption, whereas communication between the device and cloud is encrypted with secKey. secKey is only transmitted to the device once, during the pairing process, and isn't sent to the app. The only way to retrieve this key is by intercepting this pairing or by directly reading it from the device's flash memory.

We'd need to get this for a device which has an update available but hasn't yet been updated (the cloud tracks what version each device is running when it pairs and only makes an update available to those which need it).

[tsightler]

OK, I'm not sure how to help with that unfortunately. I'm still convinced that all they've done is made it so that entire local command is encrypted using the key, otherwise local control would not work as that's the only data the app has and it still needs to discover the devices via the now encrypted UDP packet and talk to the devices using encryption. That means my localKey should still be able to decrypt the messages, so I'm going to try a more brute force method to figure this out, just trying to decrypt the packets with the key. But it might take a little time as I'm super busy for the next few weeks, unless it just turns out to be really easy!

[namgk]

I'm having the same issue, it was a stealth update without consent. Now my plug doesn't seem to open any port (I ran a quick port scan with nmap, yielding no open port at all).

This really sucks, updates these days are scary, either they bring more ads, or broke the hacks.

[tsightler]

That sucks! At least it my case it was not a silent upgrade, but that makes me think that silent upgrades are coming for my devices as well. I guess I'll have to make my choice, either convert the devices I have to open firmware with tuya-convert, or live with cloud control only. Due to lack of time, I'll probably end up choosing the latter, at least for now.

I did try to decrypt the UDP packet that is now being sent by my device, but it is obviously encrypted somehow differently than the previous encryption method. It doesn't look base64, but I still think it has to be something to do with the localKey as, at least for my devices, local control still works, it's just that these commands are now fully encrypted.

[kueblc]

That sucks! At least it my case it was not a silent upgrade, but that makes me think that silent upgrades are coming for my devices as well.

Yes, this is what I've been afraid of 😞

I guess I'll have to make my choice, either convert the devices I have to open firmware with tuya-convert, or live with cloud control only. Due to lack of time, I'll probably end up choosing the latter, at least for now.

Would it help if I released a sandboxing application to prevent upgrades?

I did try to decrypt the UDP packet that is now being sent by my device, but it is obviously encrypted somehow differently than the previous encryption method. It doesn't look base64, but I still think it has to be something to do with the localKey as, at least for my devices, local control still works, it's just that these commands are now fully encrypted.

If you are comfortable sharing your device keys with me I can try my hand at it and see what we find. Feel free to email me directly at my Gmail account of the same name.

[Apollon77]

One question ... I know this thread is about Tuya FW 2.0, but I have reports for some Teckin lamps with MCI/Wifi 1.0.5 and also no longer working ... (see Apollon77/ioBroker.tuya#32) My assumption is that Port 6668 is closed there ... but still need to verify. That's why I was wondering that 1.0.5 seems to work fine for other devices ... hm ...

[kueblc]

Firmware is per product, version 2.0 on one device is not the same as 2.0 on another. It has been established that Teckin 1.0.5 does indeed include the TLS upgrade which has been breaking compatibility. I have a fix shortly forthcoming.

[Apollon77]

Cool. Looking forward.

[ChristianHoefer]

@kueblc any news on this Issue?

[kueblc]

Almost there. Discovery and local control have been decrypted thanks to data from @tsightler, new command types are already implemented. I imagine we'll roll this out with the next major release. Thanks for your patience as we are updating several tuya related projects.

[Tollbringer]

I take it this addresses my earlier post about the TeckinSp20 also? This is very exciting if I am understanding correctly!

EDIT: Sorry just saw you linked my post-- Still, Thanks in order!

Thank you all!

[-=SrZ=-]

[thirug010]

@kueblc any update on this issue ?

[Apollon77]

You saw the post two messages above and 3days ago ?! Things need time ... ;-)

[digiblur]

@kueblc feel free to let me know if you need any help with things. Have two test subjects here. A dimmer and switch that came patched out the box.

[Tollbringer]

@kueblc I too will offer to contribute to tests - I have more factory flashed SP20 devices, once updated will be blocked.

[-=SrZ=-]

[kueblc]

Thanks for your patience, I had to take a few days off due to real life. We are all volunteers here, working on projects we love out of passion and not for profit.

Thank you @digiblur and @SirRedZ for the very kind offer. I would be very happy to take you up on that offer. Would you please reach out to me via my gmail of the same name?

[sylwester-]

Just bought 4 plugs today and upgraded the firmware straight away... Happy to help with testing if needed.

[EricPalmquist]

I too am having the same issue with a 3 plug "Foxnova Outdoor Smart Plug". Mine must have upgraded firmware because this API was working fine a week ago. Through wireshark captures and running this API in a debugger I can confirm that the UDP broadcast is now port 6667 instead of 6666, and that the size, prefix, postfix, and data length is the same as before, but the ASCII payload is encoded. Note that my WiFi and MCU firmware both state they are V1.0.4, so not all brands are calling it 2.0.

[Jalleronline]

Hello, same here with a brand new fish feeder with this version:
bv: 5.42
cdv: 1.0.0
pv: 2.2
verSw: 1.1.14
Will wait for your updates and thank you for this project.
Regards.

[bcaro]

Hello,
You will find the protocol details for 2.0 firmware in the attached file.
Only 4 bytes are unknown when sending controls to devices
Some example of packets are given + the Localkey to code/encode data for tests purposes.

The goal is to understand the 4 unknown bytes

Regards
tuya_F2.0_protocol.txt

[Apollon77]

Great progress! And the 4 Bytes need to match device specific to work?

[bcaro]

yes : specific to localkey and message sent, probably
-> in fact bytes 00 01 are always the same
-> so only 2 bytes are unknown
-> they are the same during a certain time and changes regularely
And I don't know how they are generated
But with the above informations, it is possible to obtain data sent by Tuya devices as,in that case, the protocol is fully decoded

[codetheweb]

Thanks to the great work of @kueblc, support for v2/3.3 protocol firmware has been merged and published.

You can try it out by updating to v5.1.0 of TuyAPI. If you use .find() to resolve device IPs, everything should work normally. If you manually pass in the IP to the constructor, you'll have to also pass in the new version number:

const device = new TuyAPI({
  id: 'xxxxxxxxxxxxxxxxxxxx',
  key: 'xxxxxxxxxxxxxxxx',
  ip: 'xxx.xxx.xxx.xxx',
  version: 3.3});

[codetheweb]

Closing this issue for now, feel free to continue discussing changes.

If there are any problems with the upgrade to v5.1.0, please open a new issue.

[SynAckFin]

@bcaro
In the 3.1 version the unknown bytes were a substring of the MD5 hash of the following string:
"data=base64str||lpv=3.1||localkey"
Where base64str is the encrypted data converted to base64 and localkey is the localkey.
It is probably something similar in in the 3.3 version