Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for the new sdk #273

Closed
M4dmartig4n opened this issue Sep 18, 2019 · 365 comments
Closed

Support for the new sdk #273

M4dmartig4n opened this issue Sep 18, 2019 · 365 comments

Comments

@M4dmartig4n
Copy link
Contributor

Hi all,

I did succeed in flashing a device with the new tuya sdk with tuya-convert.
Changes needed are in my own fork, for reference

https://github.com/M4dmartig4n/tuya-convert

It is not ready for a pull-request, since it drops support for older sdk atm.

Enjoy !

@EverythingSmartHome
Copy link

Hey, are you saying this works for new firmware that was previously not compatible?

@M4dmartig4n
Copy link
Contributor Author

M4dmartig4n commented Sep 19, 2019

Yes.
Obviously, I haven't tested it against all new firmware, so your mileage may vary.

@cr0ky
Copy link

cr0ky commented Sep 19, 2019

I hope it will work for the latest tuya devices, bought some door sensors and stuff that would be handy to flash like this.

@naxxxe
Copy link

naxxxe commented Sep 19, 2019

I just tried this on a Nedis WIFIP310FWT that couldn't be flashed using the old method. I can confirm that this worked very well

@EverythingSmartHome
Copy link

Amazing job well done! Care to give a brief overview of how you figured it out? Just to satisfy my curiosity, I'm really interested! Thanks!

@Chaotic
Copy link

Chaotic commented Sep 19, 2019

I'll be checking this out tonight got 4 gosunds with the new firmware on them I'll report back if they work.

@cr0ky
Copy link

cr0ky commented Sep 19, 2019

So just use the build from CT? And import the recent files from you?

@kueblc
Copy link
Collaborator

kueblc commented Sep 19, 2019

@M4dmartig4n Go ahead and open a PR and we can work together to get this merged. Will require a few small tweaks to be compatible with #226. You can reach me directly at my gmail of the same name.

@M4dmartig4n
Copy link
Contributor Author

@kueblc I will be afk for a moment, but will do that.

Apart from my coding style ;) , we need to figure a way to identify devices with previous/new sdk in order to provide them the right answers (for example, mqtt messages, and upgrade answers are quite different).

Maybe there is a parameter in the request, I dunno.

A "hack" would be to check the source ip in fake_registration server, since new sdk requests are proxyfied through psk-frontend and thus come from 127.0.0.1.

@M4dmartig4n
Copy link
Contributor Author

@EverythingSmartHome I will probably write a small blog post about it in the near future

@kueblc
Copy link
Collaborator

kueblc commented Sep 19, 2019

@kueblc I will be afk for a moment, but will do that.

Do you have Telegram or Signal?

Apart from my coding style ;) , we need to figure a way to identify devices with previous/new sdk in order to provide them the right answers (for example, mqtt messages, and upgrade answers are quite different).

Maybe there is a parameter in the request, I dunno.

Sure, check for et=1

A "hack" would be to check the source ip in fake_registration server, since new sdk requests are proxyfied through psk-frontend and thus come from 127.0.0.1.

This would work fine too, this entire project is a "hack" 😛

@Tollbringer
Copy link

Tollbringer commented Sep 19, 2019

Thanks for the efforts. I was unable to install as the pre-req libssl-dev seemed to be missing/absent. I tried a Kali-linux environment, along with my own Proxmox/Debian LXC container script previously working well for Tuya 1.0.

I fixed the error, and was able to proceed.

I attempted conversion with Teckin Sp20 outlet with no success. I can see it finds the device and talks to it, and I would say it engages stage one - but never actually backs up or proceeds. It does the leave the device crippled/bricked and unable to proceed. (that is ok it is a test unit and has serial connections ready to re-flash back to stock). I am able to re-flash back to stock firmware which DOES work, and I am able to add it to the APP and it detects v1.0.3 and updates it, blocking Tuya Convert v1 -- as expected.

I have another AliExpress plug that doesn't have a firmware update for it, and therefor I can't test with.

[-=SrZ=-]

@kueblc
Copy link
Collaborator

kueblc commented Sep 19, 2019

@SirRedZ can you elaborate on what you mean by "bricked"? You can't put it back into EZ config mode after attempting the process?

@Janghou
Copy link

Janghou commented Sep 19, 2019

Another missing prerequisite is git ;)

@Tollbringer
Copy link

@SirRedZ can you elaborate on what you mean by "bricked"? You can't put it back into EZ config mode after attempting the process?

Correct. The module's behavior is modified, and button access is unresponsive. Unplugging and plugging in flashes the light once, but that is it. I recognize this behavior after the bootloader has been applied??? mid-state? Only serial flashing allows me to recover.

@kueblc
Copy link
Collaborator

kueblc commented Sep 19, 2019

Since you already have it connected to serial, it would be great to get a backup of the firmware in this state. Did you back it up before reflashing?

@ThaStealth
Copy link

ThaStealth commented Sep 19, 2019

Along with the missing libssl-dev pre-req which was already mentioned I noticed another issue with my Smart Connect bulbs.

After connecting it is posting to the following URL a couple of times (and failing):

POST /d.json?a=tuya.device.config.get&et=1&other={"token":"00000000","region":"US"}&t=14&uuid=some UUID&v=4.1&sign=some hex string

It seems to be asking for some sort of configuration, but I'm not sure which

@haubke
Copy link

haubke commented Sep 19, 2019

i'm trying for an hour. i got the smart bulb in config mode. tuya convert is searching. but no luck. do i need to start pairing this device thrue my phone?if yes wich app do you use?

@ThaStealth
Copy link

@haubke,

  • is the lamp flashing quickly (2 times per sec) when you start?
  • does it stop flashing after you start the process?
  • what does the logs tell

You only need to connect to the WiFi with your phone, no apps or anything else

If it doesn’t switch to continuous and keeps on flashing, repeat the process. It took me 10 times for it to finally to connect

@haubke
Copy link

haubke commented Sep 19, 2019

yes flashing 2 times a sec.
no it wont stop flashing. i will check logs when i tried a few times. i will post logs in a minute.
I checked the logs, nothing usefull there. Everything seems ok. i will dig deeper.

i guess i found a problem.
log PSK
Traceback (most recent call last):
File "./psk-frontend.py", line 8, in
from Crypto.Cipher import AES
ImportError: No module named 'Crypto'

@pjv
Copy link

pjv commented Sep 19, 2019

sudo apt-get install libssl-dev

...and then boom, it worked. Saved me from having to try to solder pins that are about 1/8 away from a tall plastic switch soldered to the pcb.

THANK YOU!!!

@haubke
Copy link

haubke commented Sep 19, 2019

still no luck, thanks anyhow. i will keep trying and post my findings.

@smadhukarHacs
Copy link

Hi All,
seeing this in the smarthack-web.log with the new sdk setup

URI:/gw.json?a=tuya.device.upgrade.silent.get&gwId=63120466c44f33863260&t=26&v=4.1&sign=31d615cf31fbc6a63e0031d0685a06c7
Answer s.gw.upgrade.get
b'{"result":{"auto":3,"size":"506587","type":9,"pskUrl":"https://10.42.42.1/files/upgrade.bin","hmac":"7BA1C83AD95ED48CBAB9A3BA731B79DE1CFC61DEDC12CE84BBD480F49833F5F9","version":"9.0.0"},"t":10,"e":false,"success":true}'
[I 190919 17:29:55 web:2246] 200 POST /gw.json?a=tuya.device.upgrade.silent.get&gwId=63120466c44f33863260&t=26&v=4.1&sign=31d615cf31fbc6a63e0031d0685a06c7 (10.42.42.29) 24.65ms

URI:/gw.json?a=s.gw.dev.timer.count&gwId=63120466c44f33863260&t=167&sign=28d676260773c01b4dbfba9d8a94d556
WARN: unknown request: s.gw.dev.timer.count (/gw.json?a=s.gw.dev.timer.count&gwId=63120466c44f33863260&t=167&sign=28d676260773c01b4dbfba9d8a94d556)
[I 190919 17:32:32 web:2246] 200 POST /gw.json?a=s.gw.dev.timer.count&gwId=63120466c44f33863260&t=167&sign=28d676260773c01b4dbfba9d8a94d556 (10.42.42.29) 5.93ms

The last line keeps repeating

@kueblc
Copy link
Collaborator

kueblc commented Sep 19, 2019

This should be resolved when we merge this with #226

@Tollbringer
Copy link

Colin, do you still want a copy of firmware in the 'frozen state' ?

@Tollbringer
Copy link

I thought after reading the comments above I should check my logs.... Maybe it was working and crashed for me?>

URI:/gw.json?a=tuya.device.upgrade.silent.get&gwId=88020554cc50e3cf1954&t=7&v=4.1&sign=fab950f88c43806fa17e9bcca40bdf34
Answer s.gw.upgrade.get
b'{"result":{"auto":3,"size":"506587","type":9,"pskUrl":"https://10.42.42.1/files/upgrade.bin","hmac":"7BA1C83AD95ED48CBAB9A3BA731B79DE1CFC61DEDC12CE84BBD480F$
^[[32m[I 190919 07:53:07 web:2246]^[[m^O 200 POST /gw.json?a=tuya.device.upgrade.silent.get&gwId=88020554cc50e3cf1954&t=7&v=4.1&sign=fab950f88c43806fa17e9bcca$
^CTraceback (most recent call last):
File "./fake-registration-server.py", line 160, in
main()
File "./fake-registration-server.py", line 156, in main
tornado.ioloop.IOLoop.current().start()
File "/usr/local/lib/python3.5/dist-packages/tornado/platform/asyncio.py", line 148, in start
self.asyncio_loop.run_forever()
File "/usr/lib/python3.5/asyncio/base_events.py", line 421, in run_forever
self._run_once()
File "/usr/lib/python3.5/asyncio/base_events.py", line 1388, in _run_once
event_list = self._selector.select(timeout)
File "/usr/lib/python3.5/selectors.py", line 445, in select
fd_event_list = self._epoll.poll(timeout, max_ev)
KeyboardInterrupt

@kueblc
Copy link
Collaborator

kueblc commented Sep 20, 2019

@SirRedZ Yes, I can use the backup to hopefully debug what went wrong. Doesn't look like the server crashed, it's a KeyboardInterrupt exception, which is what Ctrl-C throws (the ^C at the beginning of the error message).

@smadhukarHacs
Copy link

This should be resolved when we merge this with #226

Worked with the changes. Thanks guys for all the great work.

@Tollbringer
Copy link

teckin_sp20_afterfw_convertattempt.zip

Here is the FW in "bricked" state. Here are the exact steps.

  1. Module defaulted to original FW (rev =unknown - via serial backup made before touching)

  2. Module tested for pairing mode/operation via manual button.

  3. Module put in to pairing mode, 6 seconds on power button.

  4. Device added to Tuya app per normal use.

  5. New FW update v1.0.3 identified, accepted and module upgraded.

  6. Confirm normal operation of device.

  7. Remove device from APP

  8. Attempt Tuya convert 2.0 process.

  9. Connect mobile device to vtrust-flash

  10. Device in pairing mode as a result of APP removal - GO
    ..... endless dots, connection made to module lights stop flashing, never flash again.
    process aborted, - UNPLUG module

  11. Restart Tuya-Convert process, reconnect to Vtrust-flash, start process and then plugin module
    F/W backup actually starts - it tries, no data received. Enter a few times, multiple lines showing no data and speed of transfer declining.

  12. Restart process, Vtrust-recovery visible, module reports flashed with userspace1 please flash with userspace 2 (after an attempt to flash 3rd party firmware)

  13. Flashed Userspace 2- rebooted.

  14. Restarted flashing process, unplugged module and plugged back in. Mopdule reports online. Firmware backup fails again (expected - file exists?)

  15. JUMP - UP - AND - DOWN - SUCCESS !!!!!

Sonoff-xxx access point visible. IT WORKS. I had to jump through hoops - it works.

This was the device I originally used to report the failure- block also!

GREAT WORK!.

[-=SrZ=-]

@Tollbringer
Copy link

My current conclusion is brute force (unplug/plugin/restart tuya convert process, etc...) gets it to go eventually.

Lightbulbs are a No-go after trying 4-5 different bulbs Zero response. I did my test module 4 times andDigiBlurDIY did 2 or 3 different devices and all eventually converted. There were all previously confirmed BLOCKED in the past. All were added to the app to make sure they had latest firmware, and then removed from the app to begin the conversion.

Thanks everyone- I hope this helps!

[-=SrZ=-]

@richterdc
Copy link

@Photo64 attach your logs. I will also mention that using apple devices as the other device seemed problematic at times. Due in part to how they share the SSID info, how they will disco when they sleep and how they dislike not having internet.

@Photo64
Copy link

Photo64 commented Sep 30, 2019

The process appears to start correctly, the vtrust-flash on my tablet shows connected however after putting the bulb into flash mode and it not connecting after a few minutes the vtrust-flash on my tablet shows "not connected" and then goes to a "Wifi Connection Failure"

@amiranees
Copy link

@Photo64 I can also confirm having issues with Android devices but iOS works fine.

@danmed
Copy link

danmed commented Sep 30, 2019

Please share your successes and failures on the wiki

What are they expecting in the GPIOs column?

@Photo64
Copy link

Photo64 commented Sep 30, 2019

Tried using an Amazon fireHD7 tablet and my MotoG6 Android phone. Had no issues with the FireHD tablet the other day when converting the smart plug

@Photo64
Copy link

Photo64 commented Sep 30, 2019

@kueblc
Copy link
Collaborator

kueblc commented Sep 30, 2019

Please share your successes and failures on the wiki

What are they expecting in the GPIOs column?

@danmed you can leave it blank, just fill out what you know. The GPIO column is for mapping the pins to their functions, see the other compatible devices wiki for examples.

@kueblc
Copy link
Collaborator

kueblc commented Sep 30, 2019

@Photo64 your logs indicate that the device did connect and attempt to install the intermediate firmware. It is unclear if it completed before you halted the process, you can try to resume by plugging the device back in while start_flash is still running (vtrust-flash AP should still be visible).

@Photo64
Copy link

Photo64 commented Sep 30, 2019

How would I know if it's resuming or completed? It never goes beyond just seeing the dots and then it seems the access point loses wifi connection.

@kueblc
Copy link
Collaborator

kueblc commented Sep 30, 2019

It would stop printing dots. You shouldn't be losing WiFi connection, but there will be no internet access through the connection, do not confuse the two.

Also note that you do not need to put the device back into pairing mode nor connect a third device, since this is only required to start the process, not resume it.

@Photo64
Copy link

Photo64 commented Sep 30, 2019

Well after several attempts I finally was able to flash both bulbs. The one thing I noticed and I have no idea if this impacts it in any way but I couldn't get them to flash until on both bulbs I got the slow blinking first and the smart app access point to show on my tablet. Once that appeared I turned on/off the bulb to get the fast blinking and then it decided to begin the flashing. Could have been a coincident but it only worked on both when that happened.

@kueblc
Copy link
Collaborator

kueblc commented Sep 30, 2019

Glad to hear it @Photo64

@luchtgitaar
Copy link

@amiranees I din't get the LSC LED strip to flash... I think I need to do it by wire....
I flashed the LSC PIR by wire, don't know if the firmware is needed? I want to flash more PIR but it's battery powered so it lose connection but here is the firmware I want to flash the door contacts also....
Is this possible or do this things need to go by the wire solution?
All other things I csn succesfull flash with the new software
Orginal Firmware of the LSC PIR: Sonoff_backup_01.zip

As @ggaljoen said, I am going to try the OTA version. If things go wrong I can fix it by wire flash method. I ll report here later.

Did you succeed flashing the LSC led strip via OTA? I managed to flash the LSC smart bulbs, but with the led strip no luck..

@amiranees
Copy link

@

@amiranees I din't get the LSC LED strip to flash... I think I need to do it by wire....
I flashed the LSC PIR by wire, don't know if the firmware is needed? I want to flash more PIR but it's battery powered so it lose connection but here is the firmware I want to flash the door contacts also....
Is this possible or do this things need to go by the wire solution?
All other things I csn succesfull flash with the new software
Orginal Firmware of the LSC PIR: Sonoff_backup_01.zip

As @ggaljoen said, I am going to try the OTA version. If things go wrong I can fix it by wire flash method. I ll report here later.

Did you succeed flashing the LSC led strip via OTA? I managed to flash the LSC smart bulbs, but with the led strip no luck..

I didn't try it yet, I'll think i try tomorrow.
How did u manage to flash the smart bulbs?

@luchtgitaar
Copy link

@

@amiranees I din't get the LSC LED strip to flash... I think I need to do it by wire....
I flashed the LSC PIR by wire, don't know if the firmware is needed? I want to flash more PIR but it's battery powered so it lose connection but here is the firmware I want to flash the door contacts also....
Is this possible or do this things need to go by the wire solution?
All other things I csn succesfull flash with the new software
Orginal Firmware of the LSC PIR: Sonoff_backup_01.zip

As @ggaljoen said, I am going to try the OTA version. If things go wrong I can fix it by wire flash method. I ll report here later.

Did you succeed flashing the LSC led strip via OTA? I managed to flash the LSC smart bulbs, but with the led strip no luck..

I didn't try it yet, I'll think i try tomorrow.
How did u manage to flash the smart bulbs?

I flashed the filament ones (both e14 and e27) just with the 'new' OTA method. I also have a multi color led e14, but didnt try to flash that one. There isn't a template available, and I dont have experience creating one, but maybe I will try tomorrow .

@amiranees
Copy link

amiranees commented Sep 30, 2019

@luchtgitaar
Nice to know.
During the flashing process, stays the light on or does it go off?

And also, did you chose for /flash3?

@Beanow
Copy link

Beanow commented Sep 30, 2019

@TimelessNL I see you did great work on the LSC bulbs. Have you tried the ceiling light one?

Still need to do a first attempt at any bulb, but disassembly of this one couldn't be easier. 3 screws, some removing of covers and you've got a WR3L on a breakout board with headers 😮

IMG_20190930_224239
IMG_20190930_224259

@amiranees
Copy link

@amiranees Which bulb are you trying to flash. Only the RGB E27 bulb doesn't work since it contains a RTL chip. All the others should work just fine.

EDIT:
Maybe it's time to create a success/failed table so people aren't asking the same questions each time.

@luchtgitaar I am afraid you cant flash the RGB ones.

@kueblc
Copy link
Collaborator

kueblc commented Sep 30, 2019

@Beanow unfortunately the WR3L is based on the RTL8710BN and thus can't be flashed with ESP82xx firmware like Tasmota. I am not aware of any custom firmware built for the RTL8710BN at this time, though I would be thrilled to learn otherwise.

@Beanow
Copy link

Beanow commented Sep 30, 2019

Bummer. But this may be a fun one to test with. And looks like there's a fair bit of info on https://github.com/TuyaInc/tysdk_for_rtl8710bn/blob/master/wifisdk_for_rtk_871x/component/soc/realtek/8711b/misc/rtl8710b_ota.c

@TimelessNL
Copy link

TimelessNL commented Sep 30, 2019

@Beanow yeah I also bought that ceiling light myself. I had to do allot of rework on mine since the overall production quality of these is very low... couple of dry solder joints, flux remains and solder balls floating around.

In the process I did replace the RTL chip with an ESP-12. They are pin compatible after all. So running Tasmota after replacing the SoC was quite easy.

There isn't a template available, and I dont have experience creating one, but maybe I will try tomorrow .

@luchtgitaar It isn't just a 'template' that needs to be worked on. the RGBW E14/GU10's use a rather unknown 5channel i2c chip that controls the LED power. So you can't just pwm your way out of this one😄. There is a proof of concept in Tasmota-dev if you're interested.

@Beanow
Copy link

Beanow commented Sep 30, 2019

In the process I did replace the RTL chip with an ESP-12. They are pin compatible after all. So running Tasmota after replacing the SoC was quite easy.

That's one way to do it. Guessing the one TYWE2S/TYWE3S are the ones you can use right? Like the one in the motion sensor.

@TimelessNL
Copy link

I suppose so yes. But it don't have to be a tuya module though.

@kueblc
Copy link
Collaborator

kueblc commented Sep 30, 2019

Bummer. But this may be a fun one to test with. And looks like there's a fair bit of info on https://github.com/TuyaInc/tysdk_for_rtl8710bn/blob/master/wifisdk_for_rtk_871x/component/soc/realtek/8711b/misc/rtl8710b_ota.c

It's funny you linked this file in particular @Beanow, I was just looking into it myself 😛

@Xenomes
Copy link

Xenomes commented Oct 1, 2019

@kueblc and @Beanow did you see this github? https://github.com/8devices/platformio-realtek-rtl8710b
I am no programmer, so... It would be great to be able to flash the LSC Smart Bulb RGB E27 😊

@Beanow
Copy link

Beanow commented Oct 1, 2019

I suppose so yes. But it don't have to be a tuya module though.

Well I was thinking about this in terms of easily removing Chinese spyware useful data-mining, just not useful for the buyer, from consumer IoT products OTA. So the tuya ones are interesting there.

It's funny you linked this file in particular @Beanow, I was just looking into it myself

It is the file that said OTA so yeah I would look at that first :P

@Xenomes I hadn't seen that. But looks like PlatformIO is some kind of abstraction layer to make the development process more unified. It isn't a ready-to-go firmware, it's tools to build firmware. Frankly though unless we have a financial sponsor I won't be able to pick up a project to build the firmware from scratch. A few too many projects I'm already maintaining 😅

@angomania
Copy link

angomania commented Oct 1, 2019

Successfully flashed 4x Teckin SP22. So I ve ordered again the same modules from Amazon.
But the new ones didn´t flash. After starting "start_flash.sh" the module goes off- and then again online, but the script is staying at "Waiting for the upgraded device to appear" ...

Maybe all have an upgraded rom?
Logfiles attached.
smarthack-mqtt.log
smarthack-psk.log
smarthack-smartconfig.log
smarthack-web.log
smarthack-wifi.log

@kueblc
Copy link
Collaborator

kueblc commented Oct 1, 2019

@angomania couple things, the latest can be found at #279, the fork isn't being maintained. And unfortunately your new Teckins are not using an ESP82xx so you won't be able to use Tasmota or other ESP based firmware, OTA or otherwise.

@kueblc
Copy link
Collaborator

kueblc commented Oct 1, 2019

Please do all testing and reporting against #279 so we can move forward with merge. Locking this issue so users see this and do not accidentally install an older unmaintained version.

@ct-Open-Source ct-Open-Source locked as resolved and limited conversation to collaborators Oct 1, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests