Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OTA update in ZHA #7

Open
pvvx opened this issue Nov 19, 2023 · 49 comments
Open

OTA update in ZHA #7

pvvx opened this issue Nov 19, 2023 · 49 comments
Labels
documentation Improvements or additions to documentation

Comments

@pvvx
Copy link
Owner

pvvx commented Nov 19, 2023

OTA update in ZHA.

The "configuration.yaml" file must have a directory assigned to the OTA files and the "ZHA Toolkit" installed.

For example, the directory name will be: “/zigbee_ota”

zha:
  zigpy_config:
    ota:
      enabled: true
      # Download the index over HTTPS
      z2m_remote_index: https://raw.githubusercontent.com/Koenkk/zigbee-OTA/master/index.json
      # Or, load the index offline
      z2m_local_index: ./zigbee_ota/index.json
      allow_advanced_ota_dir: I understand I can *destroy* my devices by enabling OTA
        updates from files. Some OTA updates can be mistakenly applied to the
        wrong device, breaking it. I am consciously using this at my own risk.
      advanced_ota_dir: ./zigbee_ota
      otau_directory: ./zigbee_ota
      ikea_provider: false
      ledvance_provider: false
      salus_provider: false
      sonoff_provider: false
      inovelli_provider: false
      thirdreality_provider: false

Place the new FW version files "*.zigbee" or "index.json" from bin into this directory ("./zigbee_ota").

Restart ZHA (preferably the entire Home Assistant).


Update in new versions of Home Assistant:

image

image


Update in ZHA Toolkit:

Doesn't work in the new version of ZHA!

https://github.com/mdeweerd/zha-toolkit#ota_notify---downloadtrigger-device-fw-update

Go to “Developer tools” -> “SERVICES” and select “ZHA Toolkit: Trigger Device's Firmware Update”.

image

Select “Device Reference” and click “CALL SERVICE”.

After some time, 15..20 minutes, the firmware number will change:

image


To restore to BLE, use the Zigbee OTA file with the same number device from: https://github.com/pvvx/ATC_MiThermometer/tree/master/zigbee_ota

After running Zigbee OTA, be sure to delete this file or files from the update directories for Zigbee OTA and completely restart z2m/zha. Otherwise, with the new Zigbee firmware, this update in BLE will work again.

@pvvx pvvx added the documentation Improvements or additions to documentation label Nov 19, 2023
@bs76
Copy link

bs76 commented Nov 25, 2023

Tried to follow restoring BLE but stumbled on these issues:

  • .zigbee file is not identified as OTA firmware:
2023-11-25 14:11:43.476 DEBUG (SyncWorker_1) [zigpy.ota.provider] File '/config/zigpy_ota/1141-020a-99993001-ATC_v46.zigbee' doesn't appear to be a OTA image
    return HueSBLOTAImage.deserialize(data)
    header, remaining_data = OTAImageHeader.deserialize(data)
ValueError: Wrong magic number for OTA Image: 168430090
    return OTAImage.deserialize(data)
    hdr, data = OTAImageHeader.deserialize(data)
  • as follow up the firmware is not found:
2023-11-25 14:30:10.491 DEBUG (MainThread) [zigpy.zcl] [0x6F9A:1:0x0019] OTA query_next_image handler for 'Xiaomi LYWSD03MMC': field_control=0, manufacturer_id=4417, image_type=522, current_file_version=17969153, hardware_version=None, model='LYWSD03MMC'
2023-11-25 14:30:10.492 DEBUG (MainThread) [zigpy.zcl] [0x6F9A:1:0x0019] No OTA image is available

HA:

 Core 2023.11.3
Supervisor 2023.11.3
Operating System 11.1
Frontend 20231030.2

Any advice ?

@pvvx
Copy link
Owner Author

pvvx commented Nov 25, 2023

Download the file correctly.
Not as “blob”, but as “raw”.
https://github.com/pvvx/ZigbeeTLc/raw/master/bin/1141-020a-01123001-Z03MMC.zigbee

@bs76
Copy link

bs76 commented Nov 25, 2023

D'oh ! Embarrasing mistake on my side. Thank you, succesfully reflashed

@breaver
Copy link

breaver commented Nov 26, 2023

OTA update in ZHA.

Do you have such an walkthrough for zigbee2mqtt?

@pvvx
Copy link
Owner Author

pvvx commented Nov 27, 2023

Do you have such an walkthrough for zigbee2mqtt?

https://www.zigbee2mqtt.io/guide/usage/ota_updates.html#local-ota-index-and-firmware-files

@breaver
Copy link

breaver commented Nov 28, 2023

I followed the procedure but still not able to do the OTA update back to BLE. Z2M keeps telling me
Failed to check if update available for 'TF Sensor ' (No image available for imageType '522')

@CLARENNE-Q
Copy link

CLARENNE-Q commented Dec 23, 2023

Hello,

If you have HomeAssistant OS, the configuration file must be like this:

      otau_directory: ./zigpy_ota

@pvvx
Copy link
Owner Author

pvvx commented Dec 23, 2023

https://github.com/zigpy/zigpy/wiki/OTA-Device-Firmware-Updates#enabling-ota-updates

@onurunaldi
Copy link

Hello,

If you have HomeAssistant OS, the configuration file must be like this:

      otau_directory: ./zigpy_ota

This solved my problem and this trick is not written anywhere. Thanks so much!

@rwronski
Copy link

rwronski commented Dec 28, 2023

Unfortunately, OTA in zigbee2mqtt doesn't work for me either (HA OS). I've followed the manual, created override OTA index, tried many things, but it just doesn't work. If anyone has any other experience, please share.

Only error I get is 'Failed to check if update available for 'Termometr 2' (No image available for imageType '522')' - it seems like z2m is just ignoring my local OTA index.

@rwronski
Copy link

Ok I've managed to make a progress - you don't add 'zigbee_ota_override_index_location' to the configuration.yaml file manually, but go to 'Settings' -> 'OTA Updates' -> fill in the 'OTA index override file name' field. And suddenly z2m sees it, even though the configuration.yaml looks the same as if I'd edited it manually.

@Neo2SHYAlien
Copy link

@rwronski try this guide. Personally I didn't validate it

@khayamgondal
Copy link

@pvvx where is the BLE .zigbee file for tuya TS0201?

@djbuldog
Copy link

djbuldog commented Feb 17, 2024

Hi, I have LYWSD03MMC with devbis fw 1.1.0 and I would like to migrate to ZigbeTLc fw or back to BT. I followed guides for Z2M and ZHA but without success. For example ZHA.. I added following to configuration.yaml:

zha:
  zigpy_config:
    ota:
      otau_directory: ./zigpy_ota

zha_toolkit:

the folder /root/config/zigpy_ota contains:

-rw-r--r--    1 root     root        128034 Feb 17 00:25 1141-0203-09993001-Z03MMC_v0120.zigbee
-rw-r--r--    1 root     root         82674 Feb 17 00:29 1141-0203-99993001-ATC_v46.zigbee

I called ZHA Toolkit: Trigger Device's Firmware Update service. The log contains:

2024-02-17 01:17:36.680 INFO (MainThread) [custom_components.zha_toolkit] Running ZHA Toolkit service: <ServiceCall zha_toolkit.ota_notify (c:01HPT6SXX821EAJN6CBNR4XVQS): ieee=update.xiaomi_lywsd03mmc_firmware>
2024-02-17 01:17:36.861 DEBUG (MainThread) [zigpy.application] Feeding watchdog
2024-02-17 01:17:36.939 DEBUG (MainThread) [zigpy.ota.provider] Inovelli: Finished downloading firmware update list
2024-02-17 01:17:37.170 DEBUG (MainThread) [zigpy.ota.provider] Ledvance: Finished downloading firmware update list
2024-02-17 01:17:37.616 DEBUG (MainThread) [zigpy.ota.provider] Salus: Finished downloading firmware update list
2024-02-17 01:17:37.685 DEBUG (MainThread) [zigpy.ota.provider] Sonoff: Finished downloading firmware update list
2024-02-17 01:17:38.169 DEBUG (MainThread) [zigpy.ota.provider] ThirdReality: Finished downloading firmware update list
2024-02-17 01:17:38.169 DEBUG (MainThread) [zigpy.device] [0xec8b] Extending timeout for 0x04 request
2024-02-17 01:17:44.143 DEBUG (MainThread) [zigpy.application] Received a packet: ZigbeePacket(timestamp=datetime.datetime(2024, 2, 17, 0, 17, 44, 143898, tzinfo=datetime.timezone.utc), src=AddrModeAddress(addr_mode=<AddrMode.NWK: 2>, address=0xEC8B), src_ep=0, dst=AddrModeAddress(addr_mode=<AddrMode.NWK: 2>, address=0x0000), dst_ep=0, source_route=None, extended_timeout=False, tsn=209, profile_id=0, cluster_id=32801, data=Serialized[b'\x04\x00'], tx_options=<TransmitOptions.NONE: 0>, radius=0, non_member_radius=0, lqi=244, rssi=-39)
2024-02-17 01:17:44.144 DEBUG (MainThread) [zigpy.zcl] [0xEC8B:1:0x0000] Sending request header: ZCLHeader(frame_control=FrameControl<0x00>(frame_type=<FrameType.GLOBAL_COMMAND: 0>, is_manufacturer_specific=False, direction=<Direction.Client_to_Server: 0>, disable_default_response=0, reserved=0, *is_cluster=False, *is_general=True), tsn=5, command_id=<GeneralCommand.Configure_Reporting: 6>, *direction=<Direction.Client_to_Server: 0>)
2024-02-17 01:17:44.144 DEBUG (MainThread) [zigpy.zcl] [0xEC8B:1:0x0000] Sending request: Configure_Reporting(config_records=[AttributeReportingConfig(direction=0, attrid=0x4000, datatype=66, min_interval=0, max_interval=1800, reportable_change=1)])
2024-02-17 01:17:44.144 DEBUG (MainThread) [zigpy.device] [0xec8b] Extending timeout for 0x05 request
2024-02-17 01:17:44.690 DEBUG (MainThread) [zigpy.application] Received a packet: ZigbeePacket(timestamp=datetime.datetime(2024, 2, 17, 0, 17, 44, 690278, tzinfo=datetime.timezone.utc), src=AddrModeAddress(addr_mode=<AddrMode.NWK: 2>, address=0xEC8B), src_ep=1, dst=AddrModeAddress(addr_mode=<AddrMode.NWK: 2>, address=0x0000), dst_ep=1, source_route=None, extended_timeout=False, tsn=210, profile_id=260, cluster_id=0, data=Serialized[b'\x18\x05\x07\x86\x00\x00@'], tx_options=<TransmitOptions.NONE: 0>, radius=0, non_member_radius=0, lqi=244, rssi=-39)
2024-02-17 01:17:44.690 DEBUG (MainThread) [zigpy.zcl] [0xEC8B:1:0x0000] Received ZCL frame: b'\x18\x05\x07\x86\x00\x00@'
2024-02-17 01:17:44.690 DEBUG (MainThread) [zigpy.zcl] [0xEC8B:1:0x0000] Decoded ZCL frame header: ZCLHeader(frame_control=FrameControl<0x18>(frame_type=<FrameType.GLOBAL_COMMAND: 0>, is_manufacturer_specific=0, direction=<Direction.Server_to_Client: 1>, disable_default_response=1, reserved=0, *is_cluster=False, *is_general=True), tsn=5, command_id=7, *direction=<Direction.Server_to_Client: 1>)
2024-02-17 01:17:44.690 DEBUG (MainThread) [zigpy.zcl] [0xEC8B:1:0x0000] Decoded ZCL frame: Basic:Configure_Reporting_rsp(status_records=[ConfigureReportingResponseRecord(status=<Status.UNSUPPORTED_ATTRIBUTE: 134>, direction=<ReportingDirection.SendReports: 0>, attrid=16384)])
2024-02-17 01:17:44.691 DEBUG (MainThread) [zigpy.zcl] [0xEC8B:1:0x0019] Sending reply header: ZCLHeader(frame_control=FrameControl<0x19>(frame_type=<FrameType.CLUSTER_COMMAND: 1>, is_manufacturer_specific=False, direction=<Direction.Server_to_Client: 1>, disable_default_response=1, reserved=0, *is_cluster=True, *is_general=False), tsn=6, command_id=0, *direction=<Direction.Server_to_Client: 1>)
2024-02-17 01:17:44.691 DEBUG (MainThread) [zigpy.zcl] [0xEC8B:1:0x0019] Sending reply: ImageNotifyCommand(payload_type=<PayloadType.QueryJitter: 0>, query_jitter=100)
2024-02-17 01:17:45.207 INFO (MainThread) [homeassistant.helpers.script.websocket_api_script] websocket_api script: Stop script sequence: done
2024-02-17 01:17:45.221 DEBUG (MainThread) [zigpy.application] Received a packet: ZigbeePacket(timestamp=datetime.datetime(2024, 2, 17, 0, 17, 45, 221578, tzinfo=datetime.timezone.utc), src=AddrModeAddress(addr_mode=<AddrMode.NWK: 2>, address=0xEC8B), src_ep=1, dst=AddrModeAddress(addr_mode=<AddrMode.NWK: 2>, address=0x0000), dst_ep=1, source_route=None, extended_timeout=False, tsn=211, profile_id=260, cluster_id=25, data=Serialized[b'\x01\xb0\x01\x00\x15\xdb\x03\x02\x010\x00\x11'], tx_options=<TransmitOptions.NONE: 0>, radius=0, non_member_radius=0, lqi=244, rssi=-39)
2024-02-17 01:17:45.221 DEBUG (MainThread) [zigpy.zcl] [0xEC8B:1:0x0019] Received ZCL frame: b'\x01\xb0\x01\x00\x15\xdb\x03\x02\x010\x00\x11'
2024-02-17 01:17:45.221 DEBUG (MainThread) [zigpy.zcl] [0xEC8B:1:0x0019] Decoded ZCL frame header: ZCLHeader(frame_control=FrameControl<0x01>(frame_type=<FrameType.CLUSTER_COMMAND: 1>, is_manufacturer_specific=0, direction=<Direction.Client_to_Server: 0>, disable_default_response=0, reserved=0, *is_cluster=True, *is_general=False), tsn=176, command_id=1, *direction=<Direction.Client_to_Server: 0>)
2024-02-17 01:17:45.222 DEBUG (MainThread) [zigpy.zcl] [0xEC8B:1:0x0019] Decoded ZCL frame: Ota:QueryNextImageCommand(field_control=<FieldControl: 0>, manufacturer_code=56085, image_type=515, current_file_version=285224961)
2024-02-17 01:17:45.222 DEBUG (MainThread) [zigpy.zcl] [0xEC8B:1:0x0019] Received command 0x01 (TSN 176): QueryNextImageCommand(field_control=<FieldControl: 0>, manufacturer_code=56085, image_type=515, current_file_version=285224961)
2024-02-17 01:17:45.222 DEBUG (MainThread) [zigpy.zcl] [0xEC8B:1:0x0019] OTA query_next_image handler for 'Xiaomi LYWSD03MMC': field_control=0, manufacturer_id=56085, image_type=515, current_file_version=285224961, hardware_version=None, model='LYWSD03MMC'
2024-02-17 01:17:45.222 DEBUG (MainThread) [zigpy.zcl] [0xEC8B:1:0x0019] No OTA image is available

there is one interesting message... I do not know what it means or if it is related to my issue...

2024-02-17 00:34:06.142 INFO (MainThread) [zigpy.ota] OTA file providers are currently disabled

Core 2024.2.2
Supervisor 2024.01.1
Operating System 11.5
Frontend 20240207.1

It looks like the local OTA files are ignored... Can you see anything wrong? Any ideas what to try next?

@robvanoostenrijk
Copy link

robvanoostenrijk commented Feb 17, 2024

zigpy has disabled local providers because many people are breaking their devices with wrong OTA updates.

The commit that does that is here:
zigpy/zigpy@6aeb8a6

You can still do OTA updates using ZHA with the following yaml configuration:
(Note the new allow_file_providers key and its value)

zha:
  zigpy_config:
    ota:
      allow_file_providers: "I understand I can *destroy* my devices by enabling OTA updates from files. Some OTA updates can be mistakenly applied to the wrong device, breaking it. I am consciously using this at my own risk."
      ikea_provider: true
      inovelli_provider: false
      ledvance_provider: false
      otau_directory: ./zigbee_ota
      salus_provider: false
      sonoff_provider: false
      thirdreality_provider: false

@sh-guy
Copy link

sh-guy commented Mar 19, 2024

After flashing the Zigbee firmware, is there a way to tell exactly which image was installed? I am unsure whether the image from pvvx or devbis were installed.
I have followed everything above, including the new allow_file_providers line and have download the ATC 4.7 files for restoring BT as raw and not blob. I uploaded both the 0203 and 020a images into the zigpy_ota folder. Restarted HA, initiated the OTA image_notify command. But nothing happens. My devices always say firmware is up-to-date and the debug log says no compatible images found.
My devices are on Firmware 0x01203001. I just have one that constantly loses connectivity. So, I wanted to restore the BT image to start over, just in case something went wrong during flashing. If that doesn't help, then I assume a faulty device. In which case I would also have to restore BT image to return it.
Is there something else that needs to be done to allow ZHA to detect the correct firmware file under the zigpy_ota folder?

HAOS
Core: 2024.3.1
Supervisor: 2024.03.0
OS: 12.1

@pvvx
Copy link
Owner Author

pvvx commented Mar 19, 2024

#7 (comment)

Zigbee->BLE Files
https://github.com/pvvx/ATC_MiThermometer/blob/master/zigbee_ota/1141-0203-99993001-ATC_v47.zigbee
https://github.com/pvvx/ATC_MiThermometer/blob/master/zigbee_ota/1141-020a-99993001-ATC_v47.zigbee

zha:
  zigpy_config:
    ota:
      allow_file_providers: "I understand I can *destroy* my devices by enabling OTA updates from files. Some OTA updates can be mistakenly applied to the wrong device, breaking it. I am consciously using this at my own risk."
      otau_directory: ./zigbee_ota

Have you tried using "reconfigure"?
image
image

Devices sometimes fall off if the coordinator is turned off for a long time (days) and the devices work through some Tuya Zigbee routers.

@sh-guy
Copy link

sh-guy commented Mar 19, 2024

#7 (comment)

Zigbee->BLE Files https://github.com/pvvx/ATC_MiThermometer/blob/master/zigbee_ota/1141-0203-99993001-ATC_v47.zigbee https://github.com/pvvx/ATC_MiThermometer/blob/master/zigbee_ota/1141-020a-99993001-ATC_v47.zigbee

zha:
  zigpy_config:
    ota:
      allow_file_providers: "I understand I can *destroy* my devices by enabling OTA updates from files. Some OTA updates can be mistakenly applied to the wrong device, breaking it. I am consciously using this at my own risk."
      otau_directory: ./zigbee_ota

I had that setup. But you might want to modify your text above that you copied here. The directory that is referenced is "zigpy_ota", but in your example config you have "zigbee_ota". I suppose it doesn't matter as long as config and actual directory name are the same. ;-)
And maybe your links to images should be modified, replacing "blob" with "raw" to make sure people download the raw version.
I just tried the reconfigure option (after finally successfully pairing again) and it shows the same as your screenshots except the TemperatureMeasurement has an "X" next to the measured_value. :-(

@maikvitesse
Copy link

maikvitesse commented Jul 30, 2024

Just got it working thanks to this and this.

My working code now:

zha:
  zigpy_config:
    ota:
      allow_advanced_ota_dir:
        I understand I can *destroy* my devices by enabling OTA
        updates from files. Some OTA updates can be mistakenly applied to the
        wrong device, breaking it. I am consciously using this at my own risk.
      advanced_ota_dir: /config/zigpy_ota

@cfulton
Copy link

cfulton commented Aug 28, 2024

Newer working config

zha:
  zigpy_config:
    ota:
      extra_providers:
        - type: advanced
          warning: I understand I can *destroy* my devices by enabling OTA updates from
            files. Some OTA updates can be mistakenly applied to the wrong
            device, breaking it. I am consciously using this at my own risk.
          path: /config/zigpy_ota

@fmauNeko
Copy link

fmauNeko commented Sep 22, 2024

Even simpler

zha:
  zigpy_config:
    ota:
      extra_providers:
        - type: z2m
          url: https://raw.githubusercontent.com/pvvx/ZigbeeTLc/refs/heads/master/bin/index.json

@michi2005
Copy link

michi2005 commented Sep 25, 2024

Dear all

I tried a merge of the last 2 options but without success.

zha:
  zigpy_config:
    ota:
      extra_providers:
        - type: z2m
          warning: I understand I can *destroy* my devices by enabling OTA updates from
            files. Some OTA updates can be mistakenly applied to the wrong
            device, breaking it. I am consciously using this at my own risk.
          url: https://raw.githubusercontent.com/pvvx/ZigbeeTLc/refs/heads/master/bin/index.json

After that I get this error in ZHA:

Error setting up entry Sonoff Zigbee 3.0 USB Dongle Plus - Sonoff Zigbee 3.0 USB Dongle Plus for zha
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 604, in async_setup
    result = await component.async_setup_entry(hass, self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/zha/__init__.py", line 132, in async_setup_entry
    async with radio_mgr.connect_zigpy_app() as app:
  File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/zha/radio_manager.py", line 182, in connect_zigpy_app
    app = await self.radio_type.controller.new(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/zigpy/application.py", line 251, in new
    app = cls(config)
          ^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/zigpy_znp/zigbee/application.py", line 81, in __init__
    super().__init__(config=config)
  File "/usr/local/lib/python3.12/site-packages/zigpy/application.py", line 68, in __init__
    self._config = self.SCHEMA(config)
                   ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/voluptuous/schema_builder.py", line 205, in __call__
    return self._compiled([], data)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/voluptuous/schema_builder.py", line 549, in validate_dict
    return base_validate(path, data.items(), out)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/voluptuous/schema_builder.py", line 382, in validate_mapping
    raise er.MultipleInvalid(errors)
voluptuous.error.MultipleInvalid: extra keys not allowed @ data['ota']['extra_providers'][0]['warning']

@arthurkowalsky
Copy link

arthurkowalsky commented Oct 3, 2024

If you're using Zigbee2MQTT and want to update your devices from devbis firmware to ZigbeeTLc using Over-The-Air (OTA) updates, follow these simple steps.


Prerequisites

  • Zigbee2Mqtt is installed and running.
  • You have access to the /config/zigbee2mqtt directory of your HA

Step-by-Step Guide

1. Download the Firmware File

  • Go to the GitHub Repository:

  • Find the Appropriate Firmware File:

    • The firmware filename may change with new versions.
    • Read the readme.txt file in the repository to identify the correct firmware file for your update.
      • For example:
        db15-0203-79993001-Z03MMC_v0122.zigbee - Zigbee OTA ver: devbis new ver1.1.0 to ZigbeeTLc v0.1.2.2
        
    • Download the firmware file that matches your device and desired firmware version.

2. Place the Firmware File in the Zigbee2MQTT Directory

  • Copy the Firmware File:
    • Place the downloaded firmware file (e.g., db15-0203-79993001-Z03MMC_v0122.zigbee) into your /config/zigbee2mqtt directory.

3. Create a Custom OTA Index File

  • Create a File Named my_index.json in the /config/zigbee2mqtt directory.

  • Add the Following Content to my_index.json:

    [
        {
            "url": "your_firmware_filename.zigbee",
            "force": true
        }
    ]
    • Replace "your_firmware_filename.zigbee" with the actual filename of the firmware you downloaded (e.g., "db15-0203-79993001-Z03MMC_v0122.zigbee").

4. Configure Zigbee2Mqtt to Use the Custom OTA Index

  • Open Zigbee2MQTT Settings:

    • Navigate to the Settings in the Zigbee2Mqtt interface.
    • Click the OTA Updates tab.
  • Set the OTA Index Override:

    • In the field labeled OTA index override file name, enter my_index.json.
  • Restart Zigbee2MQTT:

    • After making the change, a Restart button will appear.
    • Click Restart to apply the new settings.

image

5. Perform the OTA Update

  • Go to the OTA Tab:

    • In Zigbee2Mqtt, navigate to the OTA tab to see a list of devices.
  • Check for Updates:

    • For each device you want to update (e.g., thermometers), click on Check for update.
  • Update Your Devices:

    • If an update is found, follow the on-screen instructions to initiate the firmware update.

image

6. Clean Up After the Update

  • Delete the Firmware File:

    • Remove the firmware file (e.g., db15-0203-79993001-Z03MMC_v0122.zigbee) from the /config/zigbee2mqtt directory.
  • Delete the Custom OTA Index File:

    • Remove my_index.json from the /config/zigbee2mqtt directory.
  • Reset Zigbee2MQTT Settings:

    • Return to Settings -> OTA Updates in Zigbee2Mqtt.
    • Clear the OTA index override file name field to revert to default settings.
  • Restart Zigbee2MQTT Again:

    • Click the Restart button that appears to apply the changes.

Important Notes

  • Firmware Filename May Vary:

    • Always verify the correct firmware file by reading the readme.txt in the repository.
    • Ensure you consistently use the correct filename in all steps, especially when editing my_index.json.
  • Restarting Zigbee2MQTT:

    • Restarting Zigbee2MQTT is crucial after changing settings to ensure the changes take effect.

@sebgus
Copy link

sebgus commented Oct 8, 2024

How can I reflash the OTA image or downgrade it?

I've managed to upgrade the firmware of a TS0201-TZ3000 to '0x01233001' (reported in ZHA), but the temperature reading reports -327.7 degrees C and the humidity is 655.4%. So something has gone wrong somewhere...

@Regressor
Copy link

If you're using Zigbee2MQTT and want to update your devices from devbis firmware to ZigbeeTLc using Over-The-Air (OTA) updates, follow these simple steps.

This should be in main readme! Thank you!

@satmandu
Copy link

satmandu commented Oct 26, 2024

[Edit: Solved.]

I'm trying this method to update from devbis firmware to ZigbeeTLc and I get this message:

Failed to check if update available for '0xa4c1385f065ca6c7' (Attempt to access memory outside buffer bounds)

Am I missing something?

Firmware build date
    20240204
Firmware version
    3001-1100
Manufacturer
    [Custom devices (DiY)](https://www.zigbee2mqtt.io/supported-devices/#v=Custom%20devices%20(DiY))
Model
    [LYWSD03MMC](https://www.zigbee2mqtt.io/devices/LYWSD03MMC.html#custom%2520devices%2520(diy)-lywsd03mmc)

I'm trying to update with db15-0203-79993001-Z03MMC_v0122.zigbee

End of /homeassistant/zigbee2mqtt/configuration.yaml:

ota:
  zigbee_ota_override_index_location: my_index.json

/homeassistant/zigbee2mqtt/my_index.json:

[
    {
        "url": "db15-0203-79993001-Z03MMC_v0122.zigbee",
        "force": true
    }
]

Edit: A reminder to all to actually download the RAW firmware links, not the links that connect straight to github pages!

(My update has now started... but it will take 23 minutes...)

@Arachnid
Copy link

I have a TH03 from this Aliexpress listing. I'd like to OTA flash it to the custom firmware. I've confirmed the PCB matches the layout shown for TH03 devices, but it reports a manufacturerCode of 4742 and an imageType of 65535.

Further, the file listed in bin for upgrading Tuya/Mihome devices to the custom firmware, 1286-0202-10993607-TH03Zz_v0123.zigbee, is not included in index.json.

I've downloaded index.json, and added the following entry:

    {
        "fileVersion": 268645896,
        "fileSize": 128146,
        "manufacturerCode": 4742,
        "imageType": 65535,
        "sha512": "81eaf9113318a887793f6c62bf8e4f07487c55248dcbcd7175c26c57514f556161007f706cd92ce3ab8e12999f2c3439a95b153833cdceeee4de054ad2fc6f31",
        "url": "https://github.com/pvvx/ZigbeeTLc/raw/refs/heads/master/bin/1286-0202-10993607-TH03Zz_v0123.zigbee"
    }

I modified my configuration.yaml as described in the first entry here, and confirmed that ZHA is reading it. However, even after restarting home assistant, no firmware update shows as available for the device. Can anyone suggest why?

@pvvx
Copy link
Owner Author

pvvx commented Dec 21, 2024

it reports a manufacturerCode of 4742 and an imageType of 65535.

"imageType": 514 (0x0202)

The transition from Tuya to other variants is not included in the general index file.
The reason is that 90% of Tuya devices have the same IDs for OTA. And only the user can specify which device the OTA firmware is suitable for.
The OTA Zigbee update menu will drop down several different devices, and users will, without thinking, select them all...
Koenkk/zigbee-OTA#620 (comment)
Koenkk/zigbee-OTA#620 (comment)
...

manufacturerCode of 4742

In this case it is already "SonOff" :)

Additional Zigbee OTA identifiers are already being introduced to help select the correct firmware and device, but there is no user interface or support in ZHA/Z2M yet.

@Arachnid
Copy link

Arachnid commented Dec 21, 2024

it reports a manufacturerCode of 4742 and an imageType of 65535.

"imageType": 514 (0x0202)

Sorry, what do you mean by this?

The transition from Tuya to other variants is not included in the general index file.
The reason is that 90% of Tuya devices have the same IDs for OTA. And only the user can specify which device the OTA firmware is suitable for.
The OTA Zigbee update menu will drop down several different devices, and users will, without thinking, select them all...
Koenkk/zigbee-OTA#620 (comment)
Koenkk/zigbee-OTA#620 (comment)

I see, that makes sense. Is there something wrong with my custom configuration that would cause it not to be recognized as an update for my device?

Edit: Ah, I see - I'm misunderstanding the file description; the firmware I've been selecting is for installing Sonoff firmware on a ZigbeeTlc flashed device.

Is there a documented process for zigbee OTA updating from the manufacturer firmware to ZigbeeTlc?

@pvvx
Copy link
Owner Author

pvvx commented Dec 21, 2024

Sorry, what do you mean by this?

It worked for me with this one

index.json:

[
    {
        "fileVersion": 278476295,
        "fileSize": 128146,
        "manufacturerCode": 4742,
        "imageType": 514,
        "sha512": "81eaf9113318a887793f6c62bf8e4f07487c55248dcbcd7175c26c57514f556161007f706cd92ce3ab8e12999f2c3439a95b153833cdceeee4de054ad2fc6f31",
"url": "https://github.com/pvvx/ZigbeeTLc/raw/refs/heads/master/bin/1286-0202-10993607-TH03Zz_v0123.zigbee"
    }
]

But there is no guarantee that the firmware in your device has the same IDs.

@Arachnid
Copy link

But there is no guarantee that the firmware in your device has the same IDs.

Mine reports a manufacturerCode of 4742 and an imageType of 65535 when I check the cluster config, but this configuration doesn't show up as an available update.

@pvvx
Copy link
Owner Author

pvvx commented Dec 21, 2024

ZHA does not have an interface that would show the device IDs to the user for OTA. IDs can only be found in the log when OTA occurs.

@Arachnid
Copy link

Arachnid commented Dec 21, 2024

Oh, I see. I had assumed that the values I retrieved from the ota cluster would match. Is there any way to get the relevant values if I don't already have an OTA to send?

Screenshot from 2024-12-21 13-59-08

@pvvx
Copy link
Owner Author

pvvx commented Dec 21, 2024

There is, but there is no interface.

"image_type_id" for the given query is always 0xffff.

@Arachnid
Copy link

Sorry, can you elaborate? How can I get the correct imageType value for my device?

@pvvx
Copy link
Owner Author

pvvx commented Dec 21, 2024

The device provides identifiers when requesting the Zigbee OTA function. See the Zigbee 3.0 specification...

ZHA does everything to make OTA as difficult as possible for the user.

@Arachnid
Copy link

I see. I tried to use ZHA Toolkit as documented in your first post to prompt an OTA update. However, I get the error "sl_Status.ZIGBEE_DELIVERY_FAILED". In the system logs I also see "Could not get ota object for ota_update_images, try again", and a parsing error from zigpy trying to deserialize a package with too-short data:

Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/zigpy/device.py", line 488, in packet_received
    hdr, args = endpoint.deserialize(packet.cluster_id, data)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/zigpy/endpoint.py", line 228, in deserialize
    return cluster.deserialize(data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/zigpy/zcl/__init__.py", line 290, in deserialize
    response, data = command.schema.deserialize(data)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/zigpy/types/struct.py", line 319, in deserialize
    value, data = field_type.deserialize(data)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/zigpy/types/basic.py", line 874, in deserialize
    item, data = cls._item_type.deserialize(data)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/zigpy/zcl/foundation.py", line 891, in deserialize
    r.direction, data = ReportingDirection.deserialize(data)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/zigpy/types/basic.py", line 199, in deserialize
    raise ValueError(f"Data is too short to contain {byte_size} bytes")
ValueError: Data is too short to contain 1 bytes

None of this leaves me any closer to figuring out how to get the device's image type, however. Any advice you have to offer would be much appreciated.

@pvvx
Copy link
Owner Author

pvvx commented Dec 21, 2024

No advice.
ZHA toolkit is not updated.
ZHA uses a new version and interfaces with zigpy.
Each new release of ZHA becomes more complex. It is possible that you will not find anything in the log in the new version of ZHA.

@Arachnid
Copy link

Edit: I sent the following action, and suddenly an update shows up as available:

action: zha.issue_zigbee_cluster_command
data:
  ieee: [mac]
  endpoint_id: 1
  cluster_id: 25
  cluster_type: out
  command: 0
  command_type: client
  args:
    - 0
    - 100

At a guess, the device simply hadn't checked for an update since I updated the file?

@pvvx
Copy link
Owner Author

pvvx commented Dec 21, 2024

At a guess, the device simply hadn't checked for an update since I updated the file?

The check only occurs when the HA is completely rebooted...
Next, it is advisable to call the update menu. This speeds up the search process.
image

@Arachnid
Copy link

Unfortunately, the update still won't apply - it times out trying to OTA, even with me pressing the button to wake the device up.

@pvvx
Copy link
Owner Author

pvvx commented Dec 22, 2024

OTA process interrupted?
My OTA took over an hour, with interruptions. I ran it several times and the update continued... The OTA program works very slowly on this device. And periodically stops due to timeout. Apparently these are the timeout settings in ZHA.
It's good that Zigbee OTA can be continued from its previous place with a new start.

@StancuFlorin
Copy link

StancuFlorin commented Dec 29, 2024

I have six Xiaomi sensors (model LYWSD03MMC-z) that I successfully updated via OTA using the configuration provided below. I am running Home Assistant version 2024.12.5 with ZHA.

zha:
  zigpy_config:
    ota:
      extra_providers:
        - type: ikea
        - type: z2m # Xiaomi LYWSD03MMC https://github.com/pvvx/ZigbeeTLc/issues/7#issuecomment-2365373959
          url: https://raw.githubusercontent.com/pvvx/ZigbeeTLc/refs/heads/master/bin/index.json
          manufacturer_ids: [0x1141]

I experienced some connectivity issues with one sensor. I retried the update 3-4 times, but it got stuck at 100% at one point. To resolve this, I moved the sensor closer to the controller, removed the battery for 5 seconds, and when it came back online, the update was successfully completed.

The initial firmware version on these sensors was 0x01203001, and after the update, it is now 0x01233001. The update process was quite fast, taking around 30 minutes, much quicker compared to some Ikea sensors, which took approximately 2 hours.

I am using the SONOFF Zigbee 3.0 USB Dongle Plus V2 as the controller with a bunch Ikea smart plugs around the house.

I also have another Xiaomi sensor, model LYWSD03MMC (without the -z), running firmware version 0x11003001. It's possible that I flashed a different firmware when converting it from Bluetooth to Zigbee. For this sensor, no update was available, and I’m still investigating the reason.

@pvvx
Copy link
Owner Author

pvvx commented Dec 29, 2024

I also have another Xiaomi sensor, model LYWSD03MMC (without the -z), running firmware version 0x11003001.

https://github.com/devbis/z03mmc

@StancuFlorin
Copy link

I also have another Xiaomi sensor, model LYWSD03MMC (without the -z), running firmware version 0x11003001.

https://github.com/devbis/z03mmc

There is any way to flash your firmware over the air?

@pvvx
Copy link
Owner Author

pvvx commented Dec 29, 2024

@StancuFlorin
Copy link

https://github.com/pvvx/ZigbeeTLc/tree/master/bin/devbis

https://github.com/pvvx/ATC_MiThermometer/tree/master/zigbee_ota Zigbee -> BLE

How can I manually trigger the update? I’ve added the firmware file db15-0203-79993001-Z03MMC_v0123.zigbee to the /config/zigpy_ota directory, but my devbis LYWSD03MMC device shows "Firmware is up to date" in ZHA. Until now the firmware status was unknown.

@pvvx
Copy link
Owner Author

pvvx commented Dec 29, 2024

Follow the ZHA documentation. Or ask the ZHA writers. There, with every new version, everything changes with OTA.

@pvvx
Copy link
Owner Author

pvvx commented Dec 29, 2024

My current version of the mess for ZHA OTA is this:

zha:
  zigpy_config:
    ota:
      enabled: true
      # Download the index over HTTPS
      z2m_remote_index: https://raw.githubusercontent.com/Koenkk/zigbee-OTA/master/index.json
      # Or, load the index offline
      z2m_local_index: /home/pvvx/zigpy_ota/index.json
      allow_advanced_ota_dir: I understand I can *destroy* my devices by enabling OTA
        updates from files. Some OTA updates can be mistakenly applied to the
        wrong device, breaking it. I am consciously using this at my own risk.
      advanced_ota_dir: /home/pvvx/zigpy_ota
      otau_directory: /home/pvvx/zigpy_ota
      path: /home/pvvx/zigpy_ota
      ikea_provider: false
      ledvance_provider: false
      salus_provider: false
      sonoff_provider: false
      inovelli_provider: false
      thirdreality_provider: false

I create a local index.json and put it in /home/pvvx/zigpy_ota/
After the update I change index.json to an empty one containing:

[
]

This has worked for the last few versions of ZHA.
Before that, in older versions of ZHA, it was different.
What will be required in the new version of ZHA is known only to the authors of ZHA.


Due to the fact that the Tuya firmwares does not adhere to the Zigbee OTA standards, "Koenkk/zigbee-OTA" has problems. How it will be solved is unknown. Many Tuya devices have the same names and the same IDs for OTA. But the devices themselves are different...

If you get an update announcement for Zigbee, do not rush to use it. First, make sure that this update is suitable for your device.
How to indicate that an update is definitely suitable for a specific device has no solution yet. Say thanks to Tuya.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

No branches or pull requests