ONLY 2023.9.0 Overkiz breaks some unique_id [fixed on 2023.9.1]

[Quentame]

Note

Issue concerns only 2023.9.0
Fixed on 2023.9.1 (see #99950 (comment))

We advice skipping 2023.9.0 if not already updated.

It reopens #97705 as the fix consists on reverting a dependency bump that fixes that issue.

The problem

Having my Overkiz devices set up before 2023.9 with official core integration and changes their entity_id from about sensor.io_2429113_2_temperature, which got some protocol ID plus its type, to sensor.radiateur_bureau_temperature, to simply the device name plus its type (🇫🇷 radiateur bureau --> 🇬🇧 office heater).

So now that I am on 2023.9.0b0, my changed entities are not working anymore, as the integration duplicated entities that their entity_id is now sensor.radiateur_bureau_io_2429113_2_temperature, device name + protocol ID + type then.

Indeed, why this integration entity names & ids contain protocol ID in the first place ? (protocol is IO)
I always thought this should not be the case, and only device name + type

Screenchot:

Note: the climate entity is not affected.

Caused by #95238 ?

What version of Home Assistant Core has the issue?

core-2023.9.0b0

What was the last working version of Home Assistant Core?

core-2023.8.4

What type of installation are you running?

Home Assistant Container

Integration causing the issue

overkiz

Link to integration documentation on our website

https://www.home-assistant.io/integrations/overkiz/

Diagnostics information

overkiz-392925a41b4580113109bbeef69971ea-Radiateur-2e3b6b6300c5797400657d53181e0436.json.txt

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

[home-assistant]

Hey there @iMicknl, @vlebourl, @tetienne, @nyroDev, mind taking a look at this issue as it has been labeled with an integration (overkiz) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of overkiz can trigger bot actions by commenting:

• @home-assistant close Closes the issue.
• @home-assistant rename Awesome new title Renames the issue.
• @home-assistant reopen Reopen the issue.
• @home-assistant unassign overkiz Removes the current integration label and assignees on the issue, add the integration domain after the command.

_{^{(message by CodeOwnersMention)}}

---

overkiz documentation
overkiz source
_{^{(message by IssueLinks)}}

[Quentame]

At frist glance, look similar to #94492, but the issue is too old for me having the same as I did update to almost all HA version

[nyroDev]

@Quentame I updated my dev branch to be synced with current dev branch.
I even removed my config directory to start over.
And after reinit of my HASS and my Overkiz configuration, I have correctt entity name like sensor.pompe_a_chaleur_temp_rdc_temperature

What should I do to reproduce your problem?

[Quentame]

What should I do to reproduce your problem?

To reproduce:

• go to 2023.8.x branch
• init the integration
• (rename some entity ID) --> unneeded I think
• go to 2023.9.x branch

[Quentame]

From the lib: it seems caused by a move to Python 3.11 StrEnum
iMicknl/python-overkiz-api#923 (comment)

[iMicknl]

@Quentame I won’t be able to fix this issue before beta release due to holidays. Perhaps we should revert the PR and I can work on this when I am back?

[Quentame]

@Quentame I won’t be able to fix this issue before beta release due to holidays. Perhaps we should revert the PR and I can work on this when I am back?

Yep, I can do the revert if you want.
The PR to revert is #97916 ?

[iMicknl]

@Quentame yes!

[fetzerch]

Sorry, I didn't see the new comments. I worked on a fix and created a PR for it #99765. But the revert is safer for sure.

I'm relatively new to home assistant, is it worth to mention this issue somewhere in the 2023.9 release notes? I have not yet found out how to remove the newly created entities. UI says 'entity not available' and the delete button is greyed out.

[Quentame]

As 2023.9.0 went live with this breaking change, there is no reason to fix that anymore ... (see #99741 (review))

So, I think we are good to revert the revert #99742 and let it go like that.

Otherwise, 2023.9.0 is making a BC for users updating and then a second BC in 2023.9.1 !
People updating directly to 2023.9.1+ won't have the issue, but then what ? Should we prevent certain people from updating ?
Letting this BC permits to only make one BC in 2023.9.0

I don't think there is a log in the release note but it should ! (home-assistant/home-assistant.io#28811)

[Quentame]

That said, the issue will only hurts people that customized entity_id I think

[emontnemery]

The issue will hurt everyone who upgraded to 2023.9.0, because they will now have duplicated overkiz entities, with the old entities' state set to "UNAVAILABLE". The duplicates added by 2023.9.0 will have a new entity ID meaning any automation etc. referencing them will no longer work.

To properly fix this mess I think the following is needed:

• Decide if the library changes should be adjusted for backwards compatibility
  • For backwards compatibility, a possibility would be have the library use custom StrEnum class like this:

    from enum import StrEnum
    
    class CompabilityStrEnum(StrEnum):
        TEST = "test"
    
        def __str__(self) -> str:
            """Override StrEnum.__str__ for compability."""
            return f"{self.__class__.__name__}.{self.name}"
    
    print(f"{CompabilityStrEnum.TEST}")

  • Without the backwards compatibility, i.e. the library changes are not reverted, rewrite the unique id as in Keep overkiz unique ids stable for pyoverkiz >= 1.10.1 #99765
• Add cleanup code in overkiz async_setup_entry which searches for duplicated entities added in 2023.9.0 and removes them

[Delta-Charlie-DEV]

Hi, I found the issue by looking for the problem. I can confirm than since today and 2023.9.0 upgrade, I'm in this exact scenario (duplicate overkiz IDs and unavailable devices/entities)

What do you suggest ? Should I use search/replace function everywhere I'm thinking about or wait for a patch ?

[Quentame]

@iMicknl @fetzerch @nyroDev @emontnemery @Delta-Charlie-DEV

Fixed in 2023.9.1 (#99950 (comment))

You don't need to change anything from the state of 2023.8.x

Going from <= 2023.8.x to >= 2023.9.1 won't notice any issue.

[linuxstony]

Now that the Integration duplicated most of the entities with _2 and the "normal" ones are unavailable - what would be the best course of action to reset it to the defaults again?
Or should I rename all entities manually?

[iMicknl]

We are working on #101024 that will remove al unavailable entities.

Otherwise, if you remove your integration entry and add your hub again, it will remove all duplicate entities as well.

[linuxstony]

I appreciate the answer - will do.

Good luck and patience with the fixing :)

[iMicknl]

The only thing that will be hard to fix is the id of your entities. Due to the Python 3.11 update, it added duplicate entities with a _2 suffix. Our PR will remove the original entities, but won't change the HA ID of the new entities. There is no good option here unfortunately:

If we remove the _2 suffix, we break the automations / interface for everyone that has just changed to the new suffix.