Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't blow up if config entries have unhashable unique IDs #109966

Merged
merged 5 commits into from
Feb 8, 2024

Conversation

emontnemery
Copy link
Contributor

@emontnemery emontnemery commented Feb 8, 2024

Proposed change

Don't blow up if config entries have unhashable unique IDs, instead use a string representation of the unique_id as hash

This is a more forgiving approach than #109959 and #109962

We're guaranteed that:

  • The unique_id was possible to serialize / deserialize to JSON
  • It's possible to test the existing unique_id for equality

Hence, I don't think it's possible that there are collisions in the stringified representation

In a follow-up, we should prevent creating new config entries with non-string unique IDs.

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New integration (thank you!)
  • New feature (which adds functionality to an existing integration)
  • Deprecation (breaking change to happen in the future)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Additional information

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • I have followed the perfect PR recommendations
  • The code has been formatted using Ruff (ruff format homeassistant tests)
  • Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
  • Untested files have been added to .coveragerc.

To help with the load of incoming pull requests:

@emontnemery emontnemery requested a review from a team as a code owner February 8, 2024 09:14
@home-assistant home-assistant bot added cla-signed core small-pr PRs with less than 30 lines. labels Feb 8, 2024
@emontnemery emontnemery added this to the 2024.2.1 milestone Feb 8, 2024
@emontnemery emontnemery marked this pull request as draft February 8, 2024 09:19
@emontnemery emontnemery force-pushed the config_entries_stringify_unhashable_unique_id branch from 5a50ee4 to 1734ee5 Compare February 8, 2024 09:22
@emontnemery emontnemery marked this pull request as ready for review February 8, 2024 09:22
# Guard against integrations using unhashable unique_id
# In HA Core 2024.9, we should remove the guard and instead fail
if not isinstance(entry.unique_id, Hashable):
unique_id_hash = str(entry.unique_id) # type: ignore[unreachable]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should not load the config entry if the data is incorrect. I don't think we should try to be smart about it.

Copy link
Contributor Author

@emontnemery emontnemery Feb 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't agree, what's the point of that?
In my opinion, integration authors should be given some time to update the invalid unique id. It's not our users' fault that we do not validate the data and then suddenly made assumptions about the unvalidated data.

Copy link
Member

@bdraco bdraco Feb 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By making it a string we are relying on the custom component author who created it with a non string to be able to clean it up later.

I guess we are stuck with an assumption that they aren't actually using the unique id to store a list of data that they need to set up the integration. I'm not sure if it's a reasonable position to accommodate that use case as it's pretty far away from how the API is expected to be used.. This was wrong, the change is only to the index

The safest course would be to revert the original PR and give custom component authors a few months to fix their integrations but unfortunately there is already other work built on top of it There is probably too much risk to do that.

For most cases it probably works fine to stringify it and load it. It's the most user friendly option at least. Given how the frequency of issue reports this may be the way to go.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only semi-realistic risk I can come up with that seems possible is they have stored some object in the unique id that has a string representation that isn't unique but is somehow JSON serializable. That would be some type of impressive abuse of the system so it seems like a very contrived case and acceptable risk.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As example:
#109908 (comment)

Also, this is also still an issue for entity unique_ids (I found that out a few betas ago when I accidentally broke tellduslive)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@joostlek I don't understand your comment, your example does not point to a case where there's a risk of duplicates

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was responding to bdraco's comment about string representation and JSON serializable

Copy link
Contributor Author

@emontnemery emontnemery Feb 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should not load the config entry if the data is incorrect. I don't think we should try to be smart about it.

We discussed this in a meeting and there were more of us in favor of allowing the config entries to load (as the PR does) than to not load the config entry (as Martin suggests).

@home-assistant
Copy link

home-assistant bot commented Feb 8, 2024

Please take a look at the requested changes, and use the Ready for review button when you are done, thanks 👍

Learn more about our pull request process.

@home-assistant home-assistant bot marked this pull request as draft February 8, 2024 10:56
@emontnemery emontnemery marked this pull request as ready for review February 8, 2024 10:57
@bdraco
Copy link
Member

bdraco commented Feb 8, 2024

2024-02-08 08:17:00.034 ERROR (MainThread) [homeassistant.config_entries] Config entry 'HASS Bridge XH:21066' from integration homekit has an invalid unique_id '['homekit1']', please create a bug report at https://github.com/home-assistant/core/issues?q=is%3Aopen+is%3Aissue+label%3A%22integration%3A+homekit%22

Testing this by manually mutating some to lists seems to work as expected

@bdraco
Copy link
Member

bdraco commented Feb 8, 2024

Verified I can

  • restart
  • reload config entry
  • delete config entry
  • have the config flow update the unique id

@bdraco
Copy link
Member

bdraco commented Feb 8, 2024

I'm kinda surprised list as unique ids didn't blow up in other ways before from testing

2024-02-08 08:21:49.535 ERROR (MainThread) [homeassistant.components.apple_tv] Failed to connect
Traceback (most recent call last):
  File "/Users/bdraco/home-assistant/homeassistant/components/apple_tv/__init__.py", line 235, in connect_once
    await self._connect(conf, raise_missing_credentials)
  File "/Users/bdraco/home-assistant/homeassistant/components/apple_tv/__init__.py", line 345, in _connect
    self._async_setup_device_registry()
  File "/Users/bdraco/home-assistant/homeassistant/components/apple_tv/__init__.py", line 358, in _async_setup_device_registry
    ATTR_IDENTIFIERS: {(DOMAIN, self.config_entry.unique_id)},
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: unhashable type: 'list'

@bdraco
Copy link
Member

bdraco commented Feb 8, 2024

Verified all cases where the dict is accessed have the workaround

Copy link
Member

@bdraco bdraco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but this comment needs to be sorted before merging

@emontnemery emontnemery dismissed MartinHjelmare’s stale review February 8, 2024 14:38

We discussed this PR in a meeting and there were more of us in favor of allowing the config entries to load (as the PR does) than to not load the config entry (as Martin suggests).

@emontnemery emontnemery merged commit 2f15053 into dev Feb 8, 2024
52 of 53 checks passed
@emontnemery emontnemery deleted the config_entries_stringify_unhashable_unique_id branch February 8, 2024 14:39
frenck pushed a commit that referenced this pull request Feb 9, 2024
* Don't blow up if config entries have unhashable unique IDs

* Add test

* Add comment on when we remove the guard

* Don't stringify hashable non string unique_id
@frenck frenck mentioned this pull request Feb 9, 2024
@github-actions github-actions bot locked and limited conversation to collaborators Feb 10, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Can't upgrade from 2024.1.5 to 2024.2 2024.2.0 not starting for unknown reason
5 participants