Supervisor 22.03.2: Timeout while processing CodeNotary #3495

OhioSpyderman · 2022-03-09T15:35:55Z

Describe the issue you are experiencing

What is the used version of the Supervisor?

2022.03.2

What type of installation are you running?

Home Assistant Supervised

Which operating system are you running on?

Home Assistant Operating System

What is the version of your installed operating system?

Home Assistant OS 7.4

What version of Home Assistant Core is installed?

core-2022.3.3

Steps to reproduce the issue

Restart Home Assistant
2.
3.
...

Anything in the Supervisor logs that might be useful for us?

22-03-09 10:18:31 WARNING (MainThread) [supervisor.utils.codenotary] rpc error: code = Unknown desc = unexpected HTTP status code received from server: 500 (Internal Server Error); transport: received unexpected content-type "text/plain; charset=utf-8"

6 Times

Additional information

No response

stevenjev · 2022-03-09T16:04:39Z

Also seeing errors with CodeNotary after updating to Supervisor 2022.03.2 while running Debian 11 and Home Assistant Supervised with version core-2022.3.2:

[supervisor.utils.codenotary] Timeout while processing CodeNotary

iankaufmann · 2022-03-09T16:53:33Z

I also ran the supervisor update this morning and I'm now seeing this:

# ha core update
Error: Unknown error, see supervisor
# ha supervisor logs
22-03-09 10:44:55 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /info
22-03-09 10:44:55 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /host/info
22-03-09 10:44:55 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /store
22-03-09 10:44:55 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /core/info
22-03-09 10:44:55 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /supervisor/info
22-03-09 10:44:55 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /os/info
22-03-09 10:48:32 INFO (MainThread) [supervisor.core] Starting repair of Supervisor Environment
22-03-09 10:48:32 INFO (SyncWorker_1) [supervisor.docker] Prune stale containers
22-03-09 10:48:32 INFO (SyncWorker_1) [supervisor.docker] Prune stale images
22-03-09 10:48:36 INFO (SyncWorker_1) [supervisor.docker] Prune stale builds
22-03-09 10:48:36 INFO (SyncWorker_1) [supervisor.docker] Prune stale volumes
22-03-09 10:48:36 INFO (SyncWorker_1) [supervisor.docker] Prune stale networks
22-03-09 10:48:36 INFO (SyncWorker_1) [supervisor.docker] Fix stale container on hassio network
22-03-09 10:48:36 INFO (SyncWorker_1) [supervisor.docker] Fix stale container on host network
22-03-09 10:48:36 INFO (MainThread) [supervisor.addons] Found 1 add-ons to repair
22-03-09 10:48:36 INFO (MainThread) [supervisor.addons] Repairing for add-on: a0d7b954_aircast
22-03-09 10:48:36 INFO (SyncWorker_5) [supervisor.docker.interface] Downloading docker image ghcr.io/hassio-addons/aircast/amd64 with tag 3.2.1.
22-03-09 10:48:43 INFO (MainThread) [supervisor.supervisor] Repairing Supervisor 2022.03.2
22-03-09 10:48:43 INFO (MainThread) [supervisor.core] Finished repair of Supervisor Environment
22-03-09 10:48:57 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /available_updates
22-03-09 10:49:02 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /app/entrypoint.js
22-03-09 10:49:14 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /available_updates
22-03-09 10:49:19 INFO (MainThread) [supervisor.homeassistant.core] Updating Home Assistant to version 2022.3.3
22-03-09 10:49:19 INFO (SyncWorker_4) [supervisor.docker.interface] Updating image ghcr.io/home-assistant/qemux86-64-homeassistant:2022.3.2 to ghcr.io/home-assistant/qemux86-64-homeassistant:2022.3.3
22-03-09 10:49:19 INFO (SyncWorker_4) [supervisor.docker.interface] Downloading docker image ghcr.io/home-assistant/qemux86-64-homeassistant with tag 2022.3.3.
22-03-09 10:49:39 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /available_updates
22-03-09 10:49:45 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /app/entrypoint.js
22-03-09 10:49:55 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /info
22-03-09 10:49:55 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /host/info
22-03-09 10:49:55 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /store
22-03-09 10:49:55 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /core/info
22-03-09 10:49:55 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /supervisor/info
22-03-09 10:49:55 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /os/info
22-03-09 10:50:29 ERROR (MainThread) [supervisor.jobs] Unhandled exception:
Traceback (most recent call last):
  File "/usr/src/supervisor/supervisor/jobs/decorator.py", line 106, in wrapper
    return await self._method(*args, **kwargs)
  File "/usr/src/supervisor/supervisor/homeassistant/core.py", line 229, in update
    await _update(version)
  File "/usr/src/supervisor/supervisor/homeassistant/core.py", line 207, in _update
    await self.instance.update(
  File "/usr/src/supervisor/supervisor/utils/__init__.py", line 33, in wrap_api
    return await method(api, *args, **kwargs)
asyncio.exceptions.TimeoutError
22-03-09 10:50:29 WARNING (MainThread) [supervisor.utils.codenotary] rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 34.207.232.93:443: i/o timeout"

mwrben · 2022-03-09T17:02:59Z

I've seen the same issue after Supervisor updated to 2022.3.2 - I now can't update core to 2022.3.3
Log:-
22-03-09 16:38:37 INFO (SyncWorker_2) [supervisor.docker.interface] Updating image ghcr.io/home-assistant/generic-x86-64-homeassistant:2022.2.9 to ghcr.io/home-assistant/generic-x86-64-homeassistant:2022.3.3
22-03-09 16:38:37 INFO (SyncWorker_2) [supervisor.docker.interface] Downloading docker image ghcr.io/home-assistant/generic-x86-64-homeassistant with tag 2022.3.3.
22-03-09 16:39:33 ERROR (MainThread) [supervisor.jobs] Unhandled exception:
Traceback (most recent call last):
File "/usr/src/supervisor/supervisor/jobs/decorator.py", line 106, in wrapper
return await self._method(*args, **kwargs)
File "/usr/src/supervisor/supervisor/homeassistant/core.py", line 229, in update
await _update(version)
File "/usr/src/supervisor/supervisor/homeassistant/core.py", line 207, in _update
await self.instance.update(
File "/usr/src/supervisor/supervisor/utils/init.py", line 33, in wrap_api
return await method(api, *args, **kwargs)
asyncio.exceptions.TimeoutError
22-03-09 16:39:43 ERROR (MainThread) [supervisor.utils.codenotary] Timeout while processing CodeNotary
22-03-09 16:39:44 INFO (MainThread) [supervisor.homeassistant.core] Updating Home Assistant to version 2022.3.3
22-03-09 16:39:44 INFO (SyncWorker_6) [supervisor.docker.interface] Updating image ghcr.io/home-assistant/generic-x86-64-homeassistant:2022.2.9 to ghcr.io/home-assistant/generic-x86-64-homeassistant:2022.3.3
22-03-09 16:39:44 INFO (SyncWorker_6) [supervisor.docker.interface] Downloading docker image ghcr.io/home-assistant/generic-x86-64-homeassistant with tag 2022.3.3.

crhass · 2022-03-09T18:02:44Z

I have the same issue on rpi 3b+ following the supervisor update

Manage supervisor gives
Could not load the Supervisor panel!

Unable to restart home assistant from the settings page.

2022-03-09 18:02:33 WARNING (MainThread) [homeassistant.components.hassio] Can't read Supervisor data:
2022-03-09 18:02:33 ERROR (MainThread) [homeassistant.components.hassio.handler] /host/info return code 403
2022-03-09 18:02:33 ERROR (MainThread) [homeassistant.components.hassio.handler] /store return code 403
2022-03-09 18:02:33 ERROR (MainThread) [homeassistant.components.hassio.handler] /core/info return code 403
2022-03-09 18:02:33 ERROR (MainThread) [homeassistant.components.hassio.handler] /supervisor/info return code 403
2022-03-09 18:02:33 ERROR (MainThread) [homeassistant.components.hassio.handler] /os/info return code 403

Rep-Hueman · 2022-03-09T18:04:58Z

Logger: snitun.client.client_peer
Source: /usr/local/lib/python3.9/site-packages/snitun/client/client_peer.py:104
First occurred: 18:52:22 (1 occurrences)
Last logged: 18:52:22

Challenge/Response error with SniTun server (0 bytes read on a total of 32 expected bytes)

Logger: hass_nabucasa.remote
Source: /usr/local/lib/python3.9/site-packages/hass_nabucasa/remote.py:333
First occurred: 18:52:22 (1 occurrences)
Last logged: 18:52:22

Connection problem to snitun server

clajog · 2022-03-09T18:12:12Z

The same:
Hello,
I have problems with the upgrade to version 2022.3.3., currently 2022.3.1.
These error messages are in the log. I tried it with 2 different systems (RPi4, 4GB and RPi4, 8GB, both with Home Assistant OS 7.4, core-2022.3.1,supervisor-2022.03.2).
Restart core or Host --> same error
22-03-09 18:40:20 INFO (MainThread) [supervisor.homeassistant.module] Backup Home Assistant Core config folder done
22-03-09 18:40:20 INFO (MainThread) [supervisor.backups.manager] Creating partial backup with slug 1d3de046 completed
22-03-09 18:40:20 INFO (MainThread) [supervisor.homeassistant.core] Updating Home Assistant to version 2022.3.3
22-03-09 18:40:20 INFO (SyncWorker_0) [supervisor.docker.interface] Updating image ghcr.io/home-assistant/raspberrypi4-64-homeassistant:2022.3.1 to ghcr.io/home-assistant/raspberrypi4-64-homeassistant:2022.3.3
22-03-09 18:40:20 INFO (SyncWorker_0) [supervisor.docker.interface] Downloading docker image ghcr.io/home-assistant/raspberrypi4-64-homeassistant with tag 2022.3.3.
22-03-09 18:42:27 ERROR (MainThread) [supervisor.jobs] Unhandled exception:
Traceback (most recent call last):
File "/usr/src/supervisor/supervisor/jobs/decorator.py", line 106, in wrapper
return await self._method(*args, kwargs)
File "/usr/src/supervisor/supervisor/homeassistant/core.py", line 229, in update
await _update(version)
File "/usr/src/supervisor/supervisor/homeassistant/core.py", line 207, in _update
await self.instance.update(
File "/usr/src/supervisor/supervisor/utils/init.py", line 33, in wrap_api
return await method(api, *args, kwargs)
asyncio.exceptions.TimeoutError
22-03-09 18:42:28 WARNING (MainThread) [supervisor.utils.codenotary] rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 34.207.232.93:443: i/o timeout"
22-03-09 18:42:44 INFO (MainThread) [supervisor.backups.manager] Found 8 backup files
22-03-09 18:42:44 INFO (MainThread) [supervisor.backups.manager] Found 8 backup files

adamfeldman · 2022-03-09T18:18:47Z

I'm experiencing the reported issue as well. I'm wondering if this line from the top of the logs post-restart points more directly to the root problem.

22-03-09 12:03:20 WARNING (MainThread) [supervisor.plugins.manager] Can't update plugin multicast: 
22-03-09 12:03:20 WARNING (MainThread) [supervisor.utils.codenotary] rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 34.207.232.93:443: i/o timeout"
22-03-09 12:03:40 WARNING (MainThread) [supervisor.utils.codenotary] rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 34.207.232.93:443: i/o timeout"

vchaindz · 2022-03-09T18:48:02Z

Due to a very heavy load, the Codenotary CAS service was overwhelmed today and was down for less than 1 hour. Resources have been substantially upgraded now. Unfortunately, we also were required to do a DNS change. Therefore, some systems that still use cached DNS entries need clear the local DNS cache.
Sorry for the inconveniences, but all is back up

clajog · 2022-03-09T19:03:53Z

perfekt, it works. Thanks a lot

crhass · 2022-03-09T19:09:14Z

perfekt, it works. Thanks a lot

What did you do, I can't restart it or access SSH?

clajog · 2022-03-09T19:56:31Z

i use only ipconfig /flushdns in the commandline
https://www.heise.de/tipps-tricks/DNS-Cache-loeschen-4603703.html

iankaufmann · 2022-03-09T20:19:34Z

I rolled back to a proxmox snapshot of my whole VM (HASS OS 7.4) from this morning.

Then I tried once again to update Core to 2022.3.3

I'm back in the same boat where the Supervisor is all messed up. It won't load from the UI:

Trying to open any Add-on from the sidebar says "Unable to load the panel source: /api/hassio/app/entrypoint.js."

Logs are full of "Invalid token for access".

What do I need to do to get back to a working system?

# ha supervisor logs
22-03-09 12:17:31 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.SECURITY/ContextType.CORE
22-03-09 12:17:31 INFO (MainThread) [supervisor.resolution.check] System checks complete
22-03-09 12:17:31 INFO (MainThread) [supervisor.resolution.evaluate] Starting system evaluation with state CoreState.RUNNING
22-03-09 12:17:31 WARNING (MainThread) [supervisor.utils.codenotary] rpc error: code = Unavailable desc = unexpected HTTP status code received from server: 503 (Service Unavailable); transport: received unexpected content-type "text/plain; charset=utf-8"
22-03-09 12:17:31 INFO (MainThread) [supervisor.resolution.evaluate] System evaluation complete
22-03-09 12:17:31 INFO (MainThread) [supervisor.resolution.fixup] Starting system autofix at state CoreState.RUNNING
22-03-09 12:17:31 INFO (MainThread) [supervisor.resolution.fixup] System autofix complete
22-03-09 12:18:06 INFO (MainThread) [supervisor.homeassistant.api] Updated Home Assistant API token
22-03-09 12:18:26 INFO (MainThread) [supervisor.api.middleware.security] /supervisor/info access from cebe7a76_hassio_google_drive_backup
22-03-09 12:18:26 INFO (MainThread) [supervisor.api.middleware.security] /backups access from cebe7a76_hassio_google_drive_backup
22-03-09 12:48:12 INFO (MainThread) [supervisor.homeassistant.api] Updated Home Assistant API token
22-03-09 13:17:31 INFO (MainThread) [supervisor.resolution.check] Starting system checks with state CoreState.RUNNING
22-03-09 13:17:31 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.FREE_SPACE/ContextType.SYSTEM
22-03-09 13:17:31 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.PWNED/ContextType.ADDON
22-03-09 13:17:31 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.TRUST/ContextType.PLUGIN
22-03-09 13:17:33 WARNING (MainThread) [supervisor.utils.codenotary] rpc error: code = Unavailable desc = unexpected HTTP status code received from server: 503 (Service Unavailable); transport: received unexpected content-type "text/plain; charset=utf-8"
22-03-09 13:17:37 WARNING (MainThread) [supervisor.utils.codenotary] rpc error: code = Unavailable desc = unexpected HTTP status code received from server: 503 (Service Unavailable); transport: received unexpected content-type "text/plain; charset=utf-8"
22-03-09 13:17:39 WARNING (MainThread) [supervisor.utils.codenotary] rpc error: code = Unavailable desc = unexpected HTTP status code received from server: 503 (Service Unavailable); transport: received unexpected content-type "text/plain; charset=utf-8"
22-03-09 13:17:45 WARNING (MainThread) [supervisor.utils.codenotary] rpc error: code = Unavailable desc = unexpected HTTP status code received from server: 503 (Service Unavailable); transport: received unexpected content-type "text/plain; charset=utf-8"
22-03-09 13:17:49 WARNING (MainThread) [supervisor.utils.codenotary] rpc error: code = Unavailable desc = unexpected HTTP status code received from server: 503 (Service Unavailable); transport: received unexpected content-type "text/plain; charset=utf-8"
22-03-09 13:17:49 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.TRUST/ContextType.CORE
22-03-09 13:18:09 ERROR (MainThread) [supervisor.resolution.check] Error during processing IssueType.TRUST:
22-03-09 13:18:09 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.TRUST/ContextType.SUPERVISOR
22-03-09 13:18:09 WARNING (MainThread) [supervisor.utils.codenotary] rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 35.168.17.115:443: i/o timeout"
22-03-09 13:18:14 WARNING (MainThread) [supervisor.utils.codenotary] rpc error: code = Unavailable desc = unexpected HTTP status code received from server: 503 (Service Unavailable); transport: received unexpected content-type "text/plain; charset=utf-8"
22-03-09 13:18:14 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.SECURITY/ContextType.CORE
22-03-09 13:18:14 INFO (MainThread) [supervisor.resolution.check] System checks complete
22-03-09 13:18:14 INFO (MainThread) [supervisor.resolution.evaluate] Starting system evaluation with state CoreState.RUNNING
22-03-09 13:18:15 WARNING (MainThread) [supervisor.utils.codenotary] rpc error: code = Unavailable desc = unexpected HTTP status code received from server: 503 (Service Unavailable); transport: received unexpected content-type "text/plain; charset=utf-8"
22-03-09 13:18:16 INFO (MainThread) [supervisor.resolution.evaluate] System evaluation complete
22-03-09 13:18:16 INFO (MainThread) [supervisor.resolution.fixup] Starting system autofix at state CoreState.RUNNING
22-03-09 13:18:16 INFO (MainThread) [supervisor.resolution.fixup] System autofix complete
22-03-09 13:18:18 INFO (MainThread) [supervisor.homeassistant.api] Updated Home Assistant API token
22-03-09 13:18:27 INFO (MainThread) [supervisor.api.middleware.security] /supervisor/info access from cebe7a76_hassio_google_drive_backup
22-03-09 13:18:27 INFO (MainThread) [supervisor.api.middleware.security] /backups access from cebe7a76_hassio_google_drive_backup
22-03-09 13:18:29 INFO (MainThread) [supervisor.updater] Fetching update data from https://version.home-assistant.io/stable.json
22-03-09 13:18:33 WARNING (MainThread) [supervisor.utils.codenotary] rpc error: code = Unavailable desc = unexpected HTTP status code received from server: 503 (Service Unavailable); transport: received unexpected content-type "text/plain; charset=utf-8"
22-03-09 13:25:09 INFO (MainThread) [supervisor.host.info] Updating local host information
22-03-09 13:25:10 INFO (MainThread) [supervisor.host.services] Updating service information
22-03-09 13:25:10 INFO (MainThread) [supervisor.host.network] Updating local network information
22-03-09 13:25:10 INFO (MainThread) [supervisor.host.sound] Updating PulseAudio information
22-03-09 13:25:10 INFO (MainThread) [supervisor.host.manager] Host information reload completed
22-03-09 13:48:24 INFO (MainThread) [supervisor.homeassistant.api] Updated Home Assistant API token
22-03-09 14:07:41 INFO (SyncWorker_4) [supervisor.docker] Runing command 'python3 -m homeassistant -c /config --script check_config' on ghcr.io/home-assistant/qemux86-64-homeassistant
22-03-09 14:08:49 INFO (MainThread) [supervisor.homeassistant.core] Updating Home Assistant to version 2022.3.3
22-03-09 14:08:49 ERROR (MainThread) [supervisor.utils] Can't execute update while a task is in progress
22-03-09 14:08:51 INFO (MainThread) [supervisor.homeassistant.module] Update pulse/client.config: /data/tmp/homeassistant_pulse
22-03-09 14:08:51 ERROR (MainThread) [supervisor.utils] Can't execute run while a task is in progress
22-03-09 14:08:51 INFO (MainThread) [supervisor.homeassistant.core] Wait until Home Assistant is ready
22-03-09 14:08:56 INFO (MainThread) [supervisor.homeassistant.core] Detect a running Home Assistant instance
22-03-09 14:08:56 INFO (MainThread) [supervisor.homeassistant.core] Successfully started Home Assistant 2022.3.3
22-03-09 14:08:56 ERROR (MainThread) [supervisor.utils] Can't execute cleanup while a task is in progress
22-03-09 14:09:07 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /available_updates
22-03-09 14:09:09 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /app/entrypoint.js
22-03-09 14:09:16 INFO (MainThread) [supervisor.homeassistant.core] Home Assistant config is valid
22-03-09 14:09:19 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /app/entrypoint.js
22-03-09 14:09:27 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /app/entrypoint.js
22-03-09 14:09:42 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /app/entrypoint.js
22-03-09 14:09:47 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /available_updates
22-03-09 14:09:56 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /app/entrypoint.js
22-03-09 14:09:58 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /app/entrypoint.js
22-03-09 14:12:17 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /app/entrypoint.js
22-03-09 14:12:20 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /available_updates
22-03-09 14:12:27 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /app/entrypoint.js
22-03-09 14:12:29 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /available_updates
22-03-09 14:13:48 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /available_updates
22-03-09 14:13:50 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /app/entrypoint.js
22-03-09 14:13:51 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /info
22-03-09 14:13:51 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /host/info
22-03-09 14:13:51 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /store
22-03-09 14:13:51 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /core/info
22-03-09 14:13:51 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /supervisor/info
22-03-09 14:13:51 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /os/info
22-03-09 14:13:52 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /available_updates
22-03-09 14:14:11 ERROR (MainThread) [supervisor.api.middleware.security] Invalid token for access /app/entrypoint.js
22-03-09 14:15:32 INFO (MainThread) [supervisor.plugins.dns] Restarting CoreDNS plugin
22-03-09 14:15:32 INFO (SyncWorker_1) [supervisor.docker.interface] Restarting ghcr.io/home-assistant/amd64-hassio-dns

BebeMischa · 2022-03-09T20:24:03Z

on Debian Bullseye with Supervised HA this saved my day:

sudo systemd-resolve --flush-caches
sudo /etc/init.d/networking restart

davidrustingha · 2022-03-09T20:38:25Z

@BebeMischa did that fix not being able to update for you? For me it threw a diffrent error

22-03-09 21:35:26 ERROR (MainThread) [supervisor.jobs] Unhandled exception: 
Traceback (most recent call last):
  File "/usr/src/supervisor/supervisor/jobs/decorator.py", line 106, in wrapper
    return await self._method(*args, **kwargs)
  File "/usr/src/supervisor/supervisor/homeassistant/core.py", line 229, in update
    await _update(version)
  File "/usr/src/supervisor/supervisor/homeassistant/core.py", line 207, in _update
    await self.instance.update(
  File "/usr/src/supervisor/supervisor/utils/__init__.py", line 33, in wrap_api
    return await method(api, *args, **kwargs)
asyncio.exceptions.TimeoutError
22-03-09 21:35:36 ERROR (MainThread) [supervisor.utils.codenotary] Timeout while processing CodeNotary

gentigit · 2022-03-09T20:39:22Z

On hassio it started working after:

ha dns update
ha dns restart

might have been a coincidence too, and the upstream DNS happened to update around the same time.

BebeMischa · 2022-03-09T20:40:11Z

@BebeMischa did that fix not being able to update for you? For me it threw a diffrent error

I was able to update, but after that my system was marked unsupported. That issue is now gone.

iankaufmann · 2022-03-09T21:03:31Z

After ha dns restart (it was already up to date) I am getting the same as @davidrustingha

22-03-09 14:50:38 INFO (MainThread) [supervisor.homeassistant.core] Updating Home Assistant to version 2022.3.3
22-03-09 14:50:38 INFO (SyncWorker_0) [supervisor.docker.interface] Updating image ghcr.io/home-assistant/qemux86-64-homeassistant:2022.3.2 to ghcr.io/home-assistant/qemux86-64-homeassistant:2022.3.3
22-03-09 14:50:38 INFO (SyncWorker_0) [supervisor.docker.interface] Downloading docker image ghcr.io/home-assistant/qemux86-64-homeassistant with tag 2022.3.3.
22-03-09 14:50:58 ERROR (MainThread) [supervisor.jobs] Unhandled exception:
Traceback (most recent call last):
  File "/usr/src/supervisor/supervisor/jobs/decorator.py", line 106, in wrapper
    return await self._method(*args, **kwargs)
  File "/usr/src/supervisor/supervisor/homeassistant/core.py", line 229, in update
    await _update(version)
  File "/usr/src/supervisor/supervisor/homeassistant/core.py", line 207, in _update
    await self.instance.update(
  File "/usr/src/supervisor/supervisor/utils/__init__.py", line 33, in wrap_api
    return await method(api, *args, **kwargs)
asyncio.exceptions.TimeoutError
22-03-09 14:50:58 ERROR (MainThread) [supervisor.utils.codenotary] Timeout while processing CodeNotary

Does anybody know what DNS issue is causing this?

From my firewall, cas.codenotary.com pings to 35.168.17.115, which appears correct according to MXToolbox, but I don't really know for sure what I'm looking for here.

That was the only domain that I saw my home assistant box looking for in my DNS logs.

jackyaz · 2022-03-09T21:05:50Z

After ha dns restart (it was already up to date) I am getting the same as @davidrustingha

22-03-09 14:50:38 INFO (MainThread) [supervisor.homeassistant.core] Updating Home Assistant to version 2022.3.3
22-03-09 14:50:38 INFO (SyncWorker_0) [supervisor.docker.interface] Updating image ghcr.io/home-assistant/qemux86-64-homeassistant:2022.3.2 to ghcr.io/home-assistant/qemux86-64-homeassistant:2022.3.3
22-03-09 14:50:38 INFO (SyncWorker_0) [supervisor.docker.interface] Downloading docker image ghcr.io/home-assistant/qemux86-64-homeassistant with tag 2022.3.3.
22-03-09 14:50:58 ERROR (MainThread) [supervisor.jobs] Unhandled exception:
Traceback (most recent call last):
  File "/usr/src/supervisor/supervisor/jobs/decorator.py", line 106, in wrapper
    return await self._method(*args, **kwargs)
  File "/usr/src/supervisor/supervisor/homeassistant/core.py", line 229, in update
    await _update(version)
  File "/usr/src/supervisor/supervisor/homeassistant/core.py", line 207, in _update
    await self.instance.update(
  File "/usr/src/supervisor/supervisor/utils/__init__.py", line 33, in wrap_api
    return await method(api, *args, **kwargs)
asyncio.exceptions.TimeoutError
22-03-09 14:50:58 ERROR (MainThread) [supervisor.utils.codenotary] Timeout while processing CodeNotary

Does anybody know what DNS issue is causing this?

From my firewall, cas.codenotary.com pings to 35.168.17.115, which appears correct according to MXToolbox, but I don't really know for sure what I'm looking for here.

That was the only domain that I saw my home assistant box looking for in my DNS logs.

I'm seeing this too, I opened a separate issue as I'm not sure if its the same as this one or not
#3496

TomBeech85 · 2022-03-09T21:18:19Z

I was experiencing the OPs issue. I updated my DNS and now am getting this error:

22-03-09 21:15:46 INFO (SyncWorker_9) [supervisor.docker.interface] Updating image ghcr.io/home-assistant/qemux86-64-homeassistant:2022.2.9 to ghcr.io/home-assistant/qemux86-64-homeassistant:2022.3.3
22-03-09 21:15:46 INFO (SyncWorker_9) [supervisor.docker.interface] Downloading docker image ghcr.io/home-assistant/qemux86-64-homeassistant with tag 2022.3.3.
22-03-09 21:15:47 ERROR (MainThread) [supervisor.resolution.check] Error during processing IssueType.TRUST:
22-03-09 21:15:47 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.SECURITY/ContextType.CORE
22-03-09 21:15:47 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.TRUST/ContextType.CORE
22-03-09 21:15:47 ERROR (MainThread) [supervisor.utils] Can't execute check_trust while a task is in progress
22-03-09 21:15:47 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.PWNED/ContextType.ADDON
22-03-09 21:15:47 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.FREE_SPACE/ContextType.SYSTEM
22-03-09 21:15:47 INFO (MainThread) [supervisor.resolution.module] Create new suggestion SuggestionType.CREATE_FULL_BACKUP - ContextType.SYSTEM / None
22-03-09 21:15:47 INFO (MainThread) [supervisor.resolution.checks.base] Run check for IssueType.TRUST/ContextType.SUPERVISOR
22-03-09 21:15:49 ERROR (MainThread) [supervisor.utils.codenotary] Can't parse CodeNotary output: b'' - Expecting value: line 1 column 1 (char 0)
22-03-09 21:15:53 ERROR (MainThread) [supervisor.utils.codenotary] Can't parse CodeNotary output: b'' - Expecting value: line 1 column 1 (char 0)

adrianmihalko · 2022-03-09T21:23:25Z

I am trying to update from 2022.2.6 to 2022.3.3, but I am getting error:

22-03-09 22:18:43 ERROR (MainThread) [supervisor.jobs] Unhandled exception: 
Traceback (most recent call last):
  File "/usr/src/supervisor/supervisor/jobs/decorator.py", line 106, in wrapper
    return await self._method(*args, **kwargs)
  File "/usr/src/supervisor/supervisor/homeassistant/core.py", line 229, in update
    await _update(version)
  File "/usr/src/supervisor/supervisor/homeassistant/core.py", line 207, in _update
    await self.instance.update(
  File "/usr/src/supervisor/supervisor/utils/__init__.py", line 33, in wrap_api
    return await method(api, *args, **kwargs)
asyncio.exceptions.TimeoutError
22-03-09 22:18:43 WARNING (MainThread) [supervisor.utils.codenotary] rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 35.168.17.115:443: i/o timeout"
22-03-09 22:18:45 ERROR (MainThread) [supervisor.utils.codenotary] Timeout while processing CodeNotary
22-03-09 22:19:06 ERROR (MainThread) [supervisor.utils.codenotary] Timeout while processing CodeNotary
22-03-09 22:19:06 INFO (MainThread) [supervisor.resolution.evaluate] System evaluation complete
22-03-09 22:19:06 INFO (MainThread) [supervisor.resolution.fixup] Starting system autofix at state CoreState.RUNNING
22-03-09 22:19:06 INFO (MainThread) [supervisor.resolution.fixup] System autofix complete

Supervisor is already the latest: supervisor-2022.03.2

gentigit · 2022-03-09T21:32:26Z

Hmm, I was getting the same IP before so maybe codenotary is still getting hammered and I happened to get through.
Seeing this 35.168.17.115:443: i/o timeout
Also, ha loves to use 1.1.1.1:853 for DNS.

nodomain · 2022-03-09T21:35:22Z

Can you please reopen https://github.com/codenotary/cas/issues/263 @frenck - I now also get the rpc errors again due to timeout.
Thx

BebeMischa · 2022-03-09T21:51:18Z

jurekmff · 2022-03-09T22:32:02Z

For me its still the same issue, trying to update from core-2022.2.9 to core-2022.3.3 with supervisor-2022.03.2 and Home Assistant OS 7.4.

22-03-09 23:29:02 INFO (SyncWorker_2) [supervisor.docker.interface] Downloading docker image ghcr.io/home-assistant/raspberrypi4-64-homeassistant with tag 2022.3.3.
22-03-09 23:29:23 ERROR (MainThread) [supervisor.jobs] Unhandled exception: 
Traceback (most recent call last):
  File "/usr/src/supervisor/supervisor/jobs/decorator.py", line 106, in wrapper
    return await self._method(*args, **kwargs)
  File "/usr/src/supervisor/supervisor/homeassistant/core.py", line 229, in update
    await _update(version)
  File "/usr/src/supervisor/supervisor/homeassistant/core.py", line 207, in _update
    await self.instance.update(
  File "/usr/src/supervisor/supervisor/utils/__init__.py", line 33, in wrap_api
    return await method(api, *args, **kwargs)
asyncio.exceptions.TimeoutError
22-03-09 23:29:33 ERROR (MainThread) [supervisor.utils.codenotary] Timeout while processing CodeNotary

frenck · 2022-03-10T14:59:10Z

@ossconsulting I've marked your comment as offtopic. While extensive, it isn't part of this issue. If you want to discuss those ideas or policies, please use our community forums instead.
Additionally, it is fine not to like the managed system nature of the Home Assistant Supervisor enabled systems (like OS and Supervised installs) brings. In that case, please consider one of the other installation methods Home Assistant offers.

Thanks 👍

ossconsulting · 2022-03-10T15:05:19Z

@frenck It was not meant as 'i dont like it' it was meant as suggestion and I feel it is related. I suggest you take your time to read it and take it under consideration. Or not. Whatever feels appropriate to you.

Whatever you decide: Thank you for mitigating the issue.

frenck · 2022-03-10T15:06:16Z

@ossconsulting I did read it, it is full of incorrect assumptions. That said, this is not the place for that discussion as I said above as well, which was based on actually reading your posting. Thanks 👍

frenck · 2022-03-10T15:12:06Z

@marc115uk I have hidden your comment as off-topic as that is not related to this issue.

sermayoral · 2022-03-10T15:24:38Z

@marc115uk see #3497

jacla242 · 2022-03-10T15:40:49Z

New install on raspberry4 worked like a charm now, thanks a lot!

boomam · 2022-03-10T15:42:22Z

Whatever was done, appears to have resolved the issue for me.
Thanks!

crhass · 2022-03-10T17:02:25Z

Still not working again here since I updated the supervisor yesterday.

Core is 2022.3.1

Add-ons page shows
Could not load the Supervisor panel!

ha supervisor reload
command completed successfully
ha supervisor update
No update available 2022.03.3

Log keeps showing

2022-03-10 16:56:11 ERROR (MainThread) [homeassistant.components.system_health] Error fetching info
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/components/system_health/init.py", line 68, in get_integration_info
data = await registration.info_callback(hass)
File "/usr/src/homeassistant/homeassistant/components/homeassistant/system_health.py", line 17, in system_health_info
info = await system_info.async_get_system_info(hass)
File "/usr/src/homeassistant/homeassistant/helpers/system_info.py", line 57, in async_get_system_info
info_object["supervisor"] = info.get("supervisor")
AttributeError: 'NoneType' object has no attribute 'get'
2022-03-10 16:56:11 ERROR (MainThread) [homeassistant.components.hassio.handler] /available_updates return code 403
2022-03-10 16:56:11 ERROR (MainThread) [homeassistant.components.hassio] Failed to to call /available_updates -
2022-03-10 16:56:44 ERROR (MainThread) [homeassistant.components.hassio.handler] /info return code 403
2022-03-10 16:56:44 WARNING (MainThread) [homeassistant.components.hassio] Can't read Supervisor data:
2022-03-10 16:56:44 ERROR (MainThread) [homeassistant.components.hassio.handler] /host/info return code 403
2022-03-10 16:56:44 ERROR (MainThread) [homeassistant.components.hassio.handler] /store return code 403
2022-03-10 16:56:44 ERROR (MainThread) [homeassistant.components.hassio.handler] /core/info return code 403
2022-03-10 16:56:44 ERROR (MainThread) [homeassistant.components.hassio.handler] /supervisor/info return code 403
2022-03-10 16:56:44 ERROR (MainThread) [homeassistant.components.hassio.handler] /os/info return code 403

frenck · 2022-03-10T17:15:12Z

@crhass those logs are not from the Supervisor, but Home Assistant.
Please check the Supervisor logs.

crhass · 2022-03-10T17:21:57Z

@crhass those logs are not from the Supervisor, but Home Assistant. Please check the Supervisor logs.

I am getting
Could not load the Supervisor panel!
In add-ons and supervisor

Do I need to get the logs some other way?
CLI?

lianmengwa · 2022-03-10T17:23:32Z

Everything goes back to work. Thank you guys

frenck · 2022-03-10T17:26:04Z

@crhass ha supervisor logs or, in case you think the Supervisor is really unreachable, you could try visiting http://address-of-your-system:4357 to visit the Home Assistant system Observer

crhass · 2022-03-10T17:30:09Z

All green at http://address-of-your-system:4357
Logs show
supervisor.api.middleware.security invalid token for access

frenck · 2022-03-10T17:36:34Z

@crhass It seems like the Home Assistant Core hasn't got the right credentials to access the Supervisor API, which is kinda odd (and I don't believe related to this issue actually). A full system reboot should take care of that, as the Supervisor provides access to Home Assistant Core during startup.

sdotter · 2022-03-10T17:38:28Z

A full reboot worked for me! Thanks for fixing this issue!!

crhass · 2022-03-10T17:49:49Z

@crhass It seems like the Home Assistant Core hasn't got the right credentials to access the Supervisor API, which is kinda odd (and I don't believe related to this issue actually). A full system reboot should take care of that, as the Supervisor provides access to Home Assistant Core during startup.

Full reboot done

Still seeing the invalid token errors

Just before the errors it says
Updated home assistant API token in green

frenck · 2022-03-10T17:52:14Z

@crhass Sorry to say, but I have no idea at this point, and above all, if all is green it is not related to this issue. I would recommend joining our Discord chat and see if people around there can help you debug the issue further.

crhass · 2022-03-10T17:57:25Z

Thanks for your time, I am sorted
I updated the core from CLI and the front-end updated
I can now see the add-ons page again

Not sure if it was the front-end or the core that sorted it.

frenck · 2022-03-10T18:00:24Z

@crhass Awesome! Glad you got it sorted out 👍

Tiihokatti · 2022-03-11T07:29:00Z

Seems like the issue totally borked my system. HA OS on rpi4 with database moved to RAM and logs cut down to minimum.
Issues started right away after I went and updated supervisor during the service outage.

Supervisor panel is still broken and I can't even access the add-on page. This means I can't go activate SSH so I could try the fix the issue myself by flushing cache etc. Even the Deconz UI for my Raspbee II was similarly broken for some reason, although automations and all Zigbee devices are still working fine.

Rebooting the system via UI is also not working for some reason (I doubt it successfully restarts HA in 0,1 seconds). So I did a hard reset by unplugging the power cable, unfortunately it didn't help.
At this point the only solution I can think of is to wipe the OS and try installing a backup. But it seemed like people were having huge issues with that too...

crhass · 2022-03-11T17:06:11Z

@Tiihokatti this sound very like the issue I had, are you on the latest version of home assistant 2022.3.3.
If not, plug in a usb keyboard to your pi and type

ha core update

Probably don't need a monitor if you type carefully..

Tiihokatti · 2022-03-11T17:44:32Z

Going blind with a Logitech K400+ on the rpi4 and waiting extremely painful 10 minutes got it working. Thank god, because I couldn't find that stupid mini-hdmi adapter anywhere...

ha su repair
ha core rebuild
ha host reboot

Snawork · 2022-03-12T13:28:38Z

Home assistant supervised installed on debian11 versione of HA is 2022.3.3, I have a problem with the supervisor, and tried to reinstall Debian and then HA and load a backup, but unfortunately it all crashed " you are running an unsupported version "error message.

sator86 · 2022-03-22T17:42:42Z

Going blind with a Logitech K400+ on the rpi4 and waiting extremely painful 10 minutes got it working. Thank god, because I couldn't find that stupid mini-hdmi adapter anywhere...

ha su repair ha core rebuild ha host reboot

works for me,
thank you

frenck · 2022-03-22T18:24:47Z

The original problem in this issue has been resolved and deployed. I'll go ahead and close up this issue. If you are experiencing issues at this point, it is most likely not related to this issue anymore and thus you'd best create a new issue.

Thanks! 👍

../Frenck

OhioSpyderman added the bug label Mar 9, 2022

frenck mentioned this issue Mar 9, 2022

Update to 2022.3.3 fails (Unknown error, see supervisor) home-assistant/core#67928

Closed

This comment was marked as off-topic.

Sign in to view

mekaneck mentioned this issue Mar 10, 2022

Supervisor 22.03.2: Can't validate backup #3499

Closed

frenck closed this as completed Mar 22, 2022

home-assistant locked and limited conversation to collaborators Mar 22, 2022

Supervisor 22.03.2: Timeout while processing CodeNotary #3495

Supervisor 22.03.2: Timeout while processing CodeNotary #3495

Comments

OhioSpyderman commented Mar 9, 2022

Describe the issue you are experiencing

What is the used version of the Supervisor?

What type of installation are you running?

Which operating system are you running on?

What is the version of your installed operating system?

What version of Home Assistant Core is installed?

Steps to reproduce the issue

Anything in the Supervisor logs that might be useful for us?

Additional information

stevenjev commented Mar 9, 2022

iankaufmann commented Mar 9, 2022

mwrben commented Mar 9, 2022

crhass commented Mar 9, 2022 • edited Loading

Rep-Hueman commented Mar 9, 2022

clajog commented Mar 9, 2022

adamfeldman commented Mar 9, 2022

vchaindz commented Mar 9, 2022

clajog commented Mar 9, 2022

crhass commented Mar 9, 2022 • edited Loading

clajog commented Mar 9, 2022

iankaufmann commented Mar 9, 2022

BebeMischa commented Mar 9, 2022 • edited Loading

davidrustingha commented Mar 9, 2022

gentigit commented Mar 9, 2022

BebeMischa commented Mar 9, 2022

iankaufmann commented Mar 9, 2022

jackyaz commented Mar 9, 2022

TomBeech85 commented Mar 9, 2022 • edited Loading

adrianmihalko commented Mar 9, 2022

This comment was marked as off-topic.

gentigit commented Mar 9, 2022

nodomain commented Mar 9, 2022

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

BebeMischa commented Mar 9, 2022

jurekmff commented Mar 9, 2022 • edited Loading

This comment was marked as off-topic.

frenck commented Mar 10, 2022 • edited Loading

ossconsulting commented Mar 10, 2022

frenck commented Mar 10, 2022 • edited Loading

This comment was marked as off-topic.

frenck commented Mar 10, 2022 • edited Loading

sermayoral commented Mar 10, 2022

jacla242 commented Mar 10, 2022

boomam commented Mar 10, 2022

crhass commented Mar 10, 2022

frenck commented Mar 10, 2022

crhass commented Mar 10, 2022

lianmengwa commented Mar 10, 2022

frenck commented Mar 10, 2022

crhass commented Mar 10, 2022

frenck commented Mar 10, 2022

sdotter commented Mar 10, 2022

crhass commented Mar 10, 2022

frenck commented Mar 10, 2022

crhass commented Mar 10, 2022

frenck commented Mar 10, 2022

Tiihokatti commented Mar 11, 2022

crhass commented Mar 11, 2022

Tiihokatti commented Mar 11, 2022 • edited Loading

Snawork commented Mar 12, 2022

sator86 commented Mar 22, 2022

frenck commented Mar 22, 2022

crhass commented Mar 9, 2022 •

edited

Loading

crhass commented Mar 9, 2022 •

edited

Loading

BebeMischa commented Mar 9, 2022 •

edited

Loading

TomBeech85 commented Mar 9, 2022 •

edited

Loading

jurekmff commented Mar 9, 2022 •

edited

Loading

frenck commented Mar 10, 2022 •

edited

Loading

frenck commented Mar 10, 2022 •

edited

Loading

frenck commented Mar 10, 2022 •

edited

Loading

Tiihokatti commented Mar 11, 2022 •

edited

Loading