GB invalid token

[Hampo]

Describe the bug
After setting up a GB account following the new JWT steps, it errors after a HA reboot

To Reproduce
Steps to reproduce the behavior:

1. Configure a GB account
2. Reboot HA
3. Initialisation will fail
4. See error

Expected behavior
It to work

Screenshots
N/A

Version of the component
v0.6.2

Logging

Logger: lghorizon.lghorizon_api
Source: /usr/local/lib/python3.12/site-packages/backoff/_common.py:120
First occurred: 16:43:10 (19 occurrences)
Last logged: 20:39:52

    Giving up _authorize(...) after 3 tries (lghorizon.exceptions.LGHorizonApiConnectionError: Invalid token)
    Giving up connect(...) after 11 tries (lghorizon.exceptions.LGHorizonApiConnectionError: Invalid token)

Additional context
Add any other context about the problem here.

[ColinRobbins]

I am seeing this too. Some restarts work, but not all.
You'll need to delete the integration, and re-add. Your entities will remain the same.

The following in my DEBUG logs reveals what is happening...

2024-10-14 13:02:42.249 INFO (Thread-113 (_thread_main)) [lghorizon.lghorizon_api] Executing API call to https://spark-prod-gb.gnp.cloud.virgintvgo.......
2024-10-14 13:02:42.453 DEBUG (Thread-113 (_thread_main)) [lghorizon.lghorizon_api] Authorizing via refresh
2024-10-14 13:02:42.989 DEBUG (Thread-113 (_thread_main)) [lghorizon.lghorizon_api] Authorization succeeded
2024-10-14 13:02:42.991 INFO (Thread-113 (_thread_main)) [lghorizon.lghorizon_api] Backing off _do_api_call(...) for 0.6s (lghorizon.exceptions.LGHorizonApiConnectionError: Unable to call https://spark-prod-gb.gnp.cloud.virgintvgo.virginmedia.com/......
2024-10-14 13:02:43.618 INFO (Thread-113 (_thread_main)) [lghorizon.lghorizon_api] Executing API call to https://spark-prod-gb.gnp.cloud.virgintvgo.virginmedia.com/eng......
2024-10-14 13:02:43.775 DEBUG (Thread-113 (_thread_main)) [lghorizon.lghorizon_api] Result API call: {'imageVersion': 'mRh9TA', 'eventId'.......

When the box has been switched off, and then turns on, it gets a "401-Unauthorise error" from the API. So it reconnects using the refresh token, and this generates a new refresh token - but we are not catching the new token, and updating the config.

I am testing a fix in my development environment.

If the test goes well, I will make a PR tomorrow evening.

[ColinRobbins]

My "fix" didn't work, as it generated a new error.

[homeassistant.helpers.frame] Detected that custom integration 'lghorizon' calls hass.config_entries.async_update_entry from a thread other than the event loop, which may cause Home Assistant to crash or data to corrupt. For more information, see https://developers.home-assistant.io/docs/asyncio_thread_safety/#hassconfig_entriesasync_update_entry at custom_components/lghorizon/media_player.py, line 154: self.hass.config_entries.async_update_entry(self.entry, data=new_data), please report it to the author of the 'lghorizon' custom integration

Going to have to do some reading of the HA docs on how to avoid that.

[ColinRobbins]

I am struggling to find a solution to this, my python skills are not good enough.

To solve the issue, I need to call

            self.hass.config_entries.async_update_entry(
                self.entry, data=new_data
            )

However, this generates the error

[homeassistant.helpers.frame] Detected that custom integration 'lghorizon' calls hass.config_entries.async_update_entry from a thread other than the event loop, which may cause Home Assistant to crash or data to corrupt. For more information, see https://developers.home-assistant.io/docs/asyncio_thread_safety/#hassconfig_entriesasync_update_entry at custom_components/lghorizon/media_player.py, line 154: self.hass.config_entries.async_update_entry(self.entry, data=new_data), please report it to the author of the 'lghorizon' custom integration

so I have read the HA docs, and tried

            self.hass.add_job(
                self.hass.config_entries.async_update_entry(
                    self.entry, data=new_data
                )
            )

same error. Then tried

            self.hass.add_job(
                self.hass.config_entries.async_update_entry,
                self.entry, 
                data=new_data
            )

gives an error that data is undefined (which it is, is an attribute of config_entries.async_update_entry, not add_job).
Some docs suggest using a lambda function, but I don't really understand them!

Finding another HA doc, I then tried moving it to a separate function and adding @callable... but that still gives the thread error.
Now trying the msp1974 suggestions in https://community.home-assistant.io/t/calling-back-to-update-the-config-entry-from-a-thread/731938. This is effectively a combination of my last two attempts. If not I'll try a lambda.

Each try, requires leaving my environment running for 5-6 hours until the virgin timeout occurs, and triggers the error.

... frustrating...

[Shivy011]

Interestingly, only one of my boxes is showing as 'available'. The other one is stuck at unavailable

[ColinRobbins]

Interestingly, only one of my boxes is showing as 'available'. The other one is stuck at unavailable

Sorry, I've only got one box, so can't help there.
The code looks like it should support two boxes.

Silly question - are both boxes switched on?

Is there anything in the logs?

[Hampo]

When it worked, both my boxes were functional

[ColinRobbins]

When it worked, both my boxes were functional

I suggest we fix this issue first, then see where we have got to with multiple boxes.
If after this fix, you still get issues with multiple boxes, please open a new issue.
We can then keep this issue focused on the invalid token bit.

[ColinRobbins]

Eureka, I think I've found a way round the threading issue.
It's worked once in my dev environment.
I'll leave it testing a bit longer, then create a pull request tomorrow if all goes well.

[ChrisThomas42]

If it helps I have a "spare" Pi that I can put HA onto for testing?

[ColinRobbins]

The two pull requests above should resolve this issue.

In 24hr testing, I've seen two new tokens come through, and the stored configuration details updated.
A restart then worked just fine.

[ColinRobbins]

@ChrisThomas42

If it helps I have a "spare" Pi that I can put HA onto for testing?

Thanks - let’s see how the fix goes. I now in that “it works for me” situation - now need to see if it works for everyone else. If there are further issues, I may need more data points from other people.

[Shivy011]

Sorry all, noob here. Should the cookie refresh issue be fixed in an update. Struggling to receive any update via hacs?

[Hampo]

Sorry all, noob here. Should the cookie refresh issue be fixed in an update. Struggling to receive any update via hacs?

If you're having the same issue, a future update containing Colin's changes should fix it. Not sure on when that would be, waiting on the changes being pulled to the main version.

[gunionbag]

Hi Hampo,

You said in your first post above that:

"After setting up a GB account following the new JWT steps, it errors after a HA reboot"

What are the new JWT steps you are referring to when setting up a GB account? Is there a link you could provide with instructions on how to do this please?

Regards,

Matthew

[Hampo]

@gunionbag probably worth holding off until the next update which fixes this issue, but the configuration steps are in the README: https://github.com/Sholofly/lghorizon/blob/dev/README.md#configuration-for-virgin-gb

[gunionbag]

@Hampo Thanks for getting back to me so quickly.

I was trying to retrieve the JWT token yesterday as per those instructions but was unable to. So I wasn't sure if the bug you had reported was responsible for that or whether I wasn't doing something right when trying to retrieve the token. I originally tried to get the token using the extension in Chrome, but that didn't work, neither did Edge. But I was able to retrieve the token using Firefox.

As you suggested, I'll hold off until the next update before trying this again. But at least now I know how to get the token.

Thanks for your help.

[ColinRobbins]

As a temporary workaround until @Sholofly is able to integrate the fix into the official HACS version, if anyone using Hass OS want to try the fix, the following install process worked for me.

The instructions assume you have already downloaded a version of LG Horizon via HACS.

Use at your own risk - getting a command wrong could destabilise your HA installation.

Take a FULL HA backup first.

1. Login to the home assistant linux console. I use the Terminal and SSH add-on.

2. Run the following commands...

cd
mkdir temp
cd temp
git clone https://github.com/ColinRobbins/lghorizon.git
cd lghorizon
git checkout patch-1
cd ..
git clone https://github.com/ColinRobbins/lghorizon-python.git
cd lghorizon-python
git checkout patch-1
cd
cd config/custom_components/lghorizon/
cp ~/temp/lghorizon/custom_components/lghorizon/*.py .
cp -r ~/temp/lghorizon-python/lghorizon/ .
sed -i "s#^from lghorizon#from .lghorizon#" *.py

(The last sed line is crucial - get the syntax slightly wrong and it may fail silently, but not work).

3. Return to HA web interface and delete the LG Horizon integration from the HA integrations page
4. Restart HA
5. Get a new JWT token as per the Readme
6. Re-add the LG Horizon integration.

This worked for me, but no guarantees it will work for you.

If you are not familiar with Linux, and not sure what the commands above do, then I strongly urge waiting until the official release via HACS.

Backoff

If all goes wrong, you should be able to backoff the changes by going to HACS, finding the LG Horizon entry, and select the "redownload" option. This should reinstate the HACS version (if I have understood how HACS works correctly).

When the official HACS update to LG Horizon is made, it should download automatically and overwrite these changes.

Disclaimers

• If you are not confident with your linux abilities, you are best to wait for the HACS update.
• This worked for me without breaking anything, cannot be certain it will work for you.
• Getting one of the commands slightly wrong could break your HA installation.

[gunionbag]

@ColinRobbins Thanks Colin, I'll give it a try and post back my results. After you enter your LG Horizon login details and token, how long should it take for the login process to finish, assuming it works?

[ColinRobbins]

@gunionbag a few seconds.
If it fails, for some reason the timeout is set to 10 minutes.

[ChrisThomas42]

@gunionbag In my experience of the current configuration, when you have all the details correct it's a matter of seconds before it's successful. That is, after a HA restart it takes me a few minutes to get the new token via Firefox and enter all details and I'm back up and running.

If it takes a bit longer I'd suggest you check all the details and especially that you have all of the token. Which I didn't do when I tried to get the tokens when @ColinRobbins found this method.

[gunionbag]

@ChrisThomas42 Just to confirm, that is now working. The issue I had was that I wasn't selecting "flutter._WEB_SECURE_STORAGE_refreshToken" from the dropdown at the top of the JWT extension, as per the attached screenshot in case that helps anyone else.

@ChrisThomas42 and @ColinRobbins - Thank you both for your help, much appreciated.

[gunionbag]

Apologies, I should have also thanked @Sholofly for developing this integration in the first place - thanks.

[Sholofly]

No problem @gunionbag ;) I've released a new version. Please keep me posted on the results!

[xtruss]

Big thank you to Sholofly for this new version - Its installed and working perfectly, great job..

[Sholofly]

Thanks @xtruss but the credits are mostly for @ColinRobbins since he did the important part of the job!

[Shivy011]

Thanks very much to everyone involved, @Sholofly @ColinRobbins

Query - after a HA restart I assume I need to update the JWT token right?
Currently working, haven't tried a restart yet

[ColinRobbins]

@Shivy011, the whole point of the fix is the token should survive an HA restart.

The token has a 2 month lifetime, but is refreshed every time HA restarts or the Virgin box is restarted.

By grabbing the token from a browser, we are effectively hijacking the web session and the token used by the web browser. My experience is every time you re-load the web page, it refreshes the token without the need to logon again. This is also the mechanism used by the My Virgin GO app, which seems to stay logged on for a long time. Occasionally I need to re-authenticate the My Virgin GO app - not quite sure of the trigger - there could be some, other kind of timeout lurking. Time will tell. But is should work for several months, with HA restarts, without issue.