failed with exception: authentication attempt failed with 400

[GitGrezly]

Since recently i do get the following error message when executing garmin-backup:

2024-02-08 19:46:13,524 [INFO] backing up formats: json_summary, json_details, gpx, tcx, fit
Enter password:
2024-02-08 19:46:17,176 [INFO] using 'curl_cffi' to create HTTP sessions that impersonate web browser 'chrome110' ...
2024-02-08 19:46:17,176 [INFO] authenticating user ...
2024-02-08 19:46:17,176 [INFO] passing login credentials ...
2024-02-08 19:46:17,509 [ERROR] failed with exception: authentication attempt failed with 400: <?xml version="1.0" encoding="UTF-8" standalone="yes"?><error>com.garmin.sso.portal.service.ww.exception.InvalidReCaptchaException</error><errorText>Recaptcha token is null/empty</errorText></ErrorResponse>
`

[TerryGamon]

same problem here

[flyingflo]

A CAPTCHA was added on the login page. The current login flow doesn't suffice anymore :(

[gvb1234]

Agree with @flyingflo . For me, the solution was to start using garth. https://github.com/cyberjunky/python-garminconnect has a detailed example.py with lots of different API calls...

[flyingflo]

Sounds interesting. I'll try this as well.

After playing around, I found that after a successful login in a browser, we need

• The Authorization header, and
• the JWT_FGP cookie.

If I copy these from my browser, and put it into GarminClient.session, the API calls succeed.

[Egregius]

Same problem here. @gvb1234 What or how did you change the script to use garth?

[flyingflo]

garth looks very promising.

[gvb1234]

@Egregius garmin_download_garth.txt updates fit files in subdirectory fit/, so, basically, the same functionality as

garmin-backup --backup-dir=fit/ --format=fit

It is a small modification from example.py at python-garminconnect...

[Egregius]

OK, that was easy for the fit files but I'm mostly interested in the json and gpx files.

[Egregius]

All of the sudden it's working again. Don't think I changed anything... Strange.

[flyingflo]

OK, that was easy for the fit files but I'm mostly interested in the json and gpx files.

My draft above should do this well. Still have to make it ready, though.

All of the sudden it's working again. Don't think I changed anything... Strange.

While playing around with that yesterday, I found that the login doesn't always require a CAPTCHA. Sometimes it seams to trust us more and lets us sign in without the captcha.

[petergardfjall]

As I mentioned in the PR my reservations regarding garth remain. I don't feel comfortable using a library that reaches out to an S3 bucket to grab credentials. Whose are they? What happens when the owner suddenly removes the bucket? What are the legal implications of using them?

I would prefer another way.

[flyingflo]

If I understand correctly, in garth's login procedure they mimic the garmin smartphone app. Therefore, they need the oauth consumer secret of the app. Thus, it is the app's credential.

This (dirty) trick allows them to retrieve a long-lived (1 year or so) token without a captcha and access the 'connectapi'. The token can be stored and reused again and again, and refreshed at times. As in the smartphone app.

Whenever the consumer secret changes, the bucket needs to be updated. And of course, login only works as long as it is there.

Compared to the trick above , which takes

• the Authorization header, and
• the JWT_FGP cookie.

from the browser after a login and therefore mimics the web app, by taking it's secrets.
This way, we only get a token for about one hour, and thus have to repeat the extraction very often. But it doesn't require additional secrets (as the browser can't keep them anyways).

In the current release version, this tool mimics the behaviour of the website, which works great, until they came up with the captcha.

Basically, I think the root issue is, that garmin makes it hard and harder for us to grab our own data, by locking 3rd party tools out of their APIs. Sadly, without using a trick, there is no access to our data.

[flyingflo]

Sorry for repeating all that. I just scrolled through the old issues and saw that this was already discussed.

[ryeguard]

@flyingflo,
Could you explain a little further how you get the trick of copying Authorization header and JWT_FGP cookie to work?

I attempted to modify the code a bit, but am getting 401 from _login() function here:

garminexport/garminexport/garminclient.py

Line 191 in b298da8

raise ValueError(f'authentication attempt failed with {resp.status_code}: {resp.text}')

[flyingflo]

   def _authenticate(self):
        """
        Authenticates using a Garmin Connect username and password.

        The procedure has changed over the years. A good approach for figuring
        it out is to use the browser development tools to trace all requests
        following a sign-in.
        """
        #cj = browser_cookie3.firefox(domain_name='connect.garmin.com')
        #self.session.cookies.update(cj)
        log.info("authenticating user ...")

        token_type = 'Bearer'
        self.session.headers.update(
            {
                'Authorization': f'{token_type} {self.token}',
                'Di-Backend': 'connectapi.garmin.com',
        # This header appears to be needed on subsequent session requests or we
        # end up with a 402 response from Garmin.
                'NK': 'NT'
            })
        self.session.cookies.update({'JWT_FGP': self.jwt_fgp})

Instead of username and password, I passed these two values to GarminClient and patched the method above.
I used requests, not curl_cffi.

[seb2020]

Any news on this issue ?

[ryeguard]

@flyingflo Thanks for the reply! I finally spared some time to try it out and it works for me too.

I made a PR (#115), let's see what @petergardfjall thinks :)

@seb2020 you can try it out on my fork if you'd like: https://github.com/ryeguard/garminexport/tree/sr/add-token-auth

[seb2020]

@flyingflo Thanks for the reply! I finally spared some time to try it out and it works for me too.

I made a PR (#115), let's see what @petergardfjall thinks :)

@seb2020 you can try it out on my fork if you'd like: https://github.com/ryeguard/garminexport/tree/sr/add-token-auth

It's working with your version !

[matin]

@petergardfjall maybe it's helpful if I clarify a bit ...

You're emulating a web browser. Garth emulates the Connect mobile app. That's the main difference.

In terms of the OAuth consumer keys, they're stored in clear text in the APK.

I intentionally structured Garth in a way for anyone to use their own keys or hard code the ones from the Connect app.

Here's an example:

import garth.sso

garth.sso.OAUTH_CONSUMER = {
    "consumer_key": "...",
    "consumer_secret": "...",
}

If you hard code the keys, Garth will use the hard coded keys--instead of fetching from S3.

It's been years since Garmin updated the OAuth keys in the Android app. It's unlikely that they'll change them anytime soon. I store them in S3 to give me the ability to update them (just in case) without requiring everyone to upgrade the library.

This explanation isn't an attempt to sway you in one direction or another. I just want to clarify how Garth works and provide an example of how to hard code the keys.

From a personal standpoint, I have MFA enabled on my account. The main reason Garth works the way it does is to make MFA easier by creating a long-lived access token. I primarily run Garth from Google Colab, so saving a long-lived access token for repeat use is important to me.

I hope that provides more context.

[matin]

Btw, if you're completely against using the OAuth consumer key, you should take a look at Garth version 0.2.9.

Garth previously used a hybrid approach of app emulation + web scraping that didn't require the OAuth consumer key.

I just tested it out multiple times, and it doesn't run into captcha issues and creates a access token that's valid for two hours. It does this without using curl_cffi or cloudscraper.

It supports MFA and the ability to configure the domain to garmin.cn for use in China.

In other words, Garth 0.2.9 would solve your immediate issues without the need for the OAuth consumer keys.

Garth 0.2.9 is still on PyPi if you want to test it out: https://pypi.org/project/garth/0.2.9/

[Fingel]

I looked into this a bit myself, and @matin is correct. I think this is a case of Garmin just bending to the Oauth spec or at least whatever implementation of it they are using, which requires a consumer id and secret for the Android app. Likely they just bundle them in the app. I suppose they could change the values in an app update, but these are per-client keys, not per-user. In which case they could just be updated in Garth's s3 bucket again.