The local API has changed

[tonybfox]

It looks like google have changed the api as I'm getting 403 Forbidden for anything other than /setup/eureka_info and that now return a public_key

[rithvikvibhu]

Thanks for letting me know! Will take a look at this in a few days. Hopefully, it's just an auth system that's easy to replicate.

[stboch]

I was digging in a bit to see if I can figure out what is going on... it seems that google has moved local communication to port 8443. I tried to do a little MitM but it seems that google is doing some application cert validation. (Either that or something else I did is wrong) They also changed it to require a client ssl for authentication.

[klimov-gett]

@stboch i think you're on point regarding application certificate:

i figure :8443 was google's port for SSL so i did a GET to https://192.168.1.219:8443/setup/bluetooth/status (since auto-connecting to bt speakers is what i do with this api)
as a result i received "Error: Peer certificate cannot be authenticated with given CA certificates"

after disabling certifcate validation i got 401, which reads "similar to 403, but in this case authorization is possible"

[cmorche]

What's odd for me is that this is still all working on 8008, and I'm even part of the preview program... This is the first time learning of the local API, so I guess I shouldn't bother spending too much time mucking about.

[ludeeus]

It looks like the change has rolled back, every endpoint I have now tested returns the expected result 🎉

[raddacle]

It looks like the change has rolled back, every endpoint I have now tested returns the expected result 🎉

You're right! Though, I had to reboot my GH devices for it to revert

[jamieshaw]

I've been seeing this flip flop between available and unavailable for the past few days. Not entirely sure what's triggering it…

[Leatherface75]

I think they are preparing some sorts of authorization for higher security and maybe because of closing down NEST API and move it to the Cloud instead.

[rithvikvibhu]

So I finally got my hands on a GH and some updates.

First, the simple way of logging network traffic (with PacketCapture) didn't work because it's a different port and other problems.
Tried setting up proxies with burp and mitmproxy, but it wasn't possible to forward other ports with ProxyDroid (even with root). There is also SSL pinning, but Frida was able to bypass that.
Finally, I set up a hotspot and started to capture with Wireshark, but during all this, my GH reverted back and I didn't have a chance to capture anything :/ Must have been because of one of the reboots.
So yay that it's working right now, but auth may be back at any time. We'll just have to wait till then, I guess.

Simultaneously, I was going through the latest decompiled app and found some interesting bits. Most of the requests had an extra header cast-local-authorization-token. This is likely what we need to get. Unfortunately, the code is obfuscated and I couldn't find what this field should contain or how it is obtained. Will keep trying.

There are also more details on /setup/eureka_info than the docs, including a local-authorization-token-hash or something similar. Again, I didn't get to try this because it reverted back to not requiring any token.

Also, not sure since when, but there's a lot of Nest libraries now. Including Weave stuff (maybe https://openweave.io, haven't really checked it out yet).

That's it for now. If someone knows about the token or has any more info, feel free to comment here. I'm leaving this issue open.

[jkpe]

@rithvikvibhu All of my Google Home Mini in the UK are in a state where they return a 403, if I can help in debugging this let me know

curl -I http://bedroom-ghome.lan:8008/setup/eureka_info
HTTP/1.1 403 Forbidden
Content-Length:0

[rithvikvibhu]

Hey @6a61636b. That's interesting. /setup/eureka_info is the only one everyone could access when other endpoints returned 403.
Can you try /setup/bluetooth/status?

[jkpe]

curl -I http://bedroom-ghome.lan:8008/setup/bluetooth/status
HTTP/1.1 403 Forbidden
Content-Length:0

According to the Google Home app it is running System firmware version: 156414, Cast firmware version: 1.40.156414

[thorleifjacobsen]

I can confirm the issue here in Norway on both a Chromecast Ultra and Chromecast 3 and the old Chromecast (not key fob). They all this morning returned 403. After a reboot they started working again.While they were at 403 I could not see any http data going through wireshark when using google home app. Everything went through 8443 and 8009.

I have the pcap file from my chromecast while it had 403 error on google home app. Android -> chromecast ultra configuring and rebooting once if that helps. Would like to send it privately if you need it.

Current Cast Version where API Works
"build_version":"156414","cast_build_revision":"1.40.156414"

[Sennevds]

curl -I http://bedroom-ghome.lan:8008/setup/bluetooth/status
HTTP/1.1 403 Forbidden
Content-Length:0

According to the Google Home app it is running System firmware version: 156414, Cast firmware version: 1.40.156414

You need to use the small i.
So curl -i http://bedroom-ghome.lan:8008/setup/bluetooth/status

[thorleifjacobsen]

You need to use the small i.
So curl -i http://bedroom-ghome.lan:8008/setup/bluetooth/status

Captial i is to get the "headers only" lowercase is to include the HTTP response. He just want to see the headers (403 forbidden) so capital i is perfect.

[Sennevds]

You need to use the small i.
So curl -i http://bedroom-ghome.lan:8008/setup/bluetooth/status

Captial i is to get the "headers only" lowercase is to include the HTTP response. He just want to see the headers (403 forbidden) so capital i is perfect.

That's true but still if I use:
`PC51:~$ curl -I http://192.168.0.5:8008/setup/bluetooth/status

HTTP/1.1 405 Method Not Allowed
Access-Control-Allow-Headers:Content-Type
Cache-Control:no-cache
Content-Length:0`

and if use:
`curl -i http://192.168.0.5:8008/setup/bluetooth/status

HTTP/1.1 200 OK
Access-Control-Allow-Headers:Content-Type
Cache-Control:no-cache
Content-Length:114
Content-Type:application/json
{"audio_mode":1,"connected_devices":[],"connecting_devices":[],"discovery_enabled":false,"scanning_enabled":false}`

So definitely a difference

[thorleifjacobsen]

True, you need to add -X GET when using -I to specify method. Whereas -i somehow adds that manually. Wierd.

Well spotted though

[alertedsnake]

Nothing weird about this. The help explains:

-i, --include       Include protocol response headers in the output
-I, --head          Show document info only

A GET request is the default with curl, and using -i will modify that request.
-I means to make a HTTP HEAD request. So the 405 response is not surprising if the endpoint only responds to GET

[rithvikvibhu]

@thorleifjaocbsen, how did you capture the packets? And where did you see the requests over ports 8009 and 8443? I'm asking this because normally http proxies won't be able to see them.

I'd love to take a look at that pcap file. Please send it to [email protected], if possible.

[thorleifjacobsen]

I have a AP connected to a switch, then I have that port mirrored to my computer (lan adapter) and did dump all traffic going from and to the AP. My andorid is connected to WiFi the Chromecast is connected via ethernet.

10.0.0.9 is my Android Device
10.0.0.8 is my Chromecast Device

I'm connecting using Google Home -> Configuring my Chromecast -> Changing some settings -> Rebooting it -> done pcap.

Filter: ip.addr == 10.0.0.8 or ip.addr == 10.0.0.9
Filter 2: (ip.src == 10.0.0.9 and ip.dst == 10.0.0.8) or (ip.dst == 10.0.0.9 and ip.src == 10.0.0.8)

Not sure if it helps. Sent you the file now :)

[rithvikvibhu]

@thorleifjaocbsen Thanks for the file! There's not much to find there. 8443 uses TLS so no luck there. Any idea what those TCP packets on port 8009 are?

My GH is again back to sending 403 so I'll try some stuff with the android app.

[poma]

Mine are back to 403 too except the ones that have Preview Program turned on, looks like those have different firmware update schedule

[rithvikvibhu]

Hey everyone,

I got the API working over port 8443!
I'll keep this short.

Turns out the extra header I mentioned in the previous comment was needed.
I kept digging in the app, but couldn't find what the value should be. Tried another method with frida and got the token value. Tried this value with postman, and it worked. /setup/bluetooth/status works and others should too, haven't tested.
As of now, I have no idea how often or when this token changes or expires, and from where this token is generated. After looking into this, I'll mostly post a guide or script to get it. For those curious, the token is part of Google's new Homegraph and you might be able to find it with root.

[rithvikvibhu]

Okay, created a gist with info:
https://gist.github.com/rithvikvibhu/1a0f4937af957ef6a78453e3be482c1f

Please try it out and see if it works.

[rithvikvibhu]

Also, I can't test a few cases. People with multiple rooted phones (with different Google accounts signed in), can you post the first and last few characters of the token? I'm curious if it's different for each home member.

[klimov-gett]

@rithvikvibhu thanks a bunch, tried the Frida method - works like a charm.
However, i think it's worth adding that you need (or at least i need) to pass ��**--insecure**� option to curl for the request to go through.
Reason for that being that the certificate installed on my Home Mini has no corresponding root CA / cert chain installed on my device's trust store and hence my Home is treated as a potential thread and curl aborts the request unless that option is passed.

[rithvikvibhu]

Hah yes, thanks @klimov-gett. I'll add that bit.

[magicse]

Google home mini has 2 locales (as I saw in sources) and it could be different. One for chromecast part and other for "assistant" part. But I cant find yet how to set locale for assistant. If someone find how change locale for assistant not chromecats through GHlocalAPI please inform.

[rithvikvibhu]

@magicse I agree with most of it, but the problem is that, in this case, we don't have access to the server to enable extensions and the client (app) is obfuscated, debugging isn't easy.

That endpoint is for the public, which is google.home.graph.v1.HomeGraphApiService. We need google.internal.home.foyer.v1.StructureService to get the complete home graph.

[rithvikvibhu]

I've updated the proto file for getHomeGraph. My previous comment has examples how to run it and also extract the local auth token.

Now, this is only based on my home graph. Since I don't have a lot of devices, there might be some fields missing. I don't know what happens when the definition has missing fields. Please try it out and comment here if it works with your graphs or not.

[Leatherface75]

Works fine here

[XoMEX]

Works for me too. I have a home mini, a chromecast and two third-party vacuums. All show up. The chromecast and the home mini have a local auth token.

I also took a look at the bearer and where it is coming from:
POST https://android.googleapis.com/auth
Headers:

device:             <some id?>
app:                com.google.android.apps.chromecast.app
Accept-Encoding:    gzip
User-Agent:         GoogleAuth/1.4 (x86_64 PI); gzip
content-length:     719
content-type:       application/x-www-form-urlencoded
Host:               android.googleapis.com
Connection:         Keep-Alive

Post data

androidId:                    <some id, as in header>
lang:                         en-US
google_play_services_version: 201817032
sdk_version:                  28
device_country:               de
app:                          com.google.android.apps.chromecast.app
check_email:                  1
oauth2_foreground:            1
Email:                        <my email>
has_permission:               1
token_request_options:        <8 Alphanumerical chars>
service:                      oauth2:https://www.googleapis.com/auth/homegraph
client_sig:                   <40 hex chars/20B>
callerPkg:                    com.google.android.apps.chromecast.app
Token:                        <some token>
callerSig:                    <same as client_sig>

I haven't touched OAuth in a while, but I feel like the client sends too many fields for normal OAuth. It could be possible, that less data also works. Maybe the error when not supplying an auth token helps

ERROR:
Code: Unauthenticated
Message: Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.

[AngeloD2022]

I have seen a request similar to this when the app requests a URI for the OAuth consent screen. This was a more complex one to reverse, but I managed to make something working.
https://www.angelod.com/onhubauthtool

You can use the source code here: https://github.com/AngeloD2022/OnHubDesktop/blob/master/TemporaryAuthSolution/getToken.html

In order to make a request regarding the GH app, you must change some things. Copying the following from a Reddit conversation @rithvikvibhu and I had.

App Version: "2.23.112" (Not entirely sure this needs to be legitimate, but putting it here just in case...)

Package Name: "com.google.Chromecast" (Yes, this is the iOS app's bundle identifier 🙄)

Redirect URI: "com.google.sso.498579633514-hhlrn8mcjv1427j0s19dgfoe5cqaba4l:/authCallback"

Client ID: "498579633514-hhlrn8mcjv1427j0s19dgfoe5cqaba4l.apps.googleusercontent.com"

Mediator Client ID: "936475272427.apps.googleusercontent.com"

All of the rest is either can remain unchanged or is generated randomly–– and the funny thing is... IT WORKS!? 😂😂 Here's what the response looks like:

{ "advice_code": "EMBEDDED", "uri": "THE LOGIN FORM URI", "client_state": "RANDOM DATA", "iosguard_challenge": { "device_challenge": "RANDOM DATA" } }

[AngeloD2022]

@XoMEX And change the same for the oauthcode to refreshtoken request. (Inside of getToken())

[Drakulix]

Edit: Turns out the tokens used by the onhub app are different from what we need. I wrote a python script that does it: https://gist.github.com/rithvikvibhu/952f83ea656c6782fbd0f1645059055d

@rithvikvibhu Under which license is your script?
I would love to update my home assistant component to make use of this generation method instead of extracting the graph file from an android device.

[Leatherface75]

Yeah that would be a big improvement and would make it possible to get it in as an official component again.

[rithvikvibhu]

@XoMEX yes, for the request (to get access tokens), only the client_sig, service, app, and token fields are required, others are optional.

There was already a python package that handled getting both kinds of tokens, so I didn't have to write manual requests.

@AngeloD2022 Huh, I thought that page was only for getting refresh tokens (1/***). I'll try it out soon. Is it possible to get access tokens (ya29.***) this way?

@Drakulix It's just a wrapper script with some values that calls functions from the gpsoauth package. Feel free to use it anywhere you want, gonna add an MIT license in a while.

[magicse]

Hi @rithvikvibhu, GetAssistantRoutines work well. But I cant get GetHomeGraph. Maybe I missed something?
Error invoking method "google.internal.home.foyer.v1.HomeControlService/GetHomeGraph": service "google.internal.home.foyer.v1.HomeControlService" does not include a method named "GetHomeGraph"

[magicse]

Ah sorry I forgot change HomeControlService to StructuresService service

[luckydevil13]

How to dump (see grpc traffic?) any how to to setup mitm + grpcproxy ?

[rithvikvibhu]

@luckydevil13 to get the whole thing working, as of now, the only things needed are:

• An access token - This gist explains how to get one
• The proto file - This drive folder has it.
• grpcurl installed - GitHub repo / releases

How to actually make the request: This comment

However, if you're interested in actually looking into what's happening, then get grpc-tools and mitmproxy.
I don't see how grpcproxy would be useful in this case. Things you could do with these tools:

• Android proxied through mitmproxy to capture HTTP(s) requests and responses
• Android proxed through grpcdump to capture grpc requests and responses
• Wireshark over the VM's network adapter, can be used with either mitmproxy or grpcdump
• grpcreplay to replay dumped requests
• grpcurl to make any new grpc requests (need proto files though)

[unchartedshark]

Hey thanks for this, this has been great to follow along to.

I've been able to get the local auth token's for my devices but when I make a request via post man (Adding the Token as a Bearer Token) and making a request to /setup/assistant/alarms port 8008 return 403 forbidden and port 8443 doesn't return anything.

I'm assuming I'm doing something wrong here.

Edit: I wasn't using HTTPS when using port 8443. I'm now getting a 401 Unauthorised status.

Edit 2: My apologies. I read the gist above and I was able to get it to work :)

[magicse]

Hi @rithvikvibhu . I need some help. How did you create the proto (protos/google/internal/home/foyer/v1.proto) file? I'm trying to figure out how to transfer a some configuration parameters to a HomeGraph,
For example, if I want to change the language locale via gRPC for home device.

[rithvikvibhu]

Honestly, It's manual work. First used grpc-dump to get a sample request/response in JSON. Then used https://www.site24x7.com/tools/json-to-protobuf.html to convert it to proto. Now this will give in v2, so there's again manual work to modify it into proto v3. The v2 definition gives a starting point, but there are still required to make it v3 compatible. This includes removing the "required" keyword from everywhere, setting the correct types, etc.

Because of the way the converter works, the names don't work. So you'll have to rename all the message names. Also, create a service and method definitions.

You can use the existing file (protos/google/internal/home/foyer/v1) as a reference and kinda duplicate the services section and rename it to the one you want.

[rithvikvibhu]

Hi everyone!

It's been exactly 1 year(!) (in 3 hours actually) since this issue was opened. Now that the API can be used again, I refreshed the website with a new design (finally mobile-friendly), features like search and updated detailed docs.

It's available at the same site: https://rithvikvibhu.github.io/GHLocalApi/

It's been great to be a part of an active project like this one.
Thanks to @Leatherface75 and @Drakulix for the scripts to make use of an Android VM and creating components that can be used by everyone (sadly, I never got to use this).
@magicse who revived this issue after ~8 months with the source dump (along with other info) and for testing everything posted here.
@XoMEX for examining the dump, finding how the local auth token works, and for the first capture of the grpc request for getting HomeGraph.
@AngeloD2022 for sending the decrypted iOS GH app and the OAuth auth {website, code, info}. It's a python script for now, but being able to do this with a website will be useful.

I'd like to thank everyone in this thread for posting new findings, testing scripts, and creating new libraries to make it easier to consume the API.

With 128 comments, this issue is getting long, so I'll be closing it. If there are any issues, please create a new issue here