Method doesn't allow unregistered callers (callers without established identity). Please use API Key or other form of API consumer identity to call this API. with the no-key endpoint
#237
Comments
Benjamin-Loison
added
bug
#!/usr/bin/python3
import requests
import json
from tqdm import tqdm
with open('ytPrivate/keys.txt') as f:
lines = f.read().splitlines()
def getValue(obj, path):
if path == '':
return obj
pathParts = path.split('/')
key = pathParts[0]
key = int(key) if key.isdigit() else key
try:
newObj = obj[key]
return getValue(newObj, '/'.join(pathParts[1:]))
except:
return None
correct = 0
quotaExceeded = 0
progressBar = tqdm(lines)
for line in progressBar:
url = 'https://www.googleapis.com/youtube/v3/videos'
params = {
'part': 'snippet',
'id': '_ZPpU7774DQ',
'key': line,
}
data = requests.get(url, params = params).json()
if getValue(data, 'items/0/snippet/channelId') == 'UCWeg2Pkate69NFdBeuRFTAw':
correct += 1
elif data['error']['errors'][0]['reason'] == 'quotaExceeded':
quotaExceeded += 1
else:
print(json.dumps(data, indent = 4))
break
progressBar.set_description(f'{correct=} {quotaExceeded=}')with https://yt.lemnoslife.com key set shows that no key still have any quota. Note that I ran this script in 18 seconds on my Linux Mint Framework and I tested manually a few keys in Firefox and noticed only quota being exceeded. Are all keys somehow empty, if they can even be or there was more quota usage than usually? I should monitor key usage. In the meanwhile I should look for new keys. I communicated on Discord concerning this main issue. Above script helped me figure this no more interesting key: {
"error": {
"code": 403,
"message": "Permission denied: Consumer 'api_key:AIzaSyB2XeDcarYSa-TFY1ga_lTrlk9mOzVXIQM' has been suspended.",
"errors": [
{
"message": "Permission denied: Consumer 'api_key:AIzaSyB2XeDcarYSa-TFY1ga_lTrlk9mOzVXIQM' has been suspended.",
"domain": "global",
"reason": "forbidden"
}
],
"status": "PERMISSION_DENIED",
"details": [
{
"@type": "type.googleapis.com/google.rpc.ErrorInfo",
"reason": "CONSUMER_SUSPENDED",
"domain": "googleapis.com",
"metadata": {
"service": "youtube",
"consumer": "projects/137880112907"
}
}
]
}
}Once have treated this issue should unpin it and set grep -n -m 1 '13/Feb/2024:00:56' access.log.1wc -l access.log access.log.1Hence 15,960,445 - 15,260,612 + 9,201,889 = 9,901,722 have been performed in the last 24 hours. |
Benjamin-Loison
added
medium
|
As below script run under 16 seconds on OC3K, I programmed a crontab to run every minute to let me know once a key has quota. #!/usr/bin/python3
import requests
import json
with open('/var/www/yt/ytPrivate/keys.txt') as f:
lines = f.read().splitlines()
for line in lines:
url = 'https://www.googleapis.com/youtube/v3/videos'
params = {
'part': 'snippet',
'id': '_ZPpU7774DQ',
'key': line,
}
data = requests.get(url, params = params).json()
if data['error']['errors'][0]['reason'] != 'quotaExceeded':
print(json.dumps(data, indent = 4))
break |
|
Solving #91 would make the no-key endpoint and its maintenance no more important. |
|
On https://yt0.lemnoslife.com I noticed in Maybe this Stack Overflow answer helped making https://yt0.lemnoslife.com answer request instead of timing out.
|
|
Manually merging keys of:
Verified exhaustiveness thanks to LemnosLife VPS Search keys in logs:
grep -Poh 'AIzaSy[A-D][a-zA-Z0-9-_]{32}' | sort | uniq -u
(cat access.log{,.1} && gunzip -c access.log.*.gz) | grep -Poh 'AIzaSy[A-D][a-zA-Z0-9-_]{32}' | sort | uniq -u |
|
Consider #18. |
|
Unclear what was current assumption but let us now keep all keys even those that are quota exceeded, assuming that no key can have no quota forever. |
|
With new Previously had about 268 keys. May have issues due to file reading with so many keys. |
|
TLDR: incident solved (for the moment at least), you can use the no-key endpoint as previously |
|
Note that for unknown reason https://yt0.lemnoslife.com no-key endpoint is not working but as it is not officially supported let us do not care about it for the moment. |
|
Related to Benjamin-Loison/matrix-commander/issues/8. |
|
Another possibility to get YouTube Data API v3 could be to get those of big companies, thanks to this method. |
|
Just got this error, the second line of |
import requests
from tqdm import tqdm
ytUrl = 'https://www.googleapis.com/youtube/v3/videos'
ytParams = {
'part': 'snippet',
'id': '_ZPpU7774DQ',
}
correctKeys = []
quotaExceededKeys = []
def getValue(obj, path):
if path == '':
return obj
pathParts = path.split('/')
key = pathParts[0]
key = int(key) if key.isdigit() else key
try:
newObj = obj[key]
return getValue(newObj, '/'.join(pathParts[1:]))
except:
return None
with open('b') as f:
keys = f.read().splitlines()
for key in tqdm(keys):
ytParams['key'] = key
data = requests.get(ytUrl, params = ytParams).json()
if getValue(data, 'items/0/snippet/channelId') == 'UCWeg2Pkate69NFdBeuRFTAw':
correctKeys += [key]
print(f'{len(correctKeys)=}')
elif data['error']['errors'][0]['reason'] == 'quotaExceeded':
quotaExceededKeys += [key]
print(f'{len(quotaExceededKeys)=}')
print('\n'.join(quotaExceededKeys + correctKeys)) |
Benjamin-Loison
changed the title
Method doesn't allow unregistered callers (callers without established identity). Please use API Key or other form of API consumer identity to call this API. with no-key Method doesn't allow unregistered callers (callers without established identity). Please use API Key or other form of API consumer identity to call this API. with the no-key endpoint
{ "error": { "code": 403, "message": "Method doesn't allow unregistered callers (callers without established identity). Please use API Key or other form of API consumer identity to call this API.", "errors": [ { "message": "Method doesn't allow unregistered callers (callers without established identity). Please use API Key or other form of API consumer identity to call this API.", "domain": "global", "reason": "forbidden" } ], "status": "PERMISSION_DENIED" } }Related to #229 (hence also this initial Git issue comment).
The text was updated successfully, but these errors were encountered: