429 errors

[krystoflp]

Hi, Unfortunately this isn't working for my vault. I'm using 1.0.48

I first tried in on the openai's free plan, but have upgraded to a paid plan to see if that helps, but it doesn't.
The API key is working, as verified in your plugin, and because I see this in openai's usage logs:

13:20 Local time: 19 Jan 2023 at 13:20 text-embedding-ada-002-v2, 54 requests 3,183 prompt + 0 completion = 3,183 tokens 13:25 Local time: 19 Jan 2023 at 13:25 text-embedding-ada-002-v2, 16 requests 3,415 prompt + 0 completion = 3,415 tokens

However, there are no smart connections being created in my obsidian vault, and every single console entry reports status 429.

Here is a typical example:

failed embedding: 5 Daily Notes/2022-11-14-Mon.md plugin:smart-connections:717 undefined plugin:smart-connections:721 Error: Request failed, status 429 at new t (app.js:1:710073) at Aw (app.js:1:710265) at app.js:1:710942 at app.js:1:235505 at Object.next (app.js:1:235610) at a (app.js:1:234349)

Do you have any ideas? How can I help fault trace this?

[brianpetro]

Hi @krystoflp

I think the rate limits are still be the issue https://beta.openai.com/docs/guides/rate-limits/overview

It takes 48 hours after upgrading to a paid plan before the rate limits are significantly increased for your account.

You will have to intermittently go into the plugin settings and "Retry failed files" until the limit is raised and the process can be completed.

I recently noticed how low the rate limits are for free & new OpenAI accounts. So I have yet to figure out a workaround to make this easier.

Please let me know if the issue clears up as expected in the next day or so.

And thanks for reporting this!

[NightOwlCoder]

I'm getting those 429s too, but in my case, if it matters, no note name appears:

Error: Request failed, status 429
    at new t (app.js:1:710073)
    at Aw (app.js:1:710265)
    at app.js:1:710942
    at app.js:1:235505
    at Object.next (app.js:1:235610)
    at a (app.js:1:234349)
plugin:smart-connections:696 failed embedding: 13171e3f61bbfbf0a3576b29af83a11d
plugin:smart-connections:777 undefined
plugin:smart-connections:781 Error: Request failed, status 429
    at new t (app.js:1:710073)
    at Aw (app.js:1:710265)
    at app.js:1:710942
    at app.js:1:235505
    at Object.next (app.js:1:235610)
    at a (app.js:1:234349)
plugin:smart-connections:696 failed embedding: 598cada20ebd9688df51750f6f0450cf
plugin:smart-connections:777 undefined
plugin:smart-connections:781 Error: Request failed, status 429
    at new t (app.js:1:710073)
    at Aw (app.js:1:710265)
    at app.js:1:710942
    at app.js:1:235505
    at Object.next (app.js:1:235610)
    at a (app.js:1:234349)
plugin:smart-connections:696 failed embedding: e1e3785fb6d5642be25665554c719aaf
plugin:smart-connections:777 undefined
plugin:smart-connections:781 Error: Request failed, status 429
    at new t (app.js:1:710073)
    at Aw (app.js:1:710265)
    at app.js:1:710942
    at app.js:1:235505
    at Object.next (app.js:1:235610)
    at a (app.js:1:234349)
plugin:smart-connections:696 failed embedding: d41d8cd98f00b204e9800998ecf8427e

I also just converted my plan to a paid one, so next Monday I'll retry....

[NightOwlCoder]

Another update.
While reading the other issues here to see if there was ay workaround, the 429 started to go away!
Looks like paying accounts get lifted way earlier than the 48h.
But now I have this strange files on my .smart-connections folder:

total 164880
-rw-r--r--   1 XYZ  staff     2B Jan 19 18:53 embeddings-1674183561.json
-rw-r--r--   1 XYZ  staff    77M Jan 19 19:23 embeddings-2.json
-rw-r--r--   1 XYZ  staff   1.8K Jan 19 19:23 failed-embeddings.txt

is that normal? A -2 file?
and the empty 1674183561?

[NightOwlCoder]

Sorry for the spam, but another update.
Looks like my 429s were being caused by this weird named files (attached my failed-embeddings.txt), find result of one of them:

vault> find . -name 5a583870cbfe6dc17a25637d34cbbd24 -print

failed-embeddings.txt

[brianpetro]

re: embeddings-2.json
This is the file containing the new format of the embeddings. Make sure you keep this file.

re: embeddings-1674183561.json
This automatically created when you use the "Force Refresh" feature. It's just a copy of the current embeddings file at the time of the Force Refresh. It can be safely deleted.

re: weird file names in failed-embeddings.txt
This bug was introduced during the upgrade from the original embeddings file structure to the new embeddings-2.json structure. The bug was just fixed in version 1.1.3. After updating, the erroneous file names should go away after using the "Retry Failed Files" feature in the plugin settings.

@NightOwlCoder Thanks for reporting these issues!

[krystoflp]

Hi @krystoflp

I think the rate limits are still be the issue https://beta.openai.com/docs/guides/rate-limits/overview

It takes 48 hours after upgrading to a paid plan before the rate limits are significantly increased for your account.

You will have to intermittently go into the plugin settings and "Retry failed files" until the limit is raised and the process can be completed.

I recently noticed how low the rate limits are for free & new OpenAI accounts. So I have yet to figure out a workaround to make this easier.

Please let me know if the issue clears up as expected in the next day or so.

And thanks for reporting this!

Thank you for your super fast response :-)
I wondered if the rate limits were a factor which is what prompted me to upgrade to a paid plan.

I’ll upgrade smart connections to the latest version 1.1.3 and as you say wait a bit and see if it sorts itself out.
I’ll let you know!

[krystoflp]

Hi Brian,

Just letting you know that it's all suddenly started working. It looks like it was the 48hour warming up period with openai that was preventing it from working. Thanks very much for your assistance :-)

[brianpetro]

Awesome, thanks for the update!