"Maximum context length is 4097 tokens"

[leips]

Hello,

I have this error. I understand that is a limit about ChatGPT API. There's something we can do about the limit? Try to use FASS as an index and split the chunks?

Thanks!

[brianpetro]

Hey @leips , thank you for taking the time to bring this to my attention!

Are you able to take a screenshot of the error in the console?

Is this happening after the first, or many, messages?

Thanks for your help in solving this issue :)

[leips]

The screenshot,

I try with a few notes, everything works, but after I put the of the notes in my vault (500 notes) this error appears every time I'm asking something about the notes.

[brianpetro]

Thanks for the quick follow-up.

The plus sign in the top right of the chat window will clear the current context. This will be necessary until I implement a manual pruning of past messages.

but after I put the of the notes in my vault (500 notes)

Can you clarify what you mean by this?

Thanks,
Brian 🌴

[leips]

Same error after click the plus sign.

Can you clarify what you mean by this?

I made two test. First, with a few notes, everything works. Second, with a 500 notes. After refresh the embeddings, this message appears when I ask something.

[brianpetro]

Is the 500 note vault something you'd be comfortable sending me so I can test it out? Otherwise I'll have to think on it a bit, not sure what's up. [email protected]

[leips]

It's personal information.

Yesterday I deleted the embeddings and started over all the process. There is something about the embeddings that change every time I refresh the process. I'll continue testing.

Thanks!

[brianpetro]

@leips I think I might have an idea what's up.

Do you have some really long notes? Maybe ones that don't use a lot of headings? Like, large text blocks (>1000 chars) without and headings?

If so, the logic for limiting lengths of notes in the context window is probably the problem.

This is something that I can improve to better handle cases like I mentioned above.

Thanks for your help in figuring this out!

[rotello]

I am doing more test. it seems that the kindle clipping are working, while the mobi / loooong form MD i imported (it s basically a full book i ve ) is not indexed. I ve not got any error.
I updated the plugin. closed the Vault and it seems to work. I am excited

[brianpetro]

I am doing more test. it seems that the kindle clipping are working, while the mobi / loooong form MD i imported (it s basically a full book i ve ) is not indexed.

Great to know, thanks! This type of feedback is really helpful.

In the background, "parsing," breaking the notes into logical chunks, plays a big role in the plugin's effectiveness. So less common formatting is more likely to run into issues.

I do have a lot of improvements I want to make to the parser. A lot of them are designed to handle long-form content better. So, in time, the plugin will improve at handling these cases.

Lastly, I know it's incredibly frustrating when software doesn't work as expected, and it's easier to walk away than to contribute to its improvements. So thanks again for your continued assistance!

Brian 🌴

[rotello]

Thank for bearing my obnoxiousness :-)

[pinuke]

@rotello should we start a club? maybe a support group... "Brian Petro's Annoying Fans Anonymous"

[rotello]

i think we could :-P

[brianpetro]

I really appreciate it, but also couldn't imagine keeping this thing going if it weren't for such enthusiastic and helpful people like yourselves. Thank you @smartguy1196 @rotello 👏

[pinuke]

Issue reported again in comment: #88 (comment)

[dartungar]

Yeah, for sake of convenience here is my comment:

[brianpetro]

@dartungar which language is this occuring with?

I have two ideas for changes that will both contribute to solving this:

There is now a language setting in Smart Connections, and with that, we can adjust how the tokens are estimated. Some languages use characters that make the "divide by 4" estimate unreliable. This calculation should be easy to adjust based on the language selected in the settings.

Additionally, I plan on implementing a method to catch this error, reduce the amount of context, and resubmit the request automatically. This should prevent bad estimates from causing a fatal error like this.

After making those two changes, I think this issue will be solved.

[dartungar]

@brianpetro well, most of my notes are in Russian (hope that doesn't put you off! 😶 ). That does seem like a potential cause, will check the new setting. Cheers!

[brianpetro]

Closing in favor of this FR #126