q's: chat framework, context window, 16k model, embedding solution

[brezl8]

I have a few quick questions, i apologize, they might be naive in nature, i am still learning;

1. how well does it work to query something from multiple different notes of the vault and how is that implemented (in rough terms)?

2. what's the context window for smart chat? (with gpt4)

3. is there a way to run the smart chat feature with gpt3.5 turbo? i am asking because of the new 16k context window which is unfortunately not available for gpt4.

4. how well does the embedding process and the openAI embeddings themselves work?
   it is mentioned there are different ways to do approach this.

thank you so much for your time and work! @brianpetro

greetings

[brianpetro]

Hi @brezl8

how well does it work to query something from multiple different notes of the vault and how is that implemented (in rough terms)?

It depends, you have two options. 1) You can use multiple links using the Obsidian double-bracket notation to specify full notes to use as context for your query. Or 2) you can specify a folder using a forward slash then selecting a folder from the drop-down. This will the use parts of notes in the folder as context in your query.

what's the context window for smart chat? (with gpt4)

8K technically, but in practice it can be as low as 6K because of the way the tokens are estimated by this plugin specifically.

is there a way to run the smart chat feature with gpt3.5 turbo? i am asking because of the new 16k context window which is unfortunately not available for gpt4.

Yes, you can use the 16k gpt-turbo-3.5 model.

how well does the embedding process and the openAI embeddings themselves work?
it is mentioned there are different ways to do approach this.

Smart Connections implements "context aware" content chunking, based on the structure of your notes (currently, that's mainly the headings and file path), which makes the results better than others that use less sophisticated methods for chunking content.

Thanks for your interest in Smart Connections,
Brian 🌴

[brianpetro]

Hey @RealRavens

Are you thinking something like:

"Results (blocks/chunks) are weighted by whether the there's linking (may be forward, backward, or both) at the note-level."

I think something like that would be good. And it would probably be some sort of filter in the Smart View. This way you could, at it's simplest, toggle includes/excludes chunks with note-level links. Includes would increase relevance, while excludes would increase discovery of unlinked connections.

There's another FR that, similarly, would be something like weighting the final results based on specified headings. Similar as in the UIUX would be some sort of filter in the Smart View.

As for implementation, the calculation is relatively simple, I think the challenge is mostly presenting it in the UI. Overall, I do think this would be a big value add.

I'm mostly thinking out loud here. I hope that touches on what you were looking for.

🌴

[RealRavens]

I believe we're thinking of the same thing.

Consider these images:

If I did a search on "info on cute animals", paths will be checked, headers will be checked as well for context. In the case of the first image, the text under the header will possibly be used or considered in the result. What I'm suggesting is that "links" as shown in the second image should be considered just as much in the "path or weighing of relevancy".

This may allow a variety of better context clues for notes and their relationships to one another. It's "almost" like what the plugins Breadcrumbs/ExcaliBrain does for users (note relationships and relevance wise), but for AI. The weighting essentially being: of the most related notes/paths/headers considered, the closest links (relationally) down the "link tree" should be considered to be checked, and the further you are from the most relevant notes, the less relevant it is to the result (will probably have to do some limiting here considering performance and tokens unless the host machine will handle some of that?). Of course if the links considered are NOT useful, we can basically stop checking beyond those paths. Just thinking about this also reminds me of how Google works as a search engine, crawling through the web.

Again we can really go wild with ideas of how we want to accomplish this. But having links be more so considered directly can really enhance the experience, as well as possibly giving more design options for the users that I've seen requested a lot around here (note structures in tables and cells, notes in list structures etc)

Would be cool. And you present an awesome use case for even exclusion, for possible novel broader connections, I didn't even consider that. That's very clever.

Also about the UI, ya there's going to be a few considerations, maybe someone can input more on how that might look.

[RealRavens]

I feel like this should be a new discussion page, just so I can hear other's thoughts on this and implementation and design. My ideas on the possibilities are running rampant. xP

[brianpetro]

@RealRavens I started this feature request based on this #423

So any thoughts you have there would be helpful.

And I'm sure we could probably spin-off some of these ideas into additional features as they mature.

Thanks for the insights!
🌴

[RealRavens]

For sure, keep up the good work. 👍