V2

[miracletamer]

Brian,

I updated to V2 but my experience caused me to roll back.

Is it intended V2 functionality that the embeddings are not now sent to OpenAI? Instead it appears that the V2 smart connections plugin only sends the top 20 to 30 notes chosen from local embeddings file. I was checking my OpenAI usage stats and no embeddings were uploaded until i rolled back to the previous version. Good thing my vault is fairly small because i have had to restart the embeddings several times to get the vault sorted back to its last good state. Using V2 yielded significantly impeded query performance on my vault compared to the V1.6.48 derivative. Loss of ability to check OpenAI API connection is also a backward step. For my use case I would prefer finer grained controls over the OpenAI API calls with the ability to set temperature and frequency penalties. I also had problems with the notifications in the top right corner - they would hang showing files still outstanding for embedding but there is no way to see what is causing it. Seemed to always be the remainder of the files once the last batch of 10 is reported complete.

On a positive note - showing the text block count not the note count is a welcome update

Happy to give you more feedback - love the plug-in but, selfishly, this update is not for me as it stands

Marc

[arminta7]

I also rolled back, but because the performance (speed and result quality) is poor with V2 for me.

[brianpetro]

@miracletamer @arminta7

I just added a disclaimer to clarify how early of a release it is. I honestly didn't even think anyone would notice when I slipped the "get v2" button into the settings yesterday. I tend to err on the side of shipping too early, and this time error I sure did.

And I might've gotten a little overzealous with the local embedding model. Unfortunately, embedding a vault's worth of notes with the local model is really slow. I'm already in the process of re-adding OpenAI embeddings, which weren't meant to be removed permanently. I just had too high expectations for the local model and bad timing on when to make the new version available. Both models should have been there from the start, soorry!

As for the quality of retrieval using the local models, I'm implementing huggingface's transformer.js, so that includes many local embedding models that will be available for trying out, and individually managing the trade-off of size vs performance.

Performance, from embedding process to retrieval, is also very much a focus point of this update. While version one was acceptable performance-wise, it can be a lot better, and that will also make room for new (and to me, exciting!) features.

There's a ton I'm excited about bringing to life in SCv2, and the first step was essentially a page-one rewrite based on my experiences from the past year. While at first, this won't look like much from the outside, basically just the same features with fewer lines of code, over time, it will allow me to make many more improvements and significant feature additions.

So I'm very sorry that the premature release started on a bad foot, performance-wise especially!

In short, yes, v2beta is currently sub-par, and that should have been made clearer. My apologies for the inconvenience.

🌴

[brianpetro]

@avataraustin

Similarly to your v1 embeddings being saved, your v2 embeddings should also be saved if you switch back again.

So if you end up being able to give it another shot, any screenshots you can take that show the issues would be incredibly helpful.

Thanks for following up,
🌴

[avataraustin]

I remember the embedding process seemed to work fine or at least there were no error messages. My only complaint is when I used the smart chat to ask my documents a question that I knew worked before (and other similar questions), it was very obviously Not searching my documents data and just drawing off of chatgpt data or hallucinations, which is why I switched back to V1 Ada embeddings.

One other detail I never mentioned that may or may not be relevant is that when I opened the other computer's Obsidian, Obsidian reindexed it's vault since I had not opened Obsidian in a very long time on that device. I don't know if that would mess up anything Smart Connections would have been doing behind the scenes or not but thought I would mention it.

[brianpetro]

@avataraustin both are interesting notes.

1. Are you able to confirm that the embedding process succeeded by checking if relevant notes are appearing in the Smart View?

2. The Obsidian re-index could plausibly cause an issue, however, the embedding process isn't automatically run when you start Obsidian.

Did you observe a significant slowdown of Obsidian during the embedding process? Unfortunately, with the local model, this seems almost unavoidable and makes Obsidian unusable during the processing time. So if you didn't see this happen, my guess is that the embeddings may never have run at all.

If you do have a chance to try out the v2 again, if you can open the developer console and screenshot any errors in there that would also be really helpful.

Thanks for all your help in getting to the root of the issue!
🌴

[avataraustin]

1.) The very first time I used v2 with local embeddings I can confirm that Smart View gave me a relevant search result, however after switching between machines and going through the steps previously mentioned, when I came back to the original machine and tested Smart View search of my documents it was no longer giving relevant search results when it should have been picking content from my documents but instead was giving random chatgpt style answers.

As far as the embedding process succeeding, It always showed a progress indicator in the top right of the screen as it worked through the files to embed. I did not use the computer while this process was going on, I just let it complete so I can't say if it was a significant slowdown or not.

When the local embeddings were not giving expected output when doing a Smart Search, I ended up doing a "force_refresh" since I assumed it would just redo the local embeddings. Perhaps there is an issue related to that. In any case, it would then show the same alert in the top right of the screen showing the progress as it went through the embedding process again and it seemed to indicate that it cycled through everything fine and completed. If I go through this process again I can try with dev console up and see if there are any errors that were hidden from view perhaps.

[brianpetro]

@avataraustin thanks for insights, it's a huge help!

🌴

[brianpetro]

UPDATE The latest v2.0.35 has both OpenAI Ada & local BGE embeddings available.

Edit: the Smart Chat is broken in this latest release. Still looking into it.

[brianpetro]

UPDATE: I've been working in Obsidian all day using Smart Connections v2 and the local model.

I think it's safe to say that version v2.0.57 is now at least on par with v1.

Initially, using the local embedding model takes a while longer to generate the embeddings. However, after the initial embedding process, the performance seems much smoother.

In addition to the local embedding model:

• running the "Make Smart Connections" command when the cursor is mid-file (ex., on line 15) will render smart connections more relevant to that line (based on the current "block" that contains that line)
• collapse/expand all results buttons
• improved block parsing ("chunking" for those familiar with RAG)

🌴

[mashean]

Love the changes but had to roll back due to performance issues not only with V2 but my entire vault...looking forward to a more stable version soon. BTW, by far the only Obsidian GenAI tool I have come across that is actually usable. Appreciate all your work on this amazing product.

[brianpetro]

Hey @mashean and thanks for trying out v2.

Can you help me by telling me a little more about those performance issues? Was it during the initial embedding process? Was it with the local model or the OpenAI Ada embedding model? And anything else you can tell me that might help me improve things.

Thanks for helping me make Smart Connections the best it can be :)

🌴

[minthemiddle]

What are the plans for v2? Will it be open to anyone at some point or still be only for paying insiders?

[brianpetro]

@minthemiddle V2.0 early release is now available to everyone. You can upgrade via the settings.

The code will be moved to this repo soon. At that time, V2.1 will move into early release for supporters only until it's ready for general release, similar to v2.0.

[brianpetro]

Moved discussion to #432