How about using Langchain and integrate other type of docs like PDF or even external links

[OLH21]

Hello amazing work. I had a very quick look at the code and here is my idea.

Would be great if one could also vectorize PDF in the Obsidian paths, also external link could be integrated as they are part of the "Obsidian mind" as well.

Maybe using langchain could simplify the whole vectorisation process and document loading
https://js.langchain.com/docs/modules/indexes/text_splitters/examples/markdown

and process PDF
(https://js.langchain.com/docs/modules/indexes/document_loaders/examples/file_loaders/pdf)

Could also follow and scrap links found in .md docs to extend to scope (https://js.langchain.com/docs/modules/indexes/document_loaders/examples/web_loaders/web_cheerio)

[brianpetro]

Hi @OLH21

Maybe you could create a test plugin to see if the LangChain dependencies work within the Obsidian environment? At this point, it's so heavy on the dependency side that it's not something I want to mess around with in the project that so far has zero significant dependencies. But if you could confirm that it works by sharing a sample plugin that I could play around with, then maybe.

PDFs have been mentioned before. There is an Omni plugin in Obsidian that supports parsing PDFs, that's the way I would go right now, to keep any new dependencies within the existing Obsidian ecosystem.

External integrations is something I started working on (see #27) but it hasn't been of interest to as many people as I would've thought.

Thanks for joining the discussion
Brian 🌴

[OLH21]

Hi @brianpetro

I understand the dependancies point of view totally, i will see if i can make some proof of concept and come back to you, though i master python langchain more than JS ;)

Yes i had seen the omni search

I am very surprised about external because it's kind of the heart of a note system also to refer to exterals and those are totally part of the "Mind", obsidian is also really really link oriented so for me it would be a no brainer to integrate all dependancies in the context of the connections

[jppaolim]

Currently working with python to see if langchain is a good option in the first place. To be honest, about the potential gains you mention :

• the MD loader is nice but it splits the text into very small chunk (typically paragraphs). The good news is that it gives you finer retrieval than what I got from smart connection, the bad news is that it returns only one document , because most of the chunks are similar in reality. So you need to then further Digg into it and it kind of kills the purpose of using an integrated suite.
• the vector store is not super convienent ; you still need to implement yourself the issue of duplicates, you need to implement the update of the docs ....
• bottom line, it's a lot of effort just for the use case of importing some PDF and / or using other LLM than OpenAI suite
  But still , in reality, playing with :)

[OLH21]

LangChain is still early beta, update may come later , so you're right but defenitly the possibility of having one dependancy for multiple LLM and doc type is really interesting because OpenAi will not be the only good one and there will be local ones in a fews years ( months) ;)

[cocobeach]

All you need to do at this point is enable ollama with litellm?
I second Langchain integration.

[OLH21]

Agree that lots of things have change since may ;)