Generic mcp.el for Emacs (Model Context Protocol)

[karthink]

@ahyatt, I wanted to check if you're working on a Model Context Protocol (MCP) adapter for llm. It was described by the author as "LSP but for LLM assistants". If support for it catches on, it should make adding context to LLM queries easier in a semi-standardized way.

Last time around, we ended up reinventing the wheel with llm/gptel/chatgpt-shell etc. I'm hoping we can write a generic mcp.el library this time that has no (or few) dependencies, and use it as appropriate in different Emacs LLM client UIs. Of course, this is only relevant if the open protocol catches on, since it's only supported by Anthropic and a few editors like Zed right now.

I was planning to look into writing something early next year, assuming people continue to write MCP servers. I haven't read the full spec yet, just have a rough idea so far. But if you already have some plans or are writing something like mcp.el, I'd be happy to help.

CC'ing @xenodium and @s-kostyaev for thoughts and input.

[xenodium]

Thanks for cc'ing me @karthink!

I've only just caught up adding multi-llm support to chatgpt-shell and did indeed re-invent the wheel :/

I'm hoping we can write a generic mcp.el library this time that has no (or few) dependencies

This sounds great.

assuming people continue to write MCP servers

Looks like it's moving fast already. Looking forward to seeing usages in practice.

I may give it a little time to settle and see how the general user experiences develops. For example, I've yet to add things like function calling... which I'm guessing gets obsoleted if MCP catches on? In the case of chatgpt-shell, it took a few iterations to get a comfortable user experience. I did add (and discard) plenty of things. I'm guessing this may have been the case for other folks building Emacs packages.

Either way, I'll be following this closely :)

[karthink]

I don't think function calling is obsoleted by MCP -- but I'm not sure yet. I'll try to make some time to read the spec soon, and come back with some concrete ideas.

I did add (and discard) plenty of things. I'm guessing this may have been the case for other folks building Emacs packages.

Yeah, gptel has a long list of abandoned branches. Things usually don't work how I imagine they will, unfortunately.

[ahyatt]

Thanks for starting this conversation! Is the idea to use Emacs as a LSP host, or an LSP server? An LSP server could be interesting, but emacs has too many "capabilities", and it isn't clear to me how to best use it as a server, aside from maybe basic buffer listing and buffer content commands. As an LSP host, it's more intersting - Emacs could discover and use various sources of data and presumably pass them to . Right now it's similar to what you can do with function calling, but should be easier with service discovery (instead of having to write your own).

This is something I'm interesting in, but don't have any plans to work on. I have been thinking of pursuing a similar sort of standardization for context building in the llm package, but that's purely internal - not following the LSP spec but getting some inspiration from it. I'll mention here if I start doing something like that.

I do think that if this happens, and it needs to call out to llms, that should be done via the llm package, since it really is just a unopinionated package for abstracting amonst llms, without any UI components. It's also in GNU ELPA, which may be important to some, but unfortunately it also means that unlike other packages the llm package has to warn users about non-free software.

At any rate, let's keep this thread alive so we can coordinate and either work together or at least not do duplicate work!

[karthink]

Thanks for starting this conversation! Is the idea to use Emacs as a LSP host, or an LSP server?

LSP host, I think.

I do think that if this happens, and it needs to call out to llms, that should be done via the llm package, since it really is just a unopinionated package for abstracting amonst llms, without any UI components.

I don't think MCP involves any LLM calls at all. It returns partial or full prompts to the host (Emacs), which sends them to the LLM. There's more to it than that, I'll read the spec when I have more time and come back here with my findings. If you get around to reading it, please let us know as well!

[xenodium]

I have been thinking of pursuing a similar sort of standardization for context building in the llm package

While I've added some helpers to facilitate manually building context (usually feeding into M-x chatgpt-shell-prompt-compose), this is still an area of friction.

I've been tempted to prototype a smart context core of sorts, yet it feels like an entirely new/separate project, which I may not have the time for. Possibly run as a separate process/daemon. Maybe if I wait long enough, it gets obsoleted/solved by mcp 😅

[ahyatt]

Yes, a unified way to create context would be useful! What I did in the llm package (in llm-prompt) is to allow context generators to be either strings, lists, or generators for context, and it also allows filling context both forward and backward. It may require even more, such as filling inside-out (think of feeding in an org tree surrounding a target headline). I think all that is necessary, and I could separate that out into another project that isn't tied to the llm package.

My hope was to also bundle that with standard contexts that are useful, like one for org agenda, the org tree of the currently clocked in item, the current file (inside-out would help here too).

One other thing I'd like to see happen is to have a way to build up more complicated flows. I wrote something for this using state machine (the fsm package), and it does work, but I need to use it for more things and refine it before I'm confident enough to release it. I'd be happy not doing that and using a new module if someone writes one. One thing that would be useful is to also use it for context building, such as building up context by feeding it a project's file tree and let the LLM choose what is the best file to fill in the context. I saw a post yesterday on a similar technique: https://arcturus-labs.com/blog/2024/11/21/roaming-rag--make-_the-model_-find-the-answers/, but in the context of a single document.

OK, yet a third thing I'd like to see happen: a unified way of getting an LLM to interact with a buffer in a way that is natural to emacs. That means having it comment on text via overlays in a way the user can see and dismiss or act on. Even better, having a way to show its suggested changes. I know that gptel can do this via ediffs, but IIUC, no one, including gptel has developed a way to do this that isn't based on feeding a whole buffer or part of a buffer in and getting the altered text back. A solution might be to allow the LLM to actually perform edits (which could be real edits or show up as suggestions) via function calling, but I haven't figured out a good way to do that yet.

[karthink]

allow context generators to be either strings, lists, or generators for context, and it also allows filling context both forward and backward. It may require even more, such as filling inside-out (think of feeding in an org tree surrounding a target headline).

This sounds interesting. I'll check out how llm collects context.

This is also well ahead of what gptel does here, which is nothing at all. You can always write bespoke functions to do these things and give them to gptel, and it will run them and add the context. But there is no formal or informal spec for adding context chunks.

My hope was to also bundle that with standard contexts that are useful, like one for org agenda, the org tree of the currently clocked in item, the current file (inside-out would help here too).

My experience here has been that no matter what I think is a standard set, I find users asking to use it, or using it, in ways I would have never considered. For now I've given up on the idea of providing standard "context generators" with the package.

One other thing I'd like to see happen is to have a way to build up more complicated flows. I wrote something for this using state machine (the fsm package), and it does work, but I need to use it for more things and refine it before I'm confident enough to release it

Similar situation here -- I have a state machine driving the requests in a private branch in gptel, but it doesn't work well and the design isn't clear yet. I don't currently have the time for anything beyond bug fixes on gptel.

One thing that would be useful is to also use it for context building, such as building up context by feeding it a project's file tree and let the LLM choose what is the best file to fill in the context

Yup, chaining LLM requests was my primary reason for the fsm approach as well, but integration with other sync/async processes has suggested itself as a use case since.

I know that gptel can do this via ediffs, but IIUC, no one, including gptel has developed a way to do this that isn't based on feeding a whole buffer or part of a buffer in and getting the altered text back.

This is correct. gptel's in-place suggestion/rewrite UI got a little facelift recently though, there are some demos in this reddit thread.

A solution might be to allow the LLM to actually perform edits (which could be real edits or show up as suggestions) via function calling, but I haven't figured out a good way to do that yet.

I have a basic question about how this would work. I'm assuming you have to do one of the following:

1. Send the whole buffer + additional context, and get it to output location-specific suggestions/annotations, like a linter/editor does.
2. Make separate requests in parallel, each involving small chunks, and collect suggestions.

The former option will probably produce better quality results. But are LLMs any good at providing a list of locations and suggested changes in a structured format? I imagine that only the highest tier models can probably manage it. This seems like the kind of workflow that would break often and unexpectedly.

[xenodium]

My hope was to also bundle that with standard contexts that are useful, like one for org agenda, the org tree of the currently clocked in item, the current file (inside-out would help here too).

The likes of flymake and friends (spelling, grammar? ) can have neat integration points.

I have a very basic integration for fixing errors. It's smerged-based, with styling/tweaks.

a unified way of getting an LLM to interact with a buffer in a way that is natural to emacs. That means having it comment on text via overlays in a way the user can see and dismiss or act on.

There may be an opportunity here to build separate packages, applicable beyond the LLM space.

1. Generic infrastructure to interact/mutate buffers (UX-agnostic).
2. UX that plugs into 1. to present mutations to users, requiring intervention. User can plug in their favorite UX.

Both relate to each other how completing-read relates to vertico, helm, ivy...

I like what @karthink did with this interactive merge/insertion

I can see this being applicable beyond LLM interactions, and handy for package developers if a simple API for interactively applying changes is exposed.

[ahyatt]

Great points, I completely agree that breaking down into individually useful packages like this makes a lot of sense. I also really like that @karthink UI you show, that's pretty ideal IMHO, for many things much easier than a merge-driven UI. If think if we want to go in that direction, we should be submitting those kinds of packages to GNU ELPA, and probably discussing the general thinking in emacs-devel@.