Flow utilities

[ahyatt]

Hi all,

I've been thinking of improving the kind of interactions we can have with emacs and LLMs. I've written llm-flows to demonstrate the possibilities. Right now there's just one possible thing, but it's pretty nice and general: a way to replace content in a buffer, but be able to look at the diff before replacing it, and optionally revise by sending commands to the LLM asking for certain changes.

I'm curious what others think (especially @s-kostyaev). I'm also toying with the idea of making this more of a state machine-based approach. And I don't know where to put this - in this llm repository, or in a new repository? But I need to experiment more with different uses of this. Any feedback or thoughts would be welcome.

Here's an example of how I use it:

;; A test example

(defun ash/llm-correct-text (start end)
  (interactive "r")
  (llm-flows-replace-region ash/llm-openai
                         (make-llm-chat-prompt
                          :context
                          "You are an experienced editor that specializes in clear and correct communication.  You will be given a passage of text.  Return only a rewrite of that passage,  correct in both grammar and spelling, and clearer in meaning.  Do not return any other commentary other than the rewrite of the passage, which will be replacing the original text.  Keep the formatting style of the original text.  For example, if the text is in org-mode format, keep the formatting in the org-mode style instead of using markdown."
                          :interactions (list (make-llm-chat-prompt-interaction :role 'user
                                                                                :content (buffer-substring-no-properties start end))))
                         "Text fixer"))

[s-kostyaev]

Hi @ahyatt. I will try it later, but idea to generate diffs that user can approve is great. But heavy local models can do it for a long time. Can we to improve user experience show generation stream in overlay or in child frame for example? This should be optional. After generation done it should hide and user should see the diff. It just ideas now, no need to rush implementation now, but we can discuss it.

About repositories - as you wish. It doesn't matter to me. While it's experiment I'd leave it in separate repository.

[ahyatt]

Yeah, one of the issues is that I couldn't think of a good way to have the UI show the streaming output. Thanks for getting me thinking about it, let me consider your ideas and see what I can come up with.

[r0man]

Hi @ahyatt,

what's the state of the flow branch? It looks very interesting. I read your posts on Emacs devel/Reddit about this. Are you happy with the fsm based approach? I'm thinking of giving this a try for an extension to Magit that suggests a message when committing files.

[ahyatt]

The approach looks like a good one. I'd like to wrap what is there in macros, though, to make it easier to work with. It also makes sense to integrate function calling into it somehow, but I haven't done so. If you want to experiment, go ahead and experiment against the flow branch and let me know how it goes, that can help me refine it to a releasable version. I think it may take many weeks before I get it to that point, though.

[r0man]

Ok, will do. Thanks for your answer.

[mlemerre]

I tried to test your snippet with the latest version of llm-flows but could not get it to work. Do you intend on having a more proper release of it? I think having a structured way of integrating llm in an emacs workflow is the way to go, and "simple" things like a query-replace/M-x ispell interface where replacement is provided by an LLM for you to accept would already be very useful.

[ahyatt]

I haven't kept this up to date, but I think it's probably time to revisit it. Right now the best source for this is the llm-flows.el in the flows branch of this repository. A few things need to be rethought; for example, there's no need for a complicated flow about json generation, the llm package can request json and ensure pretty well that the JSON is generated, and according to a particular scheme.

It hasn't been released yet because I'm not really convinced there's enough use-cases for it, and whether it really is a good design for potential use-cases. I like your idea, though - I've updated the branch and will do a quick test soon.