UI Haystack almost no code solution

[CarlosFerLo]

Motivation

Over the past few months of working with Haystack, I've found pipeline design using Excalidraw to be incredibly useful (add image as an example). It's become a crucial part of my development process. However, as my pipelines grew more ambitious, I encountered issues when translating these diagrams into code. Small mistakes in the code could disrupt the entire pipeline. To mitigate this, I started using the .show() method to compare the output with my original diagram, but as noted in issue #8089, this method has its limitations. This led me to consider a more robust solution: eliminating the need to write code manually, as it's both automatable and prone to errors. Thus, I propose developing a "no code" solution for Haystack.

Proposal

Initially, I planned to create this solution for personal use, but I realized it could benefit other developers and potentially become a valuable product. My idea is to build an application or VS Code extension that allows developers to design pipelines by adding components and connections, similar to Excalidraw. Users would set initialization parameters, add custom components, and manage other files as they would in a traditional code editor. This tool would reduce boilerplate code and enable the compilation of fully functional Haystack projects, with options to export as YAML or to Deepset Cloud. Additionally, it would support testing and other development activities.

What I am looking for

I seek input and collaboration from other developers interested in this project. Insights, opinions, and direct contributions are welcome. Support from the Deepset team would also be greatly appreciated.

What next

I will update this issue over the coming days with further thoughts, ideas, and potential approaches to building this tool. If you're interested, please contact me via email or leave a comment on this issue.

Thank you for reading.

BabyAGI test diagram on a single Haystack pipeline, the one that broke Mermaid :)

[mathislucka]

We're working on a new product for the Haystack community that will allow users to build their pipelines in a visual editor. It's currently in development and we'll announce it mid-August.

I already talked to @CarlosFerLo about some details. If anyone else here is interested in helping us with some early user testing or wants to be put on the early access list, please drop a comment here or reach out to us.

[Deepan-kishore]

Hi,
Your respository and responsiveness of its issues are amazing.
I am a Javascript React developer but I believe, I could help with early user testing.

[mathislucka]

For reference: I reached out to @Deepan-kishore and he'll participate in the beta phase.

[lohit8846]

Hi @mathislucka I would also be interested in early user testing the the visual editor for pipelines. I'm a React JS Developer and would be interested in providing feedback. Please add me to the early access list if any spots exist

[mathislucka]

@lohit8846 happy to add you to the list. How can I best reach you?

[Bellk17]

I would also love to test this out; we are looking for a no/low code solution to add into our frontend.

[mathislucka]

That's great to hear @Bellk17. We now have an early access form where you can sign up to get access to the private beta: https://landing.deepset.ai/deepset-studio-waitlist

We'll start onboarding people from the list in the beginning of September. Looking forward to your feedback!

[PaulBFB]

this looks amazing, I am currently building something for fun which is quickly becoming rather large, could I also test this out?

[mathislucka]

@PaulBFB you can just sign-up here: https://landing.deepset.ai/deepset-studio-signup