⚠️ Deprecation of RAGenerator and Seq2SeqGenerator

[bilgeyucel]

As of version 1.16, RAGenerator and Seq2SeqGenerator have been deprecated in Haystack and will be removed in version 1.18.

How to use RAGenerator and Seq2SeqGenerator?

If you're currently using RAGenerator and Seq2SeqGenerator in a version older than 1.18, you don't need to make any changes and can continue to use them as before. Keep in mind that you will get a DeprecationWarning when you want to use these components starting from version 1.16. We recommend switching to the more powerful PromptNode so that you can smoothly upgrade to upcoming Haystack versions.

See how you can use PromptNode as a replacement for Seq2SeqGenerator in practice:

p = PromptNode("vblagoje/bart_lfqa")

# Start by defining a question/query
query = "Why does water heated to room temperature feel colder than the air around it?"

# Given the question above, suppose the documents below were found in some document store
documents = [
    "when the skin is completely wet. The body continuously loses water by...",
    "at greater pressures. There is an ambiguity, however, as to the meaning of the terms 'heating' and 'cooling'...",
    "are not in a relation of thermal equilibrium, heat will flow from the hotter to the colder, by whatever pathway...",
    "air condition and moving along a line of constant enthalpy toward a state of higher humidity. A simple example ...",
    "Thermal contact conductance. In physics, thermal contact conductance is the study of heat conduction between solid ...",
]

# Manually concatenate the question and support documents into BART input
# conditioned_doc = "<P> " + " <P> ".join([d for d in documents])
# query_and_docs = "question: {} context: {}".format(query, conditioned_doc)

# Or use the PromptTemplate as shown here
pt = PromptTemplate("lfqa", "question: {query} context: {join(documents, delimiter='<P>')}")

res = p.prompt(prompt_template=pt, query=query, documents=[Document(d) for d in documents])

Why are we doing this?

While these components have been valuable in the past, we have noticed that there is a scarcity of state-of-the-art models that perform well enough for Retrieval-augmented Generation with RAGenerator. Additionally, while Seq2SeqGenerator provides several model options, PromptNode enables the usage of seq2seq models and allows for greater control over the prompt with PromptTemplate.

Let us know 🙌

If you have any questions or comments regarding this deprecation, feel free to comment on this post or send a message to the #haystack channel on Discord.

[Asutos7]

Hello @bilgeyucel! I tried the code you provided in your comment and encountered the following error: PromptNotFoundError: Prompt template named 'lfqa' not available in the Prompt Hub. Therefore, I'm curious about the location on GitHub where I can find the list of available names in the Prompt Hub.

[anakin87]

Hello @MarioAvolio!
Can you better explain your issue and provide an example?

[MarioAvolio]

Image that i want to execute this:

`from haystack.nodes import RAGenerator

generator = RAGenerator(model_name_or_path="facebook/rag-token-nq",
embed_title=False, num_beams=5)`

Where "facebook/rag-token-nq" is the RAG-Token model. How i can use this model with PromptNode? If i try to insert this model into:
PromptNode(model_name_or_path="facebook/rag-token-nq", default_prompt_template=rag_prompt)
it gives an error.

[anakin87]

Now I better understand...

These types of models are no longer supported for several reasons:

• they don't perform particularly well
• they are tightly coupled with Dense Passage Retrievers (which themselves don't perform well)

So, in real-world applications, it is more common to implement Retrieval Augmented Generation using the PromptNode (tutorial).
The PromptNode allows the usage of several LLMs, including Hugging Face text and text2text-generation models.

If you need more help, feel free to ask and open a new discussion...

[MarioAvolio]

Grazie Stefano!

If I don't ask too much, can you link me some discussions or papers about this aspect?

[anakin87]

Some suggestions:

• RAG paper by Patrick Lewis et al.: uses Dense Passage Retriever, originally introduced in https://arxiv.org/abs/2004.04906
• BEIR paper: shows poor performance of the DPR, especially out-of-domain
• There are few and old models on Hugging Face implementing this architecture