Combining FAQ and ExtractiveQA in a pipeline with conditions

[SasikiranJ]

Question
Hello everyone, I have some use case where i have to provide both FAQ and ExtractiveQA functionalities in single search. I want to know is there way that i can define pipelines.yaml file so that I can switch from FAQPipeline to ExtractiveQA and viceversa based on some condition.

Additional context
Add any other context or screenshots about the question (optional).

[sophgit]

Hi #@SasikiranJ,
I think you may be looking for the QueryClassifier class. It allows you to define a condition, in which cases the extractive pipeline should be called and in which the FAQ pipeline. Maybe #902 is helpful to you?

[SasikiranJ]

Thank you for the suggestion. But I have already gone through that still not able to frame solution to my use case. I will just explain my use case:-
Given a Query
I want to run FAQ pipeline first and if similarity between user query and stored query(contains answer) is less than some threshold. I want to switch to extractiveQA and display answer of ExtractiveQA. If similarity is greater then i will just simply return FAQ answer and no need to go to ExtractiveQA. is it possible?

[tholor]

Hey @SasikiranJ ,

That's actually a cool use case and should be possible in Haystack. The QueryClassifier is just one example of a "Decision Node". You are free to write your own custom one. See this for an example. You can place your node in your pipeline after the FAQPart, write a condition in the node that checks the similarity and only branch to the Retriever-Reader nodes for similarities < X.

Maybe give it a try yourself first and then we can assist if you hit some barriers or any detailed questions come up.

[SasikiranJ]

@tholor Ok. I will try . Thank you

[SasikiranJ]

@tholor is Document2Answer class available? one you mentioned in other issue.

[tholor]

@SasikiranJ No, you are right. We don't have it yet. But you can see how it is "manually" converted (without a node) here: https://github.com/deepset-ai/haystack/blob/master/haystack/pipeline.py#L525

Let me know if you need any further help on this.

[tholor]

@tholor We have the Docs2Answers node by now:

haystack/haystack/pipeline.py

Line 1322 in 39d324e

class Docs2Answers(BaseComponent):

The default FAQpipeline is also returning answers now in the same format so that it should be easy to switch between extractive QA and FAQ

[SasikiranJ]

Hi @tholor , Would you please help me with code for my use case using Docs2Answers node by creating tutorial kind of thing for combining multiple pieplines especially FAQ and ExtractiveQA Pipelines?

[julian-risch]

Hi @SasikiranJ I think our pipelines tutorial will be helpful for you: https://colab.research.google.com/github/deepset-ai/haystack/blob/master/tutorials/Tutorial11_Pipelines.ipynb
There you can see that you can create a custom node for the classification you would like to do. You need to make some changes to the code in the tutorial though. Something along the following lines should work:

class CustomClassifier(BaseComponent):
    outgoing_edges = 2

    def run(self, query: str):
        if "?" in query:
            return {}, "output_2"
        else:
            return {}, "output_1"

# your pipeline:
p = Pipeline()
# start with your FAQ pipeline
p.add_node(component=retriever, name="Retriever", inputs=["Query"])
p.add_node(component=Docs2Answers(), name="Docs2Answers", inputs=["Retriever"])
# classify the output of the FAQ pipeline
p.add_node(component=CustomClassifier(), name="CustomClassifier", inputs=["Docs2Answers"])
# case 1: run in the ExtractiveQA pipeline
p.add_node(component=es_retriever, name="ESRetriever", inputs=["CustomClassifier.output_1"])
p.add_node(component=reader, name="QAReader", inputs=["ESRetriever"])
# case 2: nothing needs to be done
# join the results of case 1 and 2
p.add_node(component=JoinAnswers(join_mode="concatenate"), name="JoinResults", inputs=["QAReader", "CustomClassifier.output_2"])

The JoinAnswers node you would need to implement yourself similar to the JoinDocuments node here:

haystack/haystack/nodes/other/join_docs.py

Line 7 in 07cf3c6

class JoinDocuments(BaseComponent):

Further, you would need to implement the CustomClassifier node with your own logic that checks the query and the retrieved FAQ stored query.