[Feature Request]: Explore GraphInsight similarly to GraphRAG

[jjohare]

Is there an existing issue for the same feature request?

• I have checked the existing issues.

Is your feature request related to a problem?

GraphRAG is great, but a recent paper perhaps offers additional optimisation, most specifically around LLM understanding of graph structure, with potential implications for multi-step reasoning.

"GraphInsight is grounded in two key strategies: 1) placing critical graphical information in positions where LLMs exhibit stronger memory performance, and 2) investigating a lightweight external knowledge base for regions with weaker memory performance, inspired by retrieval-augmented generation (RAG)."

Describe the feature you'd like

The recent paper on GraphInsight introduces several novel techniques that could enhance GraphRAG in RAGFlow's ability to handle complex data across large corpora.

From their paper:

Next, we introduce the construction of our framework’s RAG knowledge base, called the
“GraphRAG base”, and the corresponding RAG process, termed the “GraphRAG process”. Note
that existing RAG techniques Ghosh et al. (2024); Rorseth et al. (2024); Sojitra et al. (2024); Das
et al. (2024) and optimizations are orthogonal to our framework and can further enhance RAG qual�ity, but here we
focus only on the most basic RAG methods.
GraphRAG Base. Conventional RAG base typically require substantial storage and rely on extensive structured or
unstructured data (e.g., documents, knowledge graphs). In contrast, our GraphRAG base, denoted as K, demands
minimal storage overhead. Specifically, for the graph description sequence Tˆ generated by the importance-based
description reorganization, the nodes and edges of the subgraph structures corresponding to the weak memory
regions within Tˆ will be stored as the GraphRAG base. The proportion of stored subgraph structures, denoted as
is adjustable. Since the subgraph structures corresponding to the weak memory regions have already
been ranked by importance, we can conveniently select and store only the top γ% of these structures.
---- other text ---
Finally, the two parts of the augmented information are organized into a prompt, which is then input
into LLMs to assist in the reasoning process for the task
Our GraphInsight framework incorporates two key techniques that can be seamlessly integrated into
the agent-based processes of LLMs to enhance the performance of such multi-step reasoning tasks:
• Initially, during the LLMs agent process’s inception phase, our framework’s importance�based description reorganization method can be applied to the sequence of graph descrip�tions input into the LLMs. This enhances the LLMs’ overall comprehension of the graph
structure.
Subsequently, in the multi-step reasoning phase of the LLMs agent process, our frame�work’s GraphRAG method can provide LLMs with enriched information relevant to each step of the reasoning process, thereby improving the quality of the reasoning.

1. Adaptive Graph Summation:

   • Integrate GraphInsight's adaptive graph summation technique, which balances information density and LLM token limits. This feature could help in managing large graphs more effectively within RAGFlow.

2. Multi-step Reasoning: Incorporate GraphInsight's approach to multi-step reasoning.

3. Importance-based Description Reorganization: Add support for reorganizing graph descriptions based on importance, as described in GraphInsight.

Potential benefits:

• Improved handling of large and complex graph structures
• Enhanced retrieval accuracy for sparse data?
• Better support for multi-step reasoning tasks
• More efficient use of LLM token limits

Describe implementation you've considered

No response

Documentation, adoption, use case

No response

Additional information

No response