Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request]: Metadata in documents for improve the prompt #3690

Closed
1 task done
Tracked by #4214
devMls opened this issue Nov 27, 2024 · 1 comment
Closed
1 task done
Tracked by #4214

[Feature Request]: Metadata in documents for improve the prompt #3690

devMls opened this issue Nov 27, 2024 · 1 comment
Assignees
Labels

Comments

@devMls
Copy link
Contributor

devMls commented Nov 27, 2024

Is there an existing issue for the same feature request?

  • I have checked the existing issues.

Is your feature request related to a problem?

I have several documents with different information , and I need refer the information with the context of his document, for achieve that LLM understard better the information

Describe the feature you'd like

Add metadata to documents in database documents

Describe implementation you've considered

with the document metadata, the prompt can be structured like that:

I think that the metadata of the documents and the metada of the knowledge database should be inserted in the prompt. In this way:

The knowledge database ${DATABASE_NAME} that has a description ${DATABASE_DESCRIPTION} has this documents with information:

Document ${DOCUMENT_NAME} with description ${DOCUMENT_DESCRIPTION}. This document has following attributes
${METADATA1_NAME} : ${METADATA1_VALUE}
${METADATA2_NAME} : ${METADATA2_VALUE}
This document ${DOCUMENT_NAME} has the following information:
${CHUNK 1}
${CHUNK 2}
End of ${DOCUMENT_NAME}

Document ${DOCUMENT_NAME} with description ${DOCUMENT_DESCRIPTION}. This document has following attributes
${METADATA1_NAME} : ${METADATA1_VALUE}
${METADATA2_NAME} : ${METADATA2_VALUE}
This document ${DOCUMENT_NAME} has the following information:
${CHUNK 3}
${CHUNK 4}
End of ${DOCUMENT_NAME}

End of ${DATABASE_NAME}

This other database ${DATABASE_NAME} that has a description ${DATABASE_DESCRIPTION} has this documents with information:

Document ${DOCUMENT_NAME} with description ${DOCUMENT_DESCRIPTION}. This document has following attributes
${METADATA1_NAME} : ${METADATA1_VALUE}
${METADATA2_NAME} : ${METADATA2_VALUE}
This document ${DOCUMENT_NAME} has the following information:
${CHUNK 5}
${CHUNK 6}
End of ${DOCUMENT_NAME}

Document ${DOCUMENT_NAME} with description ${DOCUMENT_DESCRIPTION}. This document has following attributes
${METADATA1_NAME} : ${METADATA1_VALUE}
${METADATA2_NAME} : ${METADATA2_VALUE}
This document ${DOCUMENT_NAME} has the following information:
${CHUNK 7}
${CHUNK 8}
End of ${DOCUMENT_NAME}

End of ${DATABASE_NAME}

Documentation, adoption, use case

No response

Additional information

I am not sure how that can affect to rerank process.

@devMls
Copy link
Contributor Author

devMls commented Dec 9, 2024

First aproximation #3925 organizing chunks by document and add tittles in the prompt:

This document ${DOCUMENT_NAME} has the following information:
${CHUNK 1}
${CHUNK 2}

@yingfeng yingfeng mentioned this issue Dec 24, 2024
8 tasks
@KevinHuSh KevinHuSh mentioned this issue Jan 10, 2025
1 task
KevinHuSh added a commit that referenced this issue Jan 10, 2025
### What problem does this PR solve?

#3690

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
KevinHuSh added a commit that referenced this issue Jan 13, 2025
### What problem does this PR solve?

#3690

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
cike8899 added a commit to cike8899/ragflow that referenced this issue Jan 13, 2025
KevinHuSh pushed a commit that referenced this issue Jan 13, 2025
### What problem does this PR solve?

Feat: Metadata in documents for improve the prompt #3690

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
KevinHuSh added a commit that referenced this issue Jan 13, 2025
### What problem does this PR solve?

#3690

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants