diff --git a/.DS_Store b/.DS_Store index 812de7c..5c6e6d0 100644 Binary files a/.DS_Store and b/.DS_Store differ diff --git a/demos/p1.png b/demos/p1.png deleted file mode 100644 index 3f2e36a..0000000 Binary files a/demos/p1.png and /dev/null differ diff --git a/demos/p2.png b/demos/p2.png deleted file mode 100644 index ef20ab1..0000000 Binary files a/demos/p2.png and /dev/null differ diff --git a/demos/p3.png b/demos/p3.png deleted file mode 100644 index ca767b4..0000000 Binary files a/demos/p3.png and /dev/null differ diff --git a/demos/p4.png b/demos/p4.png deleted file mode 100644 index 3aa7035..0000000 Binary files a/demos/p4.png and /dev/null differ diff --git a/demos/p5.png b/demos/p5.png deleted file mode 100644 index 42b3039..0000000 Binary files a/demos/p5.png and /dev/null differ diff --git a/demos/p6.png b/demos/p6.png deleted file mode 100644 index 3e4bcb1..0000000 Binary files a/demos/p6.png and /dev/null differ diff --git a/demos/p7.png b/demos/p7.png deleted file mode 100644 index 9ecc895..0000000 Binary files a/demos/p7.png and /dev/null differ diff --git a/demos/p8.png b/demos/p8.png deleted file mode 100644 index 61110a8..0000000 Binary files a/demos/p8.png and /dev/null differ diff --git a/demos/p9.png b/demos/p9.png deleted file mode 100644 index f80f03f..0000000 Binary files a/demos/p9.png and /dev/null differ diff --git a/images/.DS_Store b/images/.DS_Store index 5008ddf..2901ed1 100644 Binary files a/images/.DS_Store and b/images/.DS_Store differ diff --git a/images/Intro.png b/images/Intro.png deleted file mode 100644 index 7325af1..0000000 Binary files a/images/Intro.png and /dev/null differ diff --git a/images/VPGTrans.png b/images/VPGTrans.png deleted file mode 100644 index c6b062b..0000000 Binary files a/images/VPGTrans.png and /dev/null differ diff --git a/images/blip2.png b/images/blip2.png deleted file mode 100644 index bbb90ad..0000000 Binary files a/images/blip2.png and /dev/null differ diff --git a/images/cost.png b/images/cost.png deleted file mode 100644 index ba1bc31..0000000 Binary files a/images/cost.png and /dev/null differ diff --git a/images/icon.jpg b/images/icon.jpg new file mode 100644 index 0000000..8227e03 Binary files /dev/null and b/images/icon.jpg differ diff --git a/images/icon.png b/images/icon.png deleted file mode 100644 index df17451..0000000 Binary files a/images/icon.png and /dev/null differ diff --git a/images/overview.png b/images/overview.png deleted file mode 100644 index 10b952e..0000000 Binary files a/images/overview.png and /dev/null differ diff --git a/images/vl-llama.png b/images/vl-llama.png deleted file mode 100644 index c271c79..0000000 Binary files a/images/vl-llama.png and /dev/null differ diff --git a/index.html b/index.html index bb5d394..037fb66 100644 --- a/index.html +++ b/index.html @@ -33,10 +33,10 @@ - + - + @@ -82,7 +82,7 @@

RLHF-V

-

Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback

+

Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback

Tianyu Yu1 @@ -359,16 +359,16 @@

Abstract

Existing Multimodal Large Language Models prevalently suffer from serious hallucination problems, generating text that is not factually grounded in associated images. Our RLHF-V framework enhances MLLM trustworthiness via behavior alignment from fine-grained correctional human feedback.
  • - 1.4K Fine-Grained and Diverse Human Preference Data: We collect 1.4K pieces of segment-level corrections of human feedback on hallucinations, covering hallucination types including objects (41.2%), positions (20.3%), numbers (16.5%), attributes (10%), actions (5.3%), and others (6.8%). + 1.4K Fine-Grained and Diverse Human Preference Data: We collect 1.4K pieces of segment-level corrections of human feedback on hallucinations, covering hallucination types including objects (41.2%), positions (20.3%), numbers (16.5%), attributes (10%), actions (5.3%), and others (6.8%).
  • - High Data Efficiency and Scalability: With just 1.4K annotated data, we achieved a 34.8% reduction in model hallucinations. Moreover, the decrease in hallucinations becomes more significant as more data used. + High Data Efficiency and Scalability: With just 1.4K annotated data, we achieved a 34.8% reduction in model hallucinations. Moreover, the decrease in hallucinations becomes more significant as more data used.
  • - Enhanced Performance and Computational Efficiency with DDPO: Our Dense Direct Preference Optimization (DDPO) algorithm can better exploit the fine-grained human feedback, allowing training in under 1 hour on 8 A100 GPUs. + Enhanced Performance and Computational Efficiency with DDPO: Our Dense Direct Preference Optimization (DDPO) algorithm can better exploit the fine-grained human feedback, allowing training in under 1 hour on 8 A100 GPUs.
  • - Outstanding Trustworthiness without Compromising Helpfulness: Our model surpasses existing open-source MLLMs in reducing hallucination rates, mitigates hallucination from over-generalization, and maintains informativeness. + Outstanding Trustworthiness without Compromising Helpfulness: Our model surpasses existing open-source MLLMs in reducing hallucination rates, mitigates hallucination from over-generalization, and maintains informativeness.