[Bug] Fix VDMS retriever and apply fix to VDMS dataprep

[cwlacewe]

Description

These changes are to fix the error in Action job #33234707275. The container crashes and the run complained about the protobuf version. To fix this, the requirements.txt was modified by pinning the protobuf version, pinned opentelemetry-proto version to avoid any conflict, and provided a minimum vdms version.

Issues

Issue #923

Type of change

List the type of change like below. Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds new functionality)
• Breaking change (fix or feature that would break existing design and interface)
• Others (enhancement, documentation, validation, etc.)

Dependencies

Modified following dependency versions

opentelemetry-proto==1.23.0 (pinned to avoid a conflict)
protobuf==4.24.2 (required by vdms)
vdms>=0.0.20

Tests

Ran the VDMS retriever test locally to verify it runs successfully using the following:

cd comps/
bash ../tests/retrievers/test_retrievers_vdms_langchain.sh

VDMS is also used in dataprep so tested the services for both multimodal_langchain and langchain respectively:

cd ../
export http_proxy=${http_proxy}
export https_proxy=${https_proxy}
export host_ip=$(hostname -I | awk '{print $1}')
export VDMS_HOST=${host_ip}
export VDMS_PORT=55555
export INDEX_NAME="rag-vdms"
export your_hf_api_token="{your_hf_token}"

docker build -t opea/dataprep-vdms:latest --network host --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/dataprep/vdms/multimodal_langchain/Dockerfile .

docker compose -f comps/dataprep/vdms/multimodal_langchain/docker-compose-dataprep-vdms.yaml up -d

docker container logs -f dataprep-vdms-server

curl -X POST \
     -F 'link_list=["https://www.ces.tech/"]' \
     http://localhost:6007/v1/dataprep

docker compose -f comps/dataprep/vdms/multimodal_langchain/docker-compose-dataprep-vdms.yaml down

To test langchain:

docker build -t opea/dataprep-vdms:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/dataprep/vdms/langchain/Dockerfile .

export http_proxy=${http_proxy}
export https_proxy=${https_proxy}
export host_ip=$(hostname -I | awk '{print $1}')
export VDMS_HOST=${host_ip}
export VDMS_PORT=55555
export COLLECTION_NAME="rag-vdms"
export your_hf_api_token="{your_hf_token}"

docker compose -f comps/dataprep/vdms/langchain/docker-compose-dataprep-vdms.yaml up -d

docker container logs -f dataprep-vdms-server

curl -X POST \
     -F 'link_list=["https://www.ces.tech/"]' \
     http://localhost:6007/v1/dataprep

docker compose -f comps/dataprep/vdms/langchain/docker-compose-dataprep-vdms.yaml down