Mosec embedding RuntimeError with Milvus data-prep file upload

[CharleneHu-42]

After started dataprep-milvus-server with Mosec embedding service, when uploading file to Milvus db with:

curl -X POST "http://${host_ip}:6010/v1/dataprep" \
     -H "Content-Type: multipart/form-data" \
     -F "files=@./nke-10k-2023.pdf"

Mosec embedding service throws RuntimeError: The size of tensor a (1099) must match the size of tensor b (512) at non-singleton dimension 1:

2024-08-26 08:09:45,086 - mosec.log - WARNING - Traceback (most recent call last):   File "/usr/local/lib/python3.10/dist-packages/mosec/coordinator.py", line 280, in coordinate     self.worker.forward(data)   File "/home/user/comps/embeddings/langchain-mosec/mosec-docker/server-ipex.py", line 82, in forward     token_cnt, embeddings = self.get_embedding_with_token_count(inputs)   File "/home/user/comps/embeddings/langchain-mosec/mosec-docker/server-ipex.py", line 62, in get_embedding_with_token_count     model_output = self.model(**inputs)   File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl     return self._call_impl(*args, **kwargs)   File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1520, in _call_impl     return forward_call(*args, **kwargs) RuntimeError: The following operation failed in the TorchScript interpreter. Traceback of TorchScript (most recent call last): /usr/local/lib/python3.10/dist-packages/transformers/models/bert/modeling_bert.py(216): forward /usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py(1501): _slow_forward /usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py(1520): _call_impl /usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py(1511): _wrapped_call_impl /usr/local/lib/python3.10/dist-packages/transformers/models/bert/modeling_bert.py(1077): forward /usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py(1501): _slow_forward /usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py(1520): _call_impl /usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py(1511): _wrapped_call_impl /usr/local/lib/python3.10/dist-packages/torch/jit/_trace.py(1074): trace_module /usr/local/lib/python3.10/dist-packages/torch/jit/_trace.py(806): trace /usr/local/lib/python3.10/dist-packages/intel_extension_for_pytorch/jit/_trace.py(69): wrapper /home/user/comps/embeddings/langchain-mosec/mosec-docker/server-ipex.py(42): __init__ /usr/local/lib/python3.10/dist-packages/mosec/coordinator.py(132): __init__ /usr/lib/python3.10/multiprocessing/process.py(108): run /usr/lib/python3.10/multiprocessing/process.py(314): _bootstrap /usr/lib/python3.10/multiprocessing/spawn.py(129): _main /usr/lib/python3.10/multiprocessing/spawn.py(116): spawn_main <string>(1): <module> RuntimeError: The size of tensor a (1099) must match the size of tensor b (512) at non-singleton dimension 1

[letonghan]

Caused by max token limitation of Mosec embedding.
Fixed by PR #570. Will close this issue.