[DataPrep Microservice] Error connection to RedisDB due to the incorrect configuration usage on REDIS_URL and REDIS_HOST

[zhlsunshine]

When I deploy the data-prep microservice in the chatQnA example, I encountered below error message:

files:UploadFile(filename='gaudi.txt', size=1727, headers=Headers({'content-disposition': 'form-data; name="files"; filename="gaudi.txt"', 'content-type': 'application/octet-stream'}))
link_list:None
Parsing document ./uploaded_files/gaudi.txt.
Done preprocessing. Created  2  chunks of the original pdf
Current batch: 0
keys: ['doc:data-prep:e7b0f36035bb47d29e543d5352f0a8e1', 'doc:data-prep:c903764a790747aeb05830de421335f4']
Processed batch 1/1
[ check index existence ] checking <redis.commands.search.Search object at 0x7f3f4d4125d0>
[ check index existence ] index does not exist: Error 111 connecting to localhost:6379. Connection refused.
[ create index ] creating index file-keys
[ create index ] fail to create index file-keys: Error 111 connecting to localhost:6379. Connection refused.
INFO:     10.244.0.18:39596 - "POST /v1/dataprep HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 399, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/home/user/.local/lib/python3.11/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/user/.local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/home/user/.local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/home/user/.local/lib/python3.11/site-packages/prometheus_fastapi_instrumentator/middleware.py", line 174, in __call__
    raise exc
  File "/home/user/.local/lib/python3.11/site-packages/prometheus_fastapi_instrumentator/middleware.py", line 172, in __call__
    await self.app(scope, receive, send_wrapper)
  File "/home/user/.local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 85, in __call__
    await self.app(scope, receive, send)
  File "/home/user/.local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 65, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/home/user/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/home/user/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/home/user/.local/lib/python3.11/site-packages/starlette/routing.py", line 756, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/user/.local/lib/python3.11/site-packages/starlette/routing.py", line 776, in app
    await route.handle(scope, receive, send)
  File "/home/user/.local/lib/python3.11/site-packages/starlette/routing.py", line 297, in handle
    await self.app(scope, receive, send)
  File "/home/user/.local/lib/python3.11/site-packages/starlette/routing.py", line 77, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/home/user/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/home/user/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/home/user/.local/lib/python3.11/site-packages/starlette/routing.py", line 72, in app
    response = await func(request)
               ^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app
    raw_response = await run_endpoint_function(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/lib/python3.11/site-packages/langsmith/run_helpers.py", line 468, in async_wrapper
    raise e
  File "/home/user/.local/lib/python3.11/site-packages/langsmith/run_helpers.py", line 454, in async_wrapper
    function_result = await asyncio.create_task(  # type: ignore[call-arg]
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/comps/dataprep/redis/langchain/prepare_doc_redis.py", line 231, in ingest_documents
    ingest_data_to_redis(
  File "/home/user/comps/dataprep/redis/langchain/prepare_doc_redis.py", line 165, in ingest_data_to_redis
    assert create_index(client)
           ^^^^^^^^^^^^^^^^^^^^
AssertionError

After investigation with @Spycsh, it seems that the root cause is the incorrect configuration usage on REDIS_URL and REDIS_HOST. Even if I set correctly REDIS_URL, data-prep still connect to localhost is the default value of REDIS_HOST to create . Please refer to here for more information.

[Spycsh]

Hi @lvliang-intel @letonghan , REDIS_HOST and REDIS_PORT should be extracted directly from REDIS_URL but from this line https://github.com/opea-project/GenAIComps/blob/main/comps/dataprep/redis/langchain/config.py#L11 it seems that users also need to set these REDIS_HOST and REDIS_PORT. Is that correct?

[letonghan]

Hi @lvliang-intel @letonghan , REDIS_HOST and REDIS_PORT should be extracted directly from REDIS_URL but from this line https://github.com/opea-project/GenAIComps/blob/main/comps/dataprep/redis/langchain/config.py#L11 it seems that users also need to set these REDIS_HOST and REDIS_PORT. Is that correct?

Seems it could be solved by #359

[kevinintel]

Since the PR has been merged, please take a try

[zhlsunshine]

Hi @Spycsh and @kevinintel, I have validated that it's fixed, thanks so much for your efforts!