"whisper-gaudi" fails to start

[ctao456]

There was an error with the latest "whisper-gaudi" docker image built from https://github.com/opea-project/GenAIComps/tree/main/comps/asr/whisper

The whisper FastAPI service on gaudi fails to start. Below is the error log:

$ docker run -p 7066:7066 --runtime=habana -e HABANA_VISIBLE_DEVICES=all -e OMPI_MCA_btl_vader_single_copy_mechanism=none --cap-add=sys_nice --ipc=host -e http_proxy=$http_proxy -e https_proxy=$https_proxy opea/whisper-gaudi:comp

[WARNING|utils.py:212] 2024-10-22 22:45:21,543 >> optimum-habana v1.14.0 has been validated for SynapseAI v1.18.0 but habana-frameworks v1.16.1.7 was found, this could lead to undefined behavior!
[WARNING|utils.py:225] 2024-10-22 22:45:22,337 >> optimum-habana v1.14.0 has been validated for SynapseAI v1.18.0 but the driver version is v1.16.2, this could lead to undefined behavior!
/home/user/.local/lib/python3.10/site-packages/transformers/deepspeed.py:24: FutureWarning: transformers.deepspeed module is deprecated and will be removed in a future version. Please import deepspeed modules directly from transformers.integrations
  warnings.warn(
============================= HABANA PT BRIDGE CONFIGURATION =========================== 
 PT_HPU_LAZY_MODE = 1
 PT_RECIPE_CACHE_PATH = 
 PT_CACHE_FOLDER_DELETE = 0
 PT_HPU_RECIPE_CACHE_CONFIG = 
 PT_HPU_MAX_COMPOUND_OP_SIZE = 9223372036854775807
 PT_HPU_LAZY_ACC_PAR_MODE = 1
 PT_HPU_ENABLE_REFINE_DYNAMIC_SHAPES = 0
---------------------------: System Configuration :---------------------------
Num CPU Cores : 152
CPU RAM       : 1056439240 KB
------------------------------------------------------------------------------
Downloading model: openai/whisper-small
[ASR] fetch warmup audio...
[ASR] warmup...
Traceback (most recent call last):
  File "/home/user/comps/asr/whisper/dependency/whisper_server.py", line 62, in <module>
    asr = WhisperModel(model_name_or_path=args.model_name_or_path, language=args.language, device=args.device)
  File "/home/user/comps/asr/whisper/dependency/whisper_model.py", line 36, in __init__
    self._warmup_whisper_hpu_graph("https://github.com/Spycsh/assets/raw/main/ljspeech_60s_audio.wav")
  File "/home/user/comps/asr/whisper/dependency/whisper_model.py", line 100, in _warmup_whisper_hpu_graph
    _ = self.model.generate(
  File "/home/user/.local/lib/python3.10/site-packages/transformers/models/whisper/generation_whisper.py", line 520, in generate
    timestamp_begin = self._set_return_timestamps(
  File "/home/user/.local/lib/python3.10/site-packages/transformers/models/whisper/generation_whisper.py", line 1188, in _set_return_timestamps
    raise ValueError(
ValueError: You have passed more than 3000 mel input features (> 30 seconds) which automatically enables long-form generation which requires the model to predict timestamp tokens. Please either pass `return_timestamps=True` or make sure to pass no more than 3000 mel input features.

It relates to longform generation in the HPU graph warmup for a 60s audio.

[Spycsh]

Hi @ctao456 @chensuyue , here is the fix. #819

[ctao456]

Hi @ctao456 @chensuyue , here is the fix. #819

Thanks @Spycsh!