From 0af532a3eb8cd7d969fc097deaeeb0c823f52cbd Mon Sep 17 00:00:00 2001 From: ZePan110 Date: Fri, 20 Sep 2024 16:23:40 +0800 Subject: [PATCH] Remove useless file. (#138) Signed-off-by: ZePan110 --- examples/AudioQnA/README.md | 48 ------------------------------------- 1 file changed, 48 deletions(-) delete mode 100644 examples/AudioQnA/README.md diff --git a/examples/AudioQnA/README.md b/examples/AudioQnA/README.md deleted file mode 100644 index 45290620..00000000 --- a/examples/AudioQnA/README.md +++ /dev/null @@ -1,48 +0,0 @@ -# AudioQnA accuracy Evaluation - -## Dataset - - -We evaluate the ASR accuracy on the test set of librispeech [dataset](https://huggingface.co/datasets/andreagasparini/librispeech_test_only), which contains 2620 records of audio and texts. - -## Metrics - -We evaluate the WER (Word Error Rate) metric of the ASR microservice. - -## Evaluation - -### Launch ASR microservice - -Launch the ASR microserice with the following commands. For more details please refer to [doc](https://github.com/opea-project/GenAIComps/tree/main/comps/asr). - -```bash -git clone https://github.com/opea-project/GenAIComps -cd GenAIComps -docker build -t opea/whisper:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/asr/whisper/Dockerfile . -# change the name of model by editing model_name_or_path you want to evaluate -docker run -p 7066:7066 --ipc=host -e http_proxy=$http_proxy -e https_proxy=$https_proxy opea/whisper:latest --model_name_or_path "openai/whisper-tiny" -``` - -### Evaluate - -Install dependencies: - -``` -pip install -r requirements.txt -``` - -Evaluate the performance with the LLM: -```py -# validate the offline model -# python offline_evaluate.py -# validate the online asr microservice accuracy -python online_evaluate.py -``` - -### Performance Result -Here is the tested result for your reference -|| WER | -| --- | ---- | -|whisper-large-v2| 2.87| -|whisper-large| 2.7 | -|whisper-medium| 3.45 |