Replies: 6 comments 3 replies
-
|
Not a bother at all! I’m not sure what’s happening but can you try the same
thing with “—no-deepspeed” just to see if that works with the coqui voices?
…On Mon, Jan 8, 2024 at 2:36 PM Nikanoru ***@***.***> wrote:
Sorry it is me again again.
Whenever I try to use xtts like for example with this command: epub2tts
AN.epub --start 4 --end 4 --engine xtts --speaker "Damien Black"
I get the following errors in the terminal:
`[...]
Number of chapters to read: 23
Saving to AN-damien-black.m4b
Total characters: 10768
Using GPU
VRAM: 8358723584
Loading model:
/home/david/.local/share/tts/tts_models--multilingual--multi-dataset--xtts_v2
tts_models/multilingual/multi-dataset/xtts_v2 is already downloaded.
Using model: xtts
[2024-01-08 23:32:19,592] [INFO] [real_accelerator.py:161:get_accelerator]
Setting ds_accelerator to cuda (auto detect)
[2024-01-08 23:32:19,811] [INFO] [logging.py:96:log_dist] [Rank -1]
DeepSpeed info: version=0.12.6, git-hash=unknown, git-branch=unknown
[2024-01-08 23:32:19,811] [WARNING]
[config_utils.py:69:_process_deprecated_field] Config parameter
replace_method is deprecated. This parameter is no longer needed, please
remove from your call to DeepSpeed-inference
[2024-01-08 23:32:19,811] [WARNING]
[config_utils.py:69:_process_deprecated_field] Config parameter mp_size is
deprecated use tensor_parallel.tp_size instead
[2024-01-08 23:32:19,812] [INFO] [logging.py:96:log_dist] [Rank -1]
quantize_bits = 8 mlp_extra_grouping = False, quantize_groups = 1
Traceback (most recent call last):
File "/home/david/.local/bin/epub2tts", line 8, in
sys.exit(main())
File "/home/david/.local/lib/python3.10/site-packages/epub2tts.py", line
757, in main
mybook.read_book(
File "/home/david/.local/lib/python3.10/site-packages/epub2tts.py", line
390, in read_book
self.model.load_checkpoint(
File
"/home/david/.local/lib/python3.10/site-packages/TTS/tts/models/xtts.py",
line 783, in load_checkpoint
self.gpt.init_gpt_for_inference(kv_cache=self.args.kv_cache,
use_deepspeed=use_deepspeed)
File
"/home/david/.local/lib/python3.10/site-packages/TTS/tts/layers/xtts/gpt.py",
line 224, in init_gpt_for_inference
self.ds_engine = deepspeed.init_inference(
File "/home/david/.local/lib/python3.10/site-packages/deepspeed/*init*.py",
line 342, in init_inference
engine = InferenceEngine(model, config=ds_inference_config)
File
"/home/david/.local/lib/python3.10/site-packages/deepspeed/inference/engine.py",
line 158, in *init*
self._apply_injection_policy(config)
File
"/home/david/.local/lib/python3.10/site-packages/deepspeed/inference/engine.py",
line 418, in _apply_injection_policy
replace_transformer_layer(client_module, self.module, checkpoint, config,
self.config)
File
"/home/david/.local/lib/python3.10/site-packages/deepspeed/module_inject/replace_module.py",
line 342, in replace_transformer_layer
replaced_module = replace_module(model=model,
File
"/home/david/.local/lib/python3.10/site-packages/deepspeed/module_inject/replace_module.py",
line 586, in replace_module
replaced_module, _ = _replace_module(model, policy, state_dict=sd)
File
"/home/david/.local/lib/python3.10/site-packages/deepspeed/module_inject/replace_module.py",
line 646, in _replace_module
_, layer_id = _replace_module(child,
File
"/home/david/.local/lib/python3.10/site-packages/deepspeed/module_inject/replace_module.py",
line 646, in _replace_module
_, layer_id = _replace_module(child,
File
"/home/david/.local/lib/python3.10/site-packages/deepspeed/module_inject/replace_module.py",
line 622, in _replace_module
replaced_module = policies[child.*class*][0](child,
File
"/home/david/.local/lib/python3.10/site-packages/deepspeed/module_inject/replace_module.py",
line 298, in replace_fn
new_module = replace_with_policy(child,
File
"/home/david/.local/lib/python3.10/site-packages/deepspeed/module_inject/replace_module.py",
line 247, in replace_with_policy
_container.create_module()
File
"/home/david/.local/lib/python3.10/site-packages/deepspeed/module_inject/containers/gpt2.py",
line 20, in create_module
self.module = DeepSpeedGPTInference(_config, mp_group=self.mp_group)
File
"/home/david/.local/lib/python3.10/site-packages/deepspeed/model_implementations/transformers/ds_gpt.py",
line 20, in *init*
super().*init*(config, mp_group, quantize_scales, quantize_groups,
merge_count, mlp_extra_grouping)
File
"/home/david/.local/lib/python3.10/site-packages/deepspeed/model_implementations/transformers/ds_transformer.py",
line 58, in *init*
inference_module = builder.load()
File
"/home/david/.local/lib/python3.10/site-packages/deepspeed/ops/op_builder/builder.py",
line 458, in load
return self.jit_load(verbose)
File
"/home/david/.local/lib/python3.10/site-packages/deepspeed/ops/op_builder/builder.py",
line 461, in jit_load
if not self.is_compatible(verbose):
File
"/home/david/.local/lib/python3.10/site-packages/deepspeed/ops/op_builder/transformer_inference.py",
line 29, in is_compatible
sys_cuda_major, _ = installed_cuda_version()
File
"/home/david/.local/lib/python3.10/site-packages/deepspeed/ops/op_builder/builder.py",
line 50, in installed_cuda_version
raise MissingCUDAException("CUDA_HOME does not exist, unable to compile
CUDA op(s)")
deepspeed.ops.op_builder.builder.MissingCUDAException: CUDA_HOME does not
exist, unable to compile CUDA op(s)
`
I installed Ubuntu 22.04.3 LTS today because I was not able to get
deepspeed to install on Windows 11. I am not that experienced, but
installation seemed to have worked well and everything was much easier than
in Windows I have to say. Deepspeed installed with no issues aswell from
what I can see and I can even use the xtts v2 voices when I just use
https://github.com/coqui-ai/TTS but in epub2tts for an unknown reason to
me, it does not work and gives me the errors above. When I don't use
--engine xtss it also works fine.
In case it is important:
GPU: nVidia RTX 2070
CPU: i7-9700k
Sorry again! I hope I am not too bothersome. Perhaps someone knows the
reason for this issue?
—
Reply to this email directly, view it on GitHub
<#184>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAFBJGOSR4G4W6CEKRRAQPLYNRYIFAVCNFSM6AAAAABBSFLZPOVHI2DSMVQWIX3LMV43ERDJONRXK43TNFXW4OZWGA2DMMJZGA>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
using --no-deepspeed does indeed work :) Thank you very much. And Damien Black does sound very nice indeed. Now I just need to figure out how to use those xtts v2 ones where I need to put 3 different wav files in >.< |
Beta Was this translation helpful? Give feedback.
-
|
I've just installed CUDA Toolkit https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64 and DeepSpeed works ok. And you get a boost in speed conversion. |
Beta Was this translation helpful? Give feedback.
-
|
OK thanks! This is not a fix, but it narrows things down to the issue being with deepespeed. I feel like there may be some other dependencies that come with deepspeed which I have already installed, but are not documented or included in the install instructions (maybe cuda-toolkit or something like that). I'm glad you are up and running for now. Later this week I will probably have time to make a fully clean environment to validate all the installation steps and see if I can reproduce any of the problems you've run into. |
Beta Was this translation helpful? Give feedback.
-
|
Since xtts worked fine for me with the cuqoi/tts installation, would it be possible to just compare what they install vs what gets installed here? Not sure if what I say makes sense. I will probably spend some time googling tomorrow if I can find the solution :) |
Beta Was this translation helpful? Give feedback.
-
|
The solution to get deepspeed working has been posted here: #168 (comment) by the User "friki67" . Thank you very much. |
Beta Was this translation helpful? Give feedback.
-
|
I really appreciate these kinds of questions and answers - I have in mind a MAJOR re-do of the README page since there has been a lot of new documentation that has not really landed there, along with lots of new features since that has been written. Hopefully I'll be able to make time for that over the next week or two. |
Beta Was this translation helpful? Give feedback.
-
|
You have really created a wonderful piece of software here that builds upon cocqui-tts and expands its functionality in a super useful way. I am really glad that you made this, and it's great to hear that you even appreciate our problems and the resulting discourse :) If you will ever accept some donations for this, I would be glad to send a few $ your way. |
Beta Was this translation helpful? Give feedback.
-
|
You all are very welcome, no need to get any money for this. I am really happy to be doing something in open source that a few more people than me are able to use (I initially made this just for myself :) ) |
Beta Was this translation helpful? Give feedback.
-
EDIT: I just noticed that a post in the Issues Category regarding this problem is open. If there is anything I can do to help find the issue let me know. I have no problem if I have to wipe the partition and install Ubuntu fresh again if it helps. I find it interesting that https://github.com/coqui-ai/TTS works fine with xtts.
Sorry it is me again again.
Whenever I try to use xtts like for example with this command: epub2tts AN.epub --start 4 --end 4 --engine xtts --speaker "Damien Black"
I get the following errors in the terminal:
`[...]
Number of chapters to read: 23
Saving to AN-damien-black.m4b
Total characters: 10768
Using GPU
VRAM: 8358723584
Loading model: /home/david/.local/share/tts/tts_models--multilingual--multi-dataset--xtts_v2
I installed Ubuntu 22.04.3 LTS today because I was not able to get deepspeed to install on Windows 11. I am not that experienced, but installation seemed to have worked well and everything was much easier than in Windows I have to say. Deepspeed installed with no issues aswell from what I can see and I can even use the xtts v2 voices when I just use https://github.com/coqui-ai/TTS but in epub2tts for an unknown reason to me, it does not work and gives me the errors above. When I don't use --engine xtss it also works fine.
In case it is important:
GPU: nVidia RTX 2070
CPU: i7-9700k
Sorry again! I hope I am not too bothersome. Perhaps someone knows the reason for this issue?
Beta Was this translation helpful? Give feedback.
All reactions