Playing-with-ROCm

Here to show my experience about playing with ROCm with runable code, step-by-step tutorial to help you reproduce what I have did. If you have iGPU or dGPU of AMD, you may try Machine Learning with them.

Topics

Training

Finetuning

• LoRA with Radeon
• QLoRA with Radeon

Inference

• Deploy Deepseek-R1 in one GPU -AMD Instinct™ MI300X
• Deploy Llama 3.2 Vision quickly on AMD ROCm with Ollama
• Deploy vLLM service with Kubernetes over AMD ROCm GPU
• Deploy LLM with Radeon iGPU 780M

Application/Demo

• RAG_LLM_QnA_Assistant, Step-by-step tutorial repo project to setup RAG Apps with ROCm

• Ask4ROCm_Chatbot, An chatbot app drive by RAG solution.

• LLM_Voice_Assistant , Use STT/TTS model from Picovoice.

• Easy-Wav2Lip-ROCm, Easy run Wav2Lip with ROCm over AMD GPU. Way2Lip is a project of Generalized Lip Sync Models

• Run EchoMimic with ROCm EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditioning

• Run LLama-3.2-vision with ROCm Ollama+Llama-3.2-vision+ROCm

• Deploy vLLM service with Kubernetes over AMD ROCm GPU , Turoial with sample codes.

---

Projects work over ROCm

These projects may not offical announce to support ROCm GPU. But they work fine base on my verification.

Name	URL	Category	Hands on
CLM-4-Voice	https://github.com/THUDM/GLM-4-Voice	Conversation AI
EchoMimic	https://github.com/BadToBest/EchoMimic	Digital Human GenAI	Run EchoMimic with ROCm
Easy-Wav2Lip	https://github.com/anothermartz/Easy-Wav2Lip	Digital Human GenAI	Easy-Wav2Lip-ROCm
GOT-OCR2	https://github.com/Ucas-HaoranWei/GOT-OCR2.0	end2end OCR
Moshi	https://github.com/kyutai-labs/moshi	Conversation AI
mini-omni	https://github.com/gpt-omni/mini-omni	Conversation AI
mini-omni2	https://github.com/gpt-omni/mini-omni2	Conversation AI
Picovoice/orca	https://github.com/Picovoice/orca	Conversation AI	LLM_Voice_Assistant
Retrieval-based-Voice-Conversion-WebUI	https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI.git	Easily train a good VC model with voice data <= 10 mins!

Wish List

Name	URL	Category	Hands on
hertz-dev	https://github.com/Standard-Intelligence/hertz-dev	Conversation AI
Freeze-Omni	https://github.com/VITA-MLLM/Freeze-Omni	Conversation AI
LLaMA-Omni	https://github.com/ictnlp/LLaMA-Omni	Conversation AI
ichigo Llama 3.1	https://github.com/homebrewltd/ichigo	Conversation AI
ichigo-demo	https://github.com/homebrewltd/ichigo-demo/tree/docker
Exo	https://github.com/exo-explore/exo	heterogeneous distribute inference
Perpleica	https://github.com/ItzCrazyKns/Perplexica	AI Search Engine	issue
MiniPerplx	https://github.com/zaidmukaddam/miniperplx	A minimalistic AI-powered search engine
ollama-helm	https://github.com/otwld/ollama-helm
OpenHands	https://github.com/All-Hands-AI/OpenHands	a platform for software development agents powered by AI
HayStack	https://github.com/deepset-ai/haystack	end-to-end LLM framework that allows you to build applications powered by LLMs
Bailing	https://github.com/ictnlp/BayLing
Bailing	https://github.com/wwbin2017/bailing
BabelDuck	https://github.com/Orenoid/BabelDuck	Beginner-friendly AI conversation practice application
KubeAI	https://github.com/substratusai/kubeai	deploy and manage AI models on Kubernetes
DSPy	https://dspy.ai	the framework for programming
KServe	https://kserve.github.io/website/latest/
Freeze-Omni	https://github.com/VITA-MLLM/Freeze-Omni	A Smart and Low Latency Speech-to-speech Dialogue Model with Frozen LLM
Step-Auido	https://github.com/stepfun-ai/Step-Audio

Tracing

• https://github.com/BradyFU/Awesome-Multimodal-Large-Language-Models

---

@misc{ Playing with ROCm,
  author =   {He Ye (Alex)},
  title =    {Playing with ROCm: share my experience and practice},
  howpublished = {\url{https://alexhegit.github.io/}},
  year = {2024--}
}