LoRA with TensorRT #6856
Unanswered
benihime91
asked this question in
Q&A
LoRA with TensorRT
#6856
Replies: 1 comment
-
|
Merging LoRA is possible: check https://huggingface.co/docs/diffusers/main/en/tutorials/using_peft_for_inference |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Just came acrross the TensorRT optimized community pipelines. AFAIK they do not support loading LoRAs (please correct me if i am wrong). I know one option is to directly compile the merged LoRA and base model but that doesn't suit my workflow. Is there any way i can dynamically load and merge LoRA during inference. Any helps / tips ?
Beta Was this translation helpful? Give feedback.
All reactions