Replies: 2 comments
-
|
Hello, @b4rtaz! I've noticed that my transfer times were much smaller than those you posted here for the 8 nodes cluster running Llama 2 7B and 13B. I was getting similar transfer times at first but I realized that my Raspberry Pis were being throttled due to inadequate power supply (which seems to be a common problem). Maybe you had the same issue here? Just to be clear, this is not a critic at all, ok? I'm just curious about this difference in our transfer times and also thought that it might be helpful to keep this potential throttle issue in mind as it may significantly reduce distributed llama performance. |
Beta Was this translation helpful? Give feedback.
-
|
Hello @EntusiastaIApy, in my setup, the power supply could be a problem because I didn't use the original chargers. Additionally, you might have a better router; my TP-Link definitely is not the best you can find on the market. |
Beta Was this translation helpful? Give feedback.
-
Weights: Q40
Buffer: Q80
(I: 1307.94 ms, T: 1.81 ms)
(I: 739.00 ms, T: 52.50 ms)
(I: 458.81 ms, T: 34.06 ms)
(I: 296.69 ms, T: 289.75 ms)
(I: 1465.06 ms, T: 30.88 ms)
(I: 746.88 ms, T: 99.50 ms)
(I: 460.8 ms, T: 652.88 ms)
(I: 2121.94 ms, T: 2719.62 ms)
Beta Was this translation helpful? Give feedback.
All reactions