accelerator/rocm: add sync of default stream

when using the blocking hipMemcpy version of the _memcpy operation we seem to need a synchronization of the default stream to ensure completeness in some cases. Signed-off-by: Edgar Gabriel <[email protected]>
open-mpi · Dec 9, 2024 · 4a117d5 · 4a117d5
1 parent 6264853
commit 4a117d5
Showing 1 changed file with 6 additions and 0 deletions.
diff --git a/opal/mca/accelerator/rocm/accelerator_rocm_module.c b/opal/mca/accelerator/rocm/accelerator_rocm_module.c
@@ -390,6 +390,12 @@ static int mca_accelerator_rocm_memcpy(int dest_dev_id, int src_dev_id, void *de
                                 "error during synchronous copy\n");
             return OPAL_ERROR;
         }
+        err = hipStreamSynchronize(0);
+        if (hipSuccess != err ) {
+            opal_output_verbose(10, opal_accelerator_base_framework.framework_output,
+                                "error synchronizing default stream after hipMemcpy\n");
+            return OPAL_ERROR;
+        }
     }
 
     return OPAL_SUCCESS;