Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added safe wrappers called copy_from_async_sync and copy_to_async_syc in crates/cust/src/memory/device/device_slice.rs #140

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Adesoji1
Copy link

@Adesoji1 Adesoji1 commented Feb 2, 2025

Now i introduced Safe Wrappers:
I suppose that these new two new methods (named copy_from_async_sync and copy_to_async_sync) simply wraps the existing unsafe methods defined by the AsyncCopyDestination(
https://bheisler.github.io/RustaCUDA/rustacuda/memory/trait.CopyDestination.html) trait. So they perform the asynchronous copy and then immediately call stream.synchronize(), thereby ensuring that the copy is complete before returning right?.
Furthermore, i know that with these additions, users who do not need overlapping computation can avoid unsafe blocks and explicit synchronization. This methods return a CudaResult<()> (https://bheisler.github.io/RustaCUDA/rustacuda/error/enum.CudaError.html) so that any error from either the asynchronous copy or the stream synchronization is propagated.

Availability on DeviceBuffer:
Now Since DeviceBuffer implements Deref<Target = DeviceSlice>, these new methods are also available on DeviceBuffer.

…c in crates/cust/src/memory/device/device_slice.rs
@juntyr
Copy link
Collaborator

juntyr commented Feb 2, 2025

Thank you for the PR! What can these new wrappers do that the CopyDestination trait methods, which are sync and safe, cannot? Does the addition of an explicit stream parameter help with some problem?

@Adesoji1
Copy link
Author

Adesoji1 commented Feb 3, 2025

Thank you for the PR! What can these new wrappers do that the CopyDestination trait methods, which are sync and safe, cannot? Does the addition of an explicit stream parameter help with some problem?

thank you @juntyr i suppose that while the synchronous CopyDestination methods block until the copy is complete using the default or an internal stream, the new wrappers let the caller supply a specific stream. i believe that this is important for cases of when you want to integrate the copy into a larger asynchronous workflow or maybe coordinate it with other operations running on that stream though i stand to be corrected .Also i believe that the asynchronous with safety copy means that if needed, one could modify the usage (or write additional wrappers) to take advantage of overlapping computation and data transfer.

@LegNeato
Copy link
Contributor

LegNeato commented Feb 8, 2025

Is this a theoretical improvement or do you have code or intend to write code that needs this? I'd love to see an actual example.

@Adesoji1
Copy link
Author

Is this a theoretical improvement or do you have code or intend to write code that needs this? I'd love to see an actual example.

For a code that need this, kindly view


use cust::prelude::*; // This is for the safe wrapper around the unsafe API.
use cust::memory::*;
use cust::stream::{Stream, StreamFlags};

fn main() -> cust::error::CudaResult<()> {
    // Now let us Initialize a CUDA context.
    let _ctx = cust::quick_init()?;

    // Ghen Create a non-blocking stream. With this stream, you can interleave other asynchronous operations if desired.
    let stream = Stream::new(StreamFlags::NON_BLOCKING, None)?;

    // Now the source where the daya is stored will be called host data: Host data that we want to copy into a device buffer.
    let host_data = [10u32, 20, 30, 40, 50, 60];

    // Create a device buffer with the same length as host_data.
    let mut device_buffer = DeviceBuffer::from_slice(&[0u32; 6])?;

    // Use the new safe wrapper to asynchronously copy data from host_data to the device buffer.
    // This call internally uses an unsafe asynchronous copy but immediately synchronizes the stream.
    device_buffer
        .as_slice_mut()
        .copy_from_async_sync(&host_data, &stream)?;

    // Prepare a host array to copy the data back from the device.
    let mut host_result = [0u32; 6];

    // Use the safe wrapper to asynchronously copy the data back to the host.
    device_buffer
        .as_slice()
        .copy_to_async_sync(&mut host_result, &stream)?;

    //Then  Verify that the data round-tripped correctly.
    assert_eq!(host_data, host_result);
    println!("Async sync copy successful: {:?}", host_result);

    Ok(())
}
 Then the output should be : Async sync copy successful: [10, 20, 30, 40, 50, 60]


i await your response on this, also for the CI/CD test of the code, it failed because of the version in the github actions but after updating, you can run the code, ot will give no error, Thank you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants