Cuda draft

[danpovey]

making PR for easier commenting

[danpovey]

@csukuangfj I wonder if I could get your help/advice on this...
Bear in mind that this CUDA stuff is not just CUDA, it is a proposal for a refactoring of the entire thing.

This "Context" class is my proposal for how we interact with external toolkits like PyTorch, but there
are some details that I'm not sure about how to do.
We'd then do something like the following (in a separate file where we #include the appropriate torch stuff).

 class PytorchContext: public Context {
 // ...
private:
 Device device;           // The device, e.g. CPU or a specific GPU.
 Allocator *allocator;  // the PyTorch allocator corresponding to this device.
 std::unique_ptr<at::Tensor tensor_> tensor_;  // tensor that holds the memory; this is a special case,
                                                                             // where this Context is attached to a PyTorch tensor.
                                                                             // If Duplicate() is called on a Context with a non-NULL
                                                                            // tensor_ pointer we have to create
                                                                            // a new Context that has a null one.  Also, Deallocate() is
                                                                            // a no-op in this case (memory will be freed, if appropriate,
                                                                            // when the Context is deallocated and we let go of the unique_ptr).
};

The things that I'm a little unclear on right now are:
(a) We need a way to do the python binding. At some point we will want to turn a PyTorch array into some kind of Array1 with a Context attached that is really a PytorchContext. From that point we can pass it into other k2 things. I guess we could have some kind of TorchToK2() call that turns a Torch 1-d tensor into a K2 1-d tensor?

(b) How do we set up the build system? Bear in mind that it would be nice to make the torch stuff separable from the k2 stuff. I.e. we'd like to be able to work with other toolkits in future.

[csukuangfj]

@danpovey

I don't have much experience in CUDA.

To make the torch stuff separable from the k2 stuff, we should avoid things like at::Tensor in k2 since it pulls source code dependencies from PyTorch.

[qindazhu]

@danpovey it seems you forget to push ragged.h?

[danpovey]

Don't worry, anything using at::Tensor can be put in a separate source file and compiled optionally. It's only accessed through a virtual class.

…

On Sat, Jul 25, 2020 at 3:37 PM Fangjun Kuang ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In k2/csrc/cuda/context.h <https://github.com/danpovey/k2/pull/71#discussion_r460377615>: > +}; + + +/** + class Context is the main surface of interaction with external tensor libraries like + PyTorch; it allows us to use their notions of device and of memory allocation. + (in the case of PyTorch it would probably contain a Device and an Allocator). + + We will sub-class this in several ways: with versions that wrap external toolkits + like PyTorch, and also a "native" version that's mostly for testing purposes. + + This object should be allocated with std::shared_ptr, as that's how we store + pointers to it. +*/ +class Context { + @danpovey <https://github.com/danpovey> I don't have much experience in CUDA. To make the torch stuff separable from the k2 stuff, we should avoid things like at::Tensor in k2 since it pulls source code dependencies from PyTorch. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/danpovey/k2/pull/71#discussion_r460377615>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZFLO4575YQIYC3EATBCIDR5KDSDANCNFSM4PGKYLRA> .