Float64 conversion on GPU device

[IvanBioli]

Hi,
Is there any way to automatically convert Float64 arrays to gpu, without first downgrading precision to Float64. Currently, I do like this, but I clearly loose precision. When initializing the network's weight is fine, but sometimes this is not the case and it would be good to be able to set somehow the conversion

using Lux, LuxCUDA

const gpu = gpu_device()

x = rand(rng, 5, 3)
x_gpu = x |> gpu
x_gpu_f64 = f64(x_gpu)

[avik-pal]

Not yet, see https://github.com/LuxDL/MLDataDevices.jl/issues/61.

But if you pass in MLDataDevices.default_device_rng(dev) to the initialization functions, it will directly generate the parameters on GPU https://lux.csail.mit.edu/stable/api/Building_Blocks/WeightInitializers#Supported-RNG-Types-WeightInit. Combine it with setting the precision to Float64 to the init_* functions and you will get the desired behavior. A bit inconvenient currently but we should fix it...