[FEA] explore faster data transitions

[revans2]

When working with some of the cache/persist operations it has become very clear that moving data from CPU to GPU and back is a real performance problem. From past experience trying to optimize shuffle, part of the problem come down to the number of buffers that need to be moved. This is something that is going to become more and more of a problem with nested data types. The rest of the problem has a lot to do with the actual data access pattern. Going from row to column and column to row inherently forces one of the operations to stride through memory. This is really bad for the CPU cache.

We have tried to write a custom kernel to translate GPU columnar data into spark's unsafe row format in the past, and it did help some, but the memory format is really wasteful and resulted in bad performance because we could not allocate enough GPU memory to make it worth while.

I personally would like to see us work with the cudf team to develop a packed row based format that we could translate to/from on the GPU. Doing a row based to row based translation is not that expensive for the CPU.

Tasks:

• multi-gb joins result in hangs rapidsai/dask-cudf#102
• Accelerated columnar to row/row to columnar for decimal #1390
• Rewriting row/column conversions for Spark <-> cudf data conversions rapidsai/cudf#8444
• [FEA] Add string support to row/column conversion rapidsai/cudf#10033
• Accelerated columnar to row/row to columnar for nested types
• [FEA] Performance Updates for row/column conversions rapidsai/cudf#10032
• [FEA] Enable Strings as a supported type for GpuColumnarToRow transitions #5633
• [FEA] Update CudfUnsafeRow to include size estimates for strings and implementation for getUTF8String #5634
• [FEA] Update AcceleratedColumnarToRowIterator to support strings #5635
• [FEA] Update GeneratedInternalRowToCudfRowIterator for string transitions #5636
• [FEA] GPU columnar-row transition performance evaluation and tuning #6065

[sameerz]

Removing this from the sprint milestones as this is an overarching feature with sub-tasks for each sprint.