Using multi-lines defined in separate columns

[StevenCHowell]

It would be useful to be able to use Datashader to directly plot data from NumPy arrays, particularly for plotting lines. This Stack Overflow question and response demonstrates the workaround I used to put the data into a DataFrame. I am not sure if there is a simple way to accept both types of input. As a fundamental data structure for scientific data, I think it worth considering adding support for NumPy array.

[StevenCHowell]

Something like this could be incorporated to reformat NumPy arrays of lines into a DataFrame that will work well with datashader (following the info in #286):

def np_to_ds_format(x_array, y_arrays):
    """
    reformat data for plotting in DataShader

    Parameters
    ----------
    x_array: array of x-values (should be a one dimensional np.array of length
             N)
    y_arrays: M different arrays of y-values (should be a np.array with
              dimensions
        NxM) L)

    Returns
    -------
    ds_df: datashader DataFrame
    """

    n_points, n_arrays = y_arrays.shape
    assert n_points == len(x_array), 'mismatched x/y-data'

    dfs = []
    split = pd.DataFrame({'x': [np.nan]})
    for i in range(n_arrays):
        x = x_array
        y = y_arrays[:, i]
        df = pd.DataFrame({'x': x, 'y': y})
        dfs.append(df)
        dfs.append(split)

    ds_df = pd.concat(dfs, ignore_index=True)
    return ds_df

[jbednar]

Thanks for the example. I originally thought you were asking for something like what I've just written up in #289, so I've changed the title of this issue to reflect that you're really asking for a convenient way to work with lines defined in independent arrays/columns, as opposed to a single unified dataframe separated with NaNs.

The code is helpful, but because it accepts only a single x array, it will only work for a fairly specialized scenario where all lines have the same number of samples and all samples are stored in a single array. Datashader will happily work with any number of different lines of different sizes, which in NumPy could be represented variously as pairs of x and y arrays matching in size, or as single arrays with x and y columns. It's hard to see why we would provide a function for the above special case if we didn't also cover these other cases; I don't think any one of them is particularly more common than the others. And generalizing the above example to cover all those cases could get tricky.

So the above code is definitely useful as an example, but I'm not sure it makes sense to be part of the library as it is.

[jbednar]

Also, similar issues apply to Pandas dataframes that have dozens or hundreds of columns, where someone may reasonably want to be able to specify a list of those names and then plot them directly, rather than by constructing another dataframe concatenating them.

[jbednar]

I think this issue is addressed by the separately developed ds.utils.dataframe_from_multiple_sequences utility.