[FEA] libcudf Series correlation (Pearson)

[beckernick]

Is your feature request related to a problem? Please describe.
As a cuDF user, I want to calculate the correlation of two series. Pearson correlation is likely the most commonly used as it is the default in Pandas (API docs).

Describe the solution you'd like
I'd like to be able to do this with series1.corr(series2) and also on DataFrame and Groupby objects.

Describe alternatives you've considered
The alternative is to actually calculate the correlation manually, which is cumbersome.

[beckernick]

Updating this to explicitly refer to a libcuDF implementation now that #2719 has merged (providing Series.corr)

[beckernick]

Updating this to also refer to DataFrame level correlation

[rnyak]

@beckernick Can we now apply corr() after groupby? Basically, I want to be able to do the following with gdf.

ans = pdf[['id2', 'id4' ,'v1', 'v2']].groupby(['id2', 'id4']).apply(lambda x: pd.Series({'r2': x.corr()['v1']['v2']}))

Thanks.

[beckernick]

Assuming you mean once #4140 merges , we cannot. Groupby correlation via the standard API will require a libcuDF implementation.

[beckernick]

@jrhemstad discussed this today in the context of groupbys. Supporting correlation (and implicitly covariance) in the groupby machinery would require additional design, as the aggregation takes more than one input. I'm going to file a new issue to summarize and consolidate further discussion for the groupby aggregation