Add cudf.DataFrame.applymap
#10542
Add cudf.DataFrame.applymap
#10542
Conversation
brandon-b-miller
added
feature request
python/cudf/cudf/core/dataframe.py
Outdated
| func : callable | ||
| Python function, returns a single value from a single value. | ||
| na_action : {None, 'ignore'}, default None | ||
| If ``ignore``, propagate NaN values, without passing them to func. |
There was a problem hiding this comment.
Use quotes here, not code font.
| If ``ignore``, propagate NaN values, without passing them to func. | |
| If 'ignore', propagate NaN values, without passing them to func. |
python/cudf/cudf/core/dataframe.py
Outdated
| """ | ||
|
|
||
| if kwargs: | ||
| raise ValueError( |
There was a problem hiding this comment.
I think we usually raise NotImplementedError for this kind of thing, and ValueError for invalid values (like na_action not in {"ignore", None} below).
| raise ValueError( | |
| raise NotImplementedError( |
Codecov Report
@@ Coverage Diff @@
## branch-22.06 #10542 +/- ##
================================================
+ Coverage 86.33% 86.38% +0.04%
================================================
Files 140 142 +2
Lines 22289 22338 +49
================================================
+ Hits 19244 19296 +52
+ Misses 3045 3042 -3
Continue to review full report at Codecov.
|
brandon-b-miller
added
3 - Ready for Review
| df = pd.DataFrame({"x": [], "y": []}) | ||
| gdf = cudf.DataFrame.from_pandas(df) | ||
| dgf = dd.from_pandas(gdf, npartitions=npartitions) | ||
| return dgf |
There was a problem hiding this comment.
Seems strange that this only returns dgf while _make_random_frame and _make_random_frame_float return df, dgf. Should we symmetrize this?
There was a problem hiding this comment.
Sorry, I probably shouldn't have moved this function in the first place since it's being consumed elsewhere and not actually used in my tests. I just moved it back for now.
|
@gpucibot merge |
|
oops, needs dask review. |
| @@ -3718,6 +3720,68 @@ def apply( | |||
|
|
|||
| return self._apply(func, _get_row_kernel, *args, **kwargs) | |||
|
|
|||
| def applymap( | |||
There was a problem hiding this comment.
Could you also add this entry to this section of docs: https://github.com/rapidsai/cudf/blob/branch-22.06/docs/cudf/source/api_docs/dataframe.rst#function-application-groupby--window
|
|
||
| from dask import dataframe as dd | ||
|
|
||
| from .utils import _make_random_frame |
There was a problem hiding this comment.
Can we do an absolute import here instead of a relative import so that it is consistent with other imports here and elsewhere in the code-base?
| @@ -8,6 +10,8 @@ | |||
|
|
|||
| import cudf | |||
|
|
|||
| from .utils import _make_random_frame | |||
Naive implementation of
DataFrame.applymapthat just callsapplyin a loop over columns.This could theoretically be made much faster within our framework. This requires at worst
Ncompilations andMkernel launches, whereNis the number of different dtypes in the data, andMis the number of total columns. We could however as an improvement to this launch just one kernel that populates the entire output data. This would still suffer from the compilation bottleneck however, since the function must be compiled in order for an output dtype to be determined, and this will need to be done for each distinct dtype within the data.Part of #10169