ENH: numba engine in df.apply #54666
Conversation
lithomas1
added
Apply
|
gentle ping @mroeschke. I'm planning on following this up with general support for numba in df.apply. |
pandas/core/frame.py
Outdated
| @@ -9919,6 +9919,8 @@ def apply( | |||
| result_type: Literal["expand", "reduce", "broadcast"] | None = None, | |||
| args=(), | |||
| by_row: Literal[False, "compat"] = "compat", | |||
| engine: str = "python", | |||
| engine_kwargs: dict = {}, | |||
There was a problem hiding this comment.
I think we usually default this as engine_kwargs: dict[str, bool] | None = None
pandas/core/frame.py
Outdated
| @@ -9919,6 +9919,8 @@ def apply( | |||
| result_type: Literal["expand", "reduce", "broadcast"] | None = None, | |||
| args=(), | |||
| by_row: Literal[False, "compat"] = "compat", | |||
| engine: str = "python", | |||
There was a problem hiding this comment.
Could you type as a Literal here?
| else: | ||
| first_elem = values[0] | ||
| dim0 = values.shape[0] | ||
| res0 = nb_compat_func(first_elem) |
There was a problem hiding this comment.
Is inferring the shape from the first element similar to what we do for DataFrame.apply?
It would be good to note what type of UDFs are supported in the engine docstring
There was a problem hiding this comment.
np.apply_along_axis, which we use does this.
It would be good to note what type of UDFs are supported in the engine docstring
I'll add a note linking to numba's supported Python/numpy features.
|
pre-commit.ci autofix |
for more information, see https://pre-commit.ci
|
pre-commit.ci autofix |
|
Nice! Thanks @lithomas1 |
* ENH: numba engine in df.apply * fixes * more fixes * try to fix * address code review * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * go for green * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update type --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
|
Thanks for the review! |
doc/source/whatsnew/vX.X.X.rstfile if fixing a bug or adding a new feature.