Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Functions applied on .expanding() receive ndarrays rather than pandas objects #12950

Closed
max-sixty opened this issue Apr 21, 2016 · 2 comments
Closed
Labels
Duplicate Report Duplicate issue or pull request Reshaping Concat, Merge/Join, Stack/Unstack, Explode

Comments

@max-sixty
Copy link
Contributor

.apply(...) on window-based groupbys receive an ndarray rather than a Pandas object.
Is this intentional? Is it a performance issue?

It means that functions that work on Series can't necessarily be used on groupbys, which limits abstraction.

In [7]: pd.Series(range(10)).expanding().apply(lambda x: x.pow(2).sum())
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-7-970afaf538ff> in <module>()
----> 1 pd.Series(range(10)).expanding().apply(lambda x: x.pow(2).sum())

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pandas/core/window.py in apply(self, func, args, kwargs)
    929     @Appender(_shared_docs['apply'])
    930     def apply(self, func, args=(), kwargs={}):
--> 931         return super(Expanding, self).apply(func, args=args, kwargs=kwargs)
    932 
    933     @Substitution(name='expanding')

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pandas/core/window.py in apply(self, func, args, kwargs)
    547                                       kwargs)
    548 
--> 549         return self._apply(f, center=False)
    550 
    551     def sum(self, **kwargs):

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pandas/core/window.py in _apply(self, func, window, center, check_minp, how, **kwargs)
    487                 result = np.apply_along_axis(calc, self.axis, values)
    488             else:
--> 489                 result = calc(values)
    490 
    491             if center:

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pandas/core/window.py in calc(x)
    482 
    483                 def calc(x):
--> 484                     return func(x, window, min_periods=self.min_periods)
    485 
    486             if values.ndim > 1:

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pandas/core/window.py in f(arg, window, min_periods)
    545             minp = _use_window(min_periods, window)
    546             return algos.roll_generic(arg, window, minp, offset, func, args,
--> 547                                       kwargs)
    548 
    549         return self._apply(f, center=False)

pandas/algos.pyx in pandas.algos.roll_generic (pandas/algos.c:40613)()

<ipython-input-7-970afaf538ff> in <lambda>(x)
----> 1 pd.Series(range(10)).expanding().apply(lambda x: x.pow(2).sum())

AttributeError: 'numpy.ndarray' object has no attribute 'pow'
Expected (without the coercion):
In [9]: pd.Series(range(10)).expanding().apply(lambda x: pd.Series(x).pow(2).sum())
Out[9]: 
0      0.0
1      1.0
2      5.0
3     14.0
4     30.0
5     55.0
6     91.0
7    140.0
8    204.0
9    285.0
dtype: float64

Pandas 0.18

@jreback
Copy link
Contributor

jreback commented Apr 21, 2016

dupe of #5071 (and noted in the master issue). This is not hard to fix now that we are in the new structure.

@jreback jreback added Reshaping Concat, Merge/Join, Stack/Unstack, Explode Duplicate Report Duplicate issue or pull request labels Apr 21, 2016
@jreback jreback closed this as completed Apr 21, 2016
@max-sixty
Copy link
Contributor Author

Ah, mea culpa. I searched for numpy rather than ndarray

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Duplicate Report Duplicate issue or pull request Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Projects
None yet
Development

No branches or pull requests

2 participants