REF: Simplify quantile, remove reduction from BlockManager #24597
Conversation
pandas/core/internals/blocks.py
Outdated
| @@ -3365,3 +3324,37 @@ def _putmask_preserve(nv, n): | |||
| v = v.astype(dtype) | |||
|
|
|||
| return _putmask_preserve(v, n) | |||
|
|
|||
|
|
|||
| # TODO: belongs elsewhere? | |||
|
just moved + pushed. hopefully I'll get the docstrings in place before you have to remind me. |
Codecov Report
@@ Coverage Diff @@
## master #24597 +/- ##
===========================================
- Coverage 92.38% 43.04% -49.35%
===========================================
Files 166 166
Lines 52478 52481 +3
===========================================
- Hits 48483 22591 -25892
- Misses 3995 29890 +25895
Continue to review full report at Codecov.
|
Codecov Report
@@ Coverage Diff @@
## master #24597 +/- ##
=========================================
Coverage ? 92.38%
=========================================
Files ? 166
Lines ? 52488
Branches ? 0
=========================================
Hits ? 48491
Misses ? 3997
Partials ? 0
Continue to review full report at Codecov.
|
|
|
||
|
|
||
| def _nanpercentile1D(values, mask, q, na_value, interpolation): | ||
| # mask is Union[ExtensionArray, ndarray] |
There was a problem hiding this comment.
just pushed with docstrings. We're going to simplify the tar out of these methods if/when #24600 gets fixed.
|
one more quantile PR in the pipeline after this |
| """ | ||
|
|
||
| if consolidate: | ||
| self._consolidate_inplace() | ||
|
|
||
| def get_axe(block, qs, axes): |
There was a problem hiding this comment.
i don't think this adds anything to make it a function
| axe, block = getattr(b, f)(axis=axis, axes=self.axes, **kwargs) | ||
| block = b.quantile(axis=axis, qs=qs, interpolation=interpolation) | ||
|
|
||
| axe = get_axe(b, qs, axes=self.axes) |
There was a problem hiding this comment.
also doesn't need / take the bock arg
There was a problem hiding this comment.
it uses the block arg
There was a problem hiding this comment.
(which is why it is a function instead of just done once outside the loop).
I'd rather keep it as a function than in-line it, but not a deal-breaker. There is another PR after this that will be ripping out a bunch of code regardless.
There was a problem hiding this comment.
no i mean get_axe doens't
line 435 inside get_axe reads elif block.ndim == 1:
jreback
added
the
Numeric Operations
|
ping on green. followup to clean internals very welcome! |
|
thanks @jbrockmendel |
BlockManager.reduction is only ever called for quantile. Might as well remove the layer of indirection so we can simplify reduction (now renamed quantile). Most of the simplification comes in Block.quantile, since we can avoid passing around things we don't need.
Two nested functions currently defined inside Block.quantile are moved outside the closure so I don't have to double-check the namespace every time I look at them. Not sure if they belong somewhere else.