PERF: regression in DataFrame.shift method performance

[jorisvandenbossche]

Performance regression in shift caused by #40536

I don't know if it's a particularly useful benchmark, but there is a big slowdown potentially related to this PR: https://pandas.pydata.org/speed/pandas/#frame_methods.Shift.time_shift?python=3.8&Cython=0.29.21&p-axis=1&commits=05a0e98a-bc62e768

Originally posted by @jorisvandenbossche in #40536 (comment)

[jbrockmendel]

2 possible improvements come to mind:

1. Change concat to allow disabling consolidation

2. the axis=1 path added in REF: move shift logic from BlockManager to DataFrame #40536 may be unnecessary (there's a comment that suggests otherwise; would need to look more closely) when isinstance(self._mgr, BlockManager) and self._mgr.nblocks == 1, in which case we could do self.T.shift(n).T

[simonjayhawkins]

2 possible improvements come to mind:

because the regression is a labelled as a refactor and the test fixed is for ArrayManager which is not yet public, is reverting for 1.3 also an option? (and do the refactor for 1.4 instead)

[jbrockmendel]

is reverting for 1.3 also an option? (and do the refactor for 1.4 instead)

sure

[simonjayhawkins]

moving to 1.4, xref #42322

[jbrockmendel]

Closing. The PR that caused the perf regression was reverted, and my initial idea of eventually un-reverting looks very unlikely.