Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

REF: concat on bm_axis==0 #43626

Merged
merged 5 commits into from
Sep 17, 2021
Merged

Conversation

jbrockmendel
Copy link
Member

Makes it so we no longer need indexers in JoinUnit (and in fact, so that JoinUnit is only used for axis==1)

asv continuous -E virtualenv -f 1.01 master HEAD --append-samples --record-samples -b join_merge
[...]
       before           after         ratio
     [26064f0f]       [5dedf170]
     <master>         <ref-concat-axis0>
-      15.8±0.9ms       15.0±0.4ms     0.95  join_merge.MergeAsof.time_on_int('backward', None)
-      16.8±0.4ms       15.9±0.3ms     0.94  join_merge.MergeAsof.time_on_int('forward', None)
-      19.6±0.9ms       18.5±0.2ms     0.94  join_merge.MergeAsof.time_on_int('nearest', None)
-      16.1±0.4ms       15.1±0.3ms     0.94  join_merge.MergeAsof.time_on_int('backward', 5)
-      17.8±0.9ms       16.7±0.4ms     0.94  join_merge.MergeAsof.time_on_int32('backward', None)
-        84.4±6ms         78.6±2ms     0.93  join_merge.MergeAsof.time_by_object('backward', 5)
-        83.1±3ms         77.3±2ms     0.93  join_merge.MergeAsof.time_by_object('backward', None)
-        20.1±1ms       18.6±0.5ms     0.93  join_merge.MergeAsof.time_on_uint64('nearest', 5)
-      20.2±0.8ms       18.7±0.3ms     0.93  join_merge.MergeAsof.time_on_int('nearest', 5)
-        17.5±1ms       16.2±0.3ms     0.92  join_merge.MergeAsof.time_on_int('forward', 5)
-        22.3±2ms       20.5±0.4ms     0.92  join_merge.MergeAsof.time_on_int32('nearest', 5)
-      18.2±0.4ms       16.8±0.4ms     0.92  join_merge.MergeAsof.time_on_int32('backward', 5)
-        21.8±1ms       19.9±0.4ms     0.91  join_merge.MergeAsof.time_on_int32('nearest', None)
-      19.0±0.9ms       17.2±0.3ms     0.91  join_merge.MergeAsof.time_on_int32('forward', None)
-        17.6±1ms       15.9±0.6ms     0.90  join_merge.MergeAsof.time_on_uint64('forward', 5)
-        335±10ms          300±9ms     0.89  join_merge.Merge.time_merge_dataframes_cross(False)
-         334±7ms         299±10ms     0.89  join_merge.Merge.time_merge_dataframes_cross(True)
-        19.4±2ms         17.2±1ms     0.89  join_merge.Join.time_join_dataframe_index_multi(True)
-        17.1±1ms       15.1±0.2ms     0.88  join_merge.MergeAsof.time_on_uint64('backward', None)
-        10.3±1ms       8.97±0.4ms     0.87  join_merge.Join.time_join_dataframe_index_shuffle_key_bigger_sort(True)
-      19.9±0.7ms       17.4±0.4ms     0.87  join_merge.MergeAsof.time_on_int32('forward', 5)
-        18.6±1ms       15.9±0.2ms     0.86  join_merge.MergeAsof.time_on_uint64('forward', None)
-        8.44±1ms       7.20±0.7ms     0.85  join_merge.Join.time_join_dataframe_index_single_key_small(False)
-      1.98±0.09s       1.58±0.04s     0.80  join_merge.JoinIndex.time_left_outer_join_index
-      9.80±0.8ms       7.61±0.8ms     0.78  join_merge.Join.time_join_dataframe_index_single_key_bigger(False)
-      9.39±0.5ms       7.25±0.4ms     0.77  join_merge.Concat.time_concat_small_frames(1)
-        221±20ms          170±4ms     0.77  join_merge.MergeCategoricals.time_merge_cat
-        439±10ms          300±9ms     0.68  join_merge.MergeCategoricals.time_merge_object

@jreback jreback added Internals Related to non-user accessible pandas implementation Reshaping Concat, Merge/Join, Stack/Unstack, Explode labels Sep 17, 2021
@jreback jreback added this to the 1.4 milestone Sep 17, 2021
@jreback jreback merged commit 0de6f8b into pandas-dev:master Sep 17, 2021
@jreback
Copy link
Contributor

jreback commented Sep 17, 2021

nice!

@jbrockmendel jbrockmendel deleted the ref-concat-axis0 branch September 18, 2021 01:10
jorisvandenbossche added a commit to jorisvandenbossche/pandas that referenced this pull request Jun 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Internals Related to non-user accessible pandas implementation Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants