Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DatetimeIndex selection with .loc is orders of magnitude slower than [] on ordered frame #17754

Closed
tdpetrou opened this issue Oct 2, 2017 · 8 comments
Closed

DatetimeIndex selection with .loc is orders of magnitude slower than [] on ordered frame #17754

tdpetrou opened this issue Oct 2, 2017 · 8 comments
Labels
Error Reporting Incorrect or improved errors from pandas Indexing Related to indexing on series/frames, not to indexes themselves Performance Memory or execution speed performance

Comments

@tdpetrou
Copy link
Contributor

tdpetrou commented Oct 2, 2017 •
edited
Loading

Code Sample, a copy-pastable example if possible

>>> dates = pd.date_range('2011-1-1', periods=500000, freq='min')
>>> index = np.random.choice(dates, 500000, replace=True)
>>> df = pd.DataFrame(index=index, data={'a':1})
>>> df_sort = df.sort_index()

>>> %timeit df['2011-6-11']
1.19 ms ± 77.2 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

>>> # Sorted is three times faster
>>> %timeit df_sort['2011-6-11']
333 µs ± 17 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

>>> # Now with .loc
>>> %timeit df.loc['2011-6-11']
2.59 ms ± 238 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

>>> # 2500x slower
>>> %timeit df_sort.loc['2011-6-11']
853 ms ± 29.9 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

>>> # Slicing works fine
>>> %timeit df.loc['2011-6-11':'2011-10-1']
52 ms ± 2.29 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

>>> %timeit df_sort.loc['2011-6-11':'2011-10-1']
658 µs ± 35.1 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

Problem description

When using the .loc indexer on a large frame with sorted datetimeindex, selection is ~2500 times slower than just the indexing operator itself. It's also ~300 times slower than the unsorted .loc lookup.

Slicing appears to work as expected

Expected Output

Sorted frame should be faster when using .loc
@tdpetrou tdpetrou changed the title DatetimeIndex selection with .loc is orders of magnitude slower than [] DatetimeIndex selection with .loc is orders of magnitude slower than [] on ordered frame Oct 2, 2017
@jreback
Copy link
Contributor

jreback commented Oct 3, 2017

is this the case on master as well?

@tdpetrou
Copy link
Contributor Author

tdpetrou commented Oct 3, 2017

Yes

INSTALLED VERSIONS

commit: ad359f6
python: 3.6.1.final.0
python-bits: 64
OS: Darwin
OS-release: 15.6.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.21.0.dev+528.gad359f6

@tdpetrou
Copy link
Contributor Author

tdpetrou commented Oct 3, 2017

Ok, I think now I went to the master branch. It's still an issue

INSTALLED VERSIONS ------------------ commit: f797408 python: 3.6.1.final.0 python-bits: 64 OS: Darwin OS-release: 15.6.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: en_US.UTF-8

pandas: 0.21.0.dev+568.gf797408
pytest: 3.0.7
pip: 9.0.1
setuptools: 35.0.2
Cython: 0.25.2
numpy: 1.13.1
scipy: 0.19.0
pyarrow: None
xarray: None
IPython: 6.0.0
sphinx: 1.5.5
patsy: 0.4.1
dateutil: 2.6.1
pytz: 2017.2
blosc: None
bottleneck: 1.2.0
tables: 3.4.2
numexpr: 2.6.2
feather: None
matplotlib: 2.0.2
openpyxl: 2.4.7
xlrd: 1.0.0
xlwt: 1.2.0
xlsxwriter: 0.9.6
lxml: 3.7.3
bs4: 4.6.0
html5lib: 0.9999999
sqlalchemy: 1.1.9
pymysql: None
psycopg2: None
jinja2: 2.9.6
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: 0.3.0.post

@jschendel
Copy link
Member

jschendel commented Oct 3, 2017 •
edited
Loading

I can replicate this on master. Went back to 0.18.1 and looks like it was happening back then too.

Using cProfile, it looks like this is the biggest offender:

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     4    0.977    0.244    0.977    0.244 {method 'get_loc' of 'pandas._libs.index.DatetimeEngine' objects}

Full output for cProfile.run("df_sort.loc['2011-6-11']", sort='tottime'):

         1358 function calls (1326 primitive calls) in 0.979 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        4    0.977    0.244    0.977    0.244 {method 'get_loc' of 'pandas._libs.index.DatetimeEngine' objects}
        4    0.000    0.000    0.000    0.000 {built-in method nt.stat}
        2    0.000    0.000    0.002    0.001 {pandas._libs.tslibs.parsing.parse_time_string}
        6    0.000    0.000    0.000    0.000 offsets.py:2730(delta)
        4    0.000    0.000    0.001    0.000 <frozen importlib._bootstrap>:861(_find_spec)
     12/4    0.000    0.000    0.001    0.000 <frozen importlib._bootstrap>:958(_find_and_load)
        4    0.000    0.000    0.001    0.000 <frozen importlib._bootstrap_external>:1233(find_spec)
        8    0.000    0.000    0.000    0.000 _strptime.py:321(_strptime)
        2    0.000    0.000    0.000    0.000 datetimes.py:1253(_parsed_string_to_bounds)
       12    0.000    0.000    0.000    0.000 parser.py:68(get_token)
     12/4    0.000    0.000    0.001    0.000 <frozen importlib._bootstrap>:931(_find_and_load_unlocked)
       12    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:159(_get_module_lock)
      108    0.000    0.000    0.000    0.000 {built-in method builtins.isinstance}
        2    0.000    0.000    0.000    0.000 parser.py:622(_parse)
        1    0.000    0.000    0.000    0.000 internals.py:3049(_rebuild_blknos_and_blklocs)
        2    0.000    0.000    0.977    0.488 base.py:2469(get_loc)
       24    0.000    0.000    0.000    0.000 {built-in method _thread.allocate_lock}
       36    0.000    0.000    0.000    0.000 {built-in method builtins.getattr}
        2    0.000    0.000    0.000    0.000 datetimes.py:1318(_partial_date_slice)
       12    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:77(acquire)
        2    0.000    0.000    0.979    0.489 datetimes.py:1407(get_loc)
        4    0.000    0.000    0.001    0.000 <frozen importlib._bootstrap_external>:1117(_get_spec)
        1    0.000    0.000    0.979    0.979 {built-in method builtins.exec}
        2    0.000    0.000    0.000    0.000 offsets.py:2738(apply)
       12    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:146(__enter__)
        2    0.000    0.000    0.000    0.000 offsets.py:200(_determine_offset)
        8    0.000    0.000    0.000    0.000 {built-in method _locale.setlocale}
       12    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:57(__init__)
        1    0.000    0.000    0.463    0.463 generic.py:2206(xs)
       12    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:102(release)
        8    0.000    0.000    0.000    0.000 {method 'view' of 'numpy.ndarray' objects}
       20    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:57(_path_join)
        4    0.000    0.000    0.000    0.000 {method 'searchsorted' of 'numpy.ndarray' objects}
    62/55    0.000    0.000    0.000    0.000 {built-in method builtins.len}
        2    0.000    0.000    0.000    0.000 datetimelike.py:259(__getitem__)
        6    0.000    0.000    0.000    0.000 parser.py:401(append)
       20    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:59(<listcomp>)
        1    0.000    0.000    0.000    0.000 internals.py:275(getitem_block)
        2    0.000    0.000    0.000    0.000 datetimes.py:574(_simple_new)
        2    0.000    0.000    0.002    0.001 datetimes.py:1487(_get_string_slice)
        8    0.000    0.000    0.000    0.000 {method 'match' of '_sre.SRE_Pattern' objects}
       12    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:852(_find_spec_legacy)
       14    0.000    0.000    0.000    0.000 {built-in method builtins.hasattr}
        6    0.000    0.000    0.000    0.000 {built-in method pandas._libs.lib.isscalar}
        4    0.000    0.000    0.000    0.000 config.py:78(_get_single_key)
        6    0.000    0.000    0.000    0.000 base.py:572(values)
      8/4    0.000    0.000    0.001    0.000 {built-in method builtins.__import__}
        1    0.000    0.000    0.000    0.000 {method 'reduce' of 'numpy.ufunc' objects}
        8    0.000    0.000    0.000    0.000 locale.py:379(normalize)
        2    0.000    0.000    0.000    0.000 parser.py:190(split)
        5    0.000    0.000    0.000    0.000 generic.py:360(_get_axis_name)
        1    0.000    0.000    0.000    0.000 generic.py:1895(_slice)
        4    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:402(parent)
        4    0.000    0.000    0.000    0.000 {method 'format' of 'str' objects}
      2/1    0.000    0.000    0.979    0.979 indexing.py:1326(__getitem__)
        8    0.000    0.000    0.000    0.000 locale.py:565(getlocale)
        2    0.000    0.000    0.000    0.000 parser.py:212(__init__)
        4    0.000    0.000    0.000    0.000 config.py:547(_get_root)
        1    0.000    0.000    0.979    0.979 indexing.py:1511(_getitem_axis)
       20    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:208(_verbose_message)
       24    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:834(__enter__)
        4    0.000    0.000    0.001    0.000 <frozen importlib._bootstrap_external>:1149(find_spec)
        4    0.000    0.000    0.000    0.000 generic.py:3512(__setattr__)
       12    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:142(__init__)
       12    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:153(__exit__)
        2    0.000    0.000    0.000    0.000 offsets.py:194(__init__)
        1    0.000    0.000    0.000    0.000 internals.py:3547(get_slice)
        1    0.000    0.000    0.979    0.979 <string>:1(<module>)
        2    0.000    0.000    0.000    0.000 {built-in method numpy.core.multiarray.empty}
       40    0.000    0.000    0.000    0.000 {method 'rstrip' of 'str' objects}
       24    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:838(__exit__)
        4    0.000    0.000    0.000    0.000 base.py:1558(<lambda>)
       12    0.000    0.000    0.000    0.000 parser.py:180(__next__)
        2    0.000    0.000    0.000    0.000 parser.py:375(__init__)
        8    0.000    0.000    0.000    0.000 _strptime.py:29(_getlang)
       20    0.000    0.000    0.000    0.000 {method 'join' of 'str' objects}
       20    0.000    0.000    0.000    0.000 {method 'rpartition' of 'str' objects}
        4    0.000    0.000    0.000    0.000 config.py:223(__call__)
       15    0.000    0.000    0.000    0.000 generic.py:7(_check)
        1    0.000    0.000    0.000    0.000 datetimelike.py:597(_convert_scalar_indexer)
        2    0.000    0.000    0.000    0.000 parser.py:361(validate)
        4    0.000    0.000    0.000    0.000 config.py:96(_get_option)
       25    0.000    0.000    0.000    0.000 {method 'get' of 'dict' objects}
       22    0.000    0.000    0.000    0.000 {built-in method builtins.setattr}
        4    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:355(__init__)
        4    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:75(_path_stat)
        4    0.000    0.000    0.000    0.000 __init__.py:23(find_module)
        2    0.000    0.000    0.000    0.000 parser.py:52(__init__)
        8    0.000    0.000    0.000    0.000 config.py:562(_get_deprecated_option)
        3    0.000    0.000    0.000    0.000 base.py:3406(_maybe_cast_indexer)
        1    0.000    0.000    0.000    0.000 frame.py:307(__init__)
       22    0.000    0.000    0.000    0.000 parser.py:199(isnum)
        2    0.000    0.000    0.000    0.000 base.py:717(_get_attributes_dict)
        2    0.000    0.000    0.000    0.000 offsets.py:36(as_timestamp)
        4    0.000    0.000    0.000    0.000 {built-in method pandas._libs.tslib._delta_to_nanoseconds}
        8    0.000    0.000    0.000    0.000 locale.py:462(_parse_localename)
        4    0.000    0.000    0.000    0.000 common.py:1773(_get_dtype_type)
        5    0.000    0.000    0.000    0.000 common.py:184(is_bool_indexer)
        2    0.000    0.000    0.000    0.000 base.py:1207(is_monotonic_increasing)
        4    0.000    0.000    0.000    0.000 offsets.py:2734(nanos)
        5    0.000    0.000    0.000    0.000 generic.py:373(_get_axis)
        2    0.000    0.000    0.000    0.000 internals.py:2993(shape)
       36    0.000    0.000    0.000    0.000 {built-in method _imp.release_lock}
      8/4    0.000    0.000    0.001    0.000 <frozen importlib._bootstrap>:197(_call_with_frames_removed)
        8    0.000    0.000    0.000    0.000 six.py:184(find_module)
        8    0.000    0.000    0.000    0.000 _strptime.py:562(_strptime_datetime)
        5    0.000    0.000    0.000    0.000 {built-in method pandas._libs.lib.is_float}
       20    0.000    0.000    0.000    0.000 {method 'read' of '_io.StringIO' objects}
        2    0.000    0.000    0.000    0.000 parser.py:410(resolve_ymd)
        1    0.000    0.000    0.000    0.000 generic.py:121(__init__)
       10    0.000    0.000    0.000    0.000 parser.py:194(isword)
        1    0.000    0.000    0.000    0.000 generic.py:1850(_is_view)
        1    0.000    0.000    0.516    0.516 indexing.py:1402(_has_valid_type)
        1    0.000    0.000    0.000    0.000 internals.py:2952(__init__)
        4    0.000    0.000    0.000    0.000 {built-in method _imp.is_frozen}
        4    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1080(_path_importer_cache)
        2    0.000    0.000    0.000    0.000 common.py:85(is_object_dtype)
        7    0.000    0.000    0.000    0.000 base.py:541(__len__)
        1    0.000    0.000    0.000    0.000 numeric.py:149(_convert_scalar_indexer)
        2    0.000    0.000    0.000    0.000 offsets.py:2687(__add__)
        1    0.000    0.000    0.000    0.000 indexing.py:234(_convert_scalar_indexer)
        4    0.000    0.000    0.000    0.000 base.py:762(_values)
        2    0.000    0.000    0.000    0.000 generic.py:347(_get_axis_number)
        2    0.000    0.000    0.000    0.000 generic.py:1915(_set_is_copy)
        1    0.000    0.000    0.000    0.000 indexing.py:240(_convert_slice_indexer)
        1    0.000    0.000    0.000    0.000 internals.py:3490(is_view)
        4    0.000    0.000    0.000    0.000 {method 'split' of 'str' objects}
        2    0.000    0.000    0.000    0.000 parser.py:327(hms)
        2    0.000    0.000    0.000    0.000 numeric.py:144(asi8)
        1    0.000    0.000    0.463    0.463 indexing.py:123(_get_label)
        1    0.000    0.000    0.000    0.000 indexing.py:1700(_get_slice_axis)
        1    0.000    0.000    0.000    0.000 internals.py:3467(_consolidate_check)
       24    0.000    0.000    0.000    0.000 {built-in method _imp.acquire_lock}
       12    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:173(cb)
        4    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:771(find_spec)
        2    0.000    0.000    0.000    0.000 abc.py:178(__instancecheck__)
        2    0.000    0.000    0.000    0.000 _weakrefset.py:70(__contains__)
        4    0.000    0.000    0.000    0.000 config.py:529(_select_options)
        4    0.000    0.000    0.000    0.000 config.py:603(_warn_if_deprecated)
        1    0.000    0.000    0.000    0.000 base.py:1369(_convert_slice_indexer)
        1    0.000    0.000    0.516    0.516 datetimelike.py:249(__contains__)
        2    0.000    0.000    0.000    0.000 datetimelike.py:306(freqstr)
        1    0.000    0.000    0.000    0.000 generic.py:3570(f)
        6    0.000    0.000    0.000    0.000 internals.py:2995(<genexpr>)
        1    0.000    0.000    0.000    0.000 {built-in method numpy.core.multiarray.arange}
       24    0.000    0.000    0.000    0.000 {built-in method _thread.get_ident}
       12    0.000    0.000    0.000    0.000 {method 'lower' of 'str' objects}
        4    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:37(_relax_case)
        1    0.000    0.000    0.000    0.000 _methods.py:37(_any)
        2    0.000    0.000    0.000    0.000 offsets.py:402(__radd__)
        1    0.000    0.000    0.000    0.000 indexing.py:1731(_getitem_axis)
        1    0.000    0.000    0.000    0.000 internals.py:307(ftype)
        1    0.000    0.000    0.000    0.000 internals.py:3001(set_axis)
       28    0.000    0.000    0.000    0.000 {method 'isdigit' of 'str' objects}
        4    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:989(_handle_fromlist)
        4    0.000    0.000    0.000    0.000 parser.py:204(isspace)
        2    0.000    0.000    0.000    0.000 parser.py:308(jump)
        1    0.000    0.000    0.000    0.000 base.py:1321(_convert_scalar_indexer)
        1    0.000    0.000    0.000    0.000 generic.py:559(_set_axis)
        2    0.000    0.000    0.000    0.000 generic.py:1762(_indexer)
        1    0.000    0.000    0.000    0.000 generic.py:1889(_clear_item_cache)
        3    0.000    0.000    0.000    0.000 internals.py:185(mgr_locs)
        1    0.000    0.000    0.000    0.000 internals.py:214(make_block_same_class)
        4    0.000    0.000    0.000    0.000 {method 'pop' of 'list' objects}
        4    0.000    0.000    0.000    0.000 {method 'partition' of 'str' objects}
        2    0.000    0.000    0.000    0.000 {built-in method builtins.min}
        4    0.000    0.000    0.000    0.000 config.py:590(_translate_key)
        2    0.000    0.000    0.000    0.000 common.py:297(is_datetime64_dtype)
        2    0.000    0.000    0.000    0.000 common.py:444(_apply_if_callable)
        8    0.000    0.000    0.000    0.000 datetimes.py:1548(_get_freq)
        2    0.000    0.000    0.000    0.000 frequencies.py:125(get_reso)
        1    0.000    0.000    0.000    0.000 generic.py:163(_init_mgr)
        1    0.000    0.000    0.000    0.000 internals.py:103(__init__)
        1    0.000    0.000    0.000    0.000 internals.py:2856(make_block)
        1    0.000    0.000    0.000    0.000 internals.py:2953(<listcomp>)
        1    0.000    0.000    0.000    0.000 {method 'any' of 'numpy.ndarray' objects}
        2    0.000    0.000    0.000    0.000 {method 'fill' of 'numpy.ndarray' objects}
       10    0.000    0.000    0.000    0.000 {method 'append' of 'list' objects}
       10    0.000    0.000    0.000    0.000 {method 'isalpha' of 'str' objects}
        2    0.000    0.000    0.000    0.000 base.py:535(_reset_identity)
        2    0.000    0.000    0.000    0.000 base.py:1202(is_monotonic)
        3    0.000    0.000    0.000    0.000 base.py:3421(_validate_indexer)
        2    0.000    0.000    0.000    0.000 offsets.py:205(<genexpr>)
        2    0.000    0.000    0.000    0.000 indexing.py:2064(is_list_like_indexer)
        4    0.000    0.000    0.000    0.000 {method 'isspace' of 'str' objects}
        4    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:698(find_spec)
        2    0.000    0.000    0.000    0.000 parser.py:319(month)
        2    0.000    0.000    0.000    0.000 inference.py:234(is_list_like)
        2    0.000    0.000    0.000    0.000 base.py:719(<listcomp>)
        3    0.000    0.000    0.000    0.000 base.py:4081(_ensure_index)
        1    0.000    0.000    0.000    0.000 generic.py:377(_get_block_manager_axis)
        1    0.000    0.000    0.000    0.000 generic.py:3479(__finalize__)
        1    0.000    0.000    0.000    0.000 generic.py:3557(_protect_consolidate)
        1    0.000    0.000    0.000    0.000 generic.py:3567(_consolidate_inplace)
        1    0.000    0.000    0.000    0.000 indexing.py:141(_slice)
        1    0.000    0.000    0.000    0.000 internals.py:126(is_view)
        1    0.000    0.000    0.000    0.000 internals.py:256(_slice)
        2    0.000    0.000    0.000    0.000 internals.py:3459(is_consolidated)
        1    0.000    0.000    0.000    0.000 internals.py:3468(<listcomp>)
        1    0.000    0.000    0.000    0.000 internals.py:3557(<listcomp>)
        8    0.000    0.000    0.000    0.000 {built-in method pandas._libs.lib.is_integer}
        3    0.000    0.000    0.000    0.000 {method 'items' of 'dict' objects}
        1    0.000    0.000    0.000    0.000 indexing.py:1594(_has_valid_type)
        1    0.000    0.000    0.000    0.000 internals.py:3764(_consolidate_inplace)
        4    0.000    0.000    0.000    0.000 {method 'end' of '_sre.SRE_Match' objects}
        2    0.000    0.000    0.000    0.000 {built-in method __new__ of type object at 0x000000005748B620}
        2    0.000    0.000    0.000    0.000 {built-in method builtins.callable}
        4    0.000    0.000    0.000    0.000 {built-in method builtins.issubclass}
        2    0.000    0.000    0.000    0.000 parser.py:177(__iter__)
        2    0.000    0.000    0.000    0.000 parser.py:351(convertyear)
        1    0.000    0.000    0.000    0.000 indexing.py:2075(need_slice)
        1    0.000    0.000    0.000    0.000 internals.py:222(mgr_locs)
        1    0.000    0.000    0.000    0.000 internals.py:3748(consolidate)
        2    0.000    0.000    0.000    0.000 {built-in method pandas._libs.tslibs.timezones.maybe_get_tz}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
        1    0.000    0.000    0.000    0.000 {method 'clear' of 'dict' objects}
        1    0.000    0.000    0.000    0.000 frame.py:295(_constructor)
        1    0.000    0.000    0.000    0.000 indexing.py:1487(_get_partial_string_timestamp_match_key)
        1    0.000    0.000    0.000    0.000 internals.py:303(dtype)
        2    0.000    0.000    0.000    0.000 internals.py:2997(ndim)


Looks like df_sort['2011-6-11'] uses searchsorted as one would expect (loc above does too) but doesn't call get_loc. The output of cProfile.run("df_sort['2011-6-11']", sort='tottime'):

         644 function calls (628 primitive calls) in 0.003 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        2    0.000    0.000    0.000    0.000 {built-in method nt.stat}
        3    0.000    0.000    0.000    0.000 offsets.py:2730(delta)
        1    0.000    0.000    0.002    0.002 {pandas._libs.tslibs.parsing.parse_time_string}
        2    0.000    0.000    0.001    0.000 <frozen importlib._bootstrap>:861(_find_spec)
      6/2    0.000    0.000    0.001    0.001 <frozen importlib._bootstrap>:958(_find_and_load)
        4    0.000    0.000    0.000    0.000 _strptime.py:321(_strptime)
        2    0.000    0.000    0.001    0.000 <frozen importlib._bootstrap_external>:1233(find_spec)
        1    0.000    0.000    0.000    0.000 internals.py:3049(_rebuild_blknos_and_blklocs)
        1    0.000    0.000    0.000    0.000 datetimes.py:1253(_parsed_string_to_bounds)
        6    0.000    0.000    0.000    0.000 parser.py:68(get_token)
        1    0.000    0.000    0.003    0.003 frame.py:2057(__getitem__)
      6/2    0.000    0.000    0.001    0.001 <frozen importlib._bootstrap>:931(_find_and_load_unlocked)
        1    0.000    0.000    0.000    0.000 parser.py:622(_parse)
        6    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:102(release)
        1    0.000    0.000    0.003    0.003 {built-in method builtins.exec}
        2    0.000    0.000    0.001    0.000 <frozen importlib._bootstrap_external>:1117(_get_spec)
        1    0.000    0.000    0.000    0.000 datetimes.py:1318(_partial_date_slice)
        6    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:159(_get_module_lock)
        1    0.000    0.000    0.000    0.000 offsets.py:2738(apply)
       45    0.000    0.000    0.000    0.000 {built-in method builtins.isinstance}
        1    0.000    0.000    0.000    0.000 {method 'reduce' of 'numpy.ufunc' objects}
        1    0.000    0.000    0.000    0.000 internals.py:275(getitem_block)
        4    0.000    0.000    0.000    0.000 {built-in method _locale.setlocale}
       12    0.000    0.000    0.000    0.000 {built-in method _thread.allocate_lock}
        6    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:77(acquire)
        1    0.000    0.000    0.000    0.000 datetimelike.py:259(__getitem__)
       10    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:57(_path_join)
        1    0.000    0.000    0.000    0.000 generic.py:1895(_slice)
        2    0.000    0.000    0.000    0.000 {method 'searchsorted' of 'numpy.ndarray' objects}
        1    0.000    0.000    0.000    0.000 internals.py:3547(get_slice)
        1    0.000    0.000    0.000    0.000 datetimes.py:574(_simple_new)
       11    0.000    0.000    0.000    0.000 {built-in method builtins.getattr}
        6    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:852(_find_spec_legacy)
       10    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:59(<listcomp>)
        6    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:146(__enter__)
        1    0.000    0.000    0.002    0.002 indexing.py:1902(convert_to_index_sliceable)
        7    0.000    0.000    0.000    0.000 {built-in method builtins.hasattr}
        2    0.000    0.000    0.000    0.000 config.py:78(_get_single_key)
        4    0.000    0.000    0.000    0.000 {method 'match' of '_sre.SRE_Pattern' objects}
        1    0.000    0.000    0.000    0.000 base.py:1653(__contains__)
        4    0.000    0.000    0.000    0.000 locale.py:379(normalize)
        1    0.000    0.000    0.000    0.000 offsets.py:200(_determine_offset)
        6    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:57(__init__)
    34/30    0.000    0.000    0.000    0.000 {built-in method builtins.len}
        1    0.000    0.000    0.002    0.002 datetimes.py:1487(_get_string_slice)
        1    0.000    0.000    0.003    0.003 <string>:1(<module>)
        3    0.000    0.000    0.000    0.000 parser.py:401(append)
        4    0.000    0.000    0.000    0.000 locale.py:565(getlocale)
        1    0.000    0.000    0.000    0.000 base.py:572(values)
      4/2    0.000    0.000    0.001    0.001 {built-in method builtins.__import__}
        1    0.000    0.000    0.000    0.000 frame.py:307(__init__)
        2    0.000    0.000    0.000    0.000 config.py:547(_get_root)
        1    0.000    0.000    0.000    0.000 parser.py:190(split)
        2    0.000    0.000    0.000    0.000 {built-in method numpy.core.multiarray.empty}
       10    0.000    0.000    0.000    0.000 {method 'get' of 'dict' objects}
        6    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:153(__exit__)
        4    0.000    0.000    0.000    0.000 _strptime.py:29(_getlang)
       20    0.000    0.000    0.000    0.000 {method 'rstrip' of 'str' objects}
       12    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:834(__enter__)
       12    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:838(__exit__)
        2    0.000    0.000    0.000    0.000 __init__.py:23(find_module)
        1    0.000    0.000    0.000    0.000 parser.py:212(__init__)
       10    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:208(_verbose_message)
        2    0.000    0.000    0.000    0.000 config.py:96(_get_option)
        2    0.000    0.000    0.000    0.000 common.py:1773(_get_dtype_type)
        2    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:402(parent)
        4    0.000    0.000    0.000    0.000 config.py:562(_get_deprecated_option)
        2    0.000    0.000    0.000    0.000 internals.py:2993(shape)
        2    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:355(__init__)
        1    0.000    0.000    0.000    0.000 internals.py:2952(__init__)
        2    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:75(_path_stat)
        4    0.000    0.000    0.000    0.000 locale.py:462(_parse_localename)
        2    0.000    0.000    0.000    0.000 config.py:223(__call__)
        1    0.000    0.000    0.000    0.000 base.py:1207(is_monotonic_increasing)
        1    0.000    0.000    0.000    0.000 generic.py:347(_get_axis_number)
        2    0.000    0.000    0.001    0.000 <frozen importlib._bootstrap_external>:1149(find_spec)
        4    0.000    0.000    0.000    0.000 six.py:184(find_module)
        1    0.000    0.000    0.000    0.000 parser.py:52(__init__)
       10    0.000    0.000    0.000    0.000 {method 'rpartition' of 'str' objects}
        2    0.000    0.000    0.000    0.000 {method 'format' of 'str' objects}
        1    0.000    0.000    0.000    0.000 base.py:717(_get_attributes_dict)
        1    0.000    0.000    0.001    0.001 frame.py:2097(_getitem_slice)
       10    0.000    0.000    0.000    0.000 {method 'join' of 'str' objects}
       18    0.000    0.000    0.000    0.000 {built-in method _imp.release_lock}
        6    0.000    0.000    0.000    0.000 parser.py:180(__next__)
        1    0.000    0.000    0.000    0.000 _methods.py:37(_any)
       11    0.000    0.000    0.000    0.000 parser.py:199(isnum)
        1    0.000    0.000    0.000    0.000 offsets.py:194(__init__)
        6    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:142(__init__)
        1    0.000    0.000    0.000    0.000 common.py:85(is_object_dtype)
        1    0.000    0.000    0.000    0.000 internals.py:2856(make_block)
        1    0.000    0.000    0.000    0.000 internals.py:3467(_consolidate_check)
        4    0.000    0.000    0.000    0.000 _strptime.py:562(_strptime_datetime)
        1    0.000    0.000    0.000    0.000 {built-in method numpy.core.multiarray.arange}
      4/2    0.000    0.000    0.001    0.001 <frozen importlib._bootstrap>:197(_call_with_frames_removed)
        1    0.000    0.000    0.000    0.000 internals.py:214(make_block_same_class)
        1    0.000    0.000    0.000    0.000 {built-in method pandas._libs.lib.isscalar}
        2    0.000    0.000    0.000    0.000 {built-in method pandas._libs.tslib._delta_to_nanoseconds}
       11    0.000    0.000    0.000    0.000 {built-in method builtins.setattr}
        1    0.000    0.000    0.000    0.000 parser.py:410(resolve_ymd)
        1    0.000    0.000    0.000    0.000 offsets.py:36(as_timestamp)
        2    0.000    0.000    0.000    0.000 offsets.py:2734(nanos)
        2    0.000    0.000    0.000    0.000 {method 'view' of 'numpy.ndarray' objects}
        4    0.000    0.000    0.000    0.000 generic.py:7(_check)
        1    0.000    0.000    0.000    0.000 base.py:1202(is_monotonic)
        1    0.000    0.000    0.000    0.000 internals.py:103(__init__)
        1    0.000    0.000    0.000    0.000 {method 'any' of 'numpy.ndarray' objects}
       12    0.000    0.000    0.000    0.000 {built-in method _thread.get_ident}
        2    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1080(_path_importer_cache)
        1    0.000    0.000    0.000    0.000 parser.py:375(__init__)
        2    0.000    0.000    0.000    0.000 config.py:603(_warn_if_deprecated)
        1    0.000    0.000    0.000    0.000 internals.py:307(ftype)
       12    0.000    0.000    0.000    0.000 {built-in method _imp.acquire_lock}
        1    0.000    0.000    0.000    0.000 parser.py:327(hms)
        1    0.000    0.000    0.000    0.000 numeric.py:144(asi8)
        1    0.000    0.000    0.000    0.000 offsets.py:2687(__add__)
        1    0.000    0.000    0.000    0.000 generic.py:121(__init__)
        1    0.000    0.000    0.000    0.000 internals.py:3557(<listcomp>)
        2    0.000    0.000    0.000    0.000 {method 'split' of 'str' objects}
        2    0.000    0.000    0.000    0.000 {built-in method _imp.is_frozen}
        6    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:173(cb)
        2    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:771(find_spec)
        5    0.000    0.000    0.000    0.000 parser.py:194(isword)
        2    0.000    0.000    0.000    0.000 config.py:529(_select_options)
        1    0.000    0.000    0.000    0.000 common.py:184(is_bool_indexer)
        1    0.000    0.000    0.000    0.000 generic.py:1915(_set_is_copy)
        1    0.000    0.000    0.000    0.000 internals.py:2953(<listcomp>)
       10    0.000    0.000    0.000    0.000 {method 'read' of '_io.StringIO' objects}
        2    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:989(_handle_fromlist)
        2    0.000    0.000    0.000    0.000 parser.py:204(isspace)
        1    0.000    0.000    0.000    0.000 common.py:444(_apply_if_callable)
        1    0.000    0.000    0.000    0.000 generic.py:163(_init_mgr)
        1    0.000    0.000    0.000    0.000 generic.py:377(_get_block_manager_axis)
        1    0.000    0.000    0.000    0.000 generic.py:3479(__finalize__)
       14    0.000    0.000    0.000    0.000 {method 'isdigit' of 'str' objects}
        1    0.000    0.000    0.000    0.000 parser.py:361(validate)
        1    0.000    0.000    0.000    0.000 generic.py:3512(__setattr__)
        3    0.000    0.000    0.000    0.000 internals.py:185(mgr_locs)
        1    0.000    0.000    0.000    0.000 internals.py:256(_slice)
        6    0.000    0.000    0.000    0.000 internals.py:2995(<genexpr>)
        2    0.000    0.000    0.000    0.000 {method 'fill' of 'numpy.ndarray' objects}
        2    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:37(_relax_case)
        2    0.000    0.000    0.000    0.000 base.py:4081(_ensure_index)
        1    0.000    0.000    0.000    0.000 datetimelike.py:306(freqstr)
        1    0.000    0.000    0.000    0.000 internals.py:3468(<listcomp>)
        2    0.000    0.000    0.000    0.000 {method 'pop' of 'list' objects}
        1    0.000    0.000    0.000    0.000 base.py:535(_reset_identity)
        1    0.000    0.000    0.000    0.000 base.py:719(<listcomp>)
        4    0.000    0.000    0.000    0.000 datetimes.py:1548(_get_freq)
        1    0.000    0.000    0.000    0.000 frequencies.py:125(get_reso)
        5    0.000    0.000    0.000    0.000 {method 'append' of 'list' objects}
        6    0.000    0.000    0.000    0.000 {method 'lower' of 'str' objects}
        1    0.000    0.000    0.000    0.000 common.py:297(is_datetime64_dtype)
        4    0.000    0.000    0.000    0.000 base.py:541(__len__)
        1    0.000    0.000    0.000    0.000 base.py:1673(contains)
        1    0.000    0.000    0.000    0.000 offsets.py:402(__radd__)
        2    0.000    0.000    0.000    0.000 internals.py:2997(ndim)
        2    0.000    0.000    0.000    0.000 {method 'partition' of 'str' objects}
        5    0.000    0.000    0.000    0.000 {method 'isalpha' of 'str' objects}
        1    0.000    0.000    0.000    0.000 parser.py:308(jump)
        2    0.000    0.000    0.000    0.000 config.py:590(_translate_key)
        1    0.000    0.000    0.000    0.000 frame.py:295(_constructor)
        1    0.000    0.000    0.000    0.000 internals.py:3070(_get_items)
        1    0.000    0.000    0.000    0.000 internals.py:3764(_consolidate_inplace)
        2    0.000    0.000    0.000    0.000 {built-in method pandas._libs.lib.is_integer}
        1    0.000    0.000    0.000    0.000 {built-in method pandas._libs.tslibs.timezones.maybe_get_tz}
        2    0.000    0.000    0.000    0.000 {method 'end' of '_sre.SRE_Match' objects}
        2    0.000    0.000    0.000    0.000 {method 'isspace' of 'str' objects}
        2    0.000    0.000    0.000    0.000 {built-in method builtins.issubclass}
        2    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:698(find_spec)
        1    0.000    0.000    0.000    0.000 parser.py:319(month)
        1    0.000    0.000    0.000    0.000 parser.py:351(convertyear)
        1    0.000    0.000    0.000    0.000 datetimes.py:1672(is_all_dates)
        1    0.000    0.000    0.000    0.000 offsets.py:205(<genexpr>)
        1    0.000    0.000    0.000    0.000 internals.py:3459(is_consolidated)
        2    0.000    0.000    0.000    0.000 {method 'items' of 'dict' objects}
        1    0.000    0.000    0.000    0.000 {built-in method __new__ of type object at 0x000000005748B620}
        1    0.000    0.000    0.000    0.000 internals.py:222(mgr_locs)
        1    0.000    0.000    0.000    0.000 internals.py:303(dtype)
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
        1    0.000    0.000    0.000    0.000 {built-in method builtins.callable}
        2    0.000    0.000    0.000    0.000 {built-in method builtins.hash}
        1    0.000    0.000    0.000    0.000 parser.py:177(__iter__)


And for additional comparison if it's useful cProfile.run("df.loc['2011-6-11']", sort='tottime'):

         1426 function calls (1390 primitive calls) in 0.010 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        2    0.004    0.002    0.005    0.003 datetimes.py:1318(_partial_date_slice)
        4    0.001    0.000    0.001    0.000 {built-in method nt.stat}
        2    0.001    0.000    0.001    0.000 {method 'nonzero' of 'numpy.ndarray' objects}
        2    0.000    0.000    0.008    0.004 datetimes.py:1487(_get_string_slice)
        1    0.000    0.000    0.000    0.000 {pandas._libs.algos.take_2d_axis0_int64_int64}
        6    0.000    0.000    0.000    0.000 offsets.py:2730(delta)
        2    0.000    0.000    0.003    0.001 {pandas._libs.tslibs.parsing.parse_time_string}
        4    0.000    0.000    0.001    0.000 <frozen importlib._bootstrap>:861(_find_spec)
        8    0.000    0.000    0.000    0.000 _strptime.py:321(_strptime)
     12/4    0.000    0.000    0.002    0.000 <frozen importlib._bootstrap>:958(_find_and_load)
        4    0.000    0.000    0.001    0.000 <frozen importlib._bootstrap_external>:1233(find_spec)
       12    0.000    0.000    0.000    0.000 parser.py:68(get_token)
        2    0.000    0.000    0.000    0.000 indexing.py:1992(maybe_convert_indices)
        8    0.000    0.000    0.000    0.000 {method 'reduce' of 'numpy.ufunc' objects}
        2    0.000    0.000    0.000    0.000 datetimes.py:1253(_parsed_string_to_bounds)
        2    0.000    0.000    0.000    0.000 parser.py:622(_parse)
      107    0.000    0.000    0.000    0.000 {built-in method builtins.isinstance}
     12/4    0.000    0.000    0.002    0.000 <frozen importlib._bootstrap>:931(_find_and_load_unlocked)
       24    0.000    0.000    0.000    0.000 {built-in method _thread.allocate_lock}
       12    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:159(_get_module_lock)
       12    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:77(acquire)
        2    0.000    0.000    0.000    0.000 offsets.py:2738(apply)
       33    0.000    0.000    0.000    0.000 {built-in method builtins.getattr}
        4    0.000    0.000    0.000    0.000 {method 'get_loc' of 'pandas._libs.index.DatetimeEngine' objects}
        8    0.000    0.000    0.000    0.000 {built-in method _locale.setlocale}
       12    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:102(release)
        1    0.000    0.000    0.000    0.000 algorithms.py:1287(take_nd)
        1    0.000    0.000    0.010    0.010 {built-in method builtins.exec}
       12    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:146(__enter__)
        4    0.000    0.000    0.001    0.000 <frozen importlib._bootstrap_external>:1117(_get_spec)
        1    0.000    0.000    0.000    0.000 {method 'take' of 'numpy.ndarray' objects}
        2    0.000    0.000    0.000    0.000 offsets.py:200(_determine_offset)
    73/61    0.000    0.000    0.000    0.000 {built-in method builtins.len}
        1    0.000    0.000    0.001    0.001 internals.py:4178(take)
        2    0.000    0.000    0.008    0.004 datetimes.py:1407(get_loc)
       12    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:57(__init__)
       20    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:57(_path_join)
        1    0.000    0.000    0.005    0.005 generic.py:2206(xs)
        1    0.000    0.000    0.000    0.000 internals.py:3049(_rebuild_blknos_and_blklocs)
       20    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:59(<listcomp>)
        6    0.000    0.000    0.000    0.000 parser.py:401(append)
       20    0.000    0.000    0.000    0.000 {built-in method builtins.hasattr}
        8    0.000    0.000    0.000    0.000 locale.py:379(normalize)
        7    0.000    0.000    0.000    0.000 generic.py:360(_get_axis_name)
        1    0.000    0.000    0.000    0.000 datetimelike.py:391(take)
        1    0.000    0.000    0.001    0.001 generic.py:2099(_take)
        8    0.000    0.000    0.000    0.000 {method 'match' of '_sre.SRE_Pattern' objects}
        4    0.000    0.000    0.000    0.000 config.py:78(_get_single_key)
       12    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:852(_find_spec_legacy)
        2    0.000    0.000    0.000    0.000 base.py:2469(get_loc)
        1    0.000    0.000    0.010    0.010 indexing.py:1511(_getitem_axis)
        2    0.000    0.000    0.000    0.000 parser.py:190(split)
       11    0.000    0.000    0.000    0.000 {method 'view' of 'numpy.ndarray' objects}
        2    0.000    0.000    0.000    0.000 parser.py:212(__init__)
        8    0.000    0.000    0.000    0.000 locale.py:565(getlocale)
      8/4    0.000    0.000    0.001    0.000 {built-in method builtins.__import__}
        4    0.000    0.000    0.000    0.000 config.py:96(_get_option)
        4    0.000    0.000    0.000    0.000 config.py:547(_get_root)
        1    0.000    0.000    0.005    0.005 indexing.py:1402(_has_valid_type)
        8    0.000    0.000    0.000    0.000 _strptime.py:29(_getlang)
        1    0.000    0.000    0.000    0.000 datetimelike.py:597(_convert_scalar_indexer)
        4    0.000    0.000    0.000    0.000 {built-in method numpy.core.multiarray.array}
        1    0.000    0.000    0.000    0.000 datetimelike.py:128(equals)
        1    0.000    0.000    0.000    0.000 datetimes.py:574(_simple_new)
        4    0.000    0.000    0.000    0.000 internals.py:2993(shape)
       26    0.000    0.000    0.000    0.000 {method 'get' of 'dict' objects}
        1    0.000    0.000    0.010    0.010 <string>:1(<module>)
       40    0.000    0.000    0.000    0.000 {method 'rstrip' of 'str' objects}
       24    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:834(__enter__)
       12    0.000    0.000    0.000    0.000 parser.py:180(__next__)
       24    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:838(__exit__)
       12    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:153(__exit__)
        2    0.000    0.000    0.000    0.000 parser.py:52(__init__)
        8    0.000    0.000    0.000    0.000 _strptime.py:562(_strptime_datetime)
        1    0.000    0.000    0.005    0.005 indexing.py:123(_get_label)
        8    0.000    0.000    0.000    0.000 {method 'any' of 'numpy.ndarray' objects}
       20    0.000    0.000    0.000    0.000 {method 'rpartition' of 'str' objects}
       20    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:208(_verbose_message)
        4    0.000    0.000    0.001    0.000 <frozen importlib._bootstrap_external>:75(_path_stat)
        5    0.000    0.000    0.000    0.000 numeric.py:144(asi8)
        3    0.000    0.000    0.000    0.000 generic.py:3512(__setattr__)
        4    0.000    0.000    0.000    0.000 {method 'format' of 'str' objects}
        8    0.000    0.000    0.000    0.000 config.py:562(_get_deprecated_option)
       14    0.000    0.000    0.000    0.000 generic.py:7(_check)
       12    0.000    0.000    0.000    0.000 base.py:541(__len__)
        7    0.000    0.000    0.000    0.000 generic.py:373(_get_axis)
        3    0.000    0.000    0.000    0.000 {built-in method numpy.core.multiarray.empty}
       22    0.000    0.000    0.000    0.000 {built-in method builtins.setattr}
        8    0.000    0.000    0.000    0.000 locale.py:462(_parse_localename)
        1    0.000    0.000    0.001    0.001 internals.py:1169(take_nd)
        4    0.000    0.000    0.000    0.000 {built-in method pandas._libs.tslib._delta_to_nanoseconds}
       36    0.000    0.000    0.000    0.000 {built-in method _imp.release_lock}
        4    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:402(parent)
        1    0.000    0.000    0.005    0.005 datetimelike.py:249(__contains__)
        4    0.000    0.000    0.000    0.000 offsets.py:2734(nanos)
       20    0.000    0.000    0.000    0.000 {method 'join' of 'str' objects}
        4    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:355(__init__)
        4    0.000    0.000    0.000    0.000 __init__.py:23(find_module)
       22    0.000    0.000    0.000    0.000 parser.py:199(isnum)
        6    0.000    0.000    0.000    0.000 dtypes.py:85(is_dtype)
        2    0.000    0.000    0.000    0.000 offsets.py:194(__init__)
        2    0.000    0.000    0.000    0.000 generic.py:3570(f)
        1    0.000    0.000    0.000    0.000 internals.py:3174(_verify_integrity)
        4    0.000    0.000    0.001    0.000 <frozen importlib._bootstrap_external>:1149(find_spec)
        2    0.000    0.000    0.000    0.000 parser.py:375(__init__)
        5    0.000    0.000    0.000    0.000 base.py:572(values)
        2    0.000    0.000    0.000    0.000 offsets.py:36(as_timestamp)
       24    0.000    0.000    0.000    0.000 {built-in method _imp.acquire_lock}
      8/4    0.000    0.000    0.001    0.000 <frozen importlib._bootstrap>:197(_call_with_frames_removed)
        2    0.000    0.000    0.000    0.000 parser.py:410(resolve_ymd)
       12    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:142(__init__)
        1    0.000    0.000    0.000    0.000 fromnumeric.py:1886(any)
        1    0.000    0.000    0.000    0.000 cast.py:252(maybe_promote)
        1    0.000    0.000    0.000    0.000 internals.py:2952(__init__)
        1    0.000    0.000    0.001    0.001 internals.py:4048(reindex_indexer)
       10    0.000    0.000    0.000    0.000 parser.py:194(isword)
        4    0.000    0.000    0.000    0.000 config.py:223(__call__)
        1    0.000    0.000    0.000    0.000 base.py:417(_shallow_copy)
        2    0.000    0.000    0.000    0.000 generic.py:3557(_protect_consolidate)
        8    0.000    0.000    0.000    0.000 _methods.py:37(_any)
        1    0.000    0.000    0.000    0.000 numeric.py:149(_convert_scalar_indexer)
        2    0.000    0.000    0.000    0.000 offsets.py:2687(__add__)
        1    0.000    0.000    0.000    0.000 frame.py:307(__init__)
        1    0.000    0.000    0.000    0.000 indexing.py:234(_convert_scalar_indexer)
        2    0.000    0.000    0.000    0.000 {built-in method pandas._libs.lib.isscalar}
        8    0.000    0.000    0.000    0.000 six.py:184(find_module)
        2    0.000    0.000    0.000    0.000 parser.py:327(hms)
        2    0.000    0.000    0.000    0.000 parser.py:361(validate)
        2    0.000    0.000    0.000    0.000 common.py:184(is_bool_indexer)
        1    0.000    0.000    0.010    0.010 indexing.py:1326(__getitem__)
        4    0.000    0.000    0.000    0.000 {method 'split' of 'str' objects}
       12    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:173(cb)
        1    0.000    0.000    0.000    0.000 base.py:1804(_assert_take_fillable)
        2    0.000    0.000    0.000    0.000 generic.py:347(_get_axis_number)
       28    0.000    0.000    0.000    0.000 {method 'isdigit' of 'str' objects}
        4    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1080(_path_importer_cache)
        1    0.000    0.000    0.000    0.000 dtypes.py:370(__new__)
        1    0.000    0.000    0.000    0.000 base.py:1321(_convert_scalar_indexer)
        1    0.000    0.000    0.000    0.000 {pandas._libs.lib.maybe_indices_to_slice}
        2    0.000    0.000    0.000    0.000 common.py:1773(_get_dtype_type)
       12    0.000    0.000    0.000    0.000 internals.py:2995(<genexpr>)
       20    0.000    0.000    0.000    0.000 {method 'read' of '_io.StringIO' objects}
        4    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:771(find_spec)
        4    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:989(_handle_fromlist)
        1    0.000    0.000    0.000    0.000 algorithms.py:1254(_get_take_nd_function)
        1    0.000    0.000    0.000    0.000 base.py:717(_get_attributes_dict)
        2    0.000    0.000    0.000    0.000 base.py:1207(is_monotonic_increasing)
       24    0.000    0.000    0.000    0.000 {built-in method _thread.get_ident}
        4    0.000    0.000    0.000    0.000 {built-in method _imp.is_frozen}
        2    0.000    0.000    0.000    0.000 numeric.py:534(asanyarray)
        4    0.000    0.000    0.000    0.000 parser.py:204(isspace)
        2    0.000    0.000    0.000    0.000 generic.py:3567(_consolidate_inplace)
        1    0.000    0.000    0.000    0.000 internals.py:103(__init__)
       10    0.000    0.000    0.000    0.000 {method 'append' of 'list' objects}
        1    0.000    0.000    0.000    0.000 internals.py:3467(_consolidate_check)
        4    0.000    0.000    0.000    0.000 {method 'pop' of 'list' objects}
       12    0.000    0.000    0.000    0.000 {method 'lower' of 'str' objects}
        2    0.000    0.000    0.000    0.000 abc.py:178(__instancecheck__)
        2    0.000    0.000    0.000    0.000 _weakrefset.py:70(__contains__)
        4    0.000    0.000    0.000    0.000 config.py:603(_warn_if_deprecated)
        1    0.000    0.000    0.000    0.000 common.py:85(is_object_dtype)
        1    0.000    0.000    0.000    0.000 dtypes.py:676(is_dtype)
        4    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:37(_relax_case)
        4    0.000    0.000    0.000    0.000 config.py:529(_select_options)
        2    0.000    0.000    0.000    0.000 base.py:1202(is_monotonic)
        2    0.000    0.000    0.000    0.000 offsets.py:402(__radd__)
        2    0.000    0.000    0.000    0.000 datetimelike.py:306(freqstr)
        1    0.000    0.000    0.000    0.000 generic.py:121(__init__)
        1    0.000    0.000    0.000    0.000 internals.py:214(make_block_same_class)
        1    0.000    0.000    0.000    0.000 internals.py:307(ftype)
        3    0.000    0.000    0.000    0.000 {built-in method pandas._libs.algos.ensure_int64}
        2    0.000    0.000    0.000    0.000 common.py:657(is_dtype_equal)
        4    0.000    0.000    0.000    0.000 common.py:1722(_get_dtype)
        1    0.000    0.000    0.000    0.000 dtypes.py:428(construct_from_string)
        1    0.000    0.000    0.000    0.000 generic.py:1762(_indexer)
        1    0.000    0.000    0.000    0.000 internals.py:2856(make_block)
        1    0.000    0.000    0.000    0.000 {built-in method numpy.core.multiarray.arange}
        1    0.000    0.000    0.000    0.000 {method 'search' of '_sre.SRE_Pattern' objects}
        2    0.000    0.000    0.000    0.000 parser.py:308(jump)
        4    0.000    0.000    0.000    0.000 config.py:590(_translate_key)
        1    0.000    0.000    0.000    0.000 common.py:444(_apply_if_callable)
        1    0.000    0.000    0.000    0.000 generic.py:377(_get_block_manager_axis)
        1    0.000    0.000    0.000    0.000 generic.py:1915(_set_is_copy)
        1    0.000    0.000    0.001    0.001 internals.py:4083(<listcomp>)
        4    0.000    0.000    0.000    0.000 {method 'partition' of 'str' objects}
        1    0.000    0.000    0.000    0.000 numeric.py:2565(array_equal)
        2    0.000    0.000    0.000    0.000 common.py:478(is_categorical_dtype)
        3    0.000    0.000    0.000    0.000 base.py:3406(_maybe_cast_indexer)
        1    0.000    0.000    0.000    0.000 generic.py:163(_init_mgr)
        2    0.000    0.000    0.000    0.000 indexing.py:2064(is_list_like_indexer)
        4    0.000    0.000    0.000    0.000 internals.py:3459(is_consolidated)
        2    0.000    0.000    0.000    0.000 internals.py:3748(consolidate)
        2    0.000    0.000    0.000    0.000 internals.py:3764(_consolidate_inplace)
        2    0.000    0.000    0.000    0.000 inference.py:234(is_list_like)
        3    0.000    0.000    0.000    0.000 common.py:334(is_datetime64tz_dtype)
        1    0.000    0.000    0.000    0.000 dtypes.py:270(construct_from_string)
        3    0.000    0.000    0.000    0.000 datetimes.py:1548(_get_freq)
        6    0.000    0.000    0.000    0.000 {built-in method pandas._libs.lib.is_float}
       10    0.000    0.000    0.000    0.000 {method 'isalpha' of 'str' objects}
        1    0.000    0.000    0.000    0.000 {built-in method builtins.min}
        2    0.000    0.000    0.000    0.000 parser.py:319(month)
        3    0.000    0.000    0.000    0.000 common.py:223(is_datetimetz)
        4    0.000    0.000    0.000    0.000 {built-in method pandas._libs.lib.is_integer}
        2    0.000    0.000    0.000    0.000 {method 'fill' of 'numpy.ndarray' objects}
        4    0.000    0.000    0.000    0.000 {method 'end' of '_sre.SRE_Match' objects}
        3    0.000    0.000    0.000    0.000 {method 'items' of 'dict' objects}
        1    0.000    0.000    0.000    0.000 common.py:297(is_datetime64_dtype)
        1    0.000    0.000    0.000    0.000 base.py:535(_reset_identity)
        1    0.000    0.000    0.000    0.000 base.py:719(<listcomp>)
        2    0.000    0.000    0.000    0.000 base.py:4081(_ensure_index)
        2    0.000    0.000    0.000    0.000 offsets.py:205(<genexpr>)
        1    0.000    0.000    0.000    0.000 generic.py:3479(__finalize__)
        1    0.000    0.000    0.000    0.000 internals.py:2953(<listcomp>)
        6    0.000    0.000    0.000    0.000 {built-in method builtins.issubclass}
        1    0.000    0.000    0.000    0.000 {built-in method builtins.sum}
        4    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:698(find_spec)
        2    0.000    0.000    0.000    0.000 numeric.py:463(asarray)
        2    0.000    0.000    0.000    0.000 parser.py:177(__iter__)
        1    0.000    0.000    0.000    0.000 common.py:191(is_categorical)
        1    0.000    0.000    0.000    0.000 common.py:442(is_interval_dtype)
        2    0.000    0.000    0.000    0.000 internals.py:2997(ndim)
        2    0.000    0.000    0.000    0.000 parser.py:351(convertyear)
        1    0.000    0.000    0.000    0.000 function.py:38(__call__)
        1    0.000    0.000    0.000    0.000 base.py:515(is_)
        1    0.000    0.000    0.000    0.000 indexing.py:1487(_get_partial_string_timestamp_match_key)
        1    0.000    0.000    0.000    0.000 internals.py:181(fill_value)
        4    0.000    0.000    0.000    0.000 internals.py:185(mgr_locs)
        1    0.000    0.000    0.000    0.000 internals.py:222(mgr_locs)
        2    0.000    0.000    0.000    0.000 internals.py:3176(<genexpr>)
        1    0.000    0.000    0.000    0.000 internals.py:3468(<listcomp>)
        1    0.000    0.000    0.000    0.000 {built-in method pandas._libs.algos.ensure_platform_int}
        1    0.000    0.000    0.000    0.000 {method 'update' of 'dict' objects}
        1    0.000    0.000    0.000    0.000 {built-in method __new__ of type object at 0x000000005748B620}
        4    0.000    0.000    0.000    0.000 {method 'isspace' of 'str' objects}
        1    0.000    0.000    0.000    0.000 frame.py:295(_constructor)
        1    0.000    0.000    0.000    0.000 internals.py:295(shape)
        1    0.000    0.000    0.000    0.000 internals.py:3070(_get_items)
        1    0.000    0.000    0.000    0.000 {built-in method pandas._libs.tslibs.timezones.maybe_get_tz}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
        1    0.000    0.000    0.000    0.000 {built-in method builtins.callable}
        1    0.000    0.000    0.000    0.000 datetimes.py:1666(dtype)
        2    0.000    0.000    0.000    0.000 internals.py:303(dtype)

@jreback jreback added Indexing Related to indexing on series/frames, not to indexes themselves Performance Memory or execution speed performance labels Oct 3, 2017
@jreback jreback mentioned this issue Jan 31, 2018
Closed
@chris-b1 chris-b1 mentioned this issue Feb 19, 2018
Closed
@TomAugspurger TomAugspurger mentioned this issue Apr 27, 2018
Merged
@mzeitlin11
Copy link
Member

Timings on master look better now, with sorted about 3x faster than not:

>>> %timeit df.loc['2011-6-11']
1.49 ms ± 18.4 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

>>> %timeit df_sort.loc['2011-6-11']
550 µs ± 8.21 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

The behavior from the 1st 2 examples is deprecated now, though while

df_sort['2011-6-11']

gives a FutureError

df['2011-6-11']

raises:

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-28-abc338eb08f8> in <module>
----> 1 df['2011-6-11']

~/Code/contrib/pandas-mzeitlin11/pandas/core/frame.py in __getitem__(self, key)
   3005             # either we have a slice or we have a string that can be converted
   3006             #  to a slice for partial-string date indexing
-> 3007             return self._slice(indexer, axis=0)
   3008
   3009         # Do we have a (boolean) DataFrame?

~/Code/contrib/pandas-mzeitlin11/pandas/core/generic.py in _slice(self, slobj, axis)
   3815         Slicing with this method is *always* positional.
   3816         """
-> 3817         assert isinstance(slobj, slice), type(slobj)
   3818         axis = self._get_block_manager_axis(axis)
   3819         result = self._constructor(self._mgr.get_slice(slobj, axis=axis))

AssertionError: <class 'numpy.ndarray'>

@mzeitlin11 mzeitlin11 added Warnings Warnings that appear or should be added to pandas Error Reporting Incorrect or improved errors from pandas labels Dec 24, 2020
@jreback
Copy link
Contributor

jreback commented Dec 24, 2020

@mzeitlin11 do we have asvs for this?

@mzeitlin11
Copy link
Member

I don't see one, will open an issue for it

@mzeitlin11 mzeitlin11 mentioned this issue Dec 25, 2020
Closed
@mroeschke mroeschke removed the Warnings Warnings that appear or should be added to pandas label Jun 12, 2021
@mroeschke
Copy link
Member

Since the performance has greatly improved now going to close

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Error Reporting Incorrect or improved errors from pandas Indexing Related to indexing on series/frames, not to indexes themselves Performance Memory or execution speed performance
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@jreback @jschendel @tdpetrou @mroeschke @mzeitlin11