Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added seek to buffer to fix xlwt asv failure #19926

Merged
merged 2 commits into from
Feb 27, 2018

Conversation

WillAyd
Copy link
Member

@WillAyd WillAyd commented Feb 27, 2018

Sample ASV results from this as follows. Across 5 runs I got the upstream/master error every time and none on HEAD

(pandas_dev) williams-imac:asv_bench williamayd$ asv continuous -f 1.1 upstream/master HEAD -b io.excel.Excel.time_read_excel --show-stderr
· Creating environments
· Discovering benchmarks
·· Uninstalling from conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt.
·· Installing into conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt..
· Running 2 total benchmarks (2 commits * 1 environments * 1 benchmarks)
[  0.00%] · For pandas commit hash 15cd9d2d:
[  0.00%] ·· Building for conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt...
[  0.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[ 50.00%] ··· Running io.excel.Excel.time_read_excel                                                                                         ok
[ 50.00%] ···· 
               ============ =============
                  engine                 
               ------------ -------------
                 openpyxl      133±4ms   
                xlsxwriter    118±0.7ms  
                   xlwt      54.2±0.05ms 
               ============ =============

[ 50.00%] · For pandas commit hash f4c9d966:
[ 50.00%] ·· Building for conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt...
[ 50.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[100.00%] ··· Running io.excel.Excel.time_read_excel                                                                                 1/3 failed
[100.00%] ···· 
               ============ ===========
                  engine               
               ------------ -----------
                 openpyxl    130±0.7ms 
                xlsxwriter   117±0.4ms 
                   xlwt        failed  
               ============ ===========

[100.00%] ····· 
                
                For parameters: 'xlwt'
                Traceback (most recent call last):
                  File "/Users/williamayd/miniconda3/envs/pandas_dev/lib/python3.6/site-packages/asv/benchmark.py", line 867, in <module>
                    commands[mode](args)
                  File "/Users/williamayd/miniconda3/envs/pandas_dev/lib/python3.6/site-packages/asv/benchmark.py", line 844, in main_run
                    result = benchmark.do_run()
                  File "/Users/williamayd/miniconda3/envs/pandas_dev/lib/python3.6/site-packages/asv/benchmark.py", line 398, in do_run
                    return self.run(*self._current_params)
                  File "/Users/williamayd/miniconda3/envs/pandas_dev/lib/python3.6/site-packages/asv/benchmark.py", line 473, in run
                    samples, number = self.benchmark_timing(timer, repeat, warmup_time, number=number)
                  File "/Users/williamayd/miniconda3/envs/pandas_dev/lib/python3.6/site-packages/asv/benchmark.py", line 500, in benchmark_timing
                    timing = timer.timeit(number)
                  File "/Users/williamayd/clones/pandas/asv_bench/env/83b3be1235aa7b08e8a17448e2f70790/lib/python3.6/timeit.py", line 178, in timeit
                    timing = self.inner(it, self.timer)
                  File "<timeit-src>", line 6, in inner
                  File "/Users/williamayd/miniconda3/envs/pandas_dev/lib/python3.6/site-packages/asv/benchmark.py", line 464, in <lambda>
                    func = lambda: self.func(*param)
                  File "/Users/williamayd/clones/pandas/asv_bench/benchmarks/io/excel.py", line 29, in time_read_excel
                    read_excel(self.bio_read)
                  File "/Users/williamayd/clones/pandas/asv_bench/env/83b3be1235aa7b08e8a17448e2f70790/lib/python3.6/site-packages/pandas/util/_decorators.py", line 172, in wrapper
                    return func(*args, **kwargs)
                  File "/Users/williamayd/clones/pandas/asv_bench/env/83b3be1235aa7b08e8a17448e2f70790/lib/python3.6/site-packages/pandas/util/_decorators.py", line 172, in wrapper
                    return func(*args, **kwargs)
                  File "/Users/williamayd/clones/pandas/asv_bench/env/83b3be1235aa7b08e8a17448e2f70790/lib/python3.6/site-packages/pandas/io/excel.py", line 315, in read_excel
                    io = ExcelFile(io, engine=engine)
                  File "/Users/williamayd/clones/pandas/asv_bench/env/83b3be1235aa7b08e8a17448e2f70790/lib/python3.6/site-packages/pandas/io/excel.py", line 391, in __init__
                    self.book = xlrd.open_workbook(file_contents=data)
                  File "/Users/williamayd/clones/pandas/asv_bench/env/83b3be1235aa7b08e8a17448e2f70790/lib/python3.6/site-packages/xlrd/__init__.py", line 116, in open_workbook
                    with open(filename, "rb") as f:
                TypeError: expected str, bytes or os.PathLike object, not NoneType

SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY.

@jreback jreback added Performance Memory or execution speed performance IO Excel read_excel, to_excel labels Feb 27, 2018
@jreback jreback added this to the 0.23.0 milestone Feb 27, 2018
@jreback
Copy link
Contributor

jreback commented Feb 27, 2018

lgtm. ping on green.

@codecov
Copy link

codecov bot commented Feb 27, 2018

Codecov Report

Merging #19926 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master   #19926   +/-   ##
=======================================
  Coverage   91.66%   91.66%           
=======================================
  Files         150      150           
  Lines       48974    48974           
=======================================
  Hits        44894    44894           
  Misses       4080     4080
Flag Coverage Δ
#multiple 90.04% <ø> (ø) ⬆️
#single 41.85% <ø> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f4c9d96...c14b19e. Read the comment docs.

@TomAugspurger TomAugspurger merged commit 169af2c into pandas-dev:master Feb 27, 2018
@TomAugspurger
Copy link
Contributor

Thanks @WillAyd .

harisbal pushed a commit to harisbal/pandas that referenced this pull request Feb 28, 2018
* Added seek to buffer to fix xlwt asv failure

* Added conditional to check for seek on xlrd object
@WillAyd WillAyd deleted the excel-asv-xlwt branch February 28, 2018 00:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
IO Excel read_excel, to_excel Performance Memory or execution speed performance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ASV: occasional asv failures on xlwt
3 participants