Fix bug where list like object not returning empty Index. #26471

sumanau7 · 2019-05-20T14:12:39Z

closes setitem on empty DataFrame / Series casts to RangeIndex #17101
tests added / passed
passes git diff upstream/master -u -- "*.py" | flake8 --diff
whatsnew entry

sumanau7 · 2019-05-20T14:14:52Z

@TomAugspurger could you please share your thoughts, can you also help me locate the file where tests have to be added, i am sorry but i don't understand project structure completely. Thanks in advance.

TomAugspurger · 2019-05-20T14:19:49Z

I think a new test with the code from the original issue at the top-level of pandas/tests/indexing/test_indexing.py makes sense

pep8speaks · 2019-05-20T14:43:13Z

Hello @sumanau7! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2019-08-20 19:14:09 UTC

codecov · 2019-05-20T14:43:14Z

Codecov Report

Merging #26471 into master will decrease coverage by 50.04%.
The diff coverage is 100%.

@@             Coverage Diff             @@
##           master   #26471       +/-   ##
===========================================
- Coverage   91.75%    41.7%   -50.05%     
===========================================
  Files         174      174               
  Lines       50765    50765               
===========================================
- Hits        46578    21173    -25405     
- Misses       4187    29592    +25405

Flag	Coverage Δ
#multiple	`?`
#single	`41.7% <100%> (-0.08%)`	⬇️

Impacted Files	Coverage Δ
pandas/core/frame.py	`34.61% <100%> (-62.53%)`	⬇️
pandas/io/formats/latex.py	`0% <0%> (-100%)`	⬇️
pandas/io/sas/sas_constants.py	`0% <0%> (-100%)`	⬇️
pandas/core/groupby/categorical.py	`0% <0%> (-100%)`	⬇️
pandas/tseries/plotting.py	`0% <0%> (-100%)`	⬇️
pandas/tseries/converter.py	`0% <0%> (-100%)`	⬇️
pandas/io/formats/html.py	`0% <0%> (-99.37%)`	⬇️
pandas/io/sas/sas7bdat.py	`0% <0%> (-91.16%)`	⬇️
pandas/io/sas/sas_xport.py	`0% <0%> (-90.1%)`	⬇️
pandas/core/tools/numeric.py	`10.14% <0%> (-89.86%)`	⬇️
... and 130 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b563d45...880790c. Read the comment docs.

codecov · 2019-05-20T14:43:16Z

Codecov Report

Merging #26471 into master will increase coverage by 49.14%.
The diff coverage is 100%.

@@             Coverage Diff             @@
##           master   #26471       +/-   ##
===========================================
+ Coverage   42.56%   91.71%   +49.14%     
===========================================
  Files         189      178       -11     
  Lines       50059    50755      +696     
===========================================
+ Hits        21307    46548    +25241     
+ Misses      28752     4207    -24545

Flag	Coverage Δ
#multiple	`90.3% <100%> (?)`
#single	`41.2% <100%> (-1.36%)`	⬇️

Impacted Files	Coverage Δ
pandas/core/frame.py	`96.88% <100%> (+62.12%)`	⬆️
pandas/io/gbq.py	`78.94% <0%> (-4.39%)`	⬇️
pandas/io/feather_format.py	`21.05% <0%> (-3.95%)`	⬇️
pandas/_config/__init__.py	`100% <0%> (ø)`	⬆️
pandas/core/api.py	`100% <0%> (ø)`	⬆️
pandas/io/msgpack/exceptions.py	`83.33% <0%> (ø)`	⬆️
pandas/io/clipboard/windows.py	`11.11% <0%> (ø)`	⬆️
pandas/io/api.py	`100% <0%> (ø)`	⬆️
pandas/core/arrays/__init__.py	`100% <0%> (ø)`	⬆️
pandas/core/dtypes/generic.py	`100% <0%> (ø)`	⬆️
... and 178 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d2031d7...25a269b. Read the comment docs.

doc/source/whatsnew/v0.25.0.rst

pandas/tests/indexing/test_indexing.py

sumanau7 · 2019-05-21T06:18:44Z

@TomAugspurger changing the behaviour is breaking few tests, i have fixed one of them if the approach is right, i can check what are the other places where tests are breaking and i can fix that.

Can you also share command on how can i run tests in individual files please, for example if i only want to run tests from file test_partial.py locally.

TomAugspurger · 2019-05-21T18:32:29Z

pandas/tests/indexing/test_partial.py

@@ -451,7 +451,7 @@ def test_partial_set_empty_frame(self):
        # these work as they don't really change
        # anything but the index
        # GH5632
-        expected = DataFrame(columns=['foo'], index=Index([], dtype='int64'))
+        expected = DataFrame(columns=['foo'], index=Index([]))


I think we want to change this test. IIUC, previously the expected index was an empty Int64Index, which would compare equal to an empty RangeIndex. Now it'll be an empty object-dtype index.

cc @jreback

pandas/tests/indexing/test_indexing.py

doc/source/whatsnew/v0.25.0.rst

pandas/core/frame.py

pandas/tests/indexing/test_indexing.py

jreback · 2019-06-08T21:59:13Z

can you merge master and update to comments

2. Edit v0.25.0.rst to update changelog.

… for test func.

…subsection. 3. Update another failing test

sumanau7 · 2019-06-10T18:08:21Z

@jreback Is the PR ok ?

jreback

can you add a whatsnew note in bug fixes / indexing section

pandas/tests/indexing/test_indexing.py

jreback · 2019-07-03T19:54:49Z

can you merge master and respond to comments

TomAugspurger · 2019-07-08T13:56:00Z

@sumanau7 there's a merge conflict. Can you update to fix it?

TomAugspurger · 2019-08-20T19:14:12Z

Merged master & addressed the comments. Let's see if CI passes.

TomAugspurger · 2019-08-21T15:21:10Z

@jbrockmendel can you give this a quick sanity check? Would be nice for 0.25.1.

jbrockmendel · 2019-08-21T16:39:24Z

The new test looks good to me, but it isn't obvious that the edited test needs to be changed (not that it is wrong, just not obviously needed)

jbrockmendel · 2019-08-21T16:40:11Z

pandas/tests/indexing/test_partial.py

@@ -469,22 +469,21 @@ def f():
        expected["foo"] = expected["foo"].astype("float64")

        def f():
-            df = DataFrame()
+            df = DataFrame(index=Index([], dtype="int64"))
            df["foo"] = []
            return df

        tm.assert_frame_equal(f(), expected)

        def f():


why do these need nested functions?

Agreed it's strange, but we'll ignore that, since the modifications are following the existing structure.

TomAugspurger · 2019-08-22T11:41:02Z

Thanks @sumanau7!

…v#26471)

* master: (40 commits) DOC: Fix GL01 and GL02 errors in the docstrings (pandas-dev#27988) Remove Encoding of values in char** For Labels (pandas-dev#27618) TYPING: more type hints for io.formats.printing (pandas-dev#27765) TST: fix compression tests when run without virtualenv/condaenv (pandas-dev#28051) DOC: Start 0.25.2 (pandas-dev#28111) DOC: Fix docstrings lack of punctuation (pandas-dev#28031) DOC: Remove alias for numpy.random.randn from the docs (pandas-dev#28082) DOC: update GroupBy.head()/tail() documentation (pandas-dev#27844) BUG: timedelta merge asof with tolerance (pandas-dev#27650) BUG: Series.rename raises error on values accepted by Series construc… (pandas-dev#27814) Preserve index when setting new column on empty dataframe. (pandas-dev#26471) BUG: Fixed groupby quantile for listlike q (pandas-dev#27827) BUG: iter with readonly values, closes pandas-dev#28055 (pandas-dev#28074) TST: non-strict xfail for period test (pandas-dev#28072) DOC: Update whatsnew (pandas-dev#28073) CI: disable codecov (pandas-dev#28065) CI: Set SHA for codecov upload (pandas-dev#28067) BUG: Correct the previous bug fixing on xlim for plotting (pandas-dev#28059) CI: Add pip dependence explicitly (pandas-dev#28008) DOC: Change document code prun in a row (pandas-dev#28029) ...

sumanau7 force-pushed the BUG-17101 branch 2 times, most recently from 6f545b8 to 3f0eb24 Compare May 20, 2019 16:24

TomAugspurger reviewed May 20, 2019

View reviewed changes

doc/source/whatsnew/v0.25.0.rst Outdated Show resolved Hide resolved

pandas/tests/indexing/test_indexing.py Outdated Show resolved Hide resolved

pandas/tests/indexing/test_indexing.py Outdated Show resolved Hide resolved

sumanau7 force-pushed the BUG-17101 branch from 3df92cd to 3b08100 Compare May 21, 2019 05:29

TomAugspurger reviewed May 21, 2019

View reviewed changes

gfyoung added Dtype Conversions Unexpected or buggy dtype conversions Indexing Related to indexing on series/frames, not to indexes themselves labels May 21, 2019

gfyoung reviewed May 21, 2019

View reviewed changes

pandas/tests/indexing/test_indexing.py Outdated Show resolved Hide resolved

jreback requested changes May 24, 2019

View reviewed changes

doc/source/whatsnew/v0.25.0.rst Outdated Show resolved Hide resolved

pandas/core/frame.py Outdated Show resolved Hide resolved

pandas/tests/indexing/test_indexing.py Outdated Show resolved Hide resolved

Sumanau Sareen added 4 commits June 10, 2019 12:35

1. Fix bug where list like object not returning empty Index.

1a1baa2

2. Edit v0.25.0.rst to update changelog.

Added test for ensuring empty df has an empty index.

47fb344

1. Add documentation under right subsection. 2. Use proper test utils…

3029d86

… for test func.

Fix test as empty df should return an empty index.

830ca39

sumanau7 force-pushed the BUG-17101 branch from 3b08100 to 61ab68c Compare June 10, 2019 10:34

1. Use proper variable names in tests. 2. Moved changelog to correct …

f2557a3

…subsection. 3. Update another failing test

sumanau7 force-pushed the BUG-17101 branch from 61ab68c to f2557a3 Compare June 10, 2019 11:13

Fix failing tests, due to change in behaviour

4a3822f

sumanau7 force-pushed the BUG-17101 branch from fa3fb22 to 4a3822f Compare June 10, 2019 13:02

jreback requested changes Jun 10, 2019

View reviewed changes

pandas/tests/indexing/test_indexing.py Outdated Show resolved Hide resolved

pandas/tests/indexing/test_indexing.py Outdated Show resolved Hide resolved

pandas/tests/indexing/test_indexing.py Outdated Show resolved Hide resolved

Merge remote-tracking branch 'upstream/master' into sumanau7-BUG-17101

56166e1

TomAugspurger added 3 commits August 20, 2019 14:10

restore master

5b7f68d

readd fix

e9c5553

review comments

25a269b

TomAugspurger added this to the 0.25.1 milestone Aug 21, 2019

jbrockmendel reviewed Aug 21, 2019

View reviewed changes

TomAugspurger merged commit a76df79 into pandas-dev:master Aug 22, 2019

galuhsahid pushed a commit to galuhsahid/pandas that referenced this pull request Aug 25, 2019

Preserve index when setting new column on empty dataframe. (pandas-de…

d436728

…v#26471)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix bug where list like object not returning empty Index. #26471

Fix bug where list like object not returning empty Index. #26471

sumanau7 commented May 20, 2019 •

edited

Loading

sumanau7 commented May 20, 2019

TomAugspurger commented May 20, 2019

pep8speaks commented May 20, 2019 •

edited

Loading

codecov bot commented May 20, 2019

codecov bot commented May 20, 2019 •

edited

Loading

sumanau7 commented May 21, 2019

TomAugspurger May 21, 2019

jreback commented Jun 8, 2019

sumanau7 commented Jun 10, 2019

jreback left a comment

jreback commented Jul 3, 2019

TomAugspurger commented Jul 8, 2019

TomAugspurger commented Aug 20, 2019

TomAugspurger commented Aug 21, 2019

jbrockmendel commented Aug 21, 2019

jbrockmendel Aug 21, 2019

TomAugspurger Aug 22, 2019

TomAugspurger commented Aug 22, 2019

Fix bug where list like object not returning empty Index. #26471

Fix bug where list like object not returning empty Index. #26471

Conversation

sumanau7 commented May 20, 2019 • edited Loading

sumanau7 commented May 20, 2019

TomAugspurger commented May 20, 2019

pep8speaks commented May 20, 2019 • edited Loading

Comment last updated at 2019-08-20 19:14:09 UTC

codecov bot commented May 20, 2019

Codecov Report

codecov bot commented May 20, 2019 • edited Loading

Codecov Report

sumanau7 commented May 21, 2019

TomAugspurger May 21, 2019

Choose a reason for hiding this comment

jreback commented Jun 8, 2019

sumanau7 commented Jun 10, 2019

jreback left a comment

Choose a reason for hiding this comment

jreback commented Jul 3, 2019

TomAugspurger commented Jul 8, 2019

TomAugspurger commented Aug 20, 2019

TomAugspurger commented Aug 21, 2019

jbrockmendel commented Aug 21, 2019

jbrockmendel Aug 21, 2019

Choose a reason for hiding this comment

TomAugspurger Aug 22, 2019

Choose a reason for hiding this comment

TomAugspurger commented Aug 22, 2019

sumanau7 commented May 20, 2019 •

edited

Loading

pep8speaks commented May 20, 2019 •

edited

Loading

codecov bot commented May 20, 2019 •

edited

Loading