removed compat check for Series.items method #13920

lewisacidic · 2016-08-05T15:55:49Z

Should I add something in the whatsnew?
I couldn't find a relevant test, so this is to check if this breaks any tests in CI.

closes Pandas.items in Python2 vs Python3 #13918
tests added / passed
passes git diff upstream/master | flake8 --diff
whatsnew entry

jreback · 2016-08-05T16:34:56Z

can you add a test for this (see where Series.iteritems is test)

jreback · 2016-08-05T16:35:01Z

yes would need a release note

lewisacidic · 2016-08-05T18:05:19Z

I wasn't sure how was best to test this (still very new to pandas development) - is this OK?

jreback · 2016-08-05T19:48:02Z

ok this looks reasonable, ping when green.

jreback · 2016-08-05T19:55:57Z

doc/source/whatsnew/v0.17.1.txt

@@ -202,3 +202,4 @@ Bug Fixes
 - Bug in ``DataFrame.to_sparse()`` loses column names for MultiIndexes (:issue:`11600`)
 - Bug in ``DataFrame.round()`` with non-unique column index producing a Fatal Python error (:issue:`11611`)
 - Bug in ``DataFrame.round()`` with ``decimals`` being a non-unique indexed Series producing extra columns (:issue:`11618`)
+- Fixed PY2/PY3 incompatability in ``Series.items`` (:issue:`13918`) 


put this in the API changes section as that will gain more attention. Say Series has gained the .items() method to mirror .iteritems() and for compat with PY3 (and add for DataFrame) as well.

shoyer · 2016-08-05T19:59:17Z

It would be nice to do the same for DataFrame.items() in this PR if you have time

lewisacidic · 2016-08-06T02:20:10Z

Oh yeah, sure I can add DataFrame.items() in this PR - might take a little while as I am nearing 48 hours without sleep now (you can probably tell...), and bed is calling me.

codecov-io · 2016-08-06T04:15:01Z

Current coverage is 85.26% (diff: 100%)

No coverage report found for master at 1919e26.

Powered by Codecov. Last update 1919e26...d6d93ba

jorisvandenbossche · 2016-08-15T13:41:17Z

@richlewis42 Can you update this?

lewisacidic · 2016-08-17T00:02:36Z

Ahh sorry completely fell off my todo list. First thing tomorrow, I will finish this.

lewisacidic · 2016-08-17T16:29:35Z

@jreback This looks ready now. Is everything good?

jreback · 2016-08-18T10:53:28Z

pandas/tests/frame/test_misc_api.py

@@ -241,6 +241,11 @@ def test_iteritems(self):
        for k, v in compat.iteritems(df):
            self.assertEqual(type(v), Series)

+    def test_items(self):
+        df = DataFrame([[1, 2, 3], [4, 5, 6]], columns=['a', 'a', 'b'])


add the issue number here as a comment

jreback · 2016-08-18T10:53:57Z

minorr change. lgtm.

Closes: pandas-dev#13918

shoyer · 2016-08-18T18:59:02Z

I am concerned that .items() does not return the expected result on Python 2. It would seem more consistent to write a list of items on Python 2 rather than an iterator.

lewisacidic · 2016-08-18T19:27:41Z

@jreback I added the comment

jreback · 2016-08-18T22:48:29Z

pandas/core/series.py

@@ -1096,8 +1096,7 @@ def iteritems(self):
        """
        return zip(iter(self.index), iter(self))

-    if compat.PY3:  # pragma: no cover
-        items = iteritems


I think @shoyer is right here,; I think this will work if you do

items=compat.iteritems which already has the correct PY2 (and 3) behavior

I think compa.iteritems just calls the object's iteritems method, so that would not make a difference?

Yeah, I think @jorisvandenbossche is right - see my new comment.

lewisacidic · 2016-08-22T11:21:15Z

@shoyer I think you are right, there is a PY2/PY3 inconsistency introduced here, and this requires more thought (sorry for leaving this around for so long, been very busy at work).

How I see this:

For dictionaries:

method	PY2	PY3
`items`	not lazy	lazy
`iteritems`	lazy	not implemented

Currently in pandas for Series and DataFrames:

method	PY2	PY3
`items`	not implemented	lazy
`iteritems`	lazy	lazy

In my opinion, PY3's compat.iteritems should be deprecated (or just left as is), and we should implement a compat.items which is not lazy for PY2 but is for PY3. Is that a better solution? @jreback @jorisvandenbossche what do you think?

Python 2

>>> d = dict(a=4, b=6)

>>> d.items()
[('a', 4), ('b', 6)]

>>> d.iteritems()
<dictionary-itemiterator at 0x10bedf310>

>>> s = pd.Series(d)

>>> s.items()
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-5-615b57d0112e> in <module>()
----> 1 s.items()

/Users/rich/anaconda/envs/python2/lib/python2.7/site-packages/pandas/core/generic.pyc in __getattr__(self, name)
   2239                 or name in self._metadata
   2240                 or name in self._accessors):
-> 2241             return object.__getattribute__(self, name)
   2242         else:
   2243             if name in self._info_axis:

AttributeError: 'Series' object has no attribute 'items'

>>> s.iteritems()
<itertools.izip at 0x10bfec488>

>>> df = s.reset_index()

>>> df.items()
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-8-952a7e29c4b1> in <module>()
----> 1 df.items()

/Users/rich/anaconda/envs/python2/lib/python2.7/site-packages/pandas/core/generic.pyc in __getattr__(self, name)
   2239                 or name in self._metadata
   2240                 or name in self._accessors):
-> 2241             return object.__getattribute__(self, name)
   2242         else:
   2243             if name in self._info_axis:

AttributeError: 'DataFrame' object has no attribute 'items'

>>> df.iteritems()
<generator object iteritems at 0x10bfe96e0>

Python 3

>>> d = dict(a=4, b=6)

>>> d.items()
dict_items([('b', 6), ('a', 4)])

>>> d.iteritems()
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-3-6c1b1c343924> in <module>()
----> 1 d.iteritems()

AttributeError: 'dict' object has no attribute 'iteritems'

>>> s = pd.Series(d)

>>> s.items()
<zip at 0x10686e388>

>>> s.iteritems()
<zip at 0x11e05c788>

>>> df = s.reset_index()

>>> df.items()
<generator object DataFrame.iteritems at 0x11deeb0f8>

>>> df.iteritems()
<generator object DataFrame.iteritems at 0x11dea2888>

jreback · 2016-08-26T20:26:45Z

@richlewis42 compat.items sounds fine to me. It is possible this already exists in six and was never included in our impl (which is included inline in compat/__init__.py).

jreback · 2016-09-09T22:30:45Z

can you rebase / update

jreback · 2016-11-16T20:58:25Z

can you update

jreback · 2016-12-21T23:17:14Z

@richlewis42 can you rebase / update.

I think we do need to make a non-lazy PY2 .items() for compat. (maybe something exists in pandas.compat for this? or in six which we can bring over.

lewisacidic · 2017-01-04T15:10:36Z

Hi @jreback sorry my bad somehow wasn't getting these updates. I'll have a look as soon as I can, I'm but I'm afraid I'm writing up my PhD at the moment so really busy.

jreback · 2017-02-27T16:04:05Z

closing as stale. please ping if you'd like to continue.

lewisacidic changed the title ~~removed compat check for items method~~ removed compat check for Series.items method Aug 5, 2016

jreback added the 2/3 Compat label Aug 5, 2016

jreback reviewed Aug 5, 2016
View reviewed changes

lewisacidic force-pushed the series-items branch from be2690c to bdecfda Compare August 6, 2016 02:14

lewisacidic force-pushed the series-items branch from 8e406d0 to 66d57ad Compare August 17, 2016 14:58

jreback reviewed Aug 18, 2016
View reviewed changes

jreback added this to the 0.19.0 milestone Aug 18, 2016

ENH: added .items to Series and DataFrame for both PY2 and PY3

d6d93ba

Closes: pandas-dev#13918

lewisacidic force-pushed the series-items branch from 66d57ad to d6d93ba Compare August 18, 2016 16:04

jreback reviewed Aug 18, 2016
View reviewed changes

jreback removed this from the 0.19.0 milestone Aug 26, 2016

jreback closed this Feb 27, 2017

jreback mentioned this pull request Aug 10, 2017

FIX: define DataFrame.items for all versions of python #17214

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

removed compat check for Series.items method #13920

removed compat check for Series.items method #13920

lewisacidic commented Aug 5, 2016 •

edited

Loading

jreback commented Aug 5, 2016

jreback commented Aug 5, 2016

lewisacidic commented Aug 5, 2016

jreback commented Aug 5, 2016

jreback Aug 5, 2016

shoyer commented Aug 5, 2016

lewisacidic commented Aug 6, 2016

codecov-io commented Aug 6, 2016 •

edited

Loading

jorisvandenbossche commented Aug 15, 2016

lewisacidic commented Aug 17, 2016

lewisacidic commented Aug 17, 2016

jreback Aug 18, 2016 •

edited

Loading

jreback commented Aug 18, 2016

shoyer commented Aug 18, 2016

lewisacidic commented Aug 18, 2016 •

edited

Loading

jreback Aug 18, 2016

jorisvandenbossche Aug 19, 2016

lewisacidic Aug 22, 2016

lewisacidic commented Aug 22, 2016

jreback commented Aug 26, 2016

jreback commented Sep 9, 2016

jreback commented Nov 16, 2016

jreback commented Dec 21, 2016

lewisacidic commented Jan 4, 2017

jreback commented Feb 27, 2017

removed compat check for Series.items method #13920

removed compat check for Series.items method #13920

Conversation

lewisacidic commented Aug 5, 2016 • edited Loading

jreback commented Aug 5, 2016

jreback commented Aug 5, 2016

lewisacidic commented Aug 5, 2016

jreback commented Aug 5, 2016

jreback Aug 5, 2016

Choose a reason for hiding this comment

shoyer commented Aug 5, 2016

lewisacidic commented Aug 6, 2016

codecov-io commented Aug 6, 2016 • edited Loading

Current coverage is 85.26% (diff: 100%)

jorisvandenbossche commented Aug 15, 2016

lewisacidic commented Aug 17, 2016

lewisacidic commented Aug 17, 2016

jreback Aug 18, 2016 • edited Loading

Choose a reason for hiding this comment

jreback commented Aug 18, 2016

shoyer commented Aug 18, 2016

lewisacidic commented Aug 18, 2016 • edited Loading

jreback Aug 18, 2016

Choose a reason for hiding this comment

jorisvandenbossche Aug 19, 2016

Choose a reason for hiding this comment

lewisacidic Aug 22, 2016

Choose a reason for hiding this comment

lewisacidic commented Aug 22, 2016

Python 2

Python 3

jreback commented Aug 26, 2016

jreback commented Sep 9, 2016

jreback commented Nov 16, 2016

jreback commented Dec 21, 2016

lewisacidic commented Jan 4, 2017

jreback commented Feb 27, 2017

lewisacidic commented Aug 5, 2016 •

edited

Loading

codecov-io commented Aug 6, 2016 •

edited

Loading

jreback Aug 18, 2016 •

edited

Loading

lewisacidic commented Aug 18, 2016 •

edited

Loading