Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

.loc assignment with empty label list should not convert dtypes #29707

Closed
Tracked by #1 ...
jondo opened this issue Nov 19, 2019 · 8 comments · Fixed by #59101
Closed
Tracked by #1 ...

.loc assignment with empty label list should not convert dtypes #29707

jondo opened this issue Nov 19, 2019 · 8 comments · Fixed by #59101
Labels
Dtype Conversions Unexpected or buggy dtype conversions Indexing Related to indexing on series/frames, not to indexes themselves Needs Tests Unit test(s) needed to prevent regressions

Comments

@jondo
Copy link

jondo commented Nov 19, 2019

Code Sample

import pandas as pd

df = pd.DataFrame({'a':[2,3]})
print(df.a.dtype) # int64
df.loc[[]] = 0.1
print(df.a.dtype) # float64!

Problem description

The .loc-Assignment with empty label list does not change any dataframe row, so it should not convert the column datatype from integer to float.

Seen with the current pandas version 0.25.3.

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit : None python : 3.6.8.final.0 python-bits : 64 OS : Linux OS-release : 4.15.0-62-generic machine : x86_64 processor : x86_64 byteorder : little LC_ALL : None LANG : en_DK.utf8 LOCALE : en_DK.UTF-8

pandas : 0.25.3
numpy : 1.17.4
pytz : 2019.3
dateutil : 2.8.1
pip : 19.3.1
setuptools : 41.6.0
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : None
pandas_datareader: None
bs4 : None
bottleneck : None
fastparquet : None
gcsfs : None
lxml.etree : None
matplotlib : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pytables : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
xarray : None
xlrd : None
xlwt : None
xlsxwriter : None

@jondo
Copy link
Author

jondo commented Nov 19, 2019

A counter argument would be that the length of the label list should not influence whether the conversion should happen or not. So the behavior might be by design.

@mroeschke
Copy link
Member

Thanks for the report! At minimum this doesn't look like friendly behavior and should probably change. I would argue that df.loc[[]] should possibly even raise in this case.

@mroeschke mroeschke added Dtype Conversions Unexpected or buggy dtype conversions Indexing Related to indexing on series/frames, not to indexes themselves labels Nov 20, 2019
@SteffenMeinecke
Copy link

Same issue exists when .loc gets an boolean including only False values, e.g.:

import numpy as np
import pandas as pd

DF = pd.DataFrame([])
DF["col0"] = np.array([1, 4])
DF["col1"] = np.array([6, 5], dtype=object)
print(DF.dtypes)  # int, object
DF.col1.loc[DF.col0 > 10] = "string"
print(DF.dtypes)  # int, int

@mroeschke mroeschke added the Bug label Jul 23, 2021
@phofl phofl added Needs Tests Unit test(s) needed to prevent regressions and removed Bug labels Apr 18, 2023
@phofl
Copy link
Member

phofl commented Apr 18, 2023

Works now

@MarvinGravert
Copy link
Contributor

MarvinGravert commented Apr 18, 2023

take

@parthi-siva
Copy link
Contributor

Hi @MarvinGravert are you working on it?

@MarvinGravert
Copy link
Contributor

Hi @parthi-siva ,
I am and have posed a PR though i will need to incorporate the feedback i have received on that PR before this issue can be closed. I hope i can get to that in the next few days

@parthi-siva
Copy link
Contributor

Hi @MarvinGravert ..
Sure, Thanks for the reply..
All the best :-)

MovsisyanM added a commit to MovsisyanM/pandas that referenced this issue Jun 25, 2024
MovsisyanM added a commit to MovsisyanM/pandas that referenced this issue Jun 25, 2024
MovsisyanM added a commit to MovsisyanM/pandas that referenced this issue Jun 25, 2024
mroeschke added a commit that referenced this issue Jun 25, 2024
* Naive test case for #29707

* Compare frames when testing loc dtype change for #29707

* Update pandas/tests/dtypes/test_dtypes.py

---------

Co-authored-by: Matthew Roeschke <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Dtype Conversions Unexpected or buggy dtype conversions Indexing Related to indexing on series/frames, not to indexes themselves Needs Tests Unit test(s) needed to prevent regressions
Projects
None yet
6 participants