[BUG] mismatch in up-casting Series.__setitem__ between pandas and cudf

[wence-]

Describe the bug

import pandas as pd
import cudf
import numpy as np

series = pd.Series([1, 2, 3], dtype=np.int64)
ceries = cudf.from_pandas(series)

series[1] = 1.5
ceries[1] = 1.5

assert np.allclose(series.values, ceries.values)

series.dtype => dtype('float64)
ceries.dtype => dtype('int64')

Expected behavior

cudf should mimic pandas behaviour here.

[Aside, I hate hate hate this upcasting from ints to floats since it is lossy for round-tripping, but I suppose we must live with it.]

Things work if I specify a list of indices:

series[[1, 2]] = 1.5
ceries[[1, 2]] = 1.5
assert np.allclose(series.values, ceries.values) => Fine

The relevant code on a series does this:

https://github.com/rapidsai/cudf/blob/branch-22.12/python/cudf/cudf/core/series.py#L216-L234

Note that if the key is a single integer, we don't upcast the column to the union dtype.

This logic goes back to #2442 which introduced __setitem__ (@shwina can you remember why?). I note that the __setitem__ implementation on Column has the opposite casting rules (if col[idx] = value does value.astype(col.dtype)), so there must be an explicit choice in Series.__setitem__ to mimic pandas.

Plausibly this is a fix:

diff --git a/python/cudf/cudf/core/series.py b/python/cudf/cudf/core/series.py
index f11052096e..b1e2773a55 100644
--- a/python/cudf/cudf/core/series.py
+++ b/python/cudf/cudf/core/series.py
@@ -222,14 +222,13 @@ class _SeriesIlocIndexer(_FrameIndexer):
             and _is_non_decimal_numeric_dtype(value.dtype)
         ):
             # normalize types if necessary:
-            if not is_integer(key):
-                to_dtype = np.result_type(
-                    value.dtype, self._frame._column.dtype
-                )
-                value = value.astype(to_dtype)
-                self._frame._column._mimic_inplace(
-                    self._frame._column.astype(to_dtype), inplace=True
-                )
+            to_dtype = np.result_type(
+                value.dtype, self._frame._column.dtype
+            )
+            value = value.astype(to_dtype)
+            self._frame._column._mimic_inplace(
+                self._frame._column.astype(to_dtype), inplace=True
+            )
 
         self._frame._column[key] = value
 

[shwina]

(@shwina can you remember why?

I cannot. I agree that automatically upcasting is the better behaviour.