COMPAT/TST: Ensure that numpy 2 string dtype converts to object array in pandas 2.2.x

[lithomas1]

We should add a test for this, given that numpy 2.0 support is happening for pandas 2.2.2.

[jbrockmendel]

Wouldn’t not-converting be nicer behavior?

[lithomas1]

Wouldn’t not-converting be nicer behavior?

I was hoping we could get @ngoldbaum 's numpy string array feature in in some form for 3.0, but that's probably too big of a change to get in for 2.2.2.

The main thing I want to avoid is a weird error happening somewhere in our internals if a numpy string array is used.

[ngoldbaum]

One big issue with just accepting stringdtype arrays is that stringdtype can’t support the buffer protocol, so a lot of cython code in pandas that expects an array type that supports the buffer protocol won’t work. My fork of pandas adding stringdtype support avoids low-level cython operations by calling numpy directly.

[mroeschke]

I would also support not coercing to object by default if possible (IMO it's OK postponing 2.2.2 extra to wait for compatibility)

Just curious, in numpy 2.0, will np.array(["1"]) default to the new string type?

[ngoldbaum]

Just curious, in numpy 2.0, will np.array(["1"]) default to the new string type?

No, someone needs to explicitly pass e.g. dtype=np.dtypes.StringDType() or dtype="T" somewhere to create a stringdtype array. Maybe in numpy 3.0 we'll change the default.

If there's interest in helping me out with getting my stringdtype changes upstreamed, I'd really appreciate it. My current latest work is here. Currently it's based on the pandas 3.0 branch but backporting it wouldn't be hard. I was hoping to have a PR open already but between getting pulled into other projects and helping out with shipping numpy 2.0 I didn't quite make it in time.

[phofl]

I don't think we should back port this, that change is too large imo