[Data] Reimplement of fix memory pandas

[Bye-legumes]

Why are these changes needed?

fix of #46939 with sampling to reduce the overhead.
Key improvement:

1. Use sampling to reduce the computation overhead
2. Use pandas sample then use df.values to convert it to numpy.
3. Use numpy vectorize to speed up the loop speed.
4. Add check for tensor with number type as the nbytes can be use directly.

Related issue number

#46785

Checks

• [√] I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• [√] I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• [√] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • [√] Unit tests
  • Release tests
  • This PR is not tested :(

[richardliaw]

let's see if the tests work -- i remember last time i had to implement def take in PythonObjectArray, but maybe you get around it more easily

[richardliaw]

relevant tests failing it looks like

[Bye-legumes]

relevant tests failing it looks like

It can pass this time but two tests are retried. In fact these two test has no problem locally so I am not sure it's related to the memory problem. i.e. when run full test there will OOM but retry can succeed. Let me try to make it stable first. But currently I fixed many test by checking numeric tensor or numpy. I think in these case it's safe to use nbytes will object string need future check,

[raulchen]

looks like it should be max_sample_size? as in "max number of items to sample"

[Bye-legumes]

@richardliaw @raulchen I think it OK now.

[BitPhinix]

Thanks for this PR! I've had to hack around this exact issue so many times - it would be great if we could get a release with the fix ASAP 🙌