Support new major numpy version

[Tansito]

What is the expected enhancement?

From #1524 we discovered that our current stack doesn't support the new major version from numpy. If you try to use numpy > 2 in the client Ray returns the next error:

Failed to start supervisor actor raysubmit_er8zgdCRKuMwP2mL: 'numpy.core.multiarray failed to import'. Full traceback:
Traceback (most recent call last):
  File "/usr/local/lib64/python3.11/site-packages/ray/dashboard/modules/job/job_manager.py", line 544, in submit_job
    ).remote(submission_id, entrypoint, metadata or {}, self._gcs_address)
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib64/python3.11/site-packages/ray/actor.py", line 833, in remote
    return actor_cls._remote(args=args, kwargs=kwargs, **updated_options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib64/python3.11/site-packages/ray/_private/auto_init_hook.py", line 21, in auto_init_wrapper
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib64/python3.11/site-packages/ray/util/tracing/tracing_helper.py", line 388, in _invocation_actor_class_remote_span
    return method(self, args, kwargs, *_args, **_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib64/python3.11/site-packages/ray/actor.py", line 1161, in _remote
    actor_id = worker.core_worker.create_actor(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "python/ray/_raylet.pyx", line 4114, in ray._raylet.CoreWorker.create_actor
  File "python/ray/_raylet.pyx", line 4119, in ray._raylet.CoreWorker.create_actor
  File "python/ray/_raylet.pyx", line 835, in ray._raylet.prepare_args_and_increment_put_refs
  File "python/ray/_raylet.pyx", line 826, in ray._raylet.prepare_args_and_increment_put_refs
  File "python/ray/_raylet.pyx", line 875, in ray._raylet.prepare_args_internal
  File "/usr/local/lib64/python3.11/site-packages/ray/_private/worker.py", line 679, in get_serialization_context
    context_map[job_id] = serialization.SerializationContext(self)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib64/python3.11/site-packages/ray/_private/serialization.py", line 163, in __init__
    serialization_addons.apply(self)
  File "/usr/local/lib64/python3.11/site-packages/ray/util/serialization_addons.py", line 39, in apply
    _register_custom_datasets_serializers(serialization_context)
  File "/usr/local/lib64/python3.11/site-packages/ray/_private/arrow_serialization.py", line 46, in _register_custom_datasets_serializers
    import pyarrow as pa  # noqa: F401
    ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib64/python3.11/site-packages/pyarrow/__init__.py", line 65, in <module>
    import pyarrow.lib as _lib
  File "pyarrow/lib.pyx", line 36, in init pyarrow.lib
ImportError: numpy.core.multiarray failed to import

[Tansito]

@paaragon could you open an issue in the ray repository explaining them the problem and the error that we are getting to see if they have it detected?

[Tansito]

@robertodr as soon as the PR is merged you will be able to use the new major version from numpy in dev and in the next release you will have this ready to apply in the function for production.