[BUG] udf_cudf tests failed: EOFException DataInputStream.readInt(DataInputStream.java:392)

[NvTimLiu]

Describe the bug

FAILED ../../src/main/python/udf_cudf_test.py::test_with_column[small data]
FAILED ../../src/main/python/udf_cudf_test.py::test_with_column[large data]
FAILED ../../src/main/python/udf_cudf_test.py::test_sql - py4j.protocol.Py4JJ...
FAILED ../../src/main/python/udf_cudf_test.py::test_select - py4j.protocol.Py...
FAILED ../../src/main/python/udf_cudf_test.py::test_map_in_pandas - py4j.prot...
FAILED ../../src/main/python/udf_cudf_test.py::test_group_apply - py4j.protoc...
FAILED ../../src/main/python/udf_cudf_test.py::test_group_apply_in_pandas - p...
FAILED ../../src/main/python/udf_cudf_test.py::test_group_agg - py4j.protocol...
FAILED ../../src/main/python/udf_cudf_test.py::test_sql_group - py4j.protocol...
FAILED ../../src/main/python/udf_cudf_test.py::test_window - py4j.protocol.Py...
FAILED ../../src/main/python/udf_cudf_test.py::test_cogroup[ALLOW_NON_GPU(GpuFlatMapCoGroupsInPandasExec,PythonUDF)]
======== 11 failed, 14178 deselected, 133 warnings in 129.66s (0:02:09) ========

E                   Caused by: java.io.EOFException
E                   	at java.io.DataInputStream.readInt(DataInputStream.java:392)
E                   	at org.apache.spark.api.python.PythonWorkerFactory.createSocket$1(PythonWorkerFactory.scala:121)
E                   	at org.apache.spark.api.python.PythonWorkerFactory.liftedTree1$1(PythonWorkerFactory.scala:137)
E                   	at org.apache.spark.api.python.PythonWorkerFactory.createThroughDaemon(PythonWorkerFactory.scala:136)
E                   	at org.apache.spark.api.python.PythonWorkerFactory.create(PythonWorkerFactory.scala:106)
E                   	at org.apache.spark.SparkEnv.createPythonWorker(SparkEnv.scala:121)
E                   	at org.apache.spark.api.python.BasePythonRunner.compute(PythonRunner.scala:162)
E                   	at org.apache.spark.sql.execution.python.PandasGroupUtils$.executePython(PandasGroupUtils.scala:44)
E                   	at org.apache.spark.sql.execution.python.rapids.GpuPandasUtils$.executePython(GpuPandasUtils.scala:35)
E                   	at org.apache.spark.sql.rapids.execution.python.GpuFlatMapCoGroupsInPandasExec.$anonfun$doExecute$1(GpuFlatMapCoGroupsInPandasExec.scala:137)
E                   	at org.apache.spark.rdd.ZippedPartitionsRDD2.compute(ZippedPartitionsRDD.scala:89)

[pxLi]

[root@6ad6a03c7f13 /]# conda list | grep cudf
cudf                      22.06.00a220525 cuda_11_py38_gb4674a1b1c_311    rapidsai-nightly
libcudf                   22.06.00a220525 cuda11_gb4674a1b1c_311    rapidsai-nightly
[root@6ad6a03c7f13 /]# python
Python 3.8.13 | packaged by conda-forge | (default, Mar 25 2022, 06:04:10)
[GCC 10.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cudf
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/conda/lib/python3.8/site-packages/cudf/__init__.py", line 71, in <module>
    from cudf.io import (
  File "/opt/conda/lib/python3.8/site-packages/cudf/io/__init__.py", line 8, in <module>
    from cudf.io.orc import read_orc, read_orc_metadata, to_orc
  File "/opt/conda/lib/python3.8/site-packages/cudf/io/orc.py", line 14, in <module>
    from cudf.utils.metadata import (  # type: ignore
  File "/opt/conda/lib/python3.8/site-packages/cudf/utils/metadata/orc_column_statistics_pb2.py", line 7, in <module>
    from google.protobuf.internal import builder as _builder
ImportError: cannot import name 'builder' from 'google.protobuf.internal' (/opt/conda/lib/python3.8/site-packages/google/protobuf/internal/__init__.py)
>>>

manually upgrade protobuf would resolve this issue, I guess this was just some cudf nightly w/ bad dependency version

[GaryShen2008]

I think we don't need a latest snapshot version of cudf to test.
I wonder if we should run udf_cudf test based on a released version of cudf to avoid such issue in the development process.

[pxLi]

If that, we only have released cudf 22.04 for testing

[pxLi]

still happening, also fail 22.08. reported at rapidsai/cudf#10990

[GaryShen2008]

The cudf PR has been merged and the latest test succeeded.
I think we can close it.