Pr 640 #172
Pr 640 #172
Conversation
NVIDIA#632) 3.0.1 Signed-off-by: Thomas Graves <[email protected]> Co-authored-by: Thomas Graves <[email protected]>
* Fix leak when we throw Signed-off-by: Thomas Graves <[email protected]> * Change to use closeOnExcept Signed-off-by: Thomas Graves <[email protected]> Co-authored-by: Thomas Graves <[email protected]>
* cudf and spark version info in artifacts Add cudf and spark version info in rapids-plugin artifacts file: rapids4spark-version-info.properties Signed-off-by: Tim Liu <[email protected]> * Add cuDF and spark version info in artifacts Create the separate files in the jar files Save relevant spark versions in all the shim jars * Add cudf & spark version files Create a separate file 'dependency-info.properties' in the jar to save cudf & spark version info * Include shims SparkShimServiceProvider into META-INF * SPARK_VER-info.properties files in the rapids jar * Add prefix with spark for the version files Co-authored-by: Tim Liu <[email protected]>
* changelog generator Signed-off-by: Peixin Li <[email protected]>
python Signed-off-by: Robert (Bobby) Evans <[email protected]>
Fixing an EOFExcetpion by creating a new file object on the same socket as output of the python worker process.
- Add a new object `PythonWorkerSemaphore`. - Add a new conf `spark.rapids.python.concurrentPythonWorkers` - Change class GpuSemaphore from `private` to `private[rapids]` - Let GpuSemaphore support not to initialize the GPU
Currently the limitation only works when pool memory is enabled. - Separate the configs for Python. - Add `OptionalConfEntry` for Python configs
including 5 types SQL_MAP_PANDAS_ITER_UDF SQL_GROUPED_AGG_PANDAS_UDF SQL_GROUPED_MAP_PANDAS_UDF SQL_SCALAR_PANDAS_ITER_UDF SQL_SCALAR_PANDAS_UDF by 4 physical plans GpuMapInPandasExec GpuAggregateInPandasExec GpuFlatMapGroupsInPandasExec GpuArrowEvalPythonExec along with a API update in python/worker.
These two types are SQL_COGROUPED_MAP_PANDAS_UDF and SQL_WINDOW_AGG_PANDAS_UDF, mapping to GpuFlatMapCoGroupsInPandasExec and GPuWindowInPandasExec respectively.
to include the Python configs in config doc generation to update the file `configs.md`.
And - Move python config check to driver side. - Throw an exception when python module mismatches.
And make the uvm config of Python internal.
Signed-off-by: Peixin Li <[email protected]>
|
build |
2 similar comments
|
build |
|
build |
|
dataComponent | Vulnerability | Severity |
* merge qa test to integration test * add run control for qa test and update sql * Update build script for qatest
No description provided.