You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Today for user to specify ParquetCachedBatchSerializer, they have to specify the shim version:
--conf spark.sql.cache.serializer=com.nvidia.spark.rapids.shims.spark311.ParquetCachedBatchSerializer
Ideally we don't want user to have to say spark version and they just specify something like com.nvidia.spark.rapids.ParquetCachedBatchSerializer and we load the proper version.
Note with the 3.2 build work going on, we have created versions for each shim temporarily as well.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
Today for user to specify ParquetCachedBatchSerializer, they have to specify the shim version:
--conf spark.sql.cache.serializer=com.nvidia.spark.rapids.shims.spark311.ParquetCachedBatchSerializer
Ideally we don't want user to have to say spark version and they just specify something like com.nvidia.spark.rapids.ParquetCachedBatchSerializer and we load the proper version.
Note with the 3.2 build work going on, we have created versions for each shim temporarily as well.
The text was updated successfully, but these errors were encountered: