You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
GpuBatchScanExec is supposed to show the same types of GPU-related metrics as FileSourceScanExec (e.g.: buffer time, GPU decode time, output batches, etc.) but output rows are the only metrics shown even with spark.rapids.sql.metrics.level=DEBUG.
Steps/Code to reproduce bug
Set spark.rapids.sql.metrics.level to DEBUG
Set spark.sql.sources.useV1SourceList to ""
Run a simple query to scan a Parquet file (e.g.: spark.read.parquet("tests/src/test/resources/timestamp-date-test.parquet").collect()
Examine the SQL UI and note that only output rows are shown as a metric for the GPU batch scan, e.g.:
Expected behavior
All of the metrics setup in GpuBatchScanExecMetrics should appear in the SQL UI for the GpuBatchScan node rather than just output rows.
Environment details (please complete the following information)
Replicated the problem with Spark 3.1.2 and Spark 3.2.1
The text was updated successfully, but these errors were encountered:
Describe the bug
GpuBatchScanExec is supposed to show the same types of GPU-related metrics as FileSourceScanExec (e.g.: buffer time, GPU decode time, output batches, etc.) but output rows are the only metrics shown even with
spark.rapids.sql.metrics.level=DEBUG
.Steps/Code to reproduce bug
spark.rapids.sql.metrics.level
toDEBUG
spark.sql.sources.useV1SourceList
to""
spark.read.parquet("tests/src/test/resources/timestamp-date-test.parquet").collect()
Expected behavior
All of the metrics setup in
GpuBatchScanExecMetrics
should appear in the SQL UI for theGpuBatchScan
node rather than just output rows.Environment details (please complete the following information)
Replicated the problem with Spark 3.1.2 and Spark 3.2.1
The text was updated successfully, but these errors were encountered: