Skip to content

Commit

Permalink
column to row refactor for performance (#11063)
Browse files Browse the repository at this point in the history
Spent some time investigating performance on column to row and row to column conversion code. Lots of wins here. Split PR up, so this is column to row work only. Row to column to follow. Also, planning to add tests and benchmarks into `spark-rapids-jni` once this is all updated.

```
Comparing 22.08/ROW_CONVERSION_BENCH to mwilson/column_to_row_optimization/ROW_CONVERSION_BENCH
Benchmark                                                                               Time             CPU      Time Old      Time New       CPU Old       CPU New
--------------------------------------------------------------------------------------------------------------------------------------------------------------------
RowConversion/old_to_row_conversion/1048576/manual_time                              +0.0012         +0.0026             5             5             5             5
RowConversion/new_to_row_conversion/1048576/manual_time                              -0.5661         -0.5652            16             7            16             7
RowConversion/new_to_row_extended_conversion/1048576/manual_time                     -0.5680         -0.5655            16             7            16             7
RowConversion/string_to_row_extended_conversion/1048576/manual_time                  -0.5768         -0.5764            27            12            27            12
RowConversion/old_from_row_conversion/1048576/manual_time                            -0.0017         -0.0018             4             4             4             4
RowConversion/new_from_row_conversion/1048576/manual_time                            +1.1629         +1.1619            31            68            32            68
RowConversion/new_from_row_extended_conversion/1048576/manual_time                   +1.1540         +1.1549            31            68            32            68
RowConversion/string_from_row_extended_conversion/1048576/manual_time                +0.7185         +0.6831            39            67            40            68

Authors:
  - Mike Wilson (https://github.com/hyperbolic2346)

Approvers:
  - Nghia Truong (https://github.com/ttnghia)
  - https://github.com/nvdbaranec
  - MithunR (https://github.com/mythrocks)

URL: #11063
  • Loading branch information
hyperbolic2346 authored Jun 22, 2022
1 parent 36aa957 commit e3d3df3
Showing 1 changed file with 209 additions and 260 deletions.
Loading

0 comments on commit e3d3df3

Please sign in to comment.