[PERF] Performance impact of mixed_type_as_string JSON reader option in reading JSON lines
#15196
Assignees
Labels
Milestone
Comments
shrshi
changed the title
mixed_type_as_string JSON reader option in reading JSON lines
3 tasks
GregoryKimball
added
libcudf
|
@karthikeyann, would you please work with @shrshi on this performance hotspot? |
|
Provided @shrshi a minor patch to benchmark. This kernel could be skipped entirely for those cases. |
3 tasks
rapids-bot bot
pushed a commit
that referenced
this issue
Mar 8, 2024
…n is enabled (#15236) Addresses #15196 by applying a patch from @karthikeyann to skip the `infer_column_type_kernel` by forcing the mixed types column to be a string. With this optimization, we see a significant improvement in performance. Please refer to the [comment](#15236 (comment)) for a visualization of the results before and after applying this optimization as obtained from the [JSON lines benchmarking exercise](#15124). Authors: - Shruti Shivakumar (https://github.com/shrshi) - Karthikeyan (https://github.com/karthikeyann) Approvers: - Karthikeyan (https://github.com/karthikeyann) - Vukasin Milovanovic (https://github.com/vuule) URL: #15236
|
Closed by #15236 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This report presents some preliminary findings on the normalization, mixed types handling, byte range reading, and error recovery handling options for JSON lines input. Given a valid JSON input string i.e. with no modifications to the data generation and reading a single chunk i.e. the byte range consists of all records, we expect to see no significant impact of enabling these options.
Benchmarks were run on A100 80GB GPU, with all combinations of the above options being enabled/disabled, and a performance degradation of 98% was observed on enabling
mixed_type_as_string(keepingnormalize_single_quotes=NO row_selection=ALL recovery_mode=RECOVER_WITH_NULLconstant between the two experiments). Refer to figure for performance comparison.To investigate the impact of
mixed_type_as_stringbeing enabled, the benchmark was profiled with--axis normalize_single_quotes=NO --axis row_selection=ALL --axis mixed_types_as_string=YES --axis recovery_mode=RECOVER_WITH_NULL.The
infer_column_type_kernelappears to be the bottleneck due to stalled warps resulting in achieved occupancy of 9.3% (nsys and ncu profiles below).Next steps
Related information
The text was updated successfully, but these errors were encountered: