Skip to content

Commit

Permalink
Remove benchmark-specific use of pinned-pooled memory in Parquet mult…
Browse files Browse the repository at this point in the history
…ithreaded benchmark. (#15838)

The benchmark was manually creating and using a pinned-pool rmm allocator which is now redundant, since cuIO itself does this by default.  This PR removes it.

Authors:
  - https://github.com/nvdbaranec
  - Nghia Truong (https://github.com/ttnghia)
  - Muhammad Haseeb (https://github.com/mhaseeb123)

Approvers:
  - Paul Mattione (https://github.com/pmattione-nvidia)
  - Nghia Truong (https://github.com/ttnghia)
  - Muhammad Haseeb (https://github.com/mhaseeb123)
  - Vukasin Milovanovic (https://github.com/vuule)

URL: #15838
  • Loading branch information
nvdbaranec authored May 24, 2024
1 parent 78a0314 commit 4a3315b
Showing 1 changed file with 0 additions and 17 deletions.
17 changes: 0 additions & 17 deletions cpp/benchmarks/io/parquet/parquet_reader_multithread.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -25,25 +25,12 @@
#include <cudf/utilities/default_stream.hpp>
#include <cudf/utilities/thread_pool.hpp>

#include <rmm/mr/device/pool_memory_resource.hpp>
#include <rmm/mr/pinned_host_memory_resource.hpp>
#include <rmm/resource_ref.hpp>

#include <nvtx3/nvtx3.hpp>

#include <nvbench/nvbench.cuh>

#include <vector>

// TODO: remove this once pinned/pooled is enabled by default in cuIO
void set_cuio_host_pinned_pool()
{
using host_pooled_mr = rmm::mr::pool_memory_resource<rmm::mr::pinned_host_memory_resource>;
static std::shared_ptr<host_pooled_mr> mr = std::make_shared<host_pooled_mr>(
std::make_shared<rmm::mr::pinned_host_memory_resource>().get(), 256ul * 1024 * 1024);
cudf::io::set_host_memory_resource(*mr);
}

size_t get_num_reads(nvbench::state const& state) { return state.get_int64("num_threads"); }

size_t get_read_size(nvbench::state const& state)
Expand Down Expand Up @@ -105,8 +92,6 @@ void BM_parquet_multithreaded_read_common(nvbench::state& state,
size_t const data_size = state.get_int64("total_data_size");
auto const num_threads = state.get_int64("num_threads");

set_cuio_host_pinned_pool();

auto streams = cudf::detail::fork_streams(cudf::get_default_stream(), num_threads);
cudf::detail::thread_pool threads(num_threads);

Expand Down Expand Up @@ -186,8 +171,6 @@ void BM_parquet_multithreaded_read_chunked_common(nvbench::state& state,
size_t const input_limit = state.get_int64("input_limit");
size_t const output_limit = state.get_int64("output_limit");

set_cuio_host_pinned_pool();

auto streams = cudf::detail::fork_streams(cudf::get_default_stream(), num_threads);
cudf::detail::thread_pool threads(num_threads);
auto [source_sink_vector, total_file_size, num_files] = write_file_data(state, d_types);
Expand Down

0 comments on commit 4a3315b

Please sign in to comment.