[FEA] cuio: consolidate host decompression

[cwharris]

Host decompression is used by json/csv readers immediately after datasource read. Rather than being a part of each reader/writer, we could consolidate them in to the datasource, or a decorator of some kind. This would simplify the readers, and also enable decouple decompression from range_offset and range_size.

This might become more straightforward after #6185

Aside from keeping things DRY, abstracting the host decompression away from the readers may allow use it in more readers.

Examples:

cudf/cpp/src/io/json/reader_impl.cu

Lines 642 to 645 in 2b09075

	ingest_raw_input(range_offset, range_size);
	CUDF_EXPECTS(buffer_ != nullptr, "Ingest failed: input data is null.\n");
	decompress_input(stream);

cudf/cpp/src/io/json/reader_impl.cu

Lines 240 to 260 in 2b09075

	void reader::impl::ingest_raw_input(size_t range_offset, size_t range_size)
	{
	size_t map_range_size = 0;
	if (range_size != 0) { map_range_size = range_size + calculate_max_row_size(args_.dtype.size()); }
	// Support delayed opening of the file if using memory mapping datasource
	// This allows only mapping of a subset of the file if using byte range
	if (source_ == nullptr) {
	assert(!filepath_.empty());
	source_ = datasource::create(filepath_, range_offset, map_range_size);
	}
	if (!source_->is_empty()) {
	auto data_size = (map_range_size != 0) ? map_range_size : source_->size();
	buffer_ = source_->host_read(range_offset, data_size);
	}
	byte_range_offset_ = range_offset;
	byte_range_size_ = range_size;
	load_whole_file_ = byte_range_offset_ == 0 && byte_range_size_ == 0;
	}

cudf/cpp/src/io/json/reader_impl.cu

Lines 268 to 283 in 2b09075

	void reader::impl::decompress_input(cudaStream_t stream)
	{
	const auto compression_type = infer_compression_type(
	args_.compression, filepath_, {{"gz", "gzip"}, {"zip", "zip"}, {"bz2", "bz2"}, {"xz", "xz"}});
	if (compression_type == "none") {
	// Do not use the owner vector here to avoid extra copy
	uncomp_data_ = reinterpret_cast<const char *>(buffer_->data());
	uncomp_size_ = buffer_->size();
	} else {
	uncomp_data_owner_ = getUncompressedHostData(
	reinterpret_cast<const char *>(buffer_->data()), buffer_->size(), compression_type);
	uncomp_data_ = uncomp_data_owner_.data();
	uncomp_size_ = uncomp_data_owner_.size();
	}
	if (load_whole_file_) data_ = rmm::device_buffer(uncomp_data_, uncomp_size_, stream);
	}

[github-actions]

This issue has been marked rotten due to no recent activity in the past 90d. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed.

[vuule]

@cwharris should we close this one?