Use KvikIO as the GDS backend

[madsbk]

Replacing the current GDS datasource with KvikIO.

Background

KvikIO is a new C++ and Python interface to cuFile that combines the current GDS implementation in cuDF and the current GDS implementation in cuCIM into a common library. The idea is to avoid double work and join forces to implement high performances IO for all of RAPIDS. In addition, KvikIO eliminates the needs for all projects to write a fallback implementation when GDS and/or cuFile isn't available. KvikIO should work in any circumstance even if cufile.h is available at compile time.

Policy

This PR reads LIBCUDF_CUFILE_POLICY to determent if KvikIO should be used. "OFF" disables KvikIO and "GDS" and "ALWAYS" enables KivkIO. It doesn't differentiate between "GDS" and "ALWAYS" and the question is if it should?
The current GDS backend rewrites the cuFile's configuration file in order to set cuFile's compatibility mode explicitly. Do we still want this feature? Isn't it better to just let users configure cuFile via. the dedicated configuration file?

---

Depend on rapidsai/kvikio#43

cc. @vuule

[codecov]

Codecov Report

Merging #10468 (3b40e48) into branch-22.06 (291fbcf) will increase coverage by 0.01%.
The diff coverage is n/a.

❗ Current head 3b40e48 differs from pull request most recent head 274c44f. Consider uploading reports for the commit 274c44f to get more accurate results

@@               Coverage Diff                @@
##           branch-22.06   #10468      +/-   ##
================================================
+ Coverage         86.31%   86.33%   +0.01%     
================================================
  Files               140      140              
  Lines             22300    22300              
================================================
+ Hits              19249    19253       +4     
+ Misses             3051     3047       -4     

Impacted Files	Coverage Δ
python/cudf/cudf/core/column/string.py	89.10% <0.00%> (+0.12%)	⬆️
python/cudf/cudf/core/groupby/groupby.py	91.72% <0.00%> (+0.22%)	⬆️
python/cudf/cudf/core/tools/datetimes.py	84.49% <0.00%> (+0.30%)	⬆️
python/cudf/cudf/core/column/lists.py	90.49% <0.00%> (+0.45%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 291fbcf...274c44f. Read the comment docs.

[devavret]

Can we get a gbench on GDS before and after?

[devavret]

Are file_io_utilities.hpp/cpp obsolete now?

[vuule]

Regarding the policy: I assume that KvikIO does not interact with the cuFile config?
If we remove the config override, the behavior would probably need to be "try KvikIO, if it fails, fall back to internal IO". This is different from the ALWAYS option, where we throw if anything goes wrong in cuFile. I feel like we need to get some form of logging before making the policy change, since cuFile/KvikIO use would be completely transparent otherwise and users wouldn't be able to tell if direct path is used (there's cuFile log, maybe that's fine to defer to).
Also, we'd want to evaluate compat code vs internal IO before making the change. Even without enforcing a default, we can recommend it in the documentation.
TL;DR: I'd like to keep the policy change separate from this PR, is possible.

[vuule]

Are file_io_utilities.hpp/cpp obsolete now?

For the most part, yes (if not completely). AFAIK, KvikIO includes all the best parts of this implementation :)

[robertmaynard]

We should work out the dependency tree for CUDF and KvikIO in regards to nvcomp. Looking at rapidsai/kvikio#23 it seems that the current approach would be to use cudf installs for nvcomp, which would make this dependency tree pretty messy.

[vuule]

@madsbk I think this should be moved to 22.06, IMO we are too close to the code freeze for a PR that affects most of IO.

[madsbk]

gbench on GDS

@devavret Is there a guide on how to run this?

@vuule I agree, we need an option to enable/disable GDS but KvikIO should work in all cases, we don't need a "try KvikIO, if it fails, fall back to internal IO". KvikIO implements its own fallback implementation that uses bounce buffers and doesn't require cufile.h or libcufile.so. In the future, I hope to completely replace the bounce buffer implementation in cuDF.

For now, I suggest that we have:

• LIBCUDF_CUFILE_POLICY=OFF use cuDF's bounce buffer implementation
• LIBCUDF_CUFILE_POLICY=ALWAYS use KvikIO and allow KvikIO to run in compatibility mode.
• LIBCUDF_CUFILE_POLICY=GDS use KvikIO and do not allow KvikIO to run in compatibility mode. However, cuFile might still run in compatibility mode if cufile.json allows it.

[madsbk]

@robertmaynard

We should work out the dependency tree for CUDF and KvikIO in regards to nvcomp. Looking at rapidsai/kvikio#23 it seems that the current approach would be to use cudf installs for nvcomp, which would make this dependency tree pretty messy.

Regarding rapidsai/kvikio#23, it is only for the Python bindings, the C++ API this PR uses doesn't depend on nvcomp. The only hard dependency of the KvikIO's C++ API is CUDAToolkit.

[devavret]

gbench on GDS

@devavret Is there a guide on how to run this?

You can build and run the PARQUET_READER_BENCH and PARQUET_WRITER_BENCH targets while the appropriate environment variables are set to use GDS. They end up in <builddir>/benchmarks/

[devavret]

Are file_io_utilities.hpp/cpp obsolete now?

For the most part, yes (if not completely). AFAIK, KvikIO includes all the best parts of this implementation :)

@madsbk, could you remove those as part of this PR?

[madsbk]

Are file_io_utilities.hpp/cpp obsolete now?

For the most part, yes (if not completely). AFAIK, KvikIO includes all the best parts of this implementation :)

@madsbk, could you remove those as part of this PR?

Will do, I just need to know how much to remove. It depends on the policy: do we want to change the cufile.json configuration?

[madsbk]

I am closing this PR in favor of #10593.