Skip to content

Commit

Permalink
Mark nvcomp zstd compression stable (#12059)
Browse files Browse the repository at this point in the history
NVCOMP zstd compression was added in 22.10, but marked experimental, meaning you have to define the environment variable `LIBCUDF_NVCOMP_POLICY=ALWAYS` to enable it.  After completing validation testing using the spark rapids plugin as documented here: NVIDIA/spark-rapids#3037, we believe that we can now change the zstd compression status to stable, which will enable it in cudf by default.  `LIBCUDF_NVCOMP_POLICY=STABLE` is the default value.

Authors:
  - Jim Brennan (https://github.com/jbrennan333)

Approvers:
  - Nghia Truong (https://github.com/ttnghia)
  - David Wendt (https://github.com/davidwendt)
  - Vukasin Milovanovic (https://github.com/vuule)

URL: #12059
  • Loading branch information
jbrennan333 authored Nov 4, 2022
1 parent e788f36 commit a3e9c1c
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 2 deletions.
2 changes: 1 addition & 1 deletion cpp/src/io/comp/nvcomp_adapter.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -398,7 +398,7 @@ bool is_compression_enabled(compression_type compression)
return false;
case compression_type::SNAPPY: return detail::nvcomp_integration::is_stable_enabled();
case compression_type::ZSTD:
return NVCOMP_HAS_ZSTD_COMP and detail::nvcomp_integration::is_all_enabled();
return NVCOMP_HAS_ZSTD_COMP and detail::nvcomp_integration::is_stable_enabled();
default: return false;
}
return false;
Expand Down
2 changes: 1 addition & 1 deletion docs/cudf/source/user_guide/io.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,7 @@ If no value is set, behavior will be the same as the "STABLE" option.
+=======================+========+========+==============+==============+=========+========+==============+==============+========+
| Snappy | ❌ | ❌ | Stable | Stable | ❌ | ❌ | Stable | Stable | ❌ |
+-----------------------+--------+--------+--------------+--------------+---------+--------+--------------+--------------+--------+
| ZSTD | ❌ | ❌ | Experimental | Experimental | ❌ | ❌ | Experimental | Experimental | ❌ |
| ZSTD | ❌ | ❌ | Stable | Stable | ❌ | ❌ | Stable | Stable | ❌ |
+-----------------------+--------+--------+--------------+--------------+---------+--------+--------------+--------------+--------+
| DEFLATE | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | Experimental | Experimental | ❌ |
+-----------------------+--------+--------+--------------+--------------+---------+--------+--------------+--------------+--------+
Expand Down

0 comments on commit a3e9c1c

Please sign in to comment.