From b3cac398d362a078e79e1564c468b137dcf281ca Mon Sep 17 00:00:00 2001 From: Nick Becker Date: Fri, 11 Oct 2019 10:31:40 -0400 Subject: [PATCH 01/19] fix markdown formatting --- docs/cudf/source/guide-to-udfs.ipynb | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/cudf/source/guide-to-udfs.ipynb b/docs/cudf/source/guide-to-udfs.ipynb index 29689862d08..79de877c294 100644 --- a/docs/cudf/source/guide-to-udfs.ipynb +++ b/docs/cudf/source/guide-to-udfs.ipynb @@ -350,7 +350,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# DataFrame UDFs\n", + "## DataFrame UDFs\n", "\n", "We could apply a UDF on a DataFrame like we did above with `forall`. We'd need to write a kernel that expects multiple inputs, and pass multiple Series as arguments when we execute our kernel. Because this is fairly common and can be difficult to manage, cuDF provides two APIs to streamline this: `apply_rows` and `apply_chunks`. Below, we walk through an example of using `apply_rows`. `apply_chunks` works in a similar way, but also offers more control over low-level kernel behavior.\n", "\n", @@ -811,7 +811,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# GroupBy DataFrame UDFs\n", + "## GroupBy DataFrame UDFs\n", "\n", "We can also apply UDFs to grouped DataFrames using `apply_grouped`. This example is also drawn and adapted from the RAPIDS [API documentation](https://rapidsai.github.io/projects/cudf/en/0.10.0/api.html#cudf.groupby.legacy_groupby.Groupby.apply_grouped).\n", "\n", @@ -1132,7 +1132,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Numba Kernels on CuPy Arrays\n", + "## Numba Kernels on CuPy Arrays\n", "\n", "We can also execute Numba kernels on CuPy NDArrays, again thanks to the `__cuda_array_interface__`. We can even run the same UDF on the Series and the CuPy array. First, we define a Series and then create a CuPy array from that Series." ] @@ -1236,7 +1236,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Null Handling in UDFs\n", + "## Null Handling in UDFs\n", "\n", "Above, we covered most basic usage of UDFs with cuDF.\n", "\n", @@ -1676,7 +1676,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Summary\n", + "## Summary\n", "\n", "This guide has covered a lot of content. At this point, you should hopefully feel comfortable writing UDFs (with or without null values) that operate on\n", "\n", @@ -1708,7 +1708,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.7.3" + "version": "3.6.4" } }, "nbformat": 4, From 8c7c5ee9844e97e80aa073ca3483d5c1b35bb717 Mon Sep 17 00:00:00 2001 From: Nick Becker Date: Fri, 11 Oct 2019 10:41:53 -0400 Subject: [PATCH 02/19] revert python version manually --- docs/cudf/source/guide-to-udfs.ipynb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/cudf/source/guide-to-udfs.ipynb b/docs/cudf/source/guide-to-udfs.ipynb index 79de877c294..aafc4f6f460 100644 --- a/docs/cudf/source/guide-to-udfs.ipynb +++ b/docs/cudf/source/guide-to-udfs.ipynb @@ -1708,7 +1708,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.4" + "version": "3.7.3" } }, "nbformat": 4, From 3f48a7fc8bb8228d9f74a00ed68e562327caa46a Mon Sep 17 00:00:00 2001 From: David Wendt Date: Fri, 11 Oct 2019 11:20:00 -0400 Subject: [PATCH 03/19] Add nvstrings python build instructions to contributing.md --- CONTRIBUTING.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 3b56f0e9d4f..f83957c4b82 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -189,6 +189,12 @@ $ ./build.sh libcudf # compile the cuDF libraries and install $ make test ``` +- Build the `nvstrings` python packages, in the `python/nvstrings` folder: +```bash +$ cd $CUDF_HOME/python/nvstrings +$ python setup.py install +``` + - Build the `cudf` python package, in the `python/cudf` folder: ```bash $ cd $CUDF_HOME/python/cudf @@ -214,12 +220,6 @@ $ cd $CUDF_HOME $ ./build.sh dask_cudf ``` -- You will also need the following environment variables, including `$CUDA_HOME`. -```bash -NUMBAPRO_NVVM=$CUDA_HOME/nvvm/lib64/libnvvm.so -NUMBAPRO_LIBDEVICE=$CUDA_HOME/nvvm/libdevice -``` - - To run Python tests (Optional): ```bash $ cd $CUDF_HOME/python From a04c81f56040e7cd44443f45a79e793fb29400b4 Mon Sep 17 00:00:00 2001 From: David Wendt Date: Fri, 11 Oct 2019 11:20:11 -0400 Subject: [PATCH 04/19] update changelog --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index df543aed8e6..d9004dc08e6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -161,6 +161,7 @@ - PR #3002 Fix CUDA invalid configuration errors reported after loading an ORC file without data - PR #3035 Update update-version.sh for new docs locations - PR #3038 Fix uninitialized stream parameter in device_table deleter +- PR #3059 Add nvstrings python build instructions to contributing.md # cuDF 0.9.0 (21 Aug 2019) From 2c3cfd36863c25caf024d4858cb228f019060281 Mon Sep 17 00:00:00 2001 From: Nick Becker Date: Fri, 11 Oct 2019 11:46:25 -0400 Subject: [PATCH 05/19] changelog --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index df543aed8e6..e0067d62d50 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -161,6 +161,7 @@ - PR #3002 Fix CUDA invalid configuration errors reported after loading an ORC file without data - PR #3035 Update update-version.sh for new docs locations - PR #3038 Fix uninitialized stream parameter in device_table deleter +- PR #3058 Fix UDF doc markdown formatting # cuDF 0.9.0 (21 Aug 2019) From 612100c8799912bbc3fe90d0b2d056fa387712c5 Mon Sep 17 00:00:00 2001 From: David Wendt Date: Fri, 11 Oct 2019 12:34:24 -0400 Subject: [PATCH 06/19] Add rmmInitialize to nvstrings gtests --- CHANGELOG.md | 1 + cpp/custrings/tests/test_array.cu | 5 ++++- cpp/custrings/tests/test_attrs.cu | 7 ++++++- cpp/custrings/tests/test_case.cpp | 5 ++++- cpp/custrings/tests/test_combine.cpp | 5 ++++- cpp/custrings/tests/test_convert.cu | 5 ++++- cpp/custrings/tests/test_count.cu | 7 ++++++- cpp/custrings/tests/test_datetime.cu | 5 ++++- cpp/custrings/tests/test_extract.cpp | 5 ++++- cpp/custrings/tests/test_find.cu | 5 ++++- cpp/custrings/tests/test_modify.cu | 5 ++++- cpp/custrings/tests/test_pad.cpp | 5 ++++- cpp/custrings/tests/test_replace.cpp | 5 ++++- cpp/custrings/tests/test_split.cpp | 5 ++++- cpp/custrings/tests/test_strip.cpp | 5 ++++- cpp/custrings/tests/test_text.cu | 5 ++++- cpp/custrings/tests/test_url.cpp | 5 ++++- cpp/custrings/tests/utils.h | 2 ++ 18 files changed, 71 insertions(+), 16 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index df543aed8e6..96910c042b5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -161,6 +161,7 @@ - PR #3002 Fix CUDA invalid configuration errors reported after loading an ORC file without data - PR #3035 Update update-version.sh for new docs locations - PR #3038 Fix uninitialized stream parameter in device_table deleter +- PR #3061 Add rmmInitialize to nvstrings gtests # cuDF 0.9.0 (21 Aug 2019) diff --git a/cpp/custrings/tests/test_array.cu b/cpp/custrings/tests/test_array.cu index fe78196ed90..ba33a05b3f0 100644 --- a/cpp/custrings/tests/test_array.cu +++ b/cpp/custrings/tests/test_array.cu @@ -155,6 +155,9 @@ TEST(TestArray, OrderName) int main( int argc, char** argv ) { + rmmInitialize(nullptr); testing::InitGoogleTest(&argc,argv); - return RUN_ALL_TESTS(); + int rc = RUN_ALL_TESTS(); + rmmFinalize(); + return rc; } \ No newline at end of file diff --git a/cpp/custrings/tests/test_attrs.cu b/cpp/custrings/tests/test_attrs.cu index 35b5b460701..1ec2ef84153 100644 --- a/cpp/custrings/tests/test_attrs.cu +++ b/cpp/custrings/tests/test_attrs.cu @@ -4,6 +4,8 @@ #include "nvstrings/NVStrings.h" +#include "./utils.h" + std::vector hstrs{ "Héllo", "thesé", nullptr, "ARE THE", "tést strings", "", "1.75", "-34", "+9.8", "17¼", "x³", "2³", " 12⅝", @@ -172,6 +174,9 @@ TEST(TestAttrs, CodePoints) int main( int argc, char** argv ) { + rmmInitialize(nullptr); testing::InitGoogleTest(&argc,argv); - return RUN_ALL_TESTS(); + int rc = RUN_ALL_TESTS(); + rmmFinalize(); + return rc; } \ No newline at end of file diff --git a/cpp/custrings/tests/test_case.cpp b/cpp/custrings/tests/test_case.cpp index 560024073ac..b040526ffc8 100644 --- a/cpp/custrings/tests/test_case.cpp +++ b/cpp/custrings/tests/test_case.cpp @@ -59,6 +59,9 @@ TEST(TestCase, Title) int main( int argc, char** argv ) { + rmmInitialize(nullptr); testing::InitGoogleTest(&argc,argv); - return RUN_ALL_TESTS(); + int rc = RUN_ALL_TESTS(); + rmmFinalize(); + return rc; } \ No newline at end of file diff --git a/cpp/custrings/tests/test_combine.cpp b/cpp/custrings/tests/test_combine.cpp index c698564a1f1..d439e711817 100644 --- a/cpp/custrings/tests/test_combine.cpp +++ b/cpp/custrings/tests/test_combine.cpp @@ -114,6 +114,9 @@ TEST(TestCombine, Join) int main( int argc, char** argv ) { + rmmInitialize(nullptr); testing::InitGoogleTest(&argc,argv); - return RUN_ALL_TESTS(); + int rc = RUN_ALL_TESTS(); + rmmFinalize(); + return rc; } \ No newline at end of file diff --git a/cpp/custrings/tests/test_convert.cu b/cpp/custrings/tests/test_convert.cu index c1e6dd674fd..4a0007a9734 100644 --- a/cpp/custrings/tests/test_convert.cu +++ b/cpp/custrings/tests/test_convert.cu @@ -210,6 +210,9 @@ TEST(TestConvert, FromIPv4) int main( int argc, char** argv ) { + rmmInitialize(nullptr); testing::InitGoogleTest(&argc,argv); - return RUN_ALL_TESTS(); + int rc = RUN_ALL_TESTS(); + rmmFinalize(); + return rc; } \ No newline at end of file diff --git a/cpp/custrings/tests/test_count.cu b/cpp/custrings/tests/test_count.cu index c3023bff8b5..34865d38c23 100644 --- a/cpp/custrings/tests/test_count.cu +++ b/cpp/custrings/tests/test_count.cu @@ -5,6 +5,8 @@ #include "nvstrings/NVStrings.h" +#include "./utils.h" + std::vector hstrs{ "The quick brown @fox jumps", "ovér the", "lazy @dog", "1234", "00:0:00", nullptr, "" }; @@ -102,6 +104,9 @@ TEST(TestCount, Count) int main( int argc, char** argv ) { + rmmInitialize(nullptr); testing::InitGoogleTest(&argc,argv); - return RUN_ALL_TESTS(); + int rc = RUN_ALL_TESTS(); + rmmFinalize(); + return rc; } \ No newline at end of file diff --git a/cpp/custrings/tests/test_datetime.cu b/cpp/custrings/tests/test_datetime.cu index 01fa5569642..f20dec9de39 100644 --- a/cpp/custrings/tests/test_datetime.cu +++ b/cpp/custrings/tests/test_datetime.cu @@ -58,6 +58,9 @@ TEST(TestTimestamp, FromTimestamp) int main( int argc, char** argv ) { + rmmInitialize(nullptr); testing::InitGoogleTest(&argc,argv); - return RUN_ALL_TESTS(); + int rc = RUN_ALL_TESTS(); + rmmFinalize(); + return rc; } \ No newline at end of file diff --git a/cpp/custrings/tests/test_extract.cpp b/cpp/custrings/tests/test_extract.cpp index 95233c5cea9..81063bd49be 100644 --- a/cpp/custrings/tests/test_extract.cpp +++ b/cpp/custrings/tests/test_extract.cpp @@ -55,6 +55,9 @@ TEST(TestExtract, ExtractRecord) int main( int argc, char** argv ) { + rmmInitialize(nullptr); testing::InitGoogleTest(&argc,argv); - return RUN_ALL_TESTS(); + int rc = RUN_ALL_TESTS(); + rmmFinalize(); + return rc; } \ No newline at end of file diff --git a/cpp/custrings/tests/test_find.cu b/cpp/custrings/tests/test_find.cu index d7eac0c9a94..e6f0dbaec5b 100644 --- a/cpp/custrings/tests/test_find.cu +++ b/cpp/custrings/tests/test_find.cu @@ -174,6 +174,9 @@ TEST(TestFind, FindAllRecord) int main( int argc, char** argv ) { + rmmInitialize(nullptr); testing::InitGoogleTest(&argc,argv); - return RUN_ALL_TESTS(); + int rc = RUN_ALL_TESTS(); + rmmFinalize(); + return rc; } \ No newline at end of file diff --git a/cpp/custrings/tests/test_modify.cu b/cpp/custrings/tests/test_modify.cu index 1b0c1573d56..1788296c0bc 100644 --- a/cpp/custrings/tests/test_modify.cu +++ b/cpp/custrings/tests/test_modify.cu @@ -151,6 +151,9 @@ TEST(TestModify, Translate) int main( int argc, char** argv ) { + rmmInitialize(nullptr); testing::InitGoogleTest(&argc,argv); - return RUN_ALL_TESTS(); + int rc = RUN_ALL_TESTS(); + rmmFinalize(); + return rc; } \ No newline at end of file diff --git a/cpp/custrings/tests/test_pad.cpp b/cpp/custrings/tests/test_pad.cpp index ca669105570..2fb52f49364 100644 --- a/cpp/custrings/tests/test_pad.cpp +++ b/cpp/custrings/tests/test_pad.cpp @@ -76,6 +76,9 @@ TEST(TestPad, Wrap) int main( int argc, char** argv ) { + rmmInitialize(nullptr); testing::InitGoogleTest(&argc,argv); - return RUN_ALL_TESTS(); + int rc = RUN_ALL_TESTS(); + rmmFinalize(); + return rc; } \ No newline at end of file diff --git a/cpp/custrings/tests/test_replace.cpp b/cpp/custrings/tests/test_replace.cpp index f1e6dc07feb..a4831440159 100644 --- a/cpp/custrings/tests/test_replace.cpp +++ b/cpp/custrings/tests/test_replace.cpp @@ -151,6 +151,9 @@ TEST(TestReplace, ReplaceBackrefs) int main( int argc, char** argv ) { + rmmInitialize(nullptr); testing::InitGoogleTest(&argc,argv); - return RUN_ALL_TESTS(); + int rc = RUN_ALL_TESTS(); + rmmFinalize(); + return rc; } diff --git a/cpp/custrings/tests/test_split.cpp b/cpp/custrings/tests/test_split.cpp index 981cffc41d8..b122df86477 100644 --- a/cpp/custrings/tests/test_split.cpp +++ b/cpp/custrings/tests/test_split.cpp @@ -206,6 +206,9 @@ TEST(TestFind, Partition) int main( int argc, char** argv ) { + rmmInitialize(nullptr); testing::InitGoogleTest(&argc,argv); - return RUN_ALL_TESTS(); + int rc = RUN_ALL_TESTS(); + rmmFinalize(); + return rc; } \ No newline at end of file diff --git a/cpp/custrings/tests/test_strip.cpp b/cpp/custrings/tests/test_strip.cpp index c9a687624e4..2e9658716e2 100644 --- a/cpp/custrings/tests/test_strip.cpp +++ b/cpp/custrings/tests/test_strip.cpp @@ -36,6 +36,9 @@ TEST(TestStrip, Strip) int main( int argc, char** argv ) { + rmmInitialize(nullptr); testing::InitGoogleTest(&argc,argv); - return RUN_ALL_TESTS(); + int rc = RUN_ALL_TESTS(); + rmmFinalize(); + return rc; } \ No newline at end of file diff --git a/cpp/custrings/tests/test_text.cu b/cpp/custrings/tests/test_text.cu index 2f0c13e21f3..16e76ea7465 100644 --- a/cpp/custrings/tests/test_text.cu +++ b/cpp/custrings/tests/test_text.cu @@ -204,6 +204,9 @@ TEST(TestText, ScatterCount) int main( int argc, char** argv ) { + rmmInitialize(nullptr); testing::InitGoogleTest(&argc,argv); - return RUN_ALL_TESTS(); + int rc = RUN_ALL_TESTS(); + rmmFinalize(); + return rc; } \ No newline at end of file diff --git a/cpp/custrings/tests/test_url.cpp b/cpp/custrings/tests/test_url.cpp index 57e56577d20..37fa399b934 100644 --- a/cpp/custrings/tests/test_url.cpp +++ b/cpp/custrings/tests/test_url.cpp @@ -44,6 +44,9 @@ TEST(TestURL, UrlDecode) int main( int argc, char** argv ) { + rmmInitialize(nullptr); testing::InitGoogleTest(&argc,argv); - return RUN_ALL_TESTS(); + int rc = RUN_ALL_TESTS(); + rmmFinalize(); + return rc; } diff --git a/cpp/custrings/tests/utils.h b/cpp/custrings/tests/utils.h index 2610ba19415..cc2e4dacfc5 100644 --- a/cpp/custrings/tests/utils.h +++ b/cpp/custrings/tests/utils.h @@ -1,6 +1,8 @@ #include #include +#include + #include "nvstrings/NVStrings.h" // utility to verify strings results From e819582797e4d573cf30cb46a302e3c83054ab2d Mon Sep 17 00:00:00 2001 From: "Ram (Ramakrishna Prabhu)" Date: Fri, 11 Oct 2019 13:14:47 -0500 Subject: [PATCH 07/19] code changes to take care of situation with same indices --- python/cudf/cudf/core/groupby/groupby.py | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/python/cudf/cudf/core/groupby/groupby.py b/python/cudf/cudf/core/groupby/groupby.py index 2ee8a5cada4..30672340496 100644 --- a/python/cudf/cudf/core/groupby/groupby.py +++ b/python/cudf/cudf/core/groupby/groupby.py @@ -604,17 +604,24 @@ def _align_by_and_df(obj, by, how="inner"): 3 1.0 Name: a, dtype: float64] """ - new_obj = None - new_by = [] if not isinstance(by, (list, tuple)): by = [by] series_count = 0 + join_required = False + series = [] for by_col in by: if not is_scalar(by_col) and not isinstance(by_col, cudf.Index): sr = by_col if not isinstance(by_col, cudf.Series): sr = cudf.Series(by_col) + if not join_required and not obj.index.equals(sr.index): + join_required = True + series.append(sr) + + new_obj = None + if join_required: + for sr in series: if new_obj is None: new_obj = sr.to_frame(series_count) else: @@ -624,6 +631,7 @@ def _align_by_and_df(obj, by, how="inner"): series_count += 1 series_count = 0 + new_by = [] if new_obj is not None: new_obj = new_obj.join(obj, how=how, sort="True") columns = new_obj.columns From 55929adaf39fcba142c7bdb7c0ac3352132e51aa Mon Sep 17 00:00:00 2001 From: David Wendt Date: Fri, 11 Oct 2019 14:25:02 -0400 Subject: [PATCH 08/19] nvstrings use python weakref for freeing C++ object --- python/nvstrings/nvstrings.py | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/python/nvstrings/nvstrings.py b/python/nvstrings/nvstrings.py index 30f376cdfbf..c2905d7c384 100644 --- a/python/nvstrings/nvstrings.py +++ b/python/nvstrings/nvstrings.py @@ -1,3 +1,7 @@ +import weakref + +import rmm # noqa: F401 + import pyniNVStrings @@ -363,7 +367,7 @@ def create_from_ipc(ipc_data): def free(dstrs): """Force free resources for the specified instance.""" if dstrs is not None: - pyniNVStrings.n_destroyStrings(dstrs.m_cptr) + dstrs._finalizer() dstrs.m_cptr = 0 @@ -376,6 +380,11 @@ def bind_cpointer(cptr, own=True): return rtn +def _destroy_nvstrings_instance(own, cptr): + if own: + pyniNVStrings.n_destroyStrings(cptr) + + # this will be documented with all the public methods class nvstrings: """ @@ -395,11 +404,9 @@ def __init__(self, cptr): """ self.m_cptr = cptr self._own = True - - def __del__(self): - if self._own: - pyniNVStrings.n_destroyStrings(self.m_cptr) - self.m_cptr = 0 + self._finalizer = weakref.finalize( + self, _destroy_nvstrings_instance, self._own, self.m_cptr + ) def __str__(self): return str(pyniNVStrings.n_createHostStrings(self.m_cptr)) From 1919cafec9cddb41278a33237558440bf278a097 Mon Sep 17 00:00:00 2001 From: David Wendt Date: Fri, 11 Oct 2019 14:28:44 -0400 Subject: [PATCH 09/19] add rmm as dependency for nvstrings --- conda/recipes/nvstrings/meta.yaml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/conda/recipes/nvstrings/meta.yaml b/conda/recipes/nvstrings/meta.yaml index 57fa026ef17..e269abe141a 100644 --- a/conda/recipes/nvstrings/meta.yaml +++ b/conda/recipes/nvstrings/meta.yaml @@ -30,8 +30,10 @@ requirements: - setuptools - cmake_setuptools 0.1.3.* - libnvstrings {{ version }} + - rmm {{ version }} run: - python + - rmm {{ version }} test: commands: From ebd9120a7b35ed3371edf9c9948f6deacb24b845 Mon Sep 17 00:00:00 2001 From: David Wendt Date: Fri, 11 Oct 2019 14:33:47 -0400 Subject: [PATCH 10/19] add weakref pattern to nvcategory --- python/nvstrings/nvcategory.py | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/python/nvstrings/nvcategory.py b/python/nvstrings/nvcategory.py index fec2eaa80ab..eb30676987c 100644 --- a/python/nvstrings/nvcategory.py +++ b/python/nvstrings/nvcategory.py @@ -1,3 +1,7 @@ +import weakref + +import rmm # noqa: F401 + import nvstrings as nvs import pyniNVCategory @@ -163,6 +167,11 @@ def bind_cpointer(cptr, own=True): return rtn +def _destroy_nvcategory_instance(own, cptr): + if own: + pyniNVCategory.n_destroyCategory(cptr) + + class nvcategory: """ Instance manages a dictionary of strings (keys) in device memory @@ -177,11 +186,9 @@ def __init__(self, cptr): """For internal use only.""" self.m_cptr = cptr self._own = True - - def __del__(self): - if self._own: - pyniNVCategory.n_destroyCategory(self.m_cptr) - self.m_cptr = 0 + self._finalizer = weakref.finalize( + self, _destroy_nvcategory_instance, self._own, self.m_cptr + ) def __str__(self): return str(self.keys()) From d49bad528d7db08ba92f1b791262a1d982ea46f5 Mon Sep 17 00:00:00 2001 From: "Ram (Ramakrishna Prabhu)" Date: Fri, 11 Oct 2019 13:36:25 -0500 Subject: [PATCH 11/19] changelog --- CHANGELOG.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index df543aed8e6..3a073ba79c9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -161,6 +161,8 @@ - PR #3002 Fix CUDA invalid configuration errors reported after loading an ORC file without data - PR #3035 Update update-version.sh for new docs locations - PR #3038 Fix uninitialized stream parameter in device_table deleter +- PR #3064 Fixes groupby performance issue + # cuDF 0.9.0 (21 Aug 2019) From 131d998d3ae325f7fa2c912c1ad42dcd6addd762 Mon Sep 17 00:00:00 2001 From: David Wendt Date: Fri, 11 Oct 2019 14:37:09 -0400 Subject: [PATCH 12/19] fix rmm meta.yml reference --- conda/recipes/nvstrings/meta.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/conda/recipes/nvstrings/meta.yaml b/conda/recipes/nvstrings/meta.yaml index e269abe141a..a29acb0c958 100644 --- a/conda/recipes/nvstrings/meta.yaml +++ b/conda/recipes/nvstrings/meta.yaml @@ -30,10 +30,10 @@ requirements: - setuptools - cmake_setuptools 0.1.3.* - libnvstrings {{ version }} - - rmm {{ version }} + - rmm {{ minor_version }}.* run: - python - - rmm {{ version }} + - rmm {{ minor_version }}.* test: commands: From 4528a7850e85eb431f40985e65c7cc531fcd4130 Mon Sep 17 00:00:00 2001 From: Ashwin Srinath Date: Fri, 11 Oct 2019 18:37:20 -0400 Subject: [PATCH 13/19] Inherit from GdfTest in fixtures --- cpp/custrings/tests/test_attrs.cu | 2 ++ cpp/custrings/tests/test_case.cpp | 4 +++- cpp/custrings/tests/test_combine.cpp | 4 +++- cpp/custrings/tests/test_convert.cu | 2 ++ cpp/custrings/tests/test_count.cu | 2 ++ cpp/custrings/tests/test_datetime.cu | 2 ++ cpp/custrings/tests/test_extract.cpp | 4 +++- cpp/custrings/tests/test_find.cu | 2 ++ cpp/custrings/tests/test_modify.cu | 2 ++ cpp/custrings/tests/test_pad.cpp | 4 +++- cpp/custrings/tests/test_replace.cpp | 2 ++ cpp/custrings/tests/test_split.cpp | 4 +++- cpp/custrings/tests/test_strip.cpp | 4 +++- cpp/custrings/tests/test_text.cu | 2 ++ cpp/custrings/tests/test_url.cpp | 2 ++ cpp/custrings/tests/utils.h | 15 +++++++++++++++ cpp/tests/binaryop/unit/jit-cache-test.hpp | 7 +++++-- cpp/tests/column/legacy/column_test.cu | 6 +++--- cpp/tests/copying/nvstring_scatter_tests.cu | 3 +-- cpp/tests/copying/scatter_tests.cu | 2 +- cpp/tests/copying/scatter_to_tables_tests.cu | 2 +- cpp/tests/io/csv/csv_test.cu | 18 ++++++++++-------- cpp/tests/reductions/reduction_tests.cu | 4 +++- cpp/tests/sort/digitize_test.cu | 4 ++-- cpp/tests/utilities/base_fixture.hpp | 9 +++++++++ 25 files changed, 86 insertions(+), 26 deletions(-) diff --git a/cpp/custrings/tests/test_attrs.cu b/cpp/custrings/tests/test_attrs.cu index 1ec2ef84153..da783af7e5d 100644 --- a/cpp/custrings/tests/test_attrs.cu +++ b/cpp/custrings/tests/test_attrs.cu @@ -6,6 +6,8 @@ #include "./utils.h" +struct TestAttrs : public GdfTest{}; + std::vector hstrs{ "Héllo", "thesé", nullptr, "ARE THE", "tést strings", "", "1.75", "-34", "+9.8", "17¼", "x³", "2³", " 12⅝", diff --git a/cpp/custrings/tests/test_case.cpp b/cpp/custrings/tests/test_case.cpp index b040526ffc8..e959797a522 100644 --- a/cpp/custrings/tests/test_case.cpp +++ b/cpp/custrings/tests/test_case.cpp @@ -5,6 +5,8 @@ #include "./utils.h" +struct TestCase : public GdfTest{}; + std::vector hstrs{ "Examples aBc", "thesé", nullptr, "ARE THE", "tést strings", "" }; TEST(TestCase, ToLower) @@ -64,4 +66,4 @@ int main( int argc, char** argv ) int rc = RUN_ALL_TESTS(); rmmFinalize(); return rc; -} \ No newline at end of file +} diff --git a/cpp/custrings/tests/test_combine.cpp b/cpp/custrings/tests/test_combine.cpp index d439e711817..3e12e1cf1c5 100644 --- a/cpp/custrings/tests/test_combine.cpp +++ b/cpp/custrings/tests/test_combine.cpp @@ -5,6 +5,8 @@ #include "./utils.h" +struct TestCombine : public GdfTest{}; + std::vector hstrs1{ "thesé", nullptr, "are", "the", "tést", "strings", "" }; std::vector hstrs2{ "1234", "accénted", "", nullptr, @@ -119,4 +121,4 @@ int main( int argc, char** argv ) int rc = RUN_ALL_TESTS(); rmmFinalize(); return rc; -} \ No newline at end of file +} diff --git a/cpp/custrings/tests/test_convert.cu b/cpp/custrings/tests/test_convert.cu index 4a0007a9734..425c246a0e1 100644 --- a/cpp/custrings/tests/test_convert.cu +++ b/cpp/custrings/tests/test_convert.cu @@ -7,6 +7,8 @@ #include "./utils.h" +struct TestConvert : public GdfTest{}; + TEST(TestConvert, Hash) { std::vector hstrs{ "thesé", nullptr, "are", "the", diff --git a/cpp/custrings/tests/test_count.cu b/cpp/custrings/tests/test_count.cu index 34865d38c23..300f95bc7bc 100644 --- a/cpp/custrings/tests/test_count.cu +++ b/cpp/custrings/tests/test_count.cu @@ -7,6 +7,8 @@ #include "./utils.h" +struct TestCount : public GdfTest{}; + std::vector hstrs{ "The quick brown @fox jumps", "ovér the", "lazy @dog", "1234", "00:0:00", nullptr, "" }; diff --git a/cpp/custrings/tests/test_datetime.cu b/cpp/custrings/tests/test_datetime.cu index f20dec9de39..525a68c3ad2 100644 --- a/cpp/custrings/tests/test_datetime.cu +++ b/cpp/custrings/tests/test_datetime.cu @@ -7,6 +7,8 @@ #include "./utils.h" +struct TestTimestamp : public GdfTest{}; + TEST(TestTimestamp, ToTimestamp) { { diff --git a/cpp/custrings/tests/test_extract.cpp b/cpp/custrings/tests/test_extract.cpp index 81063bd49be..81ce24336b3 100644 --- a/cpp/custrings/tests/test_extract.cpp +++ b/cpp/custrings/tests/test_extract.cpp @@ -5,6 +5,8 @@ #include "./utils.h" +struct TestExtract : public GdfTest{}; + std::vector hstrs{"First Last", "Joe Schmoe", "John Smith", "Jane Smith", "Beyonce", "Sting", nullptr, "" }; @@ -60,4 +62,4 @@ int main( int argc, char** argv ) int rc = RUN_ALL_TESTS(); rmmFinalize(); return rc; -} \ No newline at end of file +} diff --git a/cpp/custrings/tests/test_find.cu b/cpp/custrings/tests/test_find.cu index e6f0dbaec5b..4bf2918ff27 100644 --- a/cpp/custrings/tests/test_find.cu +++ b/cpp/custrings/tests/test_find.cu @@ -6,6 +6,8 @@ #include "./utils.h" +struct TestFind : public GdfTest{}; + std::vector hstrs{ "Héllo", "thesé", nullptr, "ARE THE", "tést strings", "" }; TEST(TestFind, Compare) diff --git a/cpp/custrings/tests/test_modify.cu b/cpp/custrings/tests/test_modify.cu index 1788296c0bc..4fedc3a7d61 100644 --- a/cpp/custrings/tests/test_modify.cu +++ b/cpp/custrings/tests/test_modify.cu @@ -8,6 +8,8 @@ #include "./utils.h" +struct TestModify : public GdfTest{}; + std::vector hstrs{ "Héllo", "thesé", nullptr, "ARE THE", "tést strings", "" }; TEST(TestModify, SliceReplace) diff --git a/cpp/custrings/tests/test_pad.cpp b/cpp/custrings/tests/test_pad.cpp index 2fb52f49364..f06b68f8810 100644 --- a/cpp/custrings/tests/test_pad.cpp +++ b/cpp/custrings/tests/test_pad.cpp @@ -5,6 +5,8 @@ #include "./utils.h" +struct TestPad : public GdfTest{}; + std::vector hstrs{ "12345", "thesé", nullptr, "ARE THE", "tést strings", "" }; TEST(TestPad, Repeat) @@ -81,4 +83,4 @@ int main( int argc, char** argv ) int rc = RUN_ALL_TESTS(); rmmFinalize(); return rc; -} \ No newline at end of file +} diff --git a/cpp/custrings/tests/test_replace.cpp b/cpp/custrings/tests/test_replace.cpp index a4831440159..8dbfb959602 100644 --- a/cpp/custrings/tests/test_replace.cpp +++ b/cpp/custrings/tests/test_replace.cpp @@ -6,6 +6,8 @@ #include "./utils.h" +struct TestReplace : public GdfTest{}; + std::vector hstrs{ "the quick brown fox jumps over the lazy dog", "the fat cat lays next to the other accénted cat", "a slow moving turtlé cannot catch the bird", diff --git a/cpp/custrings/tests/test_split.cpp b/cpp/custrings/tests/test_split.cpp index b122df86477..fab21ec6401 100644 --- a/cpp/custrings/tests/test_split.cpp +++ b/cpp/custrings/tests/test_split.cpp @@ -5,6 +5,8 @@ #include "./utils.h" +struct TestSplit : public GdfTest{}; + std::vector hstrs{ "Héllo thesé", nullptr, "are some", "tést String", "" }; @@ -211,4 +213,4 @@ int main( int argc, char** argv ) int rc = RUN_ALL_TESTS(); rmmFinalize(); return rc; -} \ No newline at end of file +} diff --git a/cpp/custrings/tests/test_strip.cpp b/cpp/custrings/tests/test_strip.cpp index 2e9658716e2..d122df15377 100644 --- a/cpp/custrings/tests/test_strip.cpp +++ b/cpp/custrings/tests/test_strip.cpp @@ -5,6 +5,8 @@ #include "./utils.h" +struct TestStrip : public GdfTest{}; + std::vector hstrs{ " hello ", " thesé ", nullptr, "ARE THE", " tést strings ", "" }; TEST(TestStrip, Strip) @@ -41,4 +43,4 @@ int main( int argc, char** argv ) int rc = RUN_ALL_TESTS(); rmmFinalize(); return rc; -} \ No newline at end of file +} diff --git a/cpp/custrings/tests/test_text.cu b/cpp/custrings/tests/test_text.cu index 16e76ea7465..221d3052739 100644 --- a/cpp/custrings/tests/test_text.cu +++ b/cpp/custrings/tests/test_text.cu @@ -7,6 +7,8 @@ #include "./utils.h" +struct TestText : public GdfTest{}; + std::vector tstrs{ "the fox jumped over the dog", "the dog chased the cat", "the cat chased the mouse", diff --git a/cpp/custrings/tests/test_url.cpp b/cpp/custrings/tests/test_url.cpp index 37fa399b934..27ffdae4243 100644 --- a/cpp/custrings/tests/test_url.cpp +++ b/cpp/custrings/tests/test_url.cpp @@ -4,6 +4,8 @@ #include "./utils.h" +struct TestURL : public GdfTest{}; + TEST(TestURL, UrlEncode) { std::vector hstrs{"www.nvidia.com/rapids?p=é", "/_file-7.txt", "a b+c~d", diff --git a/cpp/custrings/tests/utils.h b/cpp/custrings/tests/utils.h index cc2e4dacfc5..01d1b430693 100644 --- a/cpp/custrings/tests/utils.h +++ b/cpp/custrings/tests/utils.h @@ -5,6 +5,21 @@ #include "nvstrings/NVStrings.h" +#define ASSERT_RMM_SUCCEEDED(expr) ASSERT_EQ(RMM_SUCCESS, expr) + +// Base class fixture for GDF google tests that initializes / finalizes the +// RAPIDS memory manager +struct GdfTest : public ::testing::Test +{ + static void SetUpTestCase() { + ASSERT_RMM_SUCCEEDED( rmmInitialize(nullptr) ); + } + + static void TearDownTestCase() { + ASSERT_RMM_SUCCEEDED( rmmFinalize() ); + } +}; + // utility to verify strings results bool verify_strings( NVStrings* d_strs, const char** h_strs ) diff --git a/cpp/tests/binaryop/unit/jit-cache-test.hpp b/cpp/tests/binaryop/unit/jit-cache-test.hpp index 9056367f710..d5dd91ca6ee 100644 --- a/cpp/tests/binaryop/unit/jit-cache-test.hpp +++ b/cpp/tests/binaryop/unit/jit-cache-test.hpp @@ -17,12 +17,15 @@ #pragma once #include +#include +#include + #include #include -#include #include -struct JitCacheTest : public ::testing::Test + +struct JitCacheTest : public cudf::test::BaseFixture , public cudf::jit::cudfJitCache { JitCacheTest() : grid(1), block(1) { } diff --git a/cpp/tests/column/legacy/column_test.cu b/cpp/tests/column/legacy/column_test.cu index 4c20fed3e25..31405a6f57a 100644 --- a/cpp/tests/column/legacy/column_test.cu +++ b/cpp/tests/column/legacy/column_test.cu @@ -1,4 +1,6 @@ #include +#include +#include #include #include @@ -6,9 +8,7 @@ #include -#include -#include #include #include @@ -26,7 +26,7 @@ struct print { }; template -struct ColumnConcatTest : public testing::Test +struct ColumnConcatTest : public cudf::test::BaseFixture { ColumnConcatTest() {} ~ColumnConcatTest() {} diff --git a/cpp/tests/copying/nvstring_scatter_tests.cu b/cpp/tests/copying/nvstring_scatter_tests.cu index 5c1fb5366e4..ffc22e56005 100644 --- a/cpp/tests/copying/nvstring_scatter_tests.cu +++ b/cpp/tests/copying/nvstring_scatter_tests.cu @@ -26,10 +26,9 @@ #include #include -template struct ScatterTest : GdfTest {}; -TEST(ScatterTest, ScatterNVString) +TEST_F(ScatterTest, ScatterNVString) { bool print = false; const int rows_size = 12; diff --git a/cpp/tests/copying/scatter_tests.cu b/cpp/tests/copying/scatter_tests.cu index ce06cf4fe9a..99458bf9590 100644 --- a/cpp/tests/copying/scatter_tests.cu +++ b/cpp/tests/copying/scatter_tests.cu @@ -279,7 +279,7 @@ TYPED_TEST(ScatterTest, EveryOtherNull) { } // The test to test against BUG #2007 -TEST(ScatterTest, PreserveDestBitmask) { +TYPED_TEST(ScatterTest, PreserveDestBitmask) { cudf::test::column_wrapper source_column( {10, -1}, [](auto index){ return false; }); diff --git a/cpp/tests/copying/scatter_to_tables_tests.cu b/cpp/tests/copying/scatter_to_tables_tests.cu index eae4f7c76a3..4fd358aa010 100644 --- a/cpp/tests/copying/scatter_to_tables_tests.cu +++ b/cpp/tests/copying/scatter_to_tables_tests.cu @@ -262,7 +262,7 @@ auto scatter_columns( output_cols_null_count); } -TEST(ScatterToTablesTest, FunctionalTest) { +TYPED_TEST(ScatterToTablesTest, FunctionalTest) { using TypeParam = gdf_index_type; constexpr gdf_size_type table_n_cols{2}; constexpr gdf_size_type table_n_rows{1000}; diff --git a/cpp/tests/io/csv/csv_test.cu b/cpp/tests/io/csv/csv_test.cu index 7083ed5b6dd..718ac7139c3 100644 --- a/cpp/tests/io/csv/csv_test.cu +++ b/cpp/tests/io/csv/csv_test.cu @@ -348,7 +348,9 @@ TEST_F(CsvTest, Timestamps) } } -TEST(gdf_csv_test, TimestampSeconds) +struct gdf_csv_test : public GdfTest{}; + +TEST_F(gdf_csv_test, TimestampSeconds) { const std::string fname = temp_env->get_temp_dir()+"CsvTimestampSeconds.csv"; @@ -376,7 +378,7 @@ TEST(gdf_csv_test, TimestampSeconds) } } -TEST(gdf_csv_test, TimestampMilliseconds) +TEST_F(gdf_csv_test, TimestampMilliseconds) { const std::string fname = temp_env->get_temp_dir()+"CsvTimestampMilliseconds.csv"; @@ -404,7 +406,7 @@ TEST(gdf_csv_test, TimestampMilliseconds) } } -TEST(gdf_csv_test, TimestampMicroseconds) +TEST_F(gdf_csv_test, TimestampMicroseconds) { const std::string fname = temp_env->get_temp_dir()+"CsvTimestampMicroseconds.csv"; @@ -432,7 +434,7 @@ TEST(gdf_csv_test, TimestampMicroseconds) } } -TEST(gdf_csv_test, TimestampNanoseconds) +TEST_F(gdf_csv_test, TimestampNanoseconds) { const std::string fname = temp_env->get_temp_dir()+"CsvTimestampNanoseconds.csv"; @@ -460,7 +462,7 @@ TEST(gdf_csv_test, TimestampNanoseconds) } } -TEST(gdf_csv_test, DatesCastToTimestampSeconds) +TEST_F(gdf_csv_test, DatesCastToTimestampSeconds) { const std::string fname = temp_env->get_temp_dir()+"CastToTimestampSeconds.csv"; @@ -492,7 +494,7 @@ TEST(gdf_csv_test, DatesCastToTimestampSeconds) } } -TEST(gdf_csv_test, DatesCastToTimestampMilliseconds) +TEST_F(gdf_csv_test, DatesCastToTimestampMilliseconds) { const std::string fname = temp_env->get_temp_dir()+"CastToTimestampMilliseconds.csv"; @@ -524,7 +526,7 @@ TEST(gdf_csv_test, DatesCastToTimestampMilliseconds) } } -TEST(gdf_csv_test, DatesCastToTimestampMicroseconds) +TEST_F(gdf_csv_test, DatesCastToTimestampMicroseconds) { const std::string fname = temp_env->get_temp_dir()+"CastToTimestampMicroseconds.csv"; @@ -556,7 +558,7 @@ TEST(gdf_csv_test, DatesCastToTimestampMicroseconds) } } -TEST(gdf_csv_test, DatesCastToTimestampNanoseconds) +TEST_F(gdf_csv_test, DatesCastToTimestampNanoseconds) { const std::string fname = temp_env->get_temp_dir()+"CastToTimestampNanoseconds.csv"; diff --git a/cpp/tests/reductions/reduction_tests.cu b/cpp/tests/reductions/reduction_tests.cu index 80b3f87adc9..7adbba970ec 100644 --- a/cpp/tests/reductions/reduction_tests.cu +++ b/cpp/tests/reductions/reduction_tests.cu @@ -514,8 +514,10 @@ TEST_F(ReductionDtypeTest, different_precision) } +struct ReductionErrorTest : public GdfTest{}; + // test case for empty input cases -TEST(ReductionErrorTest, empty_column) +TEST_F(ReductionErrorTest, empty_column) { using T = int32_t; auto statement = [](const gdf_column* col) { diff --git a/cpp/tests/sort/digitize_test.cu b/cpp/tests/sort/digitize_test.cu index 1790b242ade..be421691efa 100644 --- a/cpp/tests/sort/digitize_test.cu +++ b/cpp/tests/sort/digitize_test.cu @@ -111,12 +111,12 @@ void digitize_detail(bool right, const std::vector& expected) { } } -TEST(DigitizeTest, UpperBoundDetail) { +TYPED_TEST(DigitizeTest, UpperBoundDetail) { std::vector expected{0, 0, 1, 1, 2, 4, 5}; digitize_detail(true, expected); } -TEST(DigitizeTest, LowerBoundDetail) { +TYPED_TEST(DigitizeTest, LowerBoundDetail) { std::vector expected{0, 1, 1, 2, 2, 5, 5}; digitize_detail(false, expected); } diff --git a/cpp/tests/utilities/base_fixture.hpp b/cpp/tests/utilities/base_fixture.hpp index daeca32bceb..5459daea9b3 100644 --- a/cpp/tests/utilities/base_fixture.hpp +++ b/cpp/tests/utilities/base_fixture.hpp @@ -17,6 +17,7 @@ #pragma once #include "cudf_gtest.hpp" +#include "cudf_test_utils.cuh" #include #include @@ -41,6 +42,14 @@ class BaseFixture : public ::testing::Test { * all tests inheritng from this fixture *---------------------------------------------------------------------------**/ rmm::mr::device_memory_resource* mr() { return _mr; } + + static void SetUpTestCase() { + ASSERT_RMM_SUCCEEDED( rmmInitialize(nullptr) ); + } + + static void TearDownTestCase() { + ASSERT_RMM_SUCCEEDED( rmmFinalize() ); + } }; } // namespace test From 64178b06716e89869f5fd36912f9699f20d5140f Mon Sep 17 00:00:00 2001 From: David Wendt Date: Fri, 11 Oct 2019 21:04:17 -0400 Subject: [PATCH 14/19] remove explicit rmm-init/finalize from conftest.py --- python/cudf/cudf/tests/conftest.py | 11 +---------- 1 file changed, 1 insertion(+), 10 deletions(-) diff --git a/python/cudf/cudf/tests/conftest.py b/python/cudf/cudf/tests/conftest.py index 371018b4c06..041bd055f0a 100644 --- a/python/cudf/cudf/tests/conftest.py +++ b/python/cudf/cudf/tests/conftest.py @@ -2,16 +2,7 @@ import pytest -import rmm - - -# Set up a fixture for the RMM memory manager to initialize and finalize it -# before and after tests. -@pytest.fixture(scope="session", autouse=True) -def setup_rmm(): - rmm.initialize() - yield rmm - rmm.finalize() +import rmm # noqa: F401 @pytest.fixture(scope="session") From 29dccb59528a61b759702e09695747b96b36e085 Mon Sep 17 00:00:00 2001 From: Keith Kraus Date: Fri, 11 Oct 2019 19:50:30 -0700 Subject: [PATCH 15/19] Use GTest fixtures and remove 'int main' --- cpp/custrings/tests/test_array.cu | 29 ++++++++++---------------- cpp/custrings/tests/test_attrs.cu | 23 +++++++-------------- cpp/custrings/tests/test_case.cpp | 19 +++++------------ cpp/custrings/tests/test_combine.cpp | 15 +++----------- cpp/custrings/tests/test_convert.cu | 29 +++++++++----------------- cpp/custrings/tests/test_count.cu | 16 +++----------- cpp/custrings/tests/test_datetime.cu | 14 ++----------- cpp/custrings/tests/test_extract.cpp | 14 ++----------- cpp/custrings/tests/test_find.cu | 28 ++++++++----------------- cpp/custrings/tests/test_modify.cu | 22 ++++++-------------- cpp/custrings/tests/test_pad.cpp | 18 ++++------------ cpp/custrings/tests/test_replace.cpp | 22 ++++++-------------- cpp/custrings/tests/test_split.cpp | 16 +++----------- cpp/custrings/tests/test_strip.cpp | 12 +---------- cpp/custrings/tests/test_text.cu | 31 ++++++++++------------------ cpp/custrings/tests/test_url.cpp | 14 ++----------- 16 files changed, 85 insertions(+), 237 deletions(-) diff --git a/cpp/custrings/tests/test_array.cu b/cpp/custrings/tests/test_array.cu index ba33a05b3f0..c9f8061fa54 100644 --- a/cpp/custrings/tests/test_array.cu +++ b/cpp/custrings/tests/test_array.cu @@ -6,9 +6,11 @@ #include "./utils.h" +struct TestArray : public GdfTest{}; + std::vector hstrs{ "John Smith", "Joe Blow", "Jane Smith", nullptr, "" }; -TEST(TestArray, Sublist) +TEST_F(TestArray, Sublist) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(),hstrs.size()); @@ -21,7 +23,7 @@ TEST(TestArray, Sublist) NVStrings::destroy(strs); } -TEST(TestArray, Gather) +TEST_F(TestArray, Gather) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(),hstrs.size()); @@ -38,7 +40,7 @@ TEST(TestArray, Gather) NVStrings::destroy(strs); } -TEST(TestArray, GatherBool) +TEST_F(TestArray, GatherBool) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(),hstrs.size()); @@ -57,7 +59,7 @@ TEST(TestArray, GatherBool) NVStrings::destroy(strs); } -TEST(TestArray, Scatter) +TEST_F(TestArray, Scatter) { NVStrings* strs1 = NVStrings::create_from_array(hstrs.data(),hstrs.size()); const char* h2[] = { "", "Joe Schmoe" }; @@ -84,7 +86,7 @@ TEST(TestArray, Scatter) NVStrings::destroy(strs2); } -TEST(TestArray, RemoveStrings) +TEST_F(TestArray, RemoveStrings) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(),hstrs.size()); @@ -101,7 +103,7 @@ TEST(TestArray, RemoveStrings) NVStrings::destroy(strs); } -TEST(TestArray, SortLength) +TEST_F(TestArray, SortLength) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(),hstrs.size()); @@ -113,7 +115,7 @@ TEST(TestArray, SortLength) NVStrings::destroy(strs); } -TEST(TestArray, SortName) +TEST_F(TestArray, SortName) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(),hstrs.size()); @@ -125,7 +127,7 @@ TEST(TestArray, SortName) NVStrings::destroy(strs); } -TEST(TestArray, OrderLength) +TEST_F(TestArray, OrderLength) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(),hstrs.size()); @@ -139,7 +141,7 @@ TEST(TestArray, OrderLength) NVStrings::destroy(strs); } -TEST(TestArray, OrderName) +TEST_F(TestArray, OrderName) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(),hstrs.size()); @@ -152,12 +154,3 @@ TEST(TestArray, OrderName) NVStrings::destroy(strs); } - -int main( int argc, char** argv ) -{ - rmmInitialize(nullptr); - testing::InitGoogleTest(&argc,argv); - int rc = RUN_ALL_TESTS(); - rmmFinalize(); - return rc; -} \ No newline at end of file diff --git a/cpp/custrings/tests/test_attrs.cu b/cpp/custrings/tests/test_attrs.cu index da783af7e5d..6f6d7ff5a43 100644 --- a/cpp/custrings/tests/test_attrs.cu +++ b/cpp/custrings/tests/test_attrs.cu @@ -13,7 +13,7 @@ std::vector hstrs{ "1.75", "-34", "+9.8", "17¼", "x³", "2³", " 12⅝", "1234567890", "de", "\t\r\n\f "}; -TEST(TestAttrs, CharCounts) +TEST_F(TestAttrs, CharCounts) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(), hstrs.size()); @@ -27,7 +27,7 @@ TEST(TestAttrs, CharCounts) NVStrings::destroy(strs); } -TEST(TestAttrs, ByteCounts) +TEST_F(TestAttrs, ByteCounts) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(), hstrs.size()); @@ -41,7 +41,7 @@ TEST(TestAttrs, ByteCounts) NVStrings::destroy(strs); } -TEST(TestAttrs, IsAlpha) +TEST_F(TestAttrs, IsAlpha) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(), hstrs.size()); @@ -76,7 +76,7 @@ TEST(TestAttrs, IsAlpha) NVStrings::destroy(strs); } -TEST(TestAttrs, IsNumeric) +TEST_F(TestAttrs, IsNumeric) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(), hstrs.size()); @@ -111,7 +111,7 @@ TEST(TestAttrs, IsNumeric) NVStrings::destroy(strs); } -TEST(TestAttrs, IsSpace) +TEST_F(TestAttrs, IsSpace) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(), hstrs.size()); @@ -138,7 +138,7 @@ TEST(TestAttrs, IsSpace) NVStrings::destroy(strs); } -TEST(TestAttrs, IsUpperLower) +TEST_F(TestAttrs, IsUpperLower) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(), hstrs.size()); @@ -164,7 +164,7 @@ TEST(TestAttrs, IsUpperLower) NVStrings::destroy(strs); } -TEST(TestAttrs, CodePoints) +TEST_F(TestAttrs, CodePoints) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(), 2); thrust::device_vector results(10,0); @@ -173,12 +173,3 @@ TEST(TestAttrs, CodePoints) for( int idx = 0; idx < 10; ++idx ) EXPECT_EQ(results[idx],expected[idx]); } - -int main( int argc, char** argv ) -{ - rmmInitialize(nullptr); - testing::InitGoogleTest(&argc,argv); - int rc = RUN_ALL_TESTS(); - rmmFinalize(); - return rc; -} \ No newline at end of file diff --git a/cpp/custrings/tests/test_case.cpp b/cpp/custrings/tests/test_case.cpp index e959797a522..f785d9673e8 100644 --- a/cpp/custrings/tests/test_case.cpp +++ b/cpp/custrings/tests/test_case.cpp @@ -9,7 +9,7 @@ struct TestCase : public GdfTest{}; std::vector hstrs{ "Examples aBc", "thesé", nullptr, "ARE THE", "tést strings", "" }; -TEST(TestCase, ToLower) +TEST_F(TestCase, ToLower) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(),hstrs.size()); NVStrings* got = strs->lower(); @@ -19,7 +19,7 @@ TEST(TestCase, ToLower) NVStrings::destroy(strs); } -TEST(TestCase, ToUpper) +TEST_F(TestCase, ToUpper) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(),hstrs.size()); NVStrings* got = strs->upper(); @@ -29,7 +29,7 @@ TEST(TestCase, ToUpper) NVStrings::destroy(strs); } -TEST(TestCase, Swapcase) +TEST_F(TestCase, Swapcase) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(),hstrs.size()); NVStrings* got = strs->swapcase(); @@ -39,7 +39,7 @@ TEST(TestCase, Swapcase) NVStrings::destroy(strs); } -TEST(TestCase, Capitalize) +TEST_F(TestCase, Capitalize) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(),hstrs.size()); NVStrings* got = strs->capitalize(); @@ -49,7 +49,7 @@ TEST(TestCase, Capitalize) NVStrings::destroy(strs); } -TEST(TestCase, Title) +TEST_F(TestCase, Title) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(),hstrs.size()); NVStrings* got = strs->title(); @@ -58,12 +58,3 @@ TEST(TestCase, Title) NVStrings::destroy(got); NVStrings::destroy(strs); } - -int main( int argc, char** argv ) -{ - rmmInitialize(nullptr); - testing::InitGoogleTest(&argc,argv); - int rc = RUN_ALL_TESTS(); - rmmFinalize(); - return rc; -} diff --git a/cpp/custrings/tests/test_combine.cpp b/cpp/custrings/tests/test_combine.cpp index 3e12e1cf1c5..8d357718394 100644 --- a/cpp/custrings/tests/test_combine.cpp +++ b/cpp/custrings/tests/test_combine.cpp @@ -12,7 +12,7 @@ std::vector hstrs1{ "thesé", nullptr, "are", "the", std::vector hstrs2{ "1234", "accénted", "", nullptr, "5678", "othér", "9" }; -TEST(TestCombine, Concatenate) +TEST_F(TestCombine, Concatenate) { NVStrings* strs1 = NVStrings::create_from_array(hstrs1.data(),hstrs1.size()); NVStrings* strs2 = NVStrings::create_from_array(hstrs2.data(),hstrs2.size()); @@ -48,7 +48,7 @@ TEST(TestCombine, Concatenate) std::vector hstrs3{ "abcdéf", "", nullptr, "ghijkl", "mnop", "éach", "xyz" }; -TEST(TestCombine, ConcatenateMultiple) +TEST_F(TestCombine, ConcatenateMultiple) { NVStrings* strs1 = NVStrings::create_from_array(hstrs1.data(),hstrs1.size()); NVStrings* strs2 = NVStrings::create_from_array(hstrs2.data(),hstrs2.size()); @@ -86,7 +86,7 @@ TEST(TestCombine, ConcatenateMultiple) NVStrings::destroy(strs3); } -TEST(TestCombine, Join) +TEST_F(TestCombine, Join) { NVStrings* strs = NVStrings::create_from_array(hstrs1.data(),hstrs1.size()); @@ -113,12 +113,3 @@ TEST(TestCombine, Join) NVStrings::destroy(strs); } - -int main( int argc, char** argv ) -{ - rmmInitialize(nullptr); - testing::InitGoogleTest(&argc,argv); - int rc = RUN_ALL_TESTS(); - rmmFinalize(); - return rc; -} diff --git a/cpp/custrings/tests/test_convert.cu b/cpp/custrings/tests/test_convert.cu index 425c246a0e1..3045756edab 100644 --- a/cpp/custrings/tests/test_convert.cu +++ b/cpp/custrings/tests/test_convert.cu @@ -9,7 +9,7 @@ struct TestConvert : public GdfTest{}; -TEST(TestConvert, Hash) +TEST_F(TestConvert, Hash) { std::vector hstrs{ "thesé", nullptr, "are", "the", "tést", "strings", "" }; @@ -25,7 +25,7 @@ TEST(TestConvert, Hash) NVStrings::destroy(strs); } -TEST(TestConvert, ToInteger) +TEST_F(TestConvert, ToInteger) { std::vector hstrs{"1234", nullptr, "-876", "543.2", "-0.12", ".55", "-.002", @@ -55,7 +55,7 @@ TEST(TestConvert, ToInteger) NVStrings::destroy(strs); } -TEST(TestConvert, FromInteger) +TEST_F(TestConvert, FromInteger) { { int values[] = {100, 987654321, -12761, 0, 5, -4}; @@ -77,7 +77,7 @@ TEST(TestConvert, FromInteger) } } -TEST(TestConvert, Hex) +TEST_F(TestConvert, Hex) { std::vector hstrs{"1234", nullptr, "98BEEF", "1a5", "CAFE", "2face"}; @@ -91,7 +91,7 @@ TEST(TestConvert, Hex) NVStrings::destroy(strs); } -TEST(TestConvert, ToFloat) +TEST_F(TestConvert, ToFloat) { std::vector hstrs{"1234", nullptr, "-876", "543.2", "-0.12", ".25", "-.002", @@ -142,7 +142,7 @@ TEST(TestConvert, ToFloat) NVStrings::destroy(strs); } -TEST(TestConvert, FromFloat) +TEST_F(TestConvert, FromFloat) { { float values[] = {100, 654321.25, -12761.125, 0, 5, -4, std::numeric_limits::quiet_NaN()}; @@ -164,7 +164,7 @@ TEST(TestConvert, FromFloat) } } -TEST(TestConvert, ToBool) +TEST_F(TestConvert, ToBool) { std::vector hstrs{"false", nullptr, "", "true", "True", "False"}; NVStrings* strs = NVStrings::create_from_array(hstrs.data(),hstrs.size()); @@ -176,7 +176,7 @@ TEST(TestConvert, ToBool) NVStrings::destroy(strs); } -TEST(TestConvert, FromBool) +TEST_F(TestConvert, FromBool) { bool values[] = { true, false, false, true, true, true }; thrust::device_vector results(6); @@ -187,7 +187,7 @@ TEST(TestConvert, FromBool) NVStrings::destroy(got); } -TEST(TestConvert, ToIPv4) +TEST_F(TestConvert, ToIPv4) { std::vector hstrs{ nullptr, "", "hello", "41.168.0.1", "127.0.0.1", "41.197.0.1" }; NVStrings* strs = NVStrings::create_from_array(hstrs.data(),hstrs.size()); @@ -199,7 +199,7 @@ TEST(TestConvert, ToIPv4) NVStrings::destroy(strs); } -TEST(TestConvert, FromIPv4) +TEST_F(TestConvert, FromIPv4) { unsigned values[] = { 3232235521, 167772161, 0, 0, 700055553, 700776449 }; thrust::device_vector results(6); @@ -209,12 +209,3 @@ TEST(TestConvert, FromIPv4) EXPECT_TRUE( verify_strings(got,expected)); NVStrings::destroy(got); } - -int main( int argc, char** argv ) -{ - rmmInitialize(nullptr); - testing::InitGoogleTest(&argc,argv); - int rc = RUN_ALL_TESTS(); - rmmFinalize(); - return rc; -} \ No newline at end of file diff --git a/cpp/custrings/tests/test_count.cu b/cpp/custrings/tests/test_count.cu index 300f95bc7bc..90b1986b0c5 100644 --- a/cpp/custrings/tests/test_count.cu +++ b/cpp/custrings/tests/test_count.cu @@ -13,7 +13,7 @@ std::vector hstrs{ "The quick brown @fox jumps", "ovér the", "lazy @dog", "1234", "00:0:00", nullptr, "" }; -TEST(TestCount, Contains) +TEST_F(TestCount, Contains) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(), hstrs.size()); @@ -43,7 +43,7 @@ TEST(TestCount, Contains) NVStrings::destroy(strs); } -TEST(TestCount, Match) +TEST_F(TestCount, Match) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(), hstrs.size()); @@ -73,7 +73,7 @@ TEST(TestCount, Match) NVStrings::destroy(strs); } -TEST(TestCount, Count) +TEST_F(TestCount, Count) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(), hstrs.size()); @@ -102,13 +102,3 @@ TEST(TestCount, Count) NVStrings::destroy(strs); } - - -int main( int argc, char** argv ) -{ - rmmInitialize(nullptr); - testing::InitGoogleTest(&argc,argv); - int rc = RUN_ALL_TESTS(); - rmmFinalize(); - return rc; -} \ No newline at end of file diff --git a/cpp/custrings/tests/test_datetime.cu b/cpp/custrings/tests/test_datetime.cu index 525a68c3ad2..55408e4ab85 100644 --- a/cpp/custrings/tests/test_datetime.cu +++ b/cpp/custrings/tests/test_datetime.cu @@ -9,7 +9,7 @@ struct TestTimestamp : public GdfTest{}; -TEST(TestTimestamp, ToTimestamp) +TEST_F(TestTimestamp, ToTimestamp) { { std::vector hstrs{"1974-02-28T01:23:45Z", "2019-07-17T21:34:37Z", @@ -35,7 +35,7 @@ TEST(TestTimestamp, ToTimestamp) } } -TEST(TestTimestamp, FromTimestamp) +TEST_F(TestTimestamp, FromTimestamp) { { unsigned long values[] = {1563399273}; @@ -56,13 +56,3 @@ TEST(TestTimestamp, FromTimestamp) NVStrings::destroy(got); } } - - -int main( int argc, char** argv ) -{ - rmmInitialize(nullptr); - testing::InitGoogleTest(&argc,argv); - int rc = RUN_ALL_TESTS(); - rmmFinalize(); - return rc; -} \ No newline at end of file diff --git a/cpp/custrings/tests/test_extract.cpp b/cpp/custrings/tests/test_extract.cpp index 81ce24336b3..f4154f50e81 100644 --- a/cpp/custrings/tests/test_extract.cpp +++ b/cpp/custrings/tests/test_extract.cpp @@ -11,7 +11,7 @@ struct TestExtract : public GdfTest{}; "Beyonce", "Sting", nullptr, "" }; -TEST(TestExtract, ExtractColumn) +TEST_F(TestExtract, ExtractColumn) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(),hstrs.size()); std::vector results; @@ -27,7 +27,7 @@ TEST(TestExtract, ExtractColumn) NVStrings::destroy(strs); } -TEST(TestExtract, ExtractRecord) +TEST_F(TestExtract, ExtractRecord) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(),hstrs.size()); std::vector results; @@ -53,13 +53,3 @@ TEST(TestExtract, ExtractRecord) } NVStrings::destroy(strs); } - - -int main( int argc, char** argv ) -{ - rmmInitialize(nullptr); - testing::InitGoogleTest(&argc,argv); - int rc = RUN_ALL_TESTS(); - rmmFinalize(); - return rc; -} diff --git a/cpp/custrings/tests/test_find.cu b/cpp/custrings/tests/test_find.cu index 4bf2918ff27..e93a655e74e 100644 --- a/cpp/custrings/tests/test_find.cu +++ b/cpp/custrings/tests/test_find.cu @@ -10,7 +10,7 @@ struct TestFind : public GdfTest{}; std::vector hstrs{ "Héllo", "thesé", nullptr, "ARE THE", "tést strings", "" }; -TEST(TestFind, Compare) +TEST_F(TestFind, Compare) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(), hstrs.size()); @@ -25,7 +25,7 @@ TEST(TestFind, Compare) } -TEST(TestFind, Find) +TEST_F(TestFind, Find) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(), hstrs.size()); @@ -46,7 +46,7 @@ TEST(TestFind, Find) NVStrings::destroy(strs); } -TEST(TestFind, Match) +TEST_F(TestFind, Match) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(), hstrs.size()); std::vector hstrs2{ "héllo", "thesé", "", nullptr, "tést strings", "" }; @@ -63,7 +63,7 @@ TEST(TestFind, Match) NVStrings::destroy(strs2); } -TEST(TestFind, Contains) +TEST_F(TestFind, Contains) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(), hstrs.size()); @@ -77,7 +77,7 @@ TEST(TestFind, Contains) NVStrings::destroy(strs); } -TEST(TestFind, FindFrom) +TEST_F(TestFind, FindFrom) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(), hstrs.size()); thrust::device_vector from(hstrs.size(),3); @@ -92,7 +92,7 @@ TEST(TestFind, FindFrom) NVStrings::destroy(strs); } -TEST(TestFind, FindMultiple) +TEST_F(TestFind, FindMultiple) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(), hstrs.size()); @@ -111,7 +111,7 @@ TEST(TestFind, FindMultiple) NVStrings::destroy(strs2); } -TEST(TestFind, StartsEnds) +TEST_F(TestFind, StartsEnds) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(), hstrs.size()); @@ -132,7 +132,7 @@ TEST(TestFind, StartsEnds) NVStrings::destroy(strs); } -TEST(TestFind, FindAll) +TEST_F(TestFind, FindAll) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(),hstrs.size()); std::vector results; @@ -148,7 +148,7 @@ TEST(TestFind, FindAll) NVStrings::destroy(strs); } -TEST(TestFind, FindAllRecord) +TEST_F(TestFind, FindAllRecord) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(),hstrs.size()); std::vector results; @@ -172,13 +172,3 @@ TEST(TestFind, FindAllRecord) } NVStrings::destroy(strs); } - - -int main( int argc, char** argv ) -{ - rmmInitialize(nullptr); - testing::InitGoogleTest(&argc,argv); - int rc = RUN_ALL_TESTS(); - rmmFinalize(); - return rc; -} \ No newline at end of file diff --git a/cpp/custrings/tests/test_modify.cu b/cpp/custrings/tests/test_modify.cu index 4fedc3a7d61..4ffbbac52ac 100644 --- a/cpp/custrings/tests/test_modify.cu +++ b/cpp/custrings/tests/test_modify.cu @@ -12,7 +12,7 @@ struct TestModify : public GdfTest{}; std::vector hstrs{ "Héllo", "thesé", nullptr, "ARE THE", "tést strings", "" }; -TEST(TestModify, SliceReplace) +TEST_F(TestModify, SliceReplace) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(), hstrs.size()); @@ -38,7 +38,7 @@ TEST(TestModify, SliceReplace) NVStrings::destroy(strs); } -TEST(TestModify, Slice) +TEST_F(TestModify, Slice) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(), hstrs.size()); @@ -70,7 +70,7 @@ TEST(TestModify, Slice) NVStrings::destroy(strs); } -TEST(TestModify, SliceFrom) +TEST_F(TestModify, SliceFrom) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(), hstrs.size()); { @@ -91,7 +91,7 @@ TEST(TestModify, SliceFrom) NVStrings::destroy(strs); } -TEST(TestModify, FillNa) +TEST_F(TestModify, FillNa) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(), hstrs.size()); { @@ -112,7 +112,7 @@ TEST(TestModify, FillNa) NVStrings::destroy(strs); } -TEST(TestModify, Insert) +TEST_F(TestModify, Insert) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(), hstrs.size()); @@ -132,7 +132,7 @@ TEST(TestModify, Insert) NVStrings::destroy(strs); } -TEST(TestModify, Translate) +TEST_F(TestModify, Translate) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(), hstrs.size()); @@ -149,13 +149,3 @@ TEST(TestModify, Translate) NVStrings::destroy(strs); } - - -int main( int argc, char** argv ) -{ - rmmInitialize(nullptr); - testing::InitGoogleTest(&argc,argv); - int rc = RUN_ALL_TESTS(); - rmmFinalize(); - return rc; -} \ No newline at end of file diff --git a/cpp/custrings/tests/test_pad.cpp b/cpp/custrings/tests/test_pad.cpp index f06b68f8810..e45e9500056 100644 --- a/cpp/custrings/tests/test_pad.cpp +++ b/cpp/custrings/tests/test_pad.cpp @@ -9,7 +9,7 @@ struct TestPad : public GdfTest{}; std::vector hstrs{ "12345", "thesé", nullptr, "ARE THE", "tést strings", "" }; -TEST(TestPad, Repeat) +TEST_F(TestPad, Repeat) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(), hstrs.size()); @@ -29,7 +29,7 @@ TEST(TestPad, Repeat) NVStrings::destroy(strs); } -TEST(TestPad, Pad) +TEST_F(TestPad, Pad) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(), hstrs.size()); @@ -55,7 +55,7 @@ TEST(TestPad, Pad) NVStrings::destroy(strs); } -TEST(TestPad, ZFill) +TEST_F(TestPad, ZFill) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(), hstrs.size()); NVStrings* got = strs->zfill(8); @@ -65,7 +65,7 @@ TEST(TestPad, ZFill) NVStrings::destroy(strs); } -TEST(TestPad, Wrap) +TEST_F(TestPad, Wrap) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(), hstrs.size()); NVStrings* got = strs->wrap(3); @@ -74,13 +74,3 @@ TEST(TestPad, Wrap) NVStrings::destroy(got); NVStrings::destroy(strs); } - - -int main( int argc, char** argv ) -{ - rmmInitialize(nullptr); - testing::InitGoogleTest(&argc,argv); - int rc = RUN_ALL_TESTS(); - rmmFinalize(); - return rc; -} diff --git a/cpp/custrings/tests/test_replace.cpp b/cpp/custrings/tests/test_replace.cpp index 8dbfb959602..96b95c03c4f 100644 --- a/cpp/custrings/tests/test_replace.cpp +++ b/cpp/custrings/tests/test_replace.cpp @@ -15,7 +15,7 @@ std::vector hstrs{ "the quick brown fox jumps over the lazy dog", "thé result does not include the value in the sum in", "", "absent stop words" }; -TEST(TestReplace, Replace) +TEST_F(TestReplace, Replace) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(),hstrs.size()); @@ -35,7 +35,7 @@ TEST(TestReplace, Replace) NVStrings::destroy(strs); } -TEST(TestReplace, ReplaceRegex) +TEST_F(TestReplace, ReplaceRegex) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(),hstrs.size()); @@ -55,7 +55,7 @@ TEST(TestReplace, ReplaceRegex) NVStrings::destroy(strs); } -TEST(TestReplace, ReplaceMulti) +TEST_F(TestReplace, ReplaceMulti) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(),hstrs.size()); @@ -81,7 +81,7 @@ TEST(TestReplace, ReplaceMulti) NVStrings::destroy(strs); } -TEST(TestReplace, ReplaceMultiRegex) +TEST_F(TestReplace, ReplaceMultiRegex) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(),hstrs.size()); @@ -105,7 +105,7 @@ TEST(TestReplace, ReplaceMultiRegex) NVStrings::destroy(strs); } -TEST(TestReplace, ReplaceTokens) +TEST_F(TestReplace, ReplaceTokens) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(),hstrs.size()); @@ -131,7 +131,7 @@ TEST(TestReplace, ReplaceTokens) NVStrings::destroy(strs); } -TEST(TestReplace, ReplaceBackrefs) +TEST_F(TestReplace, ReplaceBackrefs) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(),hstrs.size()); @@ -149,13 +149,3 @@ TEST(TestReplace, ReplaceBackrefs) NVStrings::destroy(got); NVStrings::destroy(strs); } - - -int main( int argc, char** argv ) -{ - rmmInitialize(nullptr); - testing::InitGoogleTest(&argc,argv); - int rc = RUN_ALL_TESTS(); - rmmFinalize(); - return rc; -} diff --git a/cpp/custrings/tests/test_split.cpp b/cpp/custrings/tests/test_split.cpp index fab21ec6401..01004806104 100644 --- a/cpp/custrings/tests/test_split.cpp +++ b/cpp/custrings/tests/test_split.cpp @@ -10,7 +10,7 @@ struct TestSplit : public GdfTest{}; std::vector hstrs{ "Héllo thesé", nullptr, "are some", "tést String", "" }; -TEST(TestSplit, Split) +TEST_F(TestSplit, Split) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(),hstrs.size()); { @@ -61,7 +61,7 @@ TEST(TestSplit, Split) NVStrings::destroy(strs); } -TEST(TestFind, SplitRecord) +TEST_F(TestSplit, SplitRecord) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(),hstrs.size()); { @@ -155,7 +155,7 @@ TEST(TestFind, SplitRecord) NVStrings::destroy(strs); } -TEST(TestFind, Partition) +TEST_F(TestSplit, Partition) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(),hstrs.size()); { @@ -204,13 +204,3 @@ TEST(TestFind, Partition) } NVStrings::destroy(strs); } - - -int main( int argc, char** argv ) -{ - rmmInitialize(nullptr); - testing::InitGoogleTest(&argc,argv); - int rc = RUN_ALL_TESTS(); - rmmFinalize(); - return rc; -} diff --git a/cpp/custrings/tests/test_strip.cpp b/cpp/custrings/tests/test_strip.cpp index d122df15377..c8f1d8e8fb0 100644 --- a/cpp/custrings/tests/test_strip.cpp +++ b/cpp/custrings/tests/test_strip.cpp @@ -9,7 +9,7 @@ struct TestStrip : public GdfTest{}; std::vector hstrs{ " hello ", " thesé ", nullptr, "ARE THE", " tést strings ", "" }; -TEST(TestStrip, Strip) +TEST_F(TestStrip, Strip) { NVStrings* strs = NVStrings::create_from_array(hstrs.data(), hstrs.size()); @@ -34,13 +34,3 @@ TEST(TestStrip, Strip) NVStrings::destroy(strs); } - - -int main( int argc, char** argv ) -{ - rmmInitialize(nullptr); - testing::InitGoogleTest(&argc,argv); - int rc = RUN_ALL_TESTS(); - rmmFinalize(); - return rc; -} diff --git a/cpp/custrings/tests/test_text.cu b/cpp/custrings/tests/test_text.cu index 221d3052739..28dc9b7cfab 100644 --- a/cpp/custrings/tests/test_text.cu +++ b/cpp/custrings/tests/test_text.cu @@ -15,7 +15,7 @@ std::vector tstrs{ "the fox jumped over the dog", nullptr, "", "the mouse ate the cheese" }; -TEST(TestText, Tokenize) +TEST_F(TestText, Tokenize) { NVStrings* strs = NVStrings::create_from_array(tstrs.data(),tstrs.size()); NVStrings* got = NVText::tokenize(*strs); @@ -28,7 +28,7 @@ TEST(TestText, Tokenize) NVStrings::destroy(strs); } -TEST(TestText, TokenCount) +TEST_F(TestText, TokenCount) { NVStrings* strs = NVStrings::create_from_array(tstrs.data(),tstrs.size()); thrust::device_vector results(tstrs.size(),0); @@ -41,7 +41,7 @@ TEST(TestText, TokenCount) NVStrings::destroy(strs); } -TEST(TestText, UniqueTokens) +TEST_F(TestText, UniqueTokens) { NVStrings* strs = NVStrings::create_from_array(tstrs.data(),tstrs.size()); NVStrings* got = NVText::unique_tokens(*strs); @@ -52,7 +52,7 @@ TEST(TestText, UniqueTokens) NVStrings::destroy(strs); } -TEST(TestText, ContainsStrings) +TEST_F(TestText, ContainsStrings) { NVStrings* strs = NVStrings::create_from_array(tstrs.data(),tstrs.size()); std::vector hstrs{ "the", "cat" }; @@ -69,7 +69,7 @@ TEST(TestText, ContainsStrings) NVStrings::destroy(strs); } -TEST(TestText, StringsCounts) +TEST_F(TestText, StringsCounts) { NVStrings* strs = NVStrings::create_from_array(tstrs.data(),tstrs.size()); std::vector hstrs{ "cat ", "dog " }; @@ -86,7 +86,7 @@ TEST(TestText, StringsCounts) NVStrings::destroy(strs); } -TEST(TestText, TokensCounts) +TEST_F(TestText, TokensCounts) { NVStrings* strs = NVStrings::create_from_array(tstrs.data(),tstrs.size()); std::vector hstrs{ "cat", "dog" }; @@ -103,7 +103,7 @@ TEST(TestText, TokensCounts) NVStrings::destroy(strs); } -TEST(TestText, EditDistance) +TEST_F(TestText, EditDistance) { std::vector hstrs{ "dog", nullptr, "cat", "mouse", "pup", "", "puppy" }; @@ -129,7 +129,7 @@ TEST(TestText, EditDistance) NVStrings::destroy(strs); } -TEST(TestText, EditDistanceMatrix) +TEST_F(TestText, EditDistanceMatrix) { std::vector hstrs{ "dog", nullptr, "cat", "mouse", "pup", "", "puppy" }; @@ -150,7 +150,7 @@ TEST(TestText, EditDistanceMatrix) NVStrings::destroy(strs); } -TEST(TestText, NGrams) +TEST_F(TestText, NGrams) { NVStrings* strs = NVStrings::create_from_array(tstrs.data(),tstrs.size()); NVStrings* tkns = NVText::tokenize(*strs); @@ -171,7 +171,7 @@ TEST(TestText, NGrams) } -TEST(TestText, PorterStemmerMeasure) +TEST_F(TestText, PorterStemmerMeasure) { std::vector hstrs{ "abandon", nullptr, "abbey", "cleans", "trouble", "", "yearly" }; @@ -187,7 +187,7 @@ TEST(TestText, PorterStemmerMeasure) } -TEST(TestText, ScatterCount) +TEST_F(TestText, ScatterCount) { std::vector names{ "Larry", "Curly", "Moe" }; NVStrings* strs = NVStrings::create_from_array(names.data(),names.size()); @@ -203,12 +203,3 @@ TEST(TestText, ScatterCount) NVStrings::destroy(got); NVStrings::destroy(strs); } - -int main( int argc, char** argv ) -{ - rmmInitialize(nullptr); - testing::InitGoogleTest(&argc,argv); - int rc = RUN_ALL_TESTS(); - rmmFinalize(); - return rc; -} \ No newline at end of file diff --git a/cpp/custrings/tests/test_url.cpp b/cpp/custrings/tests/test_url.cpp index 27ffdae4243..324fc8248cb 100644 --- a/cpp/custrings/tests/test_url.cpp +++ b/cpp/custrings/tests/test_url.cpp @@ -6,7 +6,7 @@ struct TestURL : public GdfTest{}; -TEST(TestURL, UrlEncode) +TEST_F(TestURL, UrlEncode) { std::vector hstrs{"www.nvidia.com/rapids?p=é", "/_file-7.txt", "a b+c~d", "e\tfgh\\jklmnopqrstuvwxyz", "ABCDEFGHIJKLMNOPQRSTUVWXYZ", @@ -25,7 +25,7 @@ TEST(TestURL, UrlEncode) NVStrings::destroy(strs); } -TEST(TestURL, UrlDecode) +TEST_F(TestURL, UrlDecode) { std::vector hstrs{ "www.nvidia.com/rapids/%3Fp%3D%C3%A9", "/_file-1234567890.txt", "a%20b%2Bc~defghijklmnopqrstuvwxyz", "%25-accent%c3%a9d", "ABCDEFGHIJKLMNOPQRSTUVWXYZ", "01234567890", @@ -42,13 +42,3 @@ TEST(TestURL, UrlDecode) NVStrings::destroy(got); NVStrings::destroy(strs); } - - -int main( int argc, char** argv ) -{ - rmmInitialize(nullptr); - testing::InitGoogleTest(&argc,argv); - int rc = RUN_ALL_TESTS(); - rmmFinalize(); - return rc; -} From 698caba98666eb29432586f38062bee839e4c975 Mon Sep 17 00:00:00 2001 From: Keith Kraus Date: Mon, 14 Oct 2019 15:44:22 -0700 Subject: [PATCH 16/19] add rmm.is_initialized check to nvstrings/nvcategory destructors --- python/nvstrings/nvcategory.py | 2 +- python/nvstrings/nvstrings.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/python/nvstrings/nvcategory.py b/python/nvstrings/nvcategory.py index eb30676987c..ba58d9fa488 100644 --- a/python/nvstrings/nvcategory.py +++ b/python/nvstrings/nvcategory.py @@ -168,7 +168,7 @@ def bind_cpointer(cptr, own=True): def _destroy_nvcategory_instance(own, cptr): - if own: + if own and rmm.is_initialized(): pyniNVCategory.n_destroyCategory(cptr) diff --git a/python/nvstrings/nvstrings.py b/python/nvstrings/nvstrings.py index c2905d7c384..e94bb90ace0 100644 --- a/python/nvstrings/nvstrings.py +++ b/python/nvstrings/nvstrings.py @@ -381,7 +381,7 @@ def bind_cpointer(cptr, own=True): def _destroy_nvstrings_instance(own, cptr): - if own: + if own and rmm.is_initialized(): pyniNVStrings.n_destroyStrings(cptr) From 720becf1e22419f3589277d749ca7163ef95c2d1 Mon Sep 17 00:00:00 2001 From: Keith Kraus Date: Mon, 14 Oct 2019 18:08:19 -0700 Subject: [PATCH 17/19] Use RMM weakref instead of atexit for proper finalization order --- python/nvstrings/nvcategory.py | 2 +- python/nvstrings/nvstrings.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/python/nvstrings/nvcategory.py b/python/nvstrings/nvcategory.py index ba58d9fa488..eb30676987c 100644 --- a/python/nvstrings/nvcategory.py +++ b/python/nvstrings/nvcategory.py @@ -168,7 +168,7 @@ def bind_cpointer(cptr, own=True): def _destroy_nvcategory_instance(own, cptr): - if own and rmm.is_initialized(): + if own: pyniNVCategory.n_destroyCategory(cptr) diff --git a/python/nvstrings/nvstrings.py b/python/nvstrings/nvstrings.py index e94bb90ace0..c2905d7c384 100644 --- a/python/nvstrings/nvstrings.py +++ b/python/nvstrings/nvstrings.py @@ -381,7 +381,7 @@ def bind_cpointer(cptr, own=True): def _destroy_nvstrings_instance(own, cptr): - if own and rmm.is_initialized(): + if own: pyniNVStrings.n_destroyStrings(cptr) From 547831cb9f1a39d7ff7c70f7c3ef3ee8d7fe6826 Mon Sep 17 00:00:00 2001 From: Ashwin Srinath Date: Tue, 15 Oct 2019 15:20:05 -0400 Subject: [PATCH 18/19] Update include path --- cpp/tests/utilities/base_fixture.hpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cpp/tests/utilities/base_fixture.hpp b/cpp/tests/utilities/base_fixture.hpp index 5459daea9b3..68ab152f650 100644 --- a/cpp/tests/utilities/base_fixture.hpp +++ b/cpp/tests/utilities/base_fixture.hpp @@ -17,7 +17,7 @@ #pragma once #include "cudf_gtest.hpp" -#include "cudf_test_utils.cuh" +#include "legacy/cudf_test_utils.cuh" #include #include From 71e9b04a520c2ff1038c745ae1a7eaaa3d95a9c3 Mon Sep 17 00:00:00 2001 From: Ashwin Srinath <3190405+shwina@users.noreply.github.com> Date: Tue, 15 Oct 2019 15:27:09 -0400 Subject: [PATCH 19/19] Update cpp/tests/binaryop/unit/jit-cache-test.hpp Co-Authored-By: Paul Taylor --- cpp/tests/binaryop/unit/jit-cache-test.hpp | 1 - 1 file changed, 1 deletion(-) diff --git a/cpp/tests/binaryop/unit/jit-cache-test.hpp b/cpp/tests/binaryop/unit/jit-cache-test.hpp index 41308581aac..42d643c1e6e 100644 --- a/cpp/tests/binaryop/unit/jit-cache-test.hpp +++ b/cpp/tests/binaryop/unit/jit-cache-test.hpp @@ -17,7 +17,6 @@ #pragma once #include -#include #include #include